U.S. patent application number 09/945847 was filed with the patent office on 2002-04-18 for image data processing apparatus and electronic camera.
This patent application is currently assigned to NIKON CORPORATION. Invention is credited to Suzuki, Masahiro.
Application Number | 20020044778 09/945847 |
Document ID | / |
Family ID | 18756843 |
Filed Date | 2002-04-18 |
United States Patent
Application |
20020044778 |
Kind Code |
A1 |
Suzuki, Masahiro |
April 18, 2002 |
Image data processing apparatus and electronic camera
Abstract
An image data processing apparatus includes a data size
conversion device that changes a data size of an image data at an
optional ratio, the image data has a plurality of pixels each of
which includes any one of a plurality of color components, and the
plurality of color components being arranged in a specific order.
The data size conversion device newly calculates a value of color
component of each pixel after changing the data size based upon
values of color components of a plurality of same color pixels
before changing the data size, while maintaining the order of
arrangement of the plurality of color components.
Inventors: |
Suzuki, Masahiro;
(Inzai-shi, JP) |
Correspondence
Address: |
OLIFF & BERRIDGE, PLC
P.O. BOX 19928
ALEXANDRIA
VA
22320
US
|
Assignee: |
NIKON CORPORATION
2-3 MARUNOUCHI 3-CHOME, CHIYODA-KU
TOKYO
JP
100-8331
|
Family ID: |
18756843 |
Appl. No.: |
09/945847 |
Filed: |
September 5, 2001 |
Current U.S.
Class: |
396/429 |
Current CPC
Class: |
H04N 1/56 20130101; G06T
3/4015 20130101; H04N 1/3935 20130101; H04N 5/225 20130101 |
Class at
Publication: |
396/429 |
International
Class: |
G03B 017/48 |
Foreign Application Data
Date |
Code |
Application Number |
Sep 6, 2000 |
JP |
2000-270396 |
Claims
What is claimed is:
1. An image data processing apparatus comprising: a data size
conversion device that changes a data size of an image data at an
optional ratio, the image data having a plurality of pixels each of
which includes any one of a plurality of color components, the
plurality of color components being arranged in a specific order,
wherein said data size conversion device newly calculates a value
of color component of each pixel after changing the data size based
upon values of color components of a plurality of same color pixels
before changing the data size, while maintaining the order of
arrangement of the plurality of color components.
2. An image data processing apparatus according to claim 1, further
comprising: an interpolation processing device that performs
interpolation processing to obtain a value of color component that
a corresponding pixel does not possess, wherein said data size
conversion device changes the data size of the image data before
the image data is subjected to interpolation processing by said
interpolation processing device.
3. An image data processing apparatus according to claim 2, further
comprising: a changeover device that changes over image data to be
supplied to said interpolation device between image data a data
size of which has been changed by said data size conversion device
and image data a data size of which has not been changed by said
data size conversion device, wherein said interpolation processing
device performs the interpolation processing with a common
algorithm on both the image data the data size of which has been
changed and the image data the data size of which has not been ch
anged.
4. An image data processing apparatus according to claim 1, wherein
said data size conversion device calculates the value of color
component of each pixel after changing the data size by taking a
relative positional relationship between the each pixel after
changing the data size and the plurality of same color pixels
before changing the data size.
5. An image data processing apparatus, comprising: a color
separation device in which a plurality of color filters, each of
which passes light of one of a plurality of color components, are
arranged in a specified order, and which separates an image of a
subject into said plurality of color components; an imaging device
which images the image of the subject which has been
color-separated by said color separation device with a plurality of
pixels; an A/D conversion device which A/D converts an image signal
outputted from said imaging device; and a data size conversion
device which changes a data size of the image data after A/D
conversion at an optional ratio, wherein said data size conversion
device newly calculates a value of color component of each pixel
after changing the data size based upon values of color components
of a plurality of same color pixels before changing the data size,
while maintaining the order of arrangement of the plurality of
color components.
6. An electronic camera comprising: a color separation device in
which a plurality of color filters, each of which passes light of
one of a plurality of color components, are arranged in a specified
order, and which separates an image of a subject into said
plurality of color components; an imaging device which images the
image of the subject which has been color-separated by said color
separation device with a plurality of pixels; an A/D conversion
device which A/D converts an image signal outputted from said
imaging device; and a data size conversion device which changes a
data size of the image data after A/D conversion at an optional
ratio, wherein said data size conversion device newly calculates a
value of color component of each pixel after changing the data size
based upon values of color components of a plurality of same color
pixels before changing the data size, while maintaining the order
of arrangement of the plurality of color components.
7. An image data processing method comprising: obtaining an image
data that has a plurality of pixels each of which includes any one
of a plurality of color components which are arranged in a specific
order; and changing a data size of the image data at an optional
ratio, wherein a value of color component of each pixel after
changing the data size is newly calculated based upon values of
color components of a plurality of same color pixels before
changing the data size in order to change the data size while
maintaining the order of arrangement of the plurality of color
components.
8. A computer-readable computer program product containing a
control program for image data size conversion processing, the
control program comprising instructions of: obtaining an image data
that has a plurality of pixels each of which includes any one of a
plurality of color components which are arranged in a specific
order; and changing a data size of the image data at an optional
ratio, wherein a value of color component of each pixel after
changing the data size is newly calculated based upon values of
color components of a plurality of same color pixels before
changing the data size in order to change the data size while
maintaining the order of arrangement of the plurality of color
components.
9. A computer-readable computer program product according to claim
8, wherein the computer-readable computer program product is a
recording medium on which the control program is recorded.
10. A computer-readable computer program product according to claim
8, wherein the computer-readable computer program product is a
carrier wave in which the control program is embodied as a data
signal.
Description
[0001] The disclosure of the following priority application is
herein incorporated by reference:
[0002] Japanese Patent Application No. 2000-270396 filed Sep. 6,
2000.
BACKGROUND OF THE INVENTION
[0003] 1. Field of the Invention
[0004] The present invention relates to an image data processing
apparatus which performs data size conversion upon image data which
have been imaged, for example, via color separation filters of the
Bayer type, and to an electronic camera.
[0005] 2. Description of the Related Art
[0006] There is a per se known type of electronic still camera
which performs specified image processing upon image data produced
from an image of a photographic subject which has been imaged
through a photographic lens by an imaging device incorporating an
imaging element such as a CCD or the like and outputted by the
imaging device. In an imaging device of such an electronic still
camera, a color separation filter is provided upon the imaging
element for forming a colored image. FIG. 15 is a figure for
explanation of a Bayer type color separation filter, in which
primary color filters for R color, G color, and B color are
arranged in a checkerboard pattern in correspondence to the pixels
of the imaging element. As shown in FIG. 15, in this Bayer
arrangement, a filter for the same color component is provided at
each second pixel in both the horizontal direction and the vertical
direction in which the pixels are arranged. It is necessary to
treat image data which has been imaged through such a color
separation filter in such a manner as to preserve the Bayer
arrangement. This is because, if the Bayer arrangement is
disturbed, it becomes impossible to reproduce the colors of the
photographic subject from the image data.
[0007] In the case of performing reduction (shrinkage) size
conversion upon the above described image data, if the pixel data
are read out while subsampling or culling every second pixel, or
every fourth pixel, . . . in the horizontal direction and in the
vertical direction, i.e. at a multiple of two, the order of the
color components which correspond to the pixel data before
subsampling and the order of the color components of the pixel data
which have been subsampled and read out agree with one another. The
shaded pixels in FIG. 15 are the pixel positions in the case of
reading out at a rate of one pixel every five pixels. When
performing reduction size conversion by subsampling while reading
out in this manner, it is only possible to perform size conversion
at a reduction ratio by subsampling at a multiple of two in the
horizontal direction and the vertical direction respectively, in
other words at a fixed reduction ratio like 2/4, 2/6. Furthermore,
even if it is arranged that the Bayer arrangement after performing
reduction size conversion is not disturbed, the spatial frequency
is reduced by the subsampling, which gives rise to the problem of
undesirable generation of moire due to subsampling.
SUMMARY OF THE INVENTION
[0008] The objective of the present invention is to provide an
image data processing apparatus and an electronic camera which
perform image data size conversion processing to convert image data
size at any optional ratio without confusing the order of
arrangement of the color components of image data which have been
imaged through a color separation filter such as one of the Bayer
type.
[0009] An image data processing apparatus according to the present
invention comprises a data size conversion device that changes a
data size of an image data at an optional ratio, the image data has
a plurality of pixels each of which includes any one of a plurality
of color components, and the plurality of color components being
arranged in a specific order. The data size conversion device newly
calculates a value of color component of each pixel after changing
the data size based upon values of color components of a plurality
of same color pixels before changing the data size, while
maintaining the order of arrangement of the plurality of color
components.
[0010] In this image data processing apparatus, it is preferred
that an interpolation processing device that performs interpolation
processing to obtain a value of color component that a
corresponding pixel does not possess, is further provided. The data
size conversion device changes the data size of the image data
before the image data is subjected to interpolation processing by
the interpolation processing device. In this case, it is preferred
that a changeover device that changes over image data to be
supplied to the interpolation device between image data a data size
of which has been changed by the data size conversion device and
image data a data size of which has not been changed by the data
size conversion device, is further provided. The interpolation
processing device performs the interpolation processing with a
common algorithm on both the image data the data size of which has
been changed and the image data the data size of which has not been
changed.
[0011] Also, it is preferred that the data size conversion device
calculates the value of color component of each pixel after
changing the data size by taking a relative positional relationship
between the each pixel after changing the data size and the
plurality of same color pixels before changing the data size.
[0012] An image data processing apparatus according to the present
invention comprises: a color separation device in which a plurality
of color filters, each of which passes light of one of a plurality
of color components, are arranged in a specified order, and which
separates an image of a subject into the plurality of color
components; an imaging device which images the image of the subject
which has been color-separated by the color separation device with
a plurality of pixels; an A/D conversion device which A/D converts
an image signal outputted from the imaging device; and a data size
conversion device which changes a data size of the image data after
A/D conversion at an optional ratio. The data size conversion
device newly calculates a value of color component of each pixel
after changing the data size based upon values of color components
of a plurality of same color pixels before changing the data size,
while maintaining the order of arrangement of the plurality of
color components.
[0013] An electronic camera according to the present invention
comprises: a color separation device in which a plurality of color
filters, each of which passes light of one of a plurality of color
components, are arranged in a specified order, and which separates
an image of a subject into the plurality of color components; an
imaging device which images the image of the subject which has been
color-separated by the color separation device with a plurality of
pixels; an A/D conversion device which A/D converts an image signal
outputted from the imaging device; and a data size conversion
device which changes a data size of the image data after A/D
conversion at an optional ratio. The data size conversion device
newly calculates a value of color component of each pixel after
changing the data size based upon values of color components of a
plurality of same color pixels before changing the data size, while
maintaining the order of arrangement of the plurality of color
components.
[0014] An image data processing method according to the present
invention comprises: obtaining an image data that has a plurality
of pixels each of which includes any one of a plurality of color
components which are arranged in a specific order; and changing a
data size of the image data at an optional ratio. And a value of
color component of each pixel after changing the data size is newly
calculated based upon values of color components of a plurality of
same color pixels before changing the data size in order to change
the data size while maintaining the order of arrangement of the
plurality of color components.
[0015] A computer-readable computer program product according to
the present invention containing a control program for image data
size conversion processing. The control program comprises
instructions of: obtaining an image data that has a plurality of
pixels each of which includes any one of a plurality of color
components which are arranged in a specific order; and changing a
data size of the image data at an optional ratio. And a value of
color component of each pixel after changing the data size is newly
calculated based upon values of color components of a plurality of
same color pixels before changing the data size in order to change
the data size while maintaining the order of arrangement of the
plurality of color components.
[0016] In this computer-readable computer program product, it is
preferred that the computer-readable computer program product is a
recording medium on which the control program is recorded.
[0017] Also, it is preferred that the computer-readable computer
program product is a carrier wave in which the control program is
embodied as a data signal.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] FIG. 1 is a figure showing the structure of a single lens
reflex electronic still camera which is an embodiment of the
present invention.
[0019] FIG. 2 is a block diagram of an embodiment of a signal
processing system in this single lens reflex electronic still
camera.
[0020] FIG. 3 is a block diagram for explanation of a circuit which
performs line processing in the signal processing system shown in
FIG. 2.
[0021] FIG. 4 is a block diagram for explanation of a circuit which
performs block processing in the signal processing system shown in
FIG. 2.
[0022] FIG. 5 is a figure showing a color separation filter in the
Bayer arrangement.
[0023] FIG. 6 is a figure for explanation of the details of the
processing performed by a G interpolation circuit.
[0024] FIG. 7 is a figure for explanation of the details of the
processing performed by a band pass filter.
[0025] FIG. 8 is a figure for explanation of the details of the
processing performed by a low pass filter.
[0026] FIG. 9 is a figure for explanation of the details of the
processing performed by a color difference signal generation
circuit.
[0027] FIG. 10 is a figure showing an example of data which is
processed by an interpolation/LPF circuit.
[0028] FIG. 11 is a figure for explanation of the details of the
processing performed by this interpolation/LPF circuit.
[0029] FIG. 12 is a figure for explanation of the details of the
processing performed by a median circuit.
[0030] FIG. 13 is a flowchart shoiwg a program which is started
when a full press switch is actuated.
[0031] FIG. 14 is a figure showing a color separation filter
arranged according to the complementary color filter arrangement
method.
[0032] FIG. 15 is a figure for explanation of a subsampling
procedure for image data which have been imaged through a color
separation filter of the Bayer type.
[0033] FIG. 16 is a figure illustrating that an image data size
conversion processing program is provided via a recording medium or
a telecommunication line.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0034] In the following, embodiments of the present invention will
be explained with reference to the figures.
[0035] Embodiment 1
[0036] As shown in FIG. 1, the single lens reflex electronic still
camera according to the present invention is comprised of a camera
main body 70, a viewfinder device 80 which can be fitted to or
removed from the camera main body 70, and an interchangeable lens
90 which comprises a photographic lens 91 and an aperture 92 and
which can be fitted to or removed from the camera main body 70.
Light from a photographic subject enters into the camera main body
70 through the interchangeable lens 90, and, before shutter
release, is directed by a quick return mirror 71 which is in its
position shown by the dotted lines into the viewfinder device 80,
where it is focused into an image upon a viewfinder matte 81. This
image of the photographic subject is also directed by a pentaprism
82 into an eyepiece lens 83. On the other hand, after shutter
release, the quick return mirror 71 is rotated to its position
shown by solid lines in the figure, and the light from the
photographic subject passes through a shutter 72 and forms an image
upon an imaging device 73. Before shutter release, the image of the
photographic subject is incident via a prism 84 and an imaging lens
85 upon a white balance sensor 86, and this white balance sensor 86
detects the color temperature of the photographic subject.
[0037] FIG. 2 is a block diagram of the single lens reflex
electronic still camera according to the first preferred
embodiment. A half press signal and a full press signal are
respectively inputted to a CPU 21 from a half press switch 22 and a
full press switch 23, both of which are actuated by the operation
of a shutter release button not shown in the figures. Furthermore,
an actuation signal from a resize switch 40 for converting the data
size of the image data is inputted to the CPU 21. When the half
press signal is inputted to the CPU 21, the CPU 21 controls the
operation of a CCD 26 of the imaging device 73 via a timing
generator 24 and a driver 25. The operational timings of an analog
signal processing circuit 27 and of an A/D conversion circuit 28
are controlled by the output signal of the timing generator 24.
Furthermore, the CPU 21 controls the driving of a white balance
detection processing circuit 35. A color filter is provided upon
the pixel region of the CCD 26.
[0038] When subsequently to the actuation to ON of the half press
switch 22 the full press switch 23 is actuated to ON, the quick
return mirror 71 is rotated to its upper position. The light from
the photographic subject which passes through the interchangeable
lens 90 is focused thereby into an image upon the light reception
surface of the CCD 26, and signal electric charges are accumulated
in the CCD 26 according to the brightness of the image of the
photographic subject. These signal electric charges accumulated in
the CCD 26 are emitted by the driver 25 and are inputted to the
analog signal processing unit 27 which comprises an AGC circuit and
a CDS circuit. This analog signal processing circuit 27 performs
analog processing such as gain control, noise removal and the like
upon the analog image signal which has been inputted. The image
signal after this analog processing is converted into a digital
signal by the A/D conversion circuit 28. The image data which has
been thus converted into digital format is fed to an image
processing circuit 29 which may for example be an ASIC, which
performs image pre-processing such as white balance adjustment,
contour compensation, gamma correction, and the like.
[0039] The white balance detection processing circuit 35 comprises
a white balance sensor 35A (the white balance sensor 86 of FIG. 1)
which is a color temperature sensor, an A/D conversion circuit 35B
which converts the analog signal from the white balance sensor 35A
into a digital signal, and a CPU 35C which generates a white
balance adjustment signal based upon this digital color temperature
signal. The white balance sensor 35A may, for example, comprise a
plurality of photoelectric conversion elements for red light, blue
light and green light each of which has its own characteristic
sensitivity, and said white balance sensor 35A receives light from
the image of the photographic field as a whole. The CPU 35C
calculates a R-gain and a B-gain based upon the output of a
plurality of photoelectric conversion elements. These calculated
gains are transferred to predetermined registers of the CPU 21 and
are stored therein. Furthermore, the white balance sensor 86 of
FIG. 1 may be constituted by a two dimensional CCD of 24
columns.times.20 rows. In this case, the CCD is divided into 16
regions, and in each of the regions there is arranged a plurality
of elements, each of which has its own sensitivity to red light,
blue light or green light.
[0040] If the resize switch 40 is set for image size conversion,
the digital image data which has thus been pre-processed is further
subjected to image data size conversion processing. The image data
after resize processing is then subjected to format processing
(image post-processing) for JPEG compression, and then is
temporarily stored in a buffer memory 30.
[0041] This image data which has been stored in the buffer memory
30 is processing into image data for display by a display image
generation circuit 31, and is then displayed upon an external
monitor 32 such as a LCD or the like as the result of photography.
Furthermore, this image data which has been stored in the buffer
memory 30 is also subjected to data compression at a predetermined
compression ratio by the JPEG method, and is then stored upon a
recording medium 34 such as a compact flash memory card (CF card)
or the like.
[0042] FIGS. 3 and 4 are block diagrams showing the details of the
image processing circuit 29. FIG. 3 shows a line processing circuit
100 which performs line by line signal processing upon the image
data from the CCD 26. The 12 bit R, G, and B signals which are
outputted from the A/D conversion circuit 28 are subjected to the
abovementioned image pre-processing. And FIG. 4 shows a block
processing circuit 200 which performs signal processing upon the
image data which have been processed by the line processing circuit
100 for each n x m pixels data at a time, in other words one block
at a time. Image post-processing may be performed upon the image
data in the manner described above for each 20.times.20 pixel
region, for each 16.times.16 pixel region, for each 12.times.12
pixel region, or for each 8.times.8 pixel region, as appropriate.
It should be understood that, in this specification, the image
processing circuit 29 will be explained in terms of a hardware
implementation thereof for the convenience of description, although
in actual fact it could be implemented in software by utilizing a
plurality of processors.
[0043] In FIG. 3, the line processing circuit 100 is shown as being
comprised of a defect correction circuit 101, a digital clamp
circuit 102, again circuit 103, a white balance circuit 104, a
black level circuit 105, a gamma correction circuit 106, and an
average value and histogram calculation circuit 107.
[0044] The defect correction circuit 101 corrects the data in the
output of the CCD 26, each line at a time in point order (point by
point), for any pixels which has any defect (which are specified in
advance, and whose addresses are set into registers of the CPU 21).
For each line at a time in point order, the digital clamp circuit
102 subtracts from the signal for each pixel of this line outputted
by the CCD 26 the weighted average of the signals from a plurality
of pixels, which are thus used as so-called optical black. For each
line at a time in the output of the CCD 26 in point order, the gain
circuit 103 impartially multiplies each of the R, G, and B signals
which are outputted from the CCD 26 by a predetermined gain, and
also performs deviation correction of the sensitivity of the CCD 26
for the G signal. Moreover, the gain circuit 103 also performs
deviation correction of sensitivity ratio of the CCD 26 for the R
and B signals.
[0045] For each line at a time in the output of the CCD 26 in point
order, the white balance circuit 104 multiplies the R and B signals
by the white balance adjustment coefficients which are determined
in advance as described above and are stored in advance in the
registers of the CPU 21, in other words by the R gain and the B
gain. Based upon the image data which have been corrected by this
white balance circuit 104, the white balance is further subjected
to fine adjustment by a white balance fine adjustment circuit which
will be described hereinafter. For each line at a time in the
output of the CCD 26 in point order, the black level circuit 105
adds to each of the R, G, and B signals a value which is determined
in advance and is stored in a register of the CPU 21. And the gamma
correction circuit 106 performs gamma correction using a gradation
look up table for each line at a time in the output of the CCD 26
in point order. It should be understood that the 12 bit R, G, and B
signals are converted by this gamma correction into 8 bit RGB
data.
[0046] The average value and histogram calculation circuit 107
extracts from within the image data after gamma correction the
image data for a 512.times.512 region which is specified, for
example, by taking the central portion of the focus detection
region as a center, and calculates a RF-gain for white balance fine
adjustment for the R signal and a BF-gain for white balance fine
adjustment for the B signal using the following Equations (1) and
(2). These calculated values for the RF-gain and the BF-gain are
stored in registers of the CPU 21. FIG. 5 is a figure showing a
color separation filter which is arranged over the pixel region of
the CCD 26 and which utilizes the Bayer arrangement. For example,
if an arrangement of color filters like that shown in FIG. 5 is
provided over the above mentioned specified 512.times.512 pixel
region, the average values of the R, G, and B signals are
calculated by using the Equations (3) through (5), and, as shown in
Equations (1) and (2), the RF-gain and the BF-gain for white
balance fine adjustment are calculated from the ratio of the
average value G-ave of the G signal to the average value R-ave of
the R signal, and the ratio of the average value G-ave of the G
signal to the average value B-ave of the B signal,
respectively.
[0047] [Equations 1]
RF-gain=G-ave/R-ave (1)
BF-gain=G-ave/B-ave (2)
[0048] in which:
R-ave=R-sum/R-pixel number (3)
G-ave=G-sum/G-pixel number (4)
B-ave=B-sum/B-pixel number (5)
[0049] It has been found by experience that excellent results are
obtained for adjustment of white balance (the overall white
balance) by this average value method in which the average values
for gradation of each of the R, G, and B signals resulting from the
input data are obtained.
[0050] In FIG. 4, the block processing circuit 200 is comprised of
a white balance fine adjustment circuit 210, an image data size
conversion processing circuit 240, a changeover circuit 250, and an
interpolation/contour processing circuit 220. The white balance
fine adjustment circuit 210 performs fine adjustment of the white
balance, with respect to the R signal and the B signal which are
stored in the buffer memory 30 after the above described processing
up to the gamma correction circuit 106, by multiplying each of the
R and B signals in the specified pixel region by, respectively, the
RF-gain and the BF-gain which are used for white balance fine
adjustment and which have been calculated by the average value
circuit 107.
[0051] If data size conversion has been set by the use of the
resize switch 40, the image data size conversion processing circuit
240 converts the amount of data, in other words the data size,
which has been generated for one photographic frame of image data
without performing any subsampling process. The image data after
data size conversion is outputted as image data for each
20.times.20 pixel region. The present invention is distinguished in
particular by the fact that data size conversion is performed
without discarding the order of the color components which
corresponds to the arrangement of the color components of the color
separation filter which is disposed over the pixel region of the
CCD 26, and moreover without lowering the spatial frequency of the
image data. During conversion of the data size, it goes without
saying that information relating to the contours of the
photographic subject and so on is preserved.
[0052] In this description of the first preferred embodiment of the
present invention, the example will be employed of performing size
conversion for a single frame by an area ratio of 9/16, in other
words when resizing the data size in both the vertical direction
and the horizontal direction by a ratio of 3/4. The term of
resizing means that a new image data which has pixels a number of
which is different from and values of which are different from the
image data before resizing is generated and that a new pixel plane
which has a spatial frequency different from one in the image data
before resizing is generated. The resizing process calculates the
data for one pixel by linear interpolation by using data of the
same color signals which correspond to two adjacent pixels which
position every two pixels. In this 3/4 resizing process, the data
for three pixels is calculated per the data for each four pixels.
The term of calculating by liner interpolation means, as mentioned
hereinafter, that weighted coefficients are obtained by taking the
relative positional relationship between pixels before resizing and
new pixels generated by resizing into account and a weighted
addition is performed with the weighted coefficients. As a result,
a pixel position after resizing is not overlapped with a pixel
position after resizing and a new plane which has a spatial
frequency different from a plane before resizing is generated.
[0053] In the horizontal direction, the first RGRG . . . line shown
in FIG. 5 will be considered. For example, let the target pixel n
be the one in the first row and the first column which provides a R
signal. The values of the R component and the G component after the
resizing procedure are calculated according to the following
Equations (6) through (11):
[0054] [Equations 2]
R(1,1)={n+(n+2)}/2 (6)
G(1,2)={(n+1)+(n+3)}/2 (7)
R(1,3)={5(n+2)+27(n+4)}/32 (8)
G(1,4)={5(n+3)+27(n+5)}/32 (9)
R(1,5)={27(n+6)+5(n+8)}/32 (10)
G(1,6)={27(n+7)+5(n+9)}/32 (11)
[0055] According to the above Equations (6) through (11), for the R
component, the three R components R(1,1), R(1,3), and R(1,5) in the
group of target pixels from n to (n+8), in other words in a range
over which 5 R signals are obtained, are calculated at almost equal
intervals in the horizontal direction. Furthermore, for the G
component, the three G components G(1,2), G(1,4), and G(1,6) in the
group of target pixels from (n+1) to (n+9), in other words in a
range over which 5 G signals are obtained, are calculated at almost
equal intervals in the horizontal direction. When calculating the
data for each of the three colors using Equations (6) through (11),
the next data are calculated by taking the target pixel (n+8) as
the new target pixel. Accordingly the 3/4 resizing procedure
calculates three data elements from four, since among the groups
described above of 5 R signals and G signals the end one overlaps
the first one of the next group.
[0056] Next, again in the horizontal direction, the second GBGB . .
. line shown in FIG. 5 will be considered. Taking the target pixel
n as the one in the second row and the first column which provides
a G signal, then the values of the G component and the B component
after the resizing procedure are calculated according to the
following Equations (12) through (17):
[0057] [Equations 3]
G(2,1)={n+(n+2)}/2 (12)
B(2,2)={(n+1)+(n+3)}/2 (13)
G(2,3)={5(n+2)+27(n+4)}/32 (14)
B(2,4)={5(n+3)+27(n+5)}/32 (15)
G(2,5)={27(n+6)+5(n+8)}/32 (16)
B(2,6)={27(n+7)+5(n+9)}/32 (17)
[0058] According to the above Equations (12) through (17), for the
G component, the three G components G(2,1), G(2,3), and G(2,5) in
the group of target pixels from n to (n+8), in other words in a
range over which 5 G signals are obtained, are calculated at almost
equal intervals in the horizontal direction. Furthermore, for the B
component, the three B components B(2,2), B(2,4), and B(2,6) in the
group of target pixels from (n+1) to (n+9), in other words in a
range over which 5 B signals are obtained, are calculated at almost
equal intervals in the horizontal direction. When calculating the
data for each of the three colors using Equations (12) through
(17), the next data are calculated by taking the target pixel (n+8)
as the new target pixel n. Accordingly the 3/4 resizing procedure
calculates three data elements from four, since among the groups
described above of 5 G signals and B signals the end one overlaps
the first one of the next group. The RGRG . . . line and the GBGB .
. . line before size conversion respectively become a RGRG . . .
line and a GBGB . . . line after size conversion, and thus the
order of arrangement of the color components in each line before
and after the resizing procedure is the same.
[0059] The same procedure is performed for resizing the image data,
which has been resized in the horizontal direction, in the vertical
direction. Since the order of arrangement of the color components
is the same before and after the resizing procedure, as described
above, this procedure for the vertical direction will be explained
with reference to FIG. 5. In this figure, for the RGRG line in the
first column, taking the target pixel m as the one in the first row
and the first column which provides a R signal, then the values of
the R component and the G component after the resizing procedure
are calculated according to the following Equations (18) through
(23):
[0060] [Equations 4]
R(1,1)={m+(m+2)}/2 (18)
G(2,1)={(m+1)+(m+3)}/2 (19)
R(3,1)={5(m+2)+27(m+4)}/32 (20)
G(4,1)={5(m+3)+27(m+5)}/32 (21)
R(5,1)={27(m+6)+5(m+8)}/32 (22)
G(6,1)={27(m+7)+5(m+9)}/32 (23)
[0061] According to the above Equations (18) through (23), for the
R component, the three R components R(1,1), R(3,1), and R(5,1) in
the group of target pixels from m to (m+8), in other words in a
range over which 5 R signals are obtained, are calculated at almost
equal intervals in the vertical direction. Furthermore, for the G
component, the three G components G(2,1), G(4,1), and G(6,1) in the
group of target pixels from (m+1) to (m+9), in other words in a
range over which 5 G signals are obtained, are calculated at almost
equal intervals in the vertical direction. When calculating the
data for each of the three colors using Equations (18) through
(23), the next data are calculated by taking the target pixel (m+8)
as the new target pixel. Accordingly the 3/4 resizing procedure
calculates three data elements from four, since among the groups
described above of 5 R signals and G signals the end one overlaps
the first one of the next group.
[0062] Next, again in the vertical direction, the second GBGB . . .
column shown in FIG. 5 will be considered. Taking the target pixel
m as the one in the first row and the second column which provides
a G signal, then the values of the G component and the B component
after the resizing procedure are calculated according to the
following Equations (24) through (29):
[0063] [Equations 5]
G(1,2)={m+(m+2)}/2 (24)
B(2,2)={(m+1)+(m+3)}/2 (25)
G(3,2)={5(m+2)+27(m+4)}/32 (26)
B(4,2)={5(m+3)+27(m+5)}/32 (27)
[0064] G(5,2)={27(m+6)+5(m+8)}/32 (28)
B(6,2)={27(m+7)+5(m+9)}/32 (29)
[0065] According to the above Equations (24) through (29), for the
G component, the three G components G(1,2), G(3,2), and G(5,2) in
the group of target pixels from m to (m+8), in other words in a
range over which 5 G signals are obtained, are calculated at almost
equal intervals in the vertical direction. Furthermore, for the B
component, the three B components B(2,2), B(4,2), and B(6,2) in the
group of target pixels from (m+1) to (m+9), in other words in a
range over which 5 B signals are obtained, are calculated at almost
equal intervals in the vertical direction. When calculating the
data for each of the three colors using Equations (24) through
(29), the next data are calculated by taking the target pixel (m+8)
as the new target pixel m. Accordingly the 3/4 resizing procedure
calculates three data elements from four, since among the groups
described above of 5 G signals and B signals the end one overlaps
the first one of the next group. The RGRG . . . line and the GBGB .
. . line before size conversion respectively become a RGRG . . .
line and a GBGB . . . line after size conversion, and thus the
order of arrangement of the color components in each line before
and after the resizing procedure is the same.
[0066] As explained above, by performing resizing processing by a
ratio of 3/4 in both the horizontal direction and in the vertical
direction, it is possible to convert the amount of data which is
generated for one photographic image to 9/16 of its size. It should
be understood that, although for the convenience of explanation the
calculation procedures for the horizontal direction and for the
vertical direction have been explained as being done separately, in
actual fact, the calculation procedures in both these directions
may be performed together as a matrix calculation. The results are
the same when performing these calculations for the two directions
together, as if they were to be performed separately and
independently.
[0067] The changeover circuit 250, upon commands from the CPU 21
(see FIG. 2), outputs to the interpolation/contour processing
circuit 220 either the image data for the 20.times.20 pixel region
which is output in order from the white balance fine adjustment
circuit 210, or the image data for the 20.times.20 pixel region
which is output in order from the image data size conversion
processing circuit 240.
[0068] The interpolation/contour processing circuit 220 performs
formatting procedures for data compression according to the JPEG
method for the block data of each 20.times.20 pixel region in
order, for the image data after white balance fine adjustment, or
after image data size conversion. As results of this formatting
procedure a Y signal of a 16.times.8 pixel region, a Cb signal of
an 8.times.8 pixel region, and a Cr signal of an 8.times.8 pixel
region are generated. The luminance signal Y includes a luminance
signal Y1 for the low frequency component of the G signal and a
contour signal Y2 for its high frequency component, as will be
described hereinafter.
[0069] This interpolation/contour processing circuit 220 is
comprised of a G interpolation circuit 221, a band pass filter
(BPF) 222, a clip circuit 223, a gain circuit 224, a low pass
filter (LPF) 225, a color difference signal generation circuit 226,
an interpolation/low pass filter (LPF) circuit 228, a matrix
circuit 229, an adder 230, and a median circuit 232.
[0070] The G interpolation circuit 221 calculates by interpolation
the G component for the pixel regions of the R signal or the B
signal for the data in the 16.times.16 pixel region around the
center of each block signal for each 20.times.20 pixel region of
the image data which is inputted from the white balance fine
adjustment circuit 210, or from the image data size conversion
processing circuit 240. In other words, as shown in FIG. 6, for the
input data set D20 for a 20.times.20 pixel region, the G
interpolation circuit 221 calculates the G component of the vacancy
(which is the pixel in the third row and the third column, and
provides a B signal) in the middle of the 5.times.5 pixel data
region D51 (from row 1 column 1 to row 5 column 5), and substitutes
this value as the G component of the pixel in the third row and the
third column of the output data set D16 (in which the "B" is
surrounded by a circle) of the 16.times.16 pixel region.
[0071] Next, with regard to the input data set D20 of the
20.times.20 pixel region, the G component of the vacancy (which is
the pixel in the fourth row and the fourth column, and provides a R
signal) in the center of the 5.times.5 pixel data region D52 (from
the second row second column to the sixth row sixth column) is
calculated, and this value is substituted as the G component of the
pixel in the fourth row and the fourth column of the output data
set D16 (in which the "R" is surrounded by a circle) of the
16.times.16 pixel region. By repeating this type of procedure, the
G interpolation procedure is performed for all the vacancies of the
16.times.16 pixel region, and thus the output data set D16 is
obtained. And on the one hand the output data set D12 from this
12.times.12 pixel region is outputted respectively to the band pass
filter 222 and the low pass filter 225, while on the other hand the
output data set D16 from the 16.times.16 pixel region is outputted
to the color difference signal generation circuit 226.
[0072] The band pass filter 222 extracts the medium frequency
component from the G signal of the 12.times.12 pixel region which
is outputted from the G interpolation circuit 221 (however, this is
the frequency component which is high enough to be able to extract
the contour of the photographic subject, and it may for convenience
be termed the high frequency component) In other words, as shown in
FIG. 7, for the input data set D12 for a 12.times.12 pixel region,
the BPF output data is obtained by multiplying the 5.times.5 pixel
region data D5 (from row 5 column 5 to row 9 column 9) by the band
pass filter coefficient, and this value is substituted as the data
item (the bold "G") in the seventh row and seventh column of the
output data set D8 of the 8.times.8 pixel region. By repeating this
type of procedure, all of the pixel data for the 8.times.8 pixel
region is substituted with the G data after BPF, and thus the
output data set D8 is generated.
[0073] The clip circuit 223 clips and cuts each element in the
8.times.8 pixel data region D8 which is output from the band pass
filter 222 to a set level. The gain circuit 224 multiplies the
output of the clip circuit 223 by a gain which is set in
advance.
[0074] The low pass filter 225 extracts the low frequency component
in the G signal of the 12.times.12 pixel region which is outputted
from the G interpolation circuit 221. In other words, as shown in
FIG. 8, for the input data region D12 of the 12.times.12 pixel
region, the 5.times.5 pixel data region D5 (from the fifth row and
fifth column to the ninth row and ninth column) is multiplied by
the low pass filter coefficient and the LPF output data is
obtained, and this value is substituted as the data (the hatched
region) for the seventh row and seventh column of the 8.times.8
pixel region output data set D8.
[0075] By repeating this type of procedure, all of the pixel data
for the 8.times.8 pixel region is substituted with the G data after
LPF, and thus the output data set D8 is generated.
[0076] The color difference signal generation circuit 226, as shown
in FIG. 9, generates intermediate data D16-3 which include a (B-G)
signal and a (R-G) signal, based upon the RGB signal input data
D16-1 for the 16.times.16 pixel region and the G signal input data
D16-2 for the 16.times.16 pixel region which has been outputted
from the G interpolation circuit 221, among the image data which
has been inputted from the white balance fine adjustment circuit
210 or the image data size conversion processing circuit 240.
Furthermore, the intermediate data D16-3 is separated into output
data D16-4 of a (B-G) color difference signal and output data D16-5
of a (R-G) color difference signal.
[0077] The interpolation/LPF circuit 228 inputs the 8 bit (B-G)
signal and (R-G) signal of the 16.times.16 pixel region which are
outputted from the color difference signal generation circuit 226,
performs interpolation calculation for this (B-G) signal and (R-G)
signal one 5.times.5 pixel region at a time, and also
simultaneously performs a low pass filtering procedure in which it
extracts the low band signal therefrom, so that as a result it
outputs a (B-G) signal and a (R-G) signal of a 12.times.12 pixel
region to the Cb and Cr matrix sections of the matrix circuit 229
respectively. Furthermore, a (B-G) signal and a (R-G) signal of a
8.times.8 pixel region are outputted to a Y matrix section of the
matrix circuit 229.
[0078] When the R-G data for the 5.times.5 pixel region are as
shown in FIG. 10, the above described interpolation calculation and
low pass filtering procedure are given by the following Equation
(30):
[0079] [Equations 6] 1 [EQUATIONS 6] I n t e r p R - G ( i , j ) =
[ { R - G ( i - 2 , j - 2 ) + R - G ( i + 2 , j - 2 ) + R - G ( i -
2 , j + 2 ) + R - G ( i + 2 , j + 2 ) } .times. k c1 + { R - G ( i
- 2 , j - 2 ) + R - G ( i + 2 , j - 2 ) + R - G ( i - 1 , j + 2 ) +
R - G ( i + 1 , j + 2 ) } .times. k c2 + { R - G ( i , j - 2 ) + R
- G ( i , j + 2 ) } .times. k c3 + R - G ( i - 1 , j - 1 ) + R - G
( i + 1 , j - 1 ) + R - G ( i - 1 , j + 1 ) + R - G ( i + 1 , j + 1
) } .times. k c5 + { R - G ( i - 2 , j - 1 ) + R - G ( i + 2 , j -
1 ) + R - G ( i - 2 , j + 1 ) + R - G ( i + 2 , j + 1 ) } .times. k
c4 + { R - G ( i , j - 1 ) + R - G ( i , j + 1 ) } .times. k c6 + {
R - G ( i - 2 , j ) + R - G ( i + 2 , j ) } .times. k c7 + { R - G
( i - 1 , j ) + R - G ( i + 1 , j ) } .times. k c8 + { R - G ( i ,
j ) } .times. k c9 ] / ( 2 ^ K t r - g ) ( 30 )
[0080] where kc1-kc9 and Ktr-g are coefficients.
[0081] Generally, if both an interpolation filter and a band
limiting LPF are performed at the same time, there is a limitation
upon the filter coefficient, as follows. For the sake of clarity,
the concept will be explained in terms of a single dimension. It
will be supposed that, among the sampling points after
interpolation, the actual sampling points are at a period of N. For
example, suppose that the sampling points after interpolation are
a, a, b, b, a, a, b, b, . . . where the points a are actual
sampling points and the points b are interpolated points. Thus in
this example the period N is 4. When performing interpolation with
an odd-number degree symmetrical digital filter of (2n+1) degree
(where (2n+1) is larger than N), if the actual sampling points are
uniform, it is necessary for the sampling points after
interpolation also to be uniform, which implies that the filter
coefficients are constrained as described below.
[0082] If the k-th filter coefficient is termed C(k), it is
necessary for N sets of the sum of coefficients to be equal to one
another. 2 [ EQUATIONS 7 ] 2 C ( N .times. i ) = [ C ( N .times. i
+ 1 ) + C ( N .times. i + N - 1 ) ] = [ C ( N .times. i + k ) + C (
N .times. i + N - k ) ]
[0083] Here, as filter coefficient indices, i is an integer greater
than or equal to zero and less than or equal to (2n+1), while k is
an integer greater than or equal to zero and less than n.
[0084] In the two dimensional case, it will be acceptable to
construct a two dimensional filter by implementing together two
filters which are constrained in the same manner in the horizontal
direction and in the vertical direction respectively. In this first
preferred embodiment of the present invention, N is equal to 2
since the sampling points are interpolated at a period of two
pixels as shown in FIGS. 5 and 10, and the sums of the odd numbered
filter coefficients and the sums of the even numbered filter
coefficients must be equal. In other words,
.SIGMA.C(2*i)=.SIGMA.C(2*i+1)
[0085] And, in the two dimensional case, with a 5 degree.times.5
degree symmetrical type filter as in the above mentioned Equation
(30), a following equation is achieved.
4*kc1+2*kc3+4*kc5+2*kc7+kc9=4*kc2+4*kc4+2*kc6+2*kc8
[0086] As an example, the case will be explained of performing
interpolation/LPF processing upon the (R-G) signal while referring
to FIG. 11. For the (R-G) signal of the input data set D16 for the
16.times.16 pixel region, the data set D5 for the 5.times.5 pixel
region (from the third row third column to the seventh row seventh
column) is multiplied by the interpolation/LPF filter coefficient,
and the (R-G) data for its central region (the fifth row and the
fifth column) are calculated, and this is substituted as the fifth
row fifth column data element in the output data set D12 for the
12.times.12 pixel region By repeating this type of procedure, all
of the pixel data for the 12.times.12 pixel region for the (R-G)
signal is subjected to interpolation/LPF processing, and thus the
output data set D12 is obtained. The same procedure is performed
for the (B-G) signal, and thus the output data for the 12.times.12
pixel region is generated.
[0087] The matrix circuit 229 comprises a Y matrix section, a Cb
matrix section, and a Cr matrix section. The Y matrix section
inputs the (B-G) signal and the (R-G) signal of the 8.times.8 pixel
region from the interpolation/LPF circuit 228, and also inputs the
G signal of the 8.times.8 pixel region from the low pass filter
225, and generates a luminance signal Y1 for the low frequency
component of the 8.times.8 pixel region according to the following
Equation (31).
[0088] [Equation 8]
[0089]
Y1(i,j)=[Mkg.times.G(i,j)+Mkr1.times.R-G(i,j)+Mkb1.times.B-G(i,j)]
(31)
[0090] where Mkg, Mkr1, and Mkb1 are matrix coefficients.
[0091] The Cb matrix section and the Cr matrix section respectively
input the (B-G) signal and the (R-G) signal of the 12.times.12
pixel region from the interpolation/LPF circuit 228, and
respectively generate a Cb signal and a Cr signal of the
12.times.12 pixel region according to the following Equations (32)
and (33).
[0092] [Equations 9]
Cr(i,j)=[Mkr2.times.R-G(i,j)+Mkb2.times.B-G(i,j)] (32)
Cb(i,j)=[Mkr3.times.R-G(i,j)+Mkb3.times.B-G(i,j)] (33)
[0093] where Mkr2, Mkr3, Mkb2 and Mkb3 are matrix coefficients.
[0094] The adder 230 adds together the luminance signal Y1 of the
low frequency component of the 8.times.8 pixel region which is
outputted from the matrix circuit 229, and the contour extraction
signal Y2 of the high frequency component of the 8.times.8 pixel
region which is outputted from the gain circuit 224. This contour
extraction signal Y2 which is outputted from the gain circuit 224
consists of the high frequency component extracted from the G
signal of the 16.times.16 pixel region which has been G
interpolated, in other words of an extracted contour. Accordingly a
luminance/contour extraction signal Y(Y1+Y2) for the image as a
whole is calculated by adding together in the adder 230 the
luminance signal Y1 which is calculated according to the above
Equation (31) and the contour extraction signal Y2 which is
calculated by the gain circuit 224. The result of this addition is
stored in the buffer memory 30.
[0095] The median circuit 233 inputs the Cb signal and the Cr
signal of the 12.times.12 pixel region which are outputted from the
matrix circuit 229, performs median processing by utilizing the
nine points of 3.times.3 pixels which are included in the 5.times.5
pixel region, and outputs the Cb signal and the Cr signal of the
8.times.8 pixels.
[0096] In the median processing procedure of this first preferred
embodiment of the present invention, as shown in FIG. 12, among the
data D12 (these data elements are marked with black dots) for the
12.times.12 pixels, the median filtering procedure is performed
upon the nine data elements D3-5 (marked with "X") of 3.times.3
pixels (from the fifth row fifth column to the ninth row ninth
column) which are included in a 5.times.5 pixel region. In other
words, these nine elements of data are sorted into ascending or
descending order, and the median value thereof is taken as the data
value after median processing. And the data item after median
processing which is obtained is substituted as the output data item
D8 in the seventh row seventh column of the 8.times.8 pixels. By
repeating this type of procedure, the output data D8 of the
8.times.8 pixels is obtained for the Cb signal and the Cr signal.
The output data, consisting of the Cr signal and the Cb signal, is
stored in the buffer memory 30.
[0097] For each of the input data elements in the 20.times.20 pixel
region which has been inputted to the block processing circuit 200
as described above, based upon the Y signal of the 16.times.8
pixels which is generated by the adder circuit 230 and upon the Cr
signal and the Cb signal of the 8.times.8 pixels which are
generated by the median circuit 232, the JPEG compression circuit
33 extracts as one unit the Y, Cr, and Cb signals which have been
formatted to an 8.times.8 pixel by the JPEG compression method, and
thus compresses the entire image by repeating a per se known
compression method. The compressed image data are stored upon the
recording medium 34 via the CPU 21.
[0098] The operation of an electronic still camera according to the
above construction will now be explained. When the full press
switch 23 is actuated by the shutter release button being pressed,
the quick return mirror is raised away from the optical path, and
the execution of the photographic sequence program shown in FIG. 13
is commenced. In a step S21, each of the pixels of the CCD 26
accumulates electric charge, and after this accumulation has been
completed the accumulated electric charge for all said pixels is
read out (discharged) in order. In the next step S22, after the
read out image signal has been processed by the analog signal
processing circuit 27, it is converted into digital image data by
the A/D conversion circuit 28. In the next step S23, this image
data is inputted into the image processing circuit 29, and the
image processing described above is performed. This image
processing circuit 29 performs procedures such as white balance
adjustment, gamma gradation correction, image data size conversion
processing, JPEG formatting processing, etc. When this image
processing has been completed the flow of control proceeds to the
next step S24, in which the image data after image processing is
temporarily stored in the buffer memory 30. In the next step S25,
the image data is read out from the buffer memory 30, and this data
is compressed by the JPEG compression circuit 33 And in a
subsequent final step S26, the image data after compression is
stored upon the recording medium 34, and then the procedure shown
in FIG. 13 terminates.
[0099] According to the first preferred embodiment of the present
invention as explained above, the following beneficial effects and
results are obtained.
[0100] (1) The image data size conversion processing circuit 240
performs resizing processing upon the digital image data before the
G interpolation processing is performed by the G interpolation
circuit 221 interior to the interpolation/contour processing
circuit 220. For example, in the opposite case of performing
resizing processing after formatting processing has been performed
by the interpolation/contour processing circuit 220, it would be
necessary to perform resizing processing for the Y signal, the Cb
signal, and the Cr signal which were calculated by the formatting
processing, i.e. for an amount of image data equivalent to the
content of three images. By contrast, according to this first
preferred embodiment of the present invention, it is sufficient to
perform resizing processing for only the amount of image data
corresponding to a single image containing the R, G, and B signals.
Accordingly, as compared with the alternative case of performing
resizing processing for the Y signal, the Cb signal, and the Cr
signal and thus for an amount of image data equivalent to the
content of three images, it is possible greatly to reduce the
processing time and also the required memory capacity.
[0101] (2) The Bayer arrangement of the color components of the
image data, in other words of the R, G, and B signals, is preserved
both before and after the image data size conversion processing
circuit 240 performs the resizing processing. Accordingly, the
interpolation/contour processing circuit 220, without any
relationship with the presence or absence of resizing processing,
is able to perform block processing by unifying n.times.m pixels
(where n and m may be 20, 16, 12, or 8) and treating them a single
block as described above. In other words, the data after resizing
processing and the data which did not perform resizing processing
can perform block processing with a common algorithm. By this
means, the image data size conversion processing circuit 240 does
not impose any change upon the circuitry of a conventional
electronic still camera, and can easily be added later.
[0102] (3) The image data size conversion processing circuit 240
calculates the data for one pixel by linear interpolation using the
same color signals of two pixels each of which is from ever tow
pixels, so as to reduce the size of the data. Accordingly, it is
possible to convert at any desired reduction ratio, which is
different from the case where the data size is converted by a
subsampling procedure. Furthermore, a high quality resized image is
obtained in which there is no tendency to generation of moire due
to subsampling, since the generation of color artifact and
reduction of the spatial frequency are suppressed by the
calculation by linear interpolation.
[0103] In the above described resizing procedure, the data for one
pixel is calculated by linear interpolation by using the data of
the same color signals of the adjacent two pixels which have
another pixel between them. It would also be possible to perform an
interpolation procedure based upon a Sinc function by using the
data of the same color signals of the adjacent five or six pixels
each of which sits every two pixels. To explain this with reference
to FIG. 5, for example, in the case of performing a resizing
procedure by 3/4 in both the horizontal direction and the vertical
direction, taking the position held by the R signal in the first
row and first column as the target pixel n, the values of the R
component and of the G component after the resizing procedure in
the horizontal direction are given by the following Equations (34)
through (39):
[0104] [Equations 10]
R(1,1)={-3n-4(n+2)+70(n+4)+70(n+6)-4(n+8)-3(n+10)}/128 (34)
G(1,2)={-3(n+1)-4(n+3)+70(n+5)+70(n+7)-4(n+9)-3(n+1l)}/128 (35)
R(1,3)={-10(n+4)+42(n+6)+88(n+8)+16(n+10)-8(n+12)}/128 (36)
G(1,4)={-10(n+5)+42(n+7)+88(n+9)+16(n+11)-8(n+13)}/128 (37)
R(1,5)={-8(n+6)+16(n+8)+88(n+10)+42(n+12)-10(n+14)}/128 (38)
G(1,6)={-8(n+7)+16(n+9)+88(n+11)+42(n+13)-10(n+15)}/128 (39)
[0105] According to the above Equations (34) through (39), for the
R component, in the range in which the four R signals from the
target pixel (n+4) through (n+10) are obtained, the three R
components R(1,1), R(1,3), and R(1,5) are calculated at almost
equal intervals in the horizontal direction. Furthermore, for the G
component, in the range in which the four G signals from the target
pixel (n+5) through (n+11) are obtained, the three G components
G(1,2), G(1,4), and G(1,6) are calculated at almost equal intervals
in the horizontal direction. When three data elements for each
color are calculated according to the above Equations (34) through
(39), the next data are calculated by setting the target pixel
(n+8) as the new target pixel n. The calculations for the GBGB . .
. line in the horizontal direction are identical. The RGRG. line
and the GBGB . . . line before size conversion respectively also
become the RGRG . . . line and the GBGB . . . line after size
conversion as well, and the order of arrangement of the color
components before and after the resizing procedure is the same.
[0106] The same procedure is performed in the vertical direction
upon the image data which have been subjected to the above
described resizing procedure in the horizontal direction. For the
first RGRG . . . column in FIG. 5, taking the position held by the
R signal in the first row and first column as the target pixel m,
the values of the R component and of the G component after the
resizing procedure in the horizontal direction are given by the
following Equations (40) through (45):
[0107] [Equations 11]
R(1,1)={-3m-4(m+2)+70(m+4)+70(m+6)-4(m+8)-3(m+10)}/128 (40)
G(2,1)={-3(m+1)-4(m+3)+70(m+5)+70(m+7)-4(m+9)-3(m+11)}/128 (41)
R(3,1)={-10(m+4)+42(m+6)+88(m+8)+16(m+10)-8(m+12)}/128 (42)
G(4,1)={-10(m+5)+42(m+7)+88(m+9)+16(m+11)-8(m+13)}/128 (43)
R(5,1)={-8(m+6)+16(m+8)+88(m+10)+42(m+12)-10(m+14)}/128 (44)
G(6,1)={-8(m+7)+16(m+9)+88(m+11)+42(m+13)-10(m+15)}/128 (45)
[0108] If the calculations for the GBGB . . . line in the vertical
direction are performed in the same manner, it is possible to
convert the data which constitute a single photographic image by
3/4 in both the horizontal direction and the vertical direction, in
other words to convert the size of the entire data set by 9/16. It
should be understood that, although for the convenience of
explanation the calculation procedures for the horizontal direction
and for the vertical direction have been explained as being done
separately, in actual fact, the calculation procedures in both
these directions may be performed together as a matrix calculation.
The results are the same when performing these calculations for the
two directions together, as if they were to be performed separately
and independently.
[0109] Embodiment 2
[0110] In the following description of the second preferred
embodiment of the present invention, by way of example, the case
will be explained of converting the size of a single photographic
image by 9/4, in other words of resizing the data size of a single
image both in the horizontal direction and in the vertical
direction by 3/2. This resizing procedure calculates the data for
one pixel by linear interpolation using the data of the same color
signals of the adjacent two pixels each of which is from every two
pixels. In this 3/2 resizing procedure, the data of three pixels is
calculated based upon the data of two pixels.
[0111] In the horizontal direction, the first RGRG . . . line shown
in FIG. 5 will be considered. For example, let the position held by
the R signal upon the first line and the first column be taken as
the target pixel n. The values of the R component and of the G
component in the horizontal direction after the resizing procedure
are given by the following Equations (46) through (51):
[0112] [Equations 12]
R(1,1)={64n+64(n+2)}/128 (46)
G(1,2)={64(n+1)+64(n+3)}/128 (47)
R(1,3)={108(n+2)+20(n+4)}/128 (48)
G(1,4)={108(n+3)+20(n+5)}/128 (49)
R(1,5)={20(n+2)+108(n+4)}/128 (50)
G(1,6)={20(n+3)+108(n+5)}/128 (51)
[0113] According to the above Equations (46) through (51), for the
R component, the group from the target pixel n to (n+4), in other
words the three R components R(1,1), R(1,3), and R(1,5) are
calculated at almost equal intervals in the horizontal direction in
the range which is held by three R signals. Furthermore, for the G
component, the group from the target pixel (n+1) to (n+5), in other
words the three G components G(1,2), G(1,4), and G(1,6) are
calculated at almost equal intervals in the horizontal direction in
the range which is held by three G signals. When three data
elements for each color are calculated according to the above
Equations (46) through (51), the next data set is calculated by
setting the target pixel (n+4) as the new target pixel n.
Accordingly a 3/2 resizing procedure is performed, since, from the
group of the three R signals and three G signals described above,
the tail one overlaps the head one of the next group, thus ensuring
that three data elements are calculated from two. The calculations
for the GBGB . . . line in the horizontal direction are identical.
The RGRG . . . line and the GBGB . . . line before size conversion
respectively also become the RGRG . . . line and the GBGB . . .
line after size conversion as well, and the order of arrangement of
the color components before and after the resizing procedure is the
same.
[0114] The same procedure is performed in the vertical direction
upon the image data which have been subjected to the above
described resizing procedure in the horizontal direction. For the
first RGRG . . . column in FIG. 5, taking the position held by the
R signal in the first row and first column as the target pixel m,
the values of the R component and of the G component after the
resizing procedure in the horizontal direction are given by the
following Equations (52) through (57):
[0115] [Equations 13]
R(1,1)={64m+64(m+2)}/128 (52)
G(2,1)={64(m+1)+64(m+3)}/128 (53)
R(3,1)={108(m+2)+20(m+4)}/128 (54)
G(4,1)={108(m+3)+20(m+5)}/128 (55)
R(5,1)={20(m+2)+108(m+4)}/128 (56)
G(6,1)={20(m+3)+108(m+5)}/128 (57)
[0116] When three data elements for each color are calculated
according to the above Equations (52) through (57), the next data
set is calculated by setting the target pixel (n+4) as the new
target pixel n. If the calculations for the GBGB . . . lines in the
vertical direction are performed in the same manner, it is possible
to convert the data which constitute a single photographic image by
3/2 in both the horizontal direction and the vertical direction, in
other words to convert the size of the entire data set by 9/4. It
should be understood that, although for the convenience of
explanation the calculation procedures for the horizontal direction
and for the vertical direction have been explained as being done
separately, in actual fact, the calculation procedures in both
these directions may be performed together as a matrix calculation.
The results are the same when performing these calculations for the
two directions together, as if they were to be performed separately
and independently.
[0117] According to the second preferred embodiment of the present
invention as explained above, it is contrived to be able to
increase the image data size, since the image data size conversion
processing circuit 240 calculates the data for one pixel by linear
interpolation using the data of the same color signals of the
adjacent two pixels which have another pixel between them.
Furthermore, any desired magnification ratio may be employed. Yet
further, there is no deterioration of the image quality after
resizing, since the generation of color artifact and reduction of
the spatial frequency are suppressed by the calculation by linear
interpolation.
[0118] In the above described resizing procedure, the data for one
pixel is calculated by linear interpolation by using the data of
the same color signals of the adjacent two pixels each of which is
from every two pixels. It would also be possible to perform an
interpolation procedure based upon a Sinc function by using the
data of the same color signals of the adjacent four pixels each of
which sits every two pixels. To explain this with reference to FIG.
5, for example, in the case of performing a resizing procedure by
3/2 in both the horizontal direction and the vertical direction,
taking the position held by the R signal in the first row and first
column as the target pixel n, the values of the R component and of
the G component after the resizing procedure in the horizontal
direction are given by the following Equations (58) through
(63):
[0119] [Equations 14]
R(1,1)={-12n+76(n+2)+76(n+4)-12(n+6)}/128 (58)
G(1,2)={-12(n+1)+76(n+3)+76(n+5)-12(n+7)}/128 (59)
R(1,3)={-11(n+2)+122(n+4)+19(n+6)-2(n+8)}/128 (60)
G(1,4)={-11(n+3)+122(n+5)+19(n+7)-2(n+9)}/128 (61)
R(1,5)={-2(n+2)+19(n+4)+122(n+6)-11(n+8)}/128 (62)
G(1,6)={-2(n+3)+19(n+5)+122(n+7)-11(n+9)}/128 (63)
[0120] According to the above Equations (58) through (63), for the
R component, in the range which is held by the three R signals from
the target pixel (n+2) to (n+6), the three R components R(1,1),
R(1,3), and R(1,5) are calculated at almost equal intervals in the
horizontal direction. Furthermore, for the G component, in the
range which is held by the three G signals from the target pixel
(n+3) to (n+7), the three G components G(1,2), G(1,4), and G(1,6)
are calculated at almost equal intervals in the horizontal
direction. When three data elements for each color are calculated
according to the above Equations (58) through (63), the next data
set is calculated by setting the target pixel (n+4) as the new
target pixel n. Accordingly a 3/2 resizing procedure is performed,
since, from the group of the three R signals and three G signals
described above, the tail ones overlap the head ones of the next
group, thus ensuring that three data elements are calculated from
two. The calculations for the GBGB . . . lines in the horizontal
direction are identical. The RGRG . . . lines and the GBGB . . .
lines before size conversion respectively also become the RGRG . .
. lines and the GBGB . . . lines after size conversion as well, and
the order of arrangement of the color components before and after
the resizing procedure is the same.
[0121] The same procedure is performed in the vertical direction
upon the image data which have been subjected to the above
described resizing procedure in the horizontal direction. For the
first RGRG . . . column in FIG. 5, taking the position held by the
R signal in the first row and first column as the target pixel m,
the values of the R component and of the G component after the
resizing procedure in the horizontal direction are given by the
following Equations (64) through (69):
[0122] [Equations 15]
R(1,1)={-12m+76(m+2)+76(m+4)-12(m+6)}/128 (64)
G(2,1)={-12(m+1)+76(m+3)+76(m+5)-12(m+7)}/128 (65)
R(3,1)={-11(m+2)+122(m+4)+19(m+6)-2(m+8)}/128 (66)
G(4,1)={-11(m+3)+122(m+5)+19(m+7)-2(m+9)}/128 (67)
R(5,1)={-2(m+2)+19(m+4)+122(m+6)-11(m+8)}/128 (68)
G(6,1)={-2(m+3)+19(m+5)+122(m+7)-11(m+9)}/128 (69)
[0123] If the calculations for the GBGB . . . lines in the vertical
direction are performed in the same manner, it is possible to
convert the data which constitute a single photographic image by
3/2 in both the horizontal direction and the vertical direction, in
other words to convert the size of the entire data set by 9/4. It
should be understood that, although for the convenience of
explanation the calculation procedures for the horizontal direction
and for the vertical direction have been explained as being done
separately, in actual fact, the calculation procedures in both
these directions may be performed together as a matrix calculation.
The results are the same when performing these calculations for the
two directions together, as if they were to be performed separately
and independently.
[0124] Although the case of the use of a color separation filter of
the Bayer type has been explained in the above, it is also possible
to apply the present invention in the case of the complementary
color filter arrangement method. FIG. 14 is a figure for
explanation of a color separation filter in which G colored, Ye
colored, Cy colored, and Ma colored complementary color filters are
arranged in correspondence with the pixels of the CCD 26. It is
also possible to perform resizing procedures by the various methods
described above in this case in which the filters for the same
color component are arranged every two pixels in both the
horizontal direction and in the vertical direction in this
manner.
[0125] It should be understood that the present invention may also
be utilized for image data of a single color in which the entire
data that constitutes a single photographic image is constituted by
the G component for example.
[0126] In the above description of the first preferred embodiment
the case was explained of reduction resizing processing at an area
ratio of 9/16, while with the second preferred embodiment the case
of magnification resizing processing at an area ratio of 9/4 was
explained. The resizing procedure of the present invention can be
set at will to any resizing ratio, and in this aspect it differs
from the simple prior art subsampling procedure. Accordingly, the
resizing ratio may be set to any desired value, and is not to be
considered as being limited to the values described above.
[0127] Although in the above described preferred embodiments the
case has been explained of application to an electronic still
camera, it is also possible, when performing the resizing procedure
upon a personal computer, to store the image data size conversion
processing circuit 240 in the form of software as an image data
size conversion processing program upon a recording medium such as
a CD-ROM or a floppy disk or the like. In such a case, the image
data which has been imaged by the CCD and has been converted to
digital form is stored upon a recording medium for image data of
high capacity, and, after this recording medium has been set into a
personal computer and the image data has been read thereinto, a
resizing procedure like the one described above is performed by the
above described image data size conversion processing program. For
example, in FIG. 3, it is possible to store the raw original R, G,
and B output data from the gamma correction circuit 106 upon the
recording medium 34, to set this recording medium 34 into a
personal computer, and to perform the resizing procedure upon the
raw data.
[0128] Instead of reading in this image data size conversion
processing program from a storage medium upon which said program is
stored by using a personal computer, it would also be acceptable to
take advantage of a transmission medium such as the internet or the
like for transmitting the above described image data size
conversion processing program. In this case, the above described
conversion process for the image data size is performed after the
transmitted program has been read into a personal computer.
[0129] FIG. 16 illustrates how this may be achieved. A personal
computer 300 performs the image data size conversion processing
explained above. The personal computer 300 has a function of
connecting with a telecommunication line 301. A computer 302 is a
server computer which provides the image data size conversion
processing program and stores the image data size conversion
processing program in a recording medium such as a hard disk 303.
The telecommunication line 301 may be a telecommunication line for
connection with the Internet, for personal computer communication
or the like or it may be a dedicated telecommunication line. The
computer 302 reads out the image data size conversion processing
program stored in the hard disk 303, and transmits the image data
size conversion processing program to the personal computer 300 via
the telecommunication line 301. In other words, the image data size
conversion processing program is embodied in a carrier wave as a
data signal and is transmitted via the telecommunication line 301.
In case of providing the image data size conversion processing
program with a recording medium, a CD-ROM 304 or the like is
employed. Thus, the image data size conversion processing program
is provided as various kinds of computer-readable computer program
product, such as a recording medium, a carrier wave and the
like.
[0130] Although the present invention has been explained in terms
of its application to a single lens reflex electronic still camera,
it should be understood that the present invention can also be
applied to the case of an electronic still camera which is not
equipped with an interchangeable lens, or to a digital video camera
which takes a moving image.
[0131] Although in the above described preferred embodiments only
one example of the circuit structure has been shown, for example,
the following modification thereof is also possible. In the G
interpolation processing, the BPF processing, the LPF processing,
and the interpolation/LPF processing which are performed by the
block processing circuit 200, the explanation has been made in
terms of performing image processing in single units of any one of
20.times.20, 16.times.16, 12.times.12, and 8.times.8 blocks. In
correspondence therewith, the explanation has been made in terms of
the image data size conversion processing circuit 240 outputting
image data after resizing in units of a 20.times.20 pixel region.
However, the block size for such processing is not to be considered
as being limited to the above cited example values; it would also
be acceptable, for example, to perform the image processing in
units of one 5.times.5 pixel region.
[0132] In the above explanation, the image data size conversion
processing circuit 240 performed the resizing processing upon the
image data after white balance fine adjustment. As described above,
it is possible to reduce the resize processing time and the memory
capacity which is required for the resizing procedure by performing
the resizing procedure before performing the pixel interpolation
procedure. Accordingly, provided that the resizing procedure is
performed upon the image data before the pixel interpolation
procedure, it does not necessarily need to be performed after the
white balance fine adjustment, and it may be performed, for
example, upon the image data after it has been digitally clamped by
the line processing circuit of FIG. 3.
[0133] To explain the correspondence between the various structural
elements mentioned in the claims and the various structural
elements of the disclosed preferred embodiments of this invention:
the R component, the G component, and the B component corresponds
to the plurality of color components; the color separation filter
corresponds to the color separation means; the CCD 26 corresponds
to the imaging means; the A/D conversion circuit 28 corresponds to
the A/D conversion means; the image data size conversion processing
circuit 240 corresponds to the data size conversion means; and the
interpolation/contour processing circuit 220 corresponds to the
image processing means.
* * * * *