U.S. patent application number 14/692176 was filed with the patent office on 2015-09-03 for method, medium, and system encoding and/or decoding moving pictures by adaptively applying optimal prediction modes.
This patent application is currently assigned to SAMSUNG ELECTRONICS CO., LTD.. The applicant listed for this patent is SAMSUNG ELECTRONICS CO., LTD.. Invention is credited to Dmitri BIRINOV, Dae-sung CHO, Woong-il CHOI, Dae-hee KIM, Hyun-mun KIM, Woo-shik KIM.
Application Number | 20150249831 14/692176 |
Document ID | / |
Family ID | 38687637 |
Filed Date | 2015-09-03 |
United States Patent
Application |
20150249831 |
Kind Code |
A1 |
KIM; Woo-shik ; et
al. |
September 3, 2015 |
METHOD, MEDIUM, AND SYSTEM ENCODING AND/OR DECODING MOVING PICTURES
BY ADAPTIVELY APPLYING OPTIMAL PREDICTION MODES
Abstract
A method, medium, and system encoding and/or decoding a moving
picture. The moving picture encoding method may include selecting a
prediction mode that is optimal for the macro blocks, which
correspond to each other, of the color components of a current
image based on the characteristics of a predetermined image,
generating a predicted image for the current image according to the
selected prediction mode, and encoding a moving picture using the
predicted image. An optimal prediction mode can be adaptively
applied to the macro blocks, which correspond to each other, of the
color components, thereby increasing the moving picture's encoding
and decoding efficiencies.
Inventors: |
KIM; Woo-shik; (Yongin-si,
KR) ; KIM; Hyun-mun; (Yongin-si, KR) ; CHO;
Dae-sung; (Yongin-si, KR) ; BIRINOV; Dmitri;
(Yongin-si, KR) ; KIM; Dae-hee; (Yongin-si,
KR) ; CHOI; Woong-il; (Yongin-si, KR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
SAMSUNG ELECTRONICS CO., LTD. |
Suwon-si |
|
KR |
|
|
Assignee: |
SAMSUNG ELECTRONICS CO.,
LTD.
Suwon-si
KR
|
Family ID: |
38687637 |
Appl. No.: |
14/692176 |
Filed: |
April 21, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
11717208 |
Mar 13, 2007 |
|
|
|
14692176 |
|
|
|
|
60781379 |
Mar 13, 2006 |
|
|
|
Current U.S.
Class: |
375/240.12 |
Current CPC
Class: |
H04N 19/136 20141101;
H04N 19/11 20141101; H04N 19/109 20141101; H04N 19/176 20141101;
H04N 19/147 20141101; H04N 19/50 20141101; G06T 9/007 20130101;
H04N 19/186 20141101; H04N 19/19 20141101; H04N 19/61 20141101;
H04N 19/15 20141101; H04N 19/103 20141101; H04N 19/593
20141101 |
International
Class: |
H04N 19/136 20060101
H04N019/136; H04N 19/147 20060101 H04N019/147; H04N 19/15 20060101
H04N019/15; H04N 19/593 20060101 H04N019/593; H04N 19/176 20060101
H04N019/176; H04N 19/19 20060101 H04N019/19; H04N 19/109 20060101
H04N019/109; H04N 19/61 20060101 H04N019/61; H04N 19/186 20060101
H04N019/186; H04N 19/11 20060101 H04N019/11 |
Foreign Application Data
Date |
Code |
Application Number |
May 30, 2006 |
KR |
10-2006-0049080 |
Claims
1. A decoding method comprising: entropy decoding a bitstream at a
block level to restore intra prediction mode information;
obtaining, from the restored intra prediction mode information, an
intra luma prediction mode of a luma component of a current block;
obtaining an intra chroma prediction mode of a chroma component
corresponding to the intra luma prediction mode of the luma
component according to one of predetermined modes, wherein the
intra chroma prediction mode of the chroma component is identical
to the intra luma prediction mode of the luma component; obtaining
a predicted block using the intra chroma prediction mode of the
chroma component; obtaining residue by decoding the bitstream; and
obtaining a restored image using the obtained residue and the
obtained predicted block, wherein the intra luma prediction mode
and the intra chroma prediction mode are used to determine a
reference pixel from among adjacent pixels in a previously restored
block when obtaining the predicted block.
2. A non-transitory computer readable medium comprising computer
readable code to control at least one processing device to
implement a decoding method, the method comprising: entropy
decoding a bitstream at a block level to restore intra prediction
mode information; obtaining, from the restored intra prediction
mode information, an intra luma prediction mode of a luma component
of a current block; obtaining an intra chroma prediction mode of a
chroma component corresponding to the intra luma prediction mode of
the luma component according to one of predetermined modes, wherein
the intra chroma prediction mode of the chroma component is
identical to the intra luma prediction mode of the luma component;
obtaining a predicted block using the intra chroma prediction mode
of the chroma component; obtaining residue by decoding the
bitstream; and obtaining a restored image using the obtained
residue and the obtained predicted block, wherein the intra luma
prediction mode and the intra chroma prediction mode are used to
determine a reference pixel from among adjacent pixels in a
previously restored block when obtaining the predicted block.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a Continuation of U.S. patent
application Ser. No. 11/717,208, filed Mar. 13, 2007 and which is
pending, and claims the benefit of Provisional Application No.
60/781,379, filed Mar. 13, 2006 and Korean Application No.
10-2006-0049080, filed on May 30, 2006, in the Korean Industrial
Property Office, the disclosures of which are incorporated herein
by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] One or more embodiments of the present invention relate to a
method, medium, and system encoding and/or decoding moving
pictures, and more particularly, to a method, medium, and system
encoding and/or decoding moving pictures in the field of the
H.264/MPEG-4 AVC (Advanced Video Coding) Fidelity Range Extensions
(FRExt) standardization.
[0004] 2. Description of the Related Art
[0005] A recently new RGB encoding technology called "residual
color transformation" has been developed during the H.264/MPEG-4
AVC Fidelity Range Extensions (FRExt) standardization process. This
technology prevents picture quality deterioration from occurring
during the transformation from the RGB color space to the YCbCr
color space. However, RGB encoding and decoding technologies,
according to the H.264/MPEG-4 AVC FRExt standard, are not
satisfactory for providing high encoding and decoding efficiencies
and thus cannot be used in moving picture reproduction.
SUMMARY OF THE INVENTION
[0006] One or more embodiments of the present invention provide a
method, medium, and system encoding and/or decoding moving pictures
using RGB encoding technology, according to the H.264/MPEG-4 AVC
(Advanced Video Coding) Fidelity Range Extensions (FRExt) standard,
to provide high encoding and decoding efficiencies.
[0007] Additional aspects and/or advantages of the invention will
be set forth in part in the description which follows and, in part,
will be apparent from the description, or may be learned by
practice of the invention.
[0008] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a predicted image
generating method, including selecting a prediction mode that is
determined to be optimal for blocks of each color component of a
current image based on characteristics of a predetermined image,
and generating a predicted image for the current image according to
the selected prediction mode.
[0009] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include at least one medium
including computer readable code to control at least one processing
element to implement a predicted image generating method, the
method including selecting a prediction mode that is determined to
be optimal for blocks of each color component of a current image
based on characteristics of a predetermined image, and generating a
predicted image for the current image according to the selected
prediction mode.
[0010] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a predicted image
generating system, including a selection unit selecting a
prediction mode that is determined to be optimal for blocks of each
color component of a current image based on characteristics of a
predetermined image, and a generation unit generating a predicted
image for the current image according to the selected prediction
mode.
[0011] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include an encoding method,
including selecting a prediction mode that is determined to be
optimal for blocks of each color component of a current image based
on characteristics of a predetermined image, generating a predicted
image for the current image according to the selected prediction
mode, generating residues, of each color component, which
correspond to differences between the current image and the
predicted image, and generating a bitstream by encoding the
generated residues.
[0012] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include at least one medium
including computer readable code to control at least one processing
element to implement embodiments of the present invention.
[0013] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include an encoding system,
including a selection unit to select a prediction mode that is
determined to be optimal for blocks of each color component of a
current image based on characteristics of a predetermined image, a
generation unit to generate a predicted image for the current image
according to the selected prediction mode and to generate residues,
of each color component, which correspond to differences between
the current image and the predicted image, and an encoding unit to
generate a bitstream by encoding the generated residues.
[0014] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a predicted image
generating method to decode image information encoded in a
bitstream, with the image information being encoded according to a
prediction mode determined to be optimal for corresponding blocks
of each color component of a corresponding current image, the
method including restoring prediction mode information from the
bitstream, the restored prediction mode information identifying the
prediction mode, and generating a predicted image for the current
image according to the prediction mode identified by the restored
prediction mode information.
[0015] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include at least one medium
including computer readable code to control at least one processing
element to implement a predicted image generating method to decode
image information encoded in a bitstream, with the image
information being encoded according to a prediction mode determined
to be optimal for corresponding blocks of each color component of a
corresponding current image, the method including restoring
prediction mode information from the bitstream, the restored
prediction mode information identifying the prediction mode, and
generating a predicted image for the current image according to the
prediction mode identified by the restored prediction mode
information.
[0016] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a predicted image
generating system to decode image information encoded in a
bitstream, with the image information being encoded according to a
prediction mode determined to be optimal for corresponding blocks
of each color component of a corresponding current image, the
system including a decoding unit to restore prediction mode
information from the bitstream, the restored prediction mode
information identifying the prediction mode, and a generation unit
to generate a predicted image for the current image according to
the prediction mode identified by the restored prediction mode
information.
[0017] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a decoding method to
decode image information encoded in a bitstream, with the image
information being encoded according to a prediction mode determined
to be optimal for corresponding blocks of each color component of a
corresponding current image, the method including restoring
prediction mode information from the bitstream, the restored
prediction mode information identifying the prediction mode,
generating residues that correspond to differences between the
current image and a reference image for the current image,
according to the prediction mode identified by the restored
prediction mode information, generating a predicted image according
to the prediction mode identified by the restored prediction mode
information, and generating a restored image that corresponds to a
sum of the generated residues and the generated predicted
image.
[0018] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include at least one medium
including computer readable code to control at least one processing
element to implement a decoding method to decode image information
encoded in a bitstream, with the image information being encoded
according to a prediction mode determined to be optimal for
corresponding blocks of each color component of a corresponding
current image, the method including restoring prediction mode
information from the bitstream, the restored prediction mode
information identifying the prediction mode, generating residues
that correspond to differences between the current image and a
reference image for the current image, according to the prediction
mode identified by the restored prediction mode information,
generating a predicted image according to the prediction mode
identified by the restored prediction mode information, and
generating a restored image that corresponds to a sum of the
generated residues and the generated predicted image.
[0019] To achieve the above and/or other aspects and advantages,
embodiments of the present invention include a decoding system to
decode image information encoded in a bitstream, with the image
information being encoded according to a prediction mode determined
to be optimal for corresponding blocks of each color component of a
corresponding current image, the system including a decoding unit
to restore prediction mode information from the bitstream, the
restored prediction mode information identifying the prediction
mode, a first generation unit to generate a predicted image for the
current image according to the prediction mode identified by the
restored prediction mode information, and a second generation unit
to generate residues that correspond to differences between the
current image and the predicted image according to the prediction
mode identified by the restored prediction mode information and to
generate a restored image corresponding to a sum of the generated
residues and the generated predicted image.
BRIEF DESCRIPTION OF THE DRAWINGS
[0020] These and/or other aspects and advantages of the invention
will become apparent and more readily appreciated from the
following description of the embodiments, taken in conjunction with
the accompanying drawings of which:
[0021] FIG. 1 illustrates a moving picture encoding system,
according to an embodiment of the present invention;
[0022] FIG. 2 illustrates a residue generation unit, such as that
illustrated in FIG. 1, according to an embodiment of the present
invention;
[0023] FIG. 3 illustrates a method of dividing macro blocks for
inter prediction;
[0024] FIG. 4 illustrates prediction directions defined for intra
prediction;
[0025] FIG. 5 illustrates a residue generation unit, such as that
illustrated in FIG. 1, according to another embodiment of the
present invention;
[0026] FIG. 6 illustrates a residue generation unit, such as that
illustrated in FIG. 1, according to still another embodiment of the
present invention;
[0027] FIGS. 7A and 7B illustrate a 5-tap filter and a 3-tap
filter, respectively, which may be used in embodiments of the
present invention;
[0028] FIG. 8 illustrates a residue generation unit, such as that
illustrated in FIG. 1, according to another embodiment of the
present invention;
[0029] FIG. 9 illustrates a restored image generation unit, such as
that illustrated in FIG. 1, according to an embodiment of the
present invention;
[0030] FIG. 10 illustrates a moving picture decoding system,
according to an embodiment of the present invention;
[0031] FIG. 11 illustrates a predicted image generation unit, such
as that illustrated in FIG. 10, according to an embodiment of the
present invention;
[0032] FIG. 12 illustrates a restored image generation unit, such
as that illustrated in FIG. 10, according to an embodiment of the
present invention;
[0033] FIG. 13 illustrates a residue inverse transformation
portion, such as that illustrated in FIG. 12, according to an
embodiment of the present invention;
[0034] FIGS. 14A and 14B illustrate a moving picture encoding
method, according to an embodiment of the present invention;
and
[0035] FIG. 15 illustrates a moving picture decoding method,
according to an embodiment of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0036] Reference will now be made in detail to embodiments of the
present invention, examples of which are illustrated in the
accompanying drawings, wherein like reference numerals refer to the
like elements throughout. Embodiments are described below to
explain the present invention by referring to the figures.
[0037] In describing embodiments herein, a current image should be
considered as meaning a target image in a current moving picture
that is to be encoded and/or decoded, and a reference image should
be considered as meaning an image that is referred to when encoding
or decoding the current image. Generally, the reference image is a
previous image that precedes the current image, though the
reference image may also be a subsequent image, and/or a plurality
of reference images may be used.
[0038] FIG. 1 is a block diagram of a moving picture encoding
system, according to an embodiment of the present invention.
[0039] Referring to FIG. 1, the moving picture encoding system may
include an optimal mode selection unit 110, a residue generation
unit 120, a frequency domain transformation unit 130, a
quantization unit 400, an entropy encoding unit 150, an inverse
quantization unit 160, a frequency domain inverse transformation
unit 170, and a restored image generation unit 180, for
example.
[0040] The optimal mode selection unit 110 selects an optimal
prediction mode for the macro blocks, which correspond to each
other, of the color components of a current image based on the
characteristics of a sample image. Here, macro blocks, which
correspond to each other, of one color component also correspond to
macro blocks of all the other remaining color components.
[0041] For example, the optimal mode selection unit 110 may select
one of a single prediction mode, which may be commonly applied to
the macro blocks, which correspond to each other, of the color
components of the current image, and a multiple prediction mode,
which may be independently applied to the macro blocks, which
correspond to each other, of the color components of the current
image.
[0042] When the optimal mode selection unit 110 selects the single
prediction mode, a corresponding prediction mode for generating
second residues that correspond to differences between first
residues of the color components can be selected.
[0043] For example, when the optimal mode selection unit 110
selects the single prediction mode, the prediction mode may be, for
example, an "RCT mode" for performing Residual Color Transformation
(RCT), an "IPP transformation mode" for performing Inter-Plane
Prediction (IPP) transformation, or an "RCP transformation mode"
for performing Residual Color Prediction (RCP), noting that
alternative embodiments are equally available. Such RCT, IPP, and
RCP transformations will be described in greater detail below.
[0044] In an embodiment of the present invention, the referenced
sample image may be one of the previous images that precede the
current image. In order to select the optimal prediction mode for
the macro blocks, the optimal mode selection unit 110 may, thus,
select all possible prediction modes for the sample image one by
one, and compare the results of encoding performed according to the
selected prediction mode, thereby selecting a prediction mode that
is optimal for the macro blocks, which correspond to each other, of
the color components of the sample image. The selected prediction
mode may then be used as the optimal prediction mode for the macro
blocks, which correspond to each other, of the color components of
the current image.
[0045] In particular, the optimal mode selection unit 110 may
select a prediction mode that leads to a minimum size of the
resultant bitstream from the encoding of the sample image, and/or a
minimum amount of distortion in quality between the original sample
image and a restored sample image, for example. As will be
described later, the size of the resultant bitstream obtained as
the result of the encoding of the sample image corresponds to the
size of the bitstream generated by the entropy encoding unit 150,
for example, with the restored sample image corresponding to a
restored image generated by the restored image generation unit 180,
for example.
[0046] In an embodiment of the present invention, the optimal mode
selection unit 110 may select the optimal prediction mode using a
Lagrangian optimization method, for example. In other words, the
optimal mode selection unit 110 may calculate the distortion in the
image from the average of the sums of the products of squaring the
differences between the original image and the restored image
obtained using the below Equation 1, for example.
D = i = 0 N ( p i - q i ) 2 Equation 1 ##EQU00001##
[0047] Here, D indicates the degree of picture quality distortion,
p indicates a pixel value in the current image, q indicates a pixel
value in the previous image, and i indicates a pixel index in a
current macro block of the current image.
[0048] Thus, in an embodiment, and as described below in Equation
2, the optimal mode selection unit 110 may multiply the size of the
bitstream "R", e.g., the number of bits in the bitstream, by a
constant "A" and add the degree of picture quality distortion "D"
to the product so as to compensate for a difference, in units
between the degree of picture quality distortion and the size of
the bitstream, thereby calculating a final value "L" to be used for
selecting the optimal prediction mode for the macro blocks, which
correspond to each other, of the color components of the current
image.
L=D+.lamda.R Equation 2
[0049] Here, R indicates the size of the bitstream, and .lamda.
indicates a predetermined constant. The final value L can be
calculated for each prediction method, and the prediction method
with the smallest L may be selected from among the prediction
methods used for the calculation.
[0050] According to an experiment performed using such a Lagrangian
optimization method, the optimal mode selection unit 110 may select
the single prediction mode when the similarity between the color
components of the sample image is high, and select the multiple
prediction mode when the similarity between the color components of
the sample is low.
[0051] The residue generation unit 120 may generate a predicted
image for the macro blocks, which correspond to each other, of the
color components of the current image according to the prediction
mode selected by the optimal mode selection unit 110, and generate
residues that correspond to differences between the current image
and the predicted image. Alternatively, the residue generation unit
120 may generate the predicted image for the macro blocks, which
correspond to each other, of the color components of the current
image according to the prediction mode selected by the optimal mode
selection unit 110, generate first residues that correspond to
differences between the current image and the predicted image, and
generate second residues that correspond to differences between the
first residues for the macro blocks, which correspond to each
other, of the color components, noting that alternative embodiments
are also available.
[0052] The frequency domain transformation unit 130 may further
transform residues generated by the residue generation unit 120,
which are color space residues, into values in the frequency
domain. Alternatively, in an embodiment, the frequency domain
transformation unit 130 may transform the second residues generated
by the residue generation unit 120, which are color space residues,
into values in the frequency domain. According to the H.264/MPEG-4
AVC standard, Discrete Hadamard Transformation (DHT), Discrete
Cosine Transformation (DCT)-based integer transformation, etc., are
suggested as color space to frequency domain transformation
methods, noting that alternative embodiments may equally be
available.
[0053] The quantization unit 140 quantizes the values transformed
by the frequency domain transformation unit 130, e.g., the
quantization unit 140 divides the frequency component values, which
are the results of the transformation by the frequency domain
transformation unit 130, by a quantization parameter, and
approximates the results to integer values.
[0054] The entropy encoding unit 150 further generates a bitstream
by entropy-encoding the values quantized by the quantization unit
140. In particular, in an embodiment of the present invention, the
entropy encoding unit 150 generates a bitstream including
information representing which prediction mode was used in the
moving picture encoding, e.g., as selected by the optimal mode
selection unit 110, by entropy-encoding the quantized values
together with such prediction mode information. According to the
H.264/MPEG-4 AVC standard, for example, Context-Adaptive Variable
Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding
(CABAC), etc., have been suggested as entropy encoding methods.
[0055] In particular, the entropy encoding unit 150 generates the
bitstream including information representing the prediction mode
selected by the optimal mode selection unit 110 in a macro block
header for each color component of the current image. A moving
picture decoding system illustrated in FIG. 10 receives and decodes
this bitstream so that the prediction mode used by the moving
picture encoding system is identified.
[0056] When the prediction modes selected by the optimal mode
selection unit 110 for the macro blocks, which correspond to each
other, of the color components of the current image are identical
to each other, the entropy encoding unit 150 may generate a
bitstream including information representing only one prediction
mode identically selected by the optimal mode selection unit 110
for all the macro blocks making up a single sequence or for all the
macro blocks making up a single picture, at a sequence level or a
picture level, which are upper levels relative to the macro block
level. Accordingly, with all corresponding macro blocks being
encoded with the same prediction mode, the information recorded in
corresponding macro block headers can be omitted, thereby
increasing the encoding efficiency.
[0057] Similarly, when some of the prediction modes selected by the
optimal mode selection unit 110 for the macro blocks of the color
components of the current image are identical, the entropy encoding
unit 150 may generate a bitstream including information
representing only one prediction mode identically selected by the
optimal mode selection unit 110 for those corresponding macro
blocks making up a single sequence or a single picture, at a
sequence level or a picture level, and including information
representing the corresponding prediction modes, which are selected
by the optimal mode selection unit 110 for the remaining
corresponding macro blocks, in macro block headers, thereby
increasing the encoding efficiency.
[0058] The inverse quantization unit 160 may further perform
inverse quantization on the values quantized by the quantization
unit 140, e.g., inverse quantization unit 160 may restore the
frequency component values by multiplying the integer values
approximated by the quantization unit 140 by a quantization
parameter.
[0059] The frequency domain inverse transformation unit 170 may
then restore the residues corresponding to the differences between
the current image and the predicted image by transforming the
frequency component values, which are restored by the inverse
quantization unit 160, which are frequency domain values, into
values in the color space. Alternatively, in an embodiment, the
frequency domain inverse transformation unit 170 restores the
aforementioned second residues, which are frequency domain values
corresponding to the differences between the aforementioned first
residues of the color components, into the values in the color
space.
[0060] The restored image generation unit 180 may generate a
restored image that corresponds to the sum of the predicted image
generated by the residue generation unit 120 and the residues
restored by the frequency domain inverse transformation unit 170.
Alternatively, in an embodiment, the restored image generation unit
180 may generate first residues corresponding to the sums of the
second residues restored by the frequency domain inverse
transformation unit 170 and generate the restored image
corresponding to the sum of the predicted image generated by the
residue generation unit 120 and the generated first residues.
[0061] FIG. 2 illustrates a residue generation unit 120, according
to an embodiment of the present invention.
[0062] Referring to FIG. 2, the residue generation unit 120 may
include a single mode residue generation portion 1211 and a
multiple mode residue generation portion 1212, for example.
[0063] When the prediction mode selected by the optimal mode
selection unit 110 is the single prediction mode, the single mode
residue generation portion 1211 may perform spatial prediction,
such as single intra prediction, for removing special redundancy in
the current image, according to the single prediction mode, or
perform temporal prediction, such as single inter prediction, for
removing temporal redundancy between the current image and the
reference image, according to the single prediction mode.
Subsequently, the single mode residue generation unit 1211 may
generate a residue corresponding to a difference between the
current image and the predicted image.
[0064] In particular, the single mode residue generation portion
1211 may divide the macro blocks, which correspond to each other,
of the color components of the current image into blocks of a
predetermined size which is commonly applied to the macro blocks,
which correspond to each other, of the color components of the
current image, determine motion vectors between the reference image
and the current images, which are commonly applied to the divided
blocks, which correspond to each other, of the color components of
the current image, and generate the predicted image for the current
image from the reference image using the determined motion vectors,
thereby performing single inter prediction.
[0065] For example, the single mode residue generation unit 1211
may divide the macro blocks, which correspond to each other, of the
color components of the current image into equal blocks, determine
motion vectors that are commonly applied to the divided blocks,
which correspond to each other, of the color components, and
generate the predicted image for the current image from the
reference image using the determined motion vectors, thereby
performing single inter prediction.
[0066] FIG. 3 illustrates a method of dividing macro blocks for
inter prediction. Referring to FIG. 3, for inter prediction, a
16.times.16 macro block may be divided into blocks of various sizes
of 16.times.16, 16.times.8, 8.times.16, 8.times.16, 8.times.8,
etc., and a motion vector for each divided block can be determined.
Furthermore, after an 8.times.8 block is further divided into
blocks of sizes of 8.times.8, 8.times.4, 4.times.8, 4.times.4,
etc., a motion vector for each divided block may be determined. As
macro blocks are divided into blocks of smaller size, more refined
motions between the current image and the reference image can be
included in the residues.
[0067] For example, in a YCoCg color space, the single mode residue
generation portion 1211 may divide the macro blocks, which
correspond to each other, of a Y component, a Co component, and a
Cg component equally, for example, into an 8.times.8 size, and
determine a motion vector which is commonly applied to the divided
blocks, which correspond to each other, of the Y component, the Co
component, and the Cg component. As another example, in a RGB color
space, the single mode residue generation portion 1211 may divide
the macro blocks, which correspond to each other, of each of a R
component, a G component, and a B component equally, for example,
into an 8.times.8 size, and determine a motion vector which is
commonly applied to the divided blocks, which correspond to each
other, of the R component, the G component, and the B
component.
[0068] In addition, the single mode residue generation portion 1211
may divide the macro blocks, which correspond to each other, of the
color components of the current image into blocks of a
predetermined size which is commonly applied to the macro blocks,
which correspond to each other, of the color components of the
current image, determine a prediction direction that is commonly
applied to the divided blocks, which correspond to each other, of
the color components, predict blocks making up the current image
from adjacent pixels in the restored image, which is generated by
the restored image generation unit 180, using the determined
prediction direction, and generate a predicted image made up by the
predicted blocks, thereby performing single intra prediction.
[0069] In other words, for example, the single mode residue
generation portion 1211 may divide the macro blocks, which
correspond to each other, of the color components of the current
image into equal blocks, determine a prediction direction which is
commonly applied to the divided blocks, which correspond to each
other, of the color components, predict blocks making up the
current image from adjacent pixels in the restored image, which is
generated by the restored image generation portion 180, using the
determined prediction direction, and generate a predicted image
made up by the predicted blocks, thereby performing single intra
prediction.
[0070] FIG. 4 illustrates prediction directions defined for intra
prediction, according to an embodiment of the present
invention.
[0071] Referring to FIG. 4, after a 16.times.16 macro block has
been divided into blocks of 4.times.4 size, a predicted image may
be generated using 9 prediction directions for each divided block.
Alternatively, in an embodiment, the predicted image may be
generated using 4 prediction directions for a 16.times.16 macro
block. In particular, in the former case, in order to predict
blocks of a 4.times.4 size, i.e., Pa, Pb, through Pq, spatially
adjacent pixels P0, P1, through P12 in the restored image are used.
The blocks Pa, Pb, through Pq are predicted from the adjacent
pixels P0, P1, through P12 using 9 prediction directions from 0 to
9. For example, in a prediction direction 0, adjacent pixels P1,
P2, P3, and P4 are projected in a vertical direction corresponding
to the prediction direction 0, so that Pa, Pe, Pi, and Pm are
predicted from P1; Pb, Pf, Pj, and Pn are predicted from P2; Pc,
Pg, Pk, and Pd are predicted from P3; and Pd, Ph, PI, and Pq are
predicted from P4. Similarly, Pa, Pb, through Pq can be predicted
using other prediction directions as described above.
[0072] For example, in a YCoCg color space, for example, the single
mode residue generation portion 1211 may divide the macro blocks,
which correspond to each other, of each of a Y component, a Co
component, and a Cg component equally, for example, into blocks of
4.times.4 size, and determine a prediction direction that is
commonly applied to the divided blocks, which correspond to each
other, of the Y component, the Co component, and the Cg component.
As another example, in a RGB color space, the single mode residue
generation portion 1211 may divide the macro blocks, which
correspond to each other, of each of a R component, a G component,
and a B component equally, for example, into blocks of 4.times.4
size, and determine a prediction direction that is commonly applied
to the divided blocks, which correspond to each other, of the R
component, the G component, and the B component.
[0073] As described above, since the single mode residue generation
portion 1211 applies the same temporal prediction method and the
same spatial prediction method to all the color components, the
similarity between the residues of the color components become
higher. In addition, since a block size, a motion vector, a
prediction direction, etc., may be commonly applied to all the
color components, there is no need to encode and transmit such
information for each color component. In other words, it is
sufficient to encode and transmit such information only once for
all the color components, thereby increasing the overall encoding
efficiency.
[0074] When the prediction mode selected by the optimal mode
selection unit 110 is the multiple prediction mode, the multiple
mode residue generation portion 1212 may generate a prediction mode
by performing temporal prediction, such as multiple inter
prediction, for removing temporal redundancy between the current
image and the reference image according to the multiple prediction
mode, or by performing spatial prediction, such as multiple intra
prediction, for removing special redundancy in the current image
according to a prediction mode that is independently applied to the
macro blocks, which correspond to each other, of the color
components of the current image.
[0075] In particular, the multiple mode residue generation portion
1212 may divide the macro blocks, which correspond to each other,
of the color components into blocks of sizes which may be
independently applied to each of the macro blocks, which correspond
to each other, of the color components of the current image,
determine motion vectors between the reference image and the
current image, which are independently applied to the divided
blocks of each color component, and generate a predicted image for
the current image using the determined motion vectors, thereby
performing multiple inter prediction.
[0076] In other words, the multiple mode residue generation portion
1212 may divide the macro blocks, which correspond to each other,
of the color components of the current image into blocks of
different sizes, determine different motion vectors for the divided
blocks of each color component, and generate a predicted image for
the current image from the reference image using the determined
motion vectors. Of course, since the multiple mode residue
generation portion 1212 may perform inter prediction according to
prediction modes that are independently applied to the macro
blocks, which correspond to each other, of the color components of
the current image, the multiple mode residue generation portion
1212 may still divide all the macro blocks of the color components
of the current image equally and may determine motion vectors which
are the same for the divided blocks of each color component.
[0077] For example, in the YCoCg color space, the multiple mode
residue generation portion 1212 may divide the macro blocks of the
Y component, the Co component, and the Cg component into blocks of
different sizes, e.g., with the macro blocks of the Y component
divided into blocks of a 4.times.4 size, the macro blocks of the Co
component divided into blocks of an 8.times.8 size, and the macro
blocks of the Cg component divided into blocks of an 8.times.8
size, and determine different motion vectors for the divided blocks
of each color component, i.e., the Y component, the Co component,
and Cg component. In addition, for example, in the RGB color space,
the multiple mode residue generation portion 1212 may divide the
macro blocks of the R component, the G component, and the B
component into blocks of different sizes, e.g., with the macro
blocks of the R component divided into blocks of an 8.times.8 size,
the macro blocks of the G component divided into blocks of a
4.times.4 size, and the macro blocks of the B component divided
into blocks of an 8.times.8 size, and determine different motion
vectors for the divided blocks of each color component, i.e., the R
component, the G component, and the B component.
[0078] In addition, the multiple mode residue generation portion
1212 may divide the macro blocks, which correspond to each other,
of the color components into blocks of sizes that are independently
applied to the macro blocks, which correspond to each other, of the
color components of the current image, determine prediction
directions that are independently applied to the divided blocks of
each color component, predict blocks making up the current image
from adjacent pixels in the restored image generated by the
restored image generation unit 180 using the determined prediction
directions, and generate a predicted image made up by the predicted
blocks, thereby performing multiple intra prediction.
[0079] In other words, the multiple mode residue generation portion
1212 may divide the macro blocks, which correspond to each other,
of the color components of the current image into blocks of
different sizes, determine different prediction directions for the
divided blocks of each color component, predict blocks making up
the current image from adjacent pixels in the restored image
generated by the restored image generation unit 180 using the
generated prediction directions, and generate the predicted image
made up by the predicted blocks, thereby performing multiple intra
prediction. Of course, since the multiple mode residue generation
portion 1212 performs intra prediction according to prediction
modes that are independently applied to the macro blocks, which
correspond to each other, of the color components of the current
image, the multiple mode residue generation portion 1212 may still
divide the macro blocks, which correspond to each other, of the
color components of the current image equally and may determine
motion vectors which are commonly applied to the divided blocks of
the color components.
[0080] For example, in the YCoCg color space, the multiple mode
residue generation portion 1212 may divide the macro blocks of the
Y component, the Co component, and the Cg component into blocks of
different sizes, e.g., with the macro blocks of the Y component
divided into blocks of a 4.times.4 size, the macro blocks of the Co
component divided into blocks of a 16.times.16 size, and the macro
blocks of the Cg component divided into blocks of a 16.times.16
size, and determine different prediction directions for the divided
blocks of each color component, i.e., the Y component, the Co
component, and the Cg component. In addition, for example, in the
RGB color space, the multiple mode residue generation portion 1212
may divide the macro blocks of the R component, the G component,
and the B component into blocks of different sizes, e.g., with the
macro blocks of the R component divided into blocks of a
16.times.16 size, the macro blocks of the G component divided into
blocks of a 4.times.4 size, and the macro blocks of the B component
divided into blocks of a 16.times.16 size, and determine different
prediction directions for the divided blocks of each color
components, i.e., the R component, the G component, and the B
component.
[0081] Furthermore, the multiple mode residue generation portion
1212 may perform multiple inter prediction or multiple intra
prediction on the macro blocks, which correspond to each other, of
the color components of the current image. For example, the
multiple mode residue generation portion 1212 may perform multiple
inter prediction on the macro blocks of one of the color components
of the current image and perform multiple intra prediction on the
macro blocks of the other color components.
[0082] As described above, since the multiple mode residue
generation portion 1212 can perform temporal prediction and spatial
prediction using different temporal prediction and spatial
prediction methods on the different color components, if the
similarity between the color components is low, independent
encoding methods which are most suitable for each color component
can be used for effective prediction encoding, thereby increasing
the overall encoding efficiency. However, it may be sufficient to
encode any of the block sizes, motion vectors, and prediction
directions, which are independently applied to each color
component, only once for transmission. Thus, the encoding
efficiency increases.
[0083] FIG. 5 illustrates a residue generation unit 120, such as
that illustrated in FIG. 1, according to another embodiment of the
present invention.
[0084] Referring to FIG. 5, the residue generation unit 120 may
include a single mode residue generation portion 1221, a residue
transformation portion 1222, and a multiple mode residue generation
portion 1223. The single mode residue generation portion 1221 and
the multiple mode residue generation portion 1223 may perform
similar operations, respectively, as the single mode residue
generation portion 1211 and the multiple mode residue generation
portion 1212 illustrated in FIG. 2.
[0085] However, even after inter prediction or intra prediction is
performed in the single mode residue generation portion 1221,
redundancy between the color components still remains. The residue
transformation portion 1222 may eliminate the redundancy between
the color components. In order to distinguish the residues
generated by single mode residue generation portion 1221 and the
residues generated by the residue transformation portion 1222, the
former can be referred to as "first residues", and the latter can
be referred to as "second residues".
[0086] If the prediction mode selected by the optimal mode
selection unit 110 is the residue transformation mode, the residue
transformation portion 1222 may generate second residues that
correspond to differences between the first residues generated by
the single mode residue generation portion 1221.
[0087] FIG. 6 illustrates a residue generation unit 120, such as
that illustrated in FIG. 1, according to another embodiment of the
present invention.
[0088] Referring to FIG. 6, the residue generation unit 120 may
include a single mode residue generation portion 1231, a residual
color transformation (RCT) portion 1232, an inter-plane prediction
(IPP) transformation portion 1233, a residual color prediction
(RCP) transformation portion 1234, and a multiple mode residue
generation portion 1235, for example. The single mode residue
generation portion 1231 and the multiple mode residue generation
portion 1235 may perform similar operations, respectively, as the
single mode residue generation portion 1211 and the multiple mode
residue generation portion 1212 illustrated in FIG. 2. As described
above, in order to distinguish the residues generated by the single
mode residue generation portion 1231 and the residues generated by
the RCT portion 1232, the IPP transformation portion 1233, and the
RCP transformation portion 1234, the former can be referred to as
"first residues", and the latter can be referred to as "second
residues".
[0089] Methods of generating the second residues that correspond to
the differences between the first residues generated by the single
mode residue generation portion 1231 include, for example, RCT, IPP
transformation, and RCP transformation. Here, RCT can be used for
generating the second residues in the YCoCg color space, and IPP
transformation is for generating the second residues in the RGB
color space. In RCP transformation, which is similar to IPP
transformation, when a residue of the G component is used as a
predictor, the same transformation as IPP transformation can be
performed after noise is removed by predetermined filtering.
[0090] If the prediction mode selected by the optimal mode
selection unit 110 is the RCT mode, the RCT portion 1232 may
generate second residues that correspond to the differences between
the first residues of the Y component, the Co component, and the Cg
component in the YCoCg color space using the below Equation 3, for
example. In particular, the following relationships may be
considered: Y=R+2G+B>>2, Co=R-B>>1, and
Cg=-R+2G-B>>2.
.DELTA..sup.2B=.DELTA.R-.DELTA.B
t=.DELTA.B+(.DELTA..sup.2B>>1)
.DELTA..sup.2R=.DELTA.G-t
.DELTA..sup.2G=t+(.DELTA..sup.2R>>1) Equation 3:
[0091] Here, .DELTA.X denotes first residues, .DELTA.2X denotes
second residues, and ">>" denotes a right shift operation,
which is subsequently equivalent to division by 2, and variable t
is used for the purpose of temporary calculation.
[0092] If the prediction mode selected by the optimal mode
selection unit 110 is the IPP transformation mode, the IPP
transformation portion 1233 may generate second residues that
correspond to the differences between the first residues of the R
component, the G component, and the B component in the RGB color
space according to the IPP transformation mode in view of the below
Equation 4, for example.
.DELTA..sup.2G=.DELTA.G'
.DELTA..sup.2R=.DELTA.R-.DELTA.G'
.DELTA..sup.2B=.DELTA.B-.DELTA.G' Equation 4:
[0093] Here, .DELTA.X denotes first residues, .DELTA..sup.2X
denotes second residues, and .DELTA.X' denotes restored first
residues. In particular, Equation 4 is effective when the G
component includes a large amount of image information. The second
residues can be calculated using the R component or the B component
as a dominant component.
[0094] If the prediction mode selected by the optimal mode
selection unit 110 is the RCP transformation mode, after removing
noise from each of the R component, the G component, and the B
component in the RGB color space according to the RCP
transformation mode using a 5-tap filter, expressed in the below
Equation 5, for example, the RCP transformation portion 1234
generates second residues that correspond to the differences
between the first residues of the R component, the G component, and
the B component.
R'.sub.G,S=4R'.sub.G(i,j)+R'.sub.G(i+1,j)+R'.sub.G(i,j+1)+R'.sup.G(i-1,j-
)+R'.sub.G(i,j-1),
r'.sup.G(i,j)=F(R'.sub.G)=sign(R'.sub.G,S)[(abs(R'.sub.G,S)+4)>>3]
Equation 5:
[0095] In addition, after removing noise from each of the R
component, the G component and the B component in the RGB color
space using a 3-tap filter, expressed in the below Equation 6, for
example, the RCP transformation portion 1234 generates second
residues that correspond to the differences between the first
residues of the R component, the G component, and the B
component.
R'.sub.G,S=R'.sub.G(i-1,j)+2R'.sub.G(i,j)+R'.sub.G(i+1,j),
r'.sup.G(i,j)=F(R'.sub.G)=sign(R'.sub.G,S)[(abs(R'.sub.G,S)+2)>>2]
Equation 6:
[0096] In the above Equations 5 and 6, abs(x) denotes the absolute
value of x, and sign(x) is given by the below Equation 7, for
example.
[0097] Equation 7:
Sign ( x ) = { 1 , x .gtoreq. 0 - 1 , x < 0 . ##EQU00002##
[0098] FIGS. 7A and 7B illustrate a 5-tap filter and a 3-tap
filter, respectively, which are used in an embodiment of the
present invention.
[0099] Referring to FIGS. 7A and 7B, a 5-tap filter used in an
embodiment of the present invention uses 4 neighboring pixels
around a pixel that is used as a predictor, and a 3-tap filter used
in an embodiment of the present invention uses two, left and right
pixels on either side of a pixel that is used as a predictor. In
addition, weights may be used as in Equations 5 and 6.
[0100] FIG. 8 illustrates a residue generation unit 120, such as
that illustrated in FIG. 1, according to an embodiment of the
present invention.
[0101] Referring to FIG. 8, the residue generation unit 120 may
include a multiple mode residue generation portion 1241 and a
residue transformation portion 1242, for example. The multiple mode
residue generation portion 1241 may perform similar operations as
the multiple mode residue generation portion 1212 illustrated in
FIG. 2, for example.
[0102] However, even after inter prediction or intra prediction is
performed in the multiple mode residue generation portion 1241,
redundancy between the color components remains. The residue
transformation portion 1242 may eliminate the redundancy between
the color components. In order to distinguish the residues
generated by the single mode residue generation portion 1241 and
the residues generated by the residue transformation portion 1242,
the former residue can be referred to as "first residues", and the
latter can be referred to as "second residues".
[0103] If the prediction mode selected by the optimal mode
selection unit 110 is the residue transformation mode, the residue
transformation portion 1242 may generate second residues that
correspond to the differences between the first residues generated
by the multiple mode residue generation portion 1241, according to
the residue transformation mode. Here, even after inter prediction
or intra prediction is performed in the multiple mode residue
generation portion 1241, redundancy between the color components
may still remain. However, since the similarity between the color
components is lower than in the single prediction mode, the
compression efficiency in the residue transformation portion 1242
may be lower than the compression efficiency in the residue
transformation portion 1222 illustrated in FIG. 5.
[0104] FIG. 9 illustrates a restored image generation unit 180,
such as that illustrated in FIG. 1, according to an embodiment of
the present invention.
[0105] Referring to FIG. 9, the restored image generation unit 180
may include a residue inverse transformation portion 181 and a
prediction compensation portion 182, for example.
[0106] The residue inverse transformation portion 181 may generate
first residues that correspond to the sums of the second residues
restored by the frequency domain inverse transformation portion
170. For example, the residue inverse transformation portion 181
may generate the first residues of the Y component, the Co
component, and the Cg component, which correspond to the sums of
the second residues of the Y component, the Co component, and the
Cg component, respectively, using the below Equation 8, for
example.
t=.DELTA..sup.2G'-(.DELTA..sup.2R'>>1)
.DELTA.G'=.DELTA..sup.2R'+t
.DELTA.B'=t-(.DELTA..sup.2B'>>1)
.DELTA.R'=.DELTA.B'+.DELTA..sup.2B' Equation 8:
[0107] Here, .DELTA.X' denotes restored first residues, and
.DELTA.2X' denotes restored second residues.
[0108] Alternatively, the residue inverse transformation portion
181 may generate the first residues of the R component, the G
component, and the B component, which correspond to the sums of the
second residues of the R component, the G component, and the B
component, respectively, using the below Equation 9, for
example.
.DELTA.G'=.DELTA..sup.2G'
.DELTA.R'=.DELTA..sup.2R'+.DELTA.G'
.DELTA.B'=.DELTA..sup.2B'+.DELTA.G' Equation 9:
[0109] Here, .DELTA.X' denotes restored first residues, and
.DELTA..sup.2X' denotes restored second residues.
[0110] The prediction compensation portion 182 may generate a
restored image that corresponds to the sum of the predicted image
generated by the residue generation unit 120 and the first residues
generated by the residue inverse transformation portion 181. For
example, the prediction compensation portion 182 may generate a
restored image in the YCoCg color space by calculating the sum of
the predicted image generated by the residue generation unit 120
and the first residues generated by the residue inverse
transformation portion 181. Alternatively, the prediction
compensation portion 182 may generate a restored image in the RGB
color space by calculating the sum of the predicted image generated
by the residue generation unit 120 and the first residues generated
by the residue inverse transformation portion 181. Here, it is
further noted that alternate embodiments are also available.
[0111] FIG. 10 illustrates a moving picture decoding system,
according to an embodiment of the present invention.
[0112] Referring to FIG. 10, the moving picture decoding system may
include an entropy decoding unit 210, an inverse quantization unit
220, a frequency domain inverse transformation unit 230, a
predicted image generation unit 240, and a restored image
generation unit 250, for example.
[0113] The entropy decoding unit 210 may restore integer values
corresponding to the current image and information representing the
optimal prediction mode for the blocks of each color component of
the current image by performing entropy-decoding on a bitstream,
such as a bitstream output from the moving picture encoding system
illustrated in FIG. 1. Here, for example, the optimal prediction
mode for the blocks of each color component of the current image
information can be the prediction mode used in the moving picture
encoding system.
[0114] The inverse quantization unit 220 may restore the frequency
component values by performing inverse quantization on the integer
values restored by the entropy decoding unit 210. In other words,
the inverse quantization unit 220 restores the frequency component
values by multiplying the integer values restored by the entropy
decoding unit 210 by a quantization parameter.
[0115] The frequency domain inverse transformation unit 230 may
generate residues that correspond to differences between the
current image and the predicted image by transforming the frequency
component values restored by the inverse quantization unit 220,
which are frequency domain values, into values in the color space.
Alternatively, in an embodiment, the frequency domain inverse
transformation unit 230 restores second residues that correspond to
the differences between the first residues of the color components
by transforming the frequency component values restored by the
inverse quantization unit 220, which are frequency domain values,
into values in the color space.
[0116] The predicted image generation unit 240 may generate a
predicted image for the current image according to the prediction
mode indicated by the information restored for the macro blocks,
which correspond to each other, of the color components by the
entropy decoding unit 210.
[0117] The restored image generation unit 250 may generate a
restored image that corresponds to the sum of the predicted image
generated by the predicted image generation unit 240 and the
residues restored by the frequency domain inverse transformation
unit 230. Alternatively, in an embodiment, the restored image
generation unit 250 may generate first residues that correspond to
the sums of the second residues restored by the frequency domain
inverse transformation unit 230 and generate a restored image that
corresponds to the sum of the predicted image generated by the
predicted image generation unit 240 and the generated first
residues.
[0118] FIG. 11 illustrates a predicted image generation unit 240,
such as that illustrated in FIG. 10, according to an embodiment of
the present invention.
[0119] Referring to FIG. 11, the predicted image generation unit
240 may include a single mode prediction image generation portion
241 and a multiple mode prediction image generation portion 242,
for example.
[0120] If the information restored by the entropy decoding unit 210
represents that a single prediction mode was applied in the
encoding, the single mode prediction image generation portion 241
may generate a predicted image by performing single intra
prediction or single inter prediction, for example. In particular,
the single mode prediction image generation portion 241 may
generate a predicted image for the current image from a reference
image using motion vectors between the reference image and the
current image, which are commonly applied to the blocks, which
correspond to each other, of the color components. In other words,
the single mode prediction image generation portion 241 may
generate the predicted image for the current image from the
reference image using the motion vectors that are commonly applied
to the blocks, which are divided from the macro blocks, which
correspond to each other, of the color components of the current
image and correspond to each other.
[0121] In addition, the single mode prediction image generation
portion 241 may further predict blocks making up the current image
from adjacent pixels in the restored image generated by the
restored image generation unit 250 using prediction directions that
are commonly applied to the blocks, which correspond to each other,
of the color components, which are divided equally from the macro
blocks, which correspond to each other, of the color components of
the current image, and generate a predicted image constituted by
the predicted blocks.
[0122] If the information restored by the entropy decoding unit 210
represents that a multiple prediction mode was applied in the
encoding, the multiple mode prediction image generation portion 242
may generate a predicted image by performing multiple inter
prediction or multiple intra prediction. In particular, the
multiple mode prediction image generation portion 242 may generate
a predicted image for the current image from a reference image
using motion vectors between the reference image and the current
image, which are independently applied to the blocks of each color
component, which are divided into different sizes from the macro
blocks, which correspond to each other, of the color components of
the current image. In other words, the multiple mode prediction
image generation portion 242 may generate the predicted image for
the current image from the reference image using different motion
vectors for the blocks of each color component, which are divided
into blocks of different sizes from the macro blocks, which
correspond to each other, of the color components of the current
image.
[0123] In addition, the multiple mode prediction image generation
portion 242 may predict blocks making up the current image from
adjacent pixels in the restored image generated by the restored
image generation unit 250 using prediction directions that are
commonly applied to the blocks, which correspond to each other, of
the color components, wherein the blocks are divided to different
sizes from the macro blocks, which correspond to each other, of the
color components of the current image, and generate a predicted
image made up by the predicted blocks, thereby performing multiple
intra prediction. In other words, the multiple mode prediction
image generation portion 242 may predict blocks making up the
current image from adjacent pixels in the restored image generated
by the restored image generation unit 250 using different
prediction directions for the blocks of each color component, which
are divided to different sizes from the macro blocks, which
correspond to each other, of the color components of the current
image and generate the predicted image made up by the predicted
blocks.
[0124] FIG. 12 illustrates a restored image generation unit 250,
such as that shown in FIG. 10, according to an embodiment of the
present invention.
[0125] Referring to FIG. 12, the restored image generation unit 250
may include a residue inverse transformation portion 251 and a
prediction compensation portion 252, for example.
[0126] If the information restored by the entropy decoding unit 210
represents the residue transformation mode was applied in the
encoding, the residue inverse transformation portion 251 may
generate first residues that correspond to the sums of the second
residues restored by the frequency domain inverse transformation
unit 230. For example, the residue inverse transformation portion
251 may generate the first residues of the Y component, the Co
component, and the Cg component, which correspond to the sums of
the second residues of each respective color component, i.e., the Y
component, the Co component, and the Cg component, using the above
Equation 8, for example. Alternatively, for example, the residue
inverse transformation portion 251 may generate the first residues
of the R component, the G component, and the B component, which
correspond to the sums of the second residues of each respective
color component, i.e., the R component, the G component, and the B
component, using the above Equation 9, for example.
[0127] The prediction compensation portion 252 may generate a
restored image that corresponds to the sum of the predicted image
generated by the predicted image generation unit 240 and the first
residues generated by the residue inverse transformation portion
251. For example, the prediction compensation portion 252 may
generate a restored image in the YCoCg color space by calculating
the sums of the predicted image of each respective color component,
i.e., the Y component, the Co component, and the Cg component, and
the first residues generated by the residue inverse transformation
portion 251. Alternatively, for example, the prediction
compensation portion 252 may generate a restored image in the RGB
color space by calculating the sums of the predicted image for each
respective color component, i.e., the R component, the G component,
and the B component, which is generated by the predicted image
generation unit 240, and the first residues generated by the
residue inverse transformation portion 251.
[0128] FIG. 13 illustrates a residue inverse transformation portion
251, such as that shown in FIG. 12, according to an embodiment of
the present invention.
[0129] Referring to FIG. 13, the residue inverse transformation
portion 251 may include an RCT portion 2511, an IPP transformation
portion 2512, and an RCP transformation portion 2513, for
example.
[0130] If the information restored by the entropy decoding unit 210
represents that the RCT transformation mode was applied during the
encoding, the RCT portion 2511 may generate the first residues of
each of the Y component, the Co component, and the Cg component,
which respectively correspond to the sums of the second residues of
each of the Y component, Co component, and the Cg component, using
the above Equation 8, for example.
[0131] If the information restored by the entropy decoding unit 210
represents that the IPP transformation mode was applied during the
encoding, the IPP transformation portion 2512 may generate the
first residues of each of the R component, the G component, and the
B component, which respectively correspond to the sums of the
second residues of each of the R component, the G component, and
the B component, using the above Equation 9, for example.
[0132] If the information restored by the entropy decoding unit 210
represents that the RCP transformation mode was applied during the
encoding, the RCP transformation portion 2513 may generate the
first residues of each of the R component, the G component, and the
B component, which respectively correspond to the sums of the
second residues of each of the R component, the G component, and
the B component, using the above Equation 9, for example.
[0133] FIGS. 14A and 14B illustrate a moving picture encoding
method, according to an embodiment of the present invention.
[0134] Referring to FIGS. 14A and 14B, a moving picture encoding
method may include operations which may be time-serially performed
in the moving picture encoding system shown in FIG. 1, for example.
Therefore, for some embodiments of the present invention, the
above-descriptions of the moving picture encoding system
illustrated in FIG. 1 can also apply to the moving picture encoding
method, even if some descriptions thereof are not directly provided
below.
[0135] In operation 1401, one of the possible prediction modes for
a sample image may be selected. As illustrated in FIG. 14A,
especially, when operation 1409 returns to operation 1401, one from
among possible prediction modes except for a previously selected
prediction mode may be selected.
[0136] In operation 1402, a predicted image may be generated for
the current image for each macro block of each color component,
according to the prediction mode selected in operation 1401, and
residues that correspond to the differences between the current
image and the predicted image may be generated. Alternatively, in
operation 1402, a predicted image for the current image for each
macro block of each color component may be generated according to
the prediction mode selected in operation 1401, and second residues
that correspond to the differences between the first residues of
the color components may be generated.
[0137] In operation 1403, the residues generated in operation 1402
may be transformed, with the residues being color space residues,
into values in the frequency domain. Alternatively, in operation
1403, the second residues generated in operation 1402 may be
transformed, with the residues being color space residues, into
values in the frequency domain.
[0138] In operation 1404, the values transformed in operation 1403
may be quantized.
[0139] In operation 1405, a bitstream may be generated by
entropy-encoding the values quantized in operation 1404 and
information representing the prediction mode selected in operation
1401.
[0140] In operation 1406, frequency component values may further be
restored by inverse-quantizing the values quantized in operation
1404.
[0141] In operation 1407, residues that correspond to differences
between the current image and the predicted image may be restored
by transforming the frequency component values restored in
operation 1406, which are frequency domain values, into values in
the color space. Alternatively, in an embodiment, in operation
1407, second residues that correspond to differences between the
first residues of each color component may be restored by
transforming the frequency component values restored in operation
1406, which are frequency domain values, into values in the color
space.
[0142] In operation 1408, a restored image that corresponds to the
sum of the predicted image generated in operation 1402 and the
residues restored in operation 1407 may be generated.
Alternatively, in an embodiment, in operation 1408, first residues
that correspond to the sums of the second residues restored in
operation 1407 may be generated, and a restored image that
corresponds to the sum of the predicted image generated in
operation 1402 and the generated first residues may be
generated.
[0143] In operation 1409, in one embodiment, whether operations
1401 through 1408 have been performed for all the prediction modes
may be determined. In this embodiment, if operations 1401 through
1408 have been performed on all the prediction modes, the method
may proceed to operation 1410 illustrated in FIG. 14B. Otherwise,
the method may returns to operation 1401.
[0144] In operation 1410, an optimal prediction mode may be
selected for the macro blocks, which correspond to each other, of
the color components of the current image based on the
characteristics of the sample images, i.e., the results of
operations 1401 through 1409. For example, in operation 1410, the
moving picture encoding system may select a prediction mode that
leads to a minimum size of a resultant bitstream and/or a minimum
amount of quality distortion between the sample image and the
restored image generated in operation 1408, as an optimal
prediction mode for the macro blocks, which correspond to each
other, of the color components of the current image.
[0145] In operation 1411, a predicted image for the current image
for each macro block of each color component according to the
prediction mode selected in operation 1410 may be generated, and
residues that correspond to the differences between the current
image and the predicted image may be generated. Alternatively, in
an embodiment, in operation 1411, a predicted image for the current
image for each macro block of each color component may be generated
according to the prediction mode selected in operation 1410, first
residues that correspond to the differences between the current
image and the predicted image may be generated, and second residues
that correspond to the differences between the first residues of
each color component may be generated.
[0146] In operation 1412, the residues generated in operation 1411
may be transformed, with the residues being color space residues,
into values in the frequency domain. Alternatively, in operation
1412, the second residues generated in operation 1411 may be
transformed, with the residues being color space residues, into
values in the frequency domain.
[0147] In operation 1413, the values transformed in operation 1412
may be quantized.
[0148] In operation 1414, a bitstream may be generated by
entropy-encoding the values quantized in operation 1413 and
information representing the prediction mode selected in operation
1410.
[0149] In operation 1415, the frequency component values may be
restored by inverse-quantizing the values quantized in operation
1413.
[0150] In operation 1416, residues that correspond to the
differences between the current image and the predicted image may
be restored by transforming the frequency component values restored
in operation 1415, which are frequency domain values, into values
in the color space. Alternatively, in an embodiment, in operation
1416, second residues that correspond to differences between the
first residues of each color component may be restored by
transforming the frequency component values restored in operation
1415, which are frequency domain values, into values in the color
space.
[0151] In operation 1417, a restored image that corresponds to the
sum of the predicted image generated in operation 1411 and the
residues restored in operation 1416 may be generated.
Alternatively, in an embodiment, in operation 1417, first residues
that correspond to the sums of the second residues of each color
component restored in operation 1416 may be generated, and a
restored image that corresponds to the sum of the predicted image
generated in operation 1411 and the generated first residues may be
generated.
[0152] FIG. 15 illustrates a moving picture decoding method,
according to an embodiment of the present invention.
[0153] Referring to FIG. 15, the illustrated operations may be
time-serially performed, e.g., such as in the moving picture
decoding system shown in FIG. 10. Accordingly, for such an
embodiment, the above-descriptions of the moving picture decoding
system illustrated in FIG. 10 should also be applied to such a
moving picture decoding method, even if some descriptions thereof
are not directly provided below.
[0154] In operation 1501, integer values corresponding to the
current image and information representing the optimal prediction
mode for the macro blocks, which correspond to each other, of the
color components of the current image may be restored by
entropy-decoding a bitstream, such as that output from the moving
picture encoding system illustrated in FIG. 1.
[0155] In operation 1502, the frequency component values may be
restored by inverse-quantizing the integer values restored in
operation 1501.
[0156] In operation 1503, residues that correspond to the
differences between the current image and the predicted image may
be generated by transforming the frequency component values
restored in operation 1502, which are frequency domain values, into
values in the color space. Alternatively, in an embodiment, in
operation 1503, second residues that correspond to the differences
between the first residues of each color component may be restored
by transforming the frequency component values restored in
operation 1502, which are frequency domain values, into values in
the color space.
[0157] In operation 1504, if the information restored in operation
1501 represents that the residue transformation mode was applied
during encoding, the below operation 1505 may be performed.
Otherwise, the below operation 1506 may be performed.
[0158] In operation 1505, first residues that correspond to the
sums of the second residues of each color component restored in
operation 1503 may be generated. In particular, if the information
restored in operation 1501 represents that the RCT transformation
mode was applied during encoding, in operation 1505, the first
residues of each of the Y component, the Co component, and the Cg
component, which respectively correspond to the sums of the second
residues of each of the Y component, the Co component, and the Cg
component, may be generated using the above Equation 8, for
example. Alternatively, if the information restored in operation
1501 represents that the IPP or RCP transformation modes were
applied in encoding, in operation 1505, the first residues of each
of the R component, the G component, and the B component, which
respectively correspond to the sums of the second residues of each
of the R component, the G component, and the B component, may be
generated using the above Equation 9, for example.
[0159] In an embodiment, if the information restored by the entropy
decoding unit 210 represents that the RCP transformation mode was
applied in encoding, the RCP transformation portion 2511 may
generate the first residues of each of the R component, the G
component, and the B component, which respectively correspond to
the sums of the second residues of each of the R component, the G
component, and the B component, using the above Equation 9, for
example.
[0160] In operation 1506, if the information restored in operation
1501 is determined to represent the single prediction mode,
operation 1507 may be performed. If the information restored in
operation 1501 is determined to represent the multiple prediction
mode, operation 1508 may be performed.
[0161] In operation 1507, a predicted image may be generated
according to the single prediction mode, i.e., using motion vectors
that are commonly applied to all the divided blocks of the color
components of the current image, which are divided equally, or
using prediction directions that are independently applied to all
the divided blocks of the color components.
[0162] In operation 1508, a predicted image may be generated
according to the multiple prediction mode, i.e., using motion
vectors that are independently applied to the divided blocks of
each color component of the current image, which are divided into
blocks of different sizes, or using prediction directions that are
independently applied to the divided blocks of each color
component.
[0163] In operation 1509, a restored image that corresponds to the
sum of the predicted image generated in operation 1507 or 1508 and
the residues restored in operation 1503 may be generated.
Alternatively, a restored image that corresponds to the sum of the
predicted image generated in operation 1507 or 1508 and the first
residues generated in operation 1505 may be generated.
[0164] In addition to the above described embodiments, embodiments
of the present invention can also be implemented through computer
readable code/instructions in/on a medium, e.g., a computer
readable medium, to control at least one processing element to
implement any above described embodiment. The medium can correspond
to any medium/media permitting the storing and/or transmission of
the computer readable code.
[0165] The computer readable code can be recorded/transferred on a
medium in a variety of ways, with examples of the medium including
recording media, such as magnetic storage media (e.g., ROM, floppy
disks, hard disks, etc.) and optical recording media (e.g.,
CD-ROMs, or DVDs), and transmission media such as carrier waves, as
well as through the Internet, for example. Thus, the medium may
further be a signal, such as a resultant signal or bitstream,
according to embodiments of the present invention. The media may
also be a distributed network, so that the computer readable code
is stored/transferred and executed in a distributed fashion. Still
further, as only an example, the processing element could include a
processor or a computer processor, and processing elements may be
distributed and/or included in a single device.
[0166] According to one or more embodiments of the present
invention, encoding and/or decoding of moving pictures may be
performed by adaptively applying an optimal prediction mode to
corresponding macro blocks, which correspond to each other, of the
color components of a current image for each macro block, so that
encoding and/or decoding efficiencies increase. In particular,
various encoding methods, such as a single prediction mode, a
multiple prediction mode, inter prediction, intra prediction,
residue transformation, RCT, IPP transformation, RCP
transformation, etc., may be selectively used to encode each color
component of the current image for each macro block, thereby
maximizing encoding and decoding efficiencies.
[0167] Although a few embodiments of the present invention have
been shown and described, it would be appreciated by those skilled
in the art that changes may be made in these embodiments without
departing from the principles and spirit of the invention, the
scope of which is defined in the claims and their equivalents.
* * * * *