U.S. patent application number 13/978492 was filed with the patent office on 2013-11-28 for image processing apparatus and method.
This patent application is currently assigned to Sony Corporation. The applicant listed for this patent is Kazushi Sato. Invention is credited to Kazushi Sato.
Application Number | 20130315493 13/978492 |
Document ID | / |
Family ID | 46507101 |
Filed Date | 2013-11-28 |
United States Patent
Application |
20130315493 |
Kind Code |
A1 |
Sato; Kazushi |
November 28, 2013 |
IMAGE PROCESSING APPARATUS AND METHOD
Abstract
The present technology relates to an image processing apparatus
and method that are capable of enhancing encoding efficiency while
suppressing a decrease in the efficiency of encoding processing.
The image processing apparatus includes an encoding mode setter
that sets, in units of coding units having a hierarchical
structure, whether a non-compression mode is to be selected as an
encoding mode for encoding image data, the non-compression mode
being an encoding mode in which the image data is output as encoded
data, and an encoder that encodes the image data in units of the
coding units in accordance with a mode set by the encoding mode
setter. The present disclosure can be applied to, for example, an
image processing apparatus.
Inventors: |
Sato; Kazushi; (Kanagawa,
JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Sato; Kazushi |
Kanagawa |
|
JP |
|
|
Assignee: |
Sony Corporation
Minato-ku, Tokyo
JP
|
Family ID: |
46507101 |
Appl. No.: |
13/978492 |
Filed: |
January 4, 2012 |
PCT Filed: |
January 4, 2012 |
PCT NO: |
PCT/JP2012/050016 |
371 Date: |
August 6, 2013 |
Current U.S.
Class: |
382/232 |
Current CPC
Class: |
H04N 19/30 20141101;
H04N 19/124 20141101; H04N 19/70 20141101; G06T 9/007 20130101;
H04N 19/12 20141101; H04N 19/132 20141101; H04N 19/82 20141101;
H04N 19/184 20141101; H04N 19/85 20141101; H04N 19/44 20141101;
H04N 19/117 20141101; H04N 19/152 20141101; H04N 19/176 20141101;
H04N 19/157 20141101; H04N 19/46 20141101 |
Class at
Publication: |
382/232 |
International
Class: |
G06T 9/00 20060101
G06T009/00 |
Foreign Application Data
Date |
Code |
Application Number |
Jan 11, 2011 |
JP |
2011-003245 |
Claims
1. An image processing apparatus comprising: an encoding mode
setter that sets, in units of coding units having a hierarchical
structure, whether a non-compression mode is to be selected as an
encoding mode for encoding image data, the non-compression mode
being an encoding mode in which the image data is output as encoded
data; and an encoder that encodes the image data in units of the
coding units in accordance with a mode set by the encoding mode
setter.
2. The image processing apparatus according to claim 1, further
comprising: a controller that performs control, on a coding unit
for which the non-compression mode has been set by the encoding
mode setter, to set a bit depth for encoding.
3. The image processing apparatus according to claim 1, further
comprising: a controller that performs control, on a coding unit
for which the non-compression mode has been set by the encoding
mode setter, to skip processing of changing a bit depth for
encoding.
4. The image processing apparatus according to claim 1, further
comprising: a controller that performs control, on a coding unit
for which the non-compression mode has been set by the encoding
mode setter, to skip filter processing in which filtering is
performed on a locally decoded image.
5. The image processing apparatus according to claim 4, wherein the
controller performs control, on a coding unit for which the
non-compression mode has been set by the encoding mode setter, to
select whether to apply or skip the filter processing in which
filtering is performed on a locally decoded image.
6. The image processing apparatus according to claim 5, further
comprising: a filter identification information generator that
generates filter identification information identifying whether the
filter processing is to be performed.
7. The image processing apparatus according to claim 1, wherein, in
a case where an amount of input data, which is a data amount of the
image data corresponding to a current coding unit, is smaller than
or equal to an amount of code of encoded data, which is obtained by
encoding the image data corresponding to the current coding unit,
the encoding mode setter sets an encoding mode of the current
coding unit to be the non-compression mode.
8. The image processing apparatus according to claim 7, further
comprising: an input data amount calculator that calculates the
amount of input data, wherein the encoding mode setter compares,
regarding the current coding unit, the amount of input data
calculated by the input data amount calculator with the amount of
code.
9. The image processing apparatus according to claim 1, further
comprising: an encoding mode identification information generator
that generates encoding mode identification information indicating
whether the non-compression mode has been set by the encoding mode
setter.
10. The image processing apparatus according to claim 1, further
comprising: an identification information generator that generates
identification information identifying that a block constitute by a
plurality of coding units includes a coding unit for which the
non-compression mode has been set.
11. The image processing apparatus according to claim 10, wherein
the block constituted by a plurality of coding units is a largest
coding unit, which is a coding unit of largest size.
12-21. (canceled)
Description
TECHNICAL FIELD
[0001] The present disclosure relates to an image processing
apparatus and method, and particularly relates to an image
processing apparatus and method that enable enhancement of encoding
efficiency while suppressing a decrease in the efficiency of
encoding processing.
BACKGROUND ART
[0002] In recent years, apparatuses compliant with a compression
format such as MPEG (Moving Picture Experts Group), in which image
information is digitally handled and is compressed using an
orthogonal transform such as the discrete cosine transform and by
motion compensation by utilizing a redundancy particular to image
information for the purpose of achieving highly efficient
transmission and accumulation of information, have been becoming
prevalent for use in both distribution of information from
broadcast stations and reception of information in general
households.
[0003] In particular, MPEG-2 (ISO (International Organization for
Standardization)/IEC (international Electrotechnical Commission)
13818-2) is defined as a general-purpose image coding format, is a
standard that covers both interlaced scanned images and progressive
scanned images as well as standard-definition images and
high-definition images, and is currently widely used in a broad
range of applications for professional use and consumer use. With
the MPEG-2 compression format, a high compression rate and a
favorable image quality can be realized by, for example, allocating
an amount of code (bit rate) of 4 to 8 Mbps to a
standard-definition interlaced scanned image having 720.times.480
pixels or an amount of code (bit rate) of 18 to 22 Mbps to a
high-definition interlaced scanned image having 1920.times.1.088
pixels.
[0004] MPEG-2 is mainly used for high image quality encoding
suitable for broadcasting, but is not compatible with coding
formats of an amount of code (bit rate) lower than that of MPEG-1,
that is, a higher compression rate. With the widespread use of
mobile terminals, the need for such coding formats will increase in
the future, and in response the MPEG-4 coding format has been
standardized. Regarding an image coding format, MPEG-4 was
designated an international standard as ISO/IEC 14496-2 in December
1998.
[0005] Furthermore, standardization of the format H.26L (ITU-T
(International Telecommunication Union Telecommunication
Standardization Sector) Q6/16 VCEG (Video Coding Expert Group)),
which was initially for the purpose of image coding for video
conferences, has been progressing in recent years. It is known that
H.26L realizes higher encoding efficiency than previous coding
formats, such as MPEG-2 and MPEG-4, though encoding and decoding
according to H.26L involve a larger amount of computation. Also, as
part of MPEG-4 activities, standardization for realizing higher
encoding efficiency by introducing functions which are not
supported by H.26L on the basis of H.26L is currently progressing
as Joint Model of Enhanced-Compression Video Coding.
[0006] As the standardization schedule, a standard called H.264 and
MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as
AVC) was designated an international standard in March 2003.
[0007] Furthermore, as an extension of the above, standardization
of FRExt (Fidelity Range Extension), including encoding tools
necessary for business use, such as RGB, 4:2:2, and 4:4:4, as well
as 8.times.8 DCT (Discrete Cosine Transform) and quantization
matrices defined in MPEG-2, was completed in February 2005.
Accordingly, a coding format capable of favorably expressing even
film noise included in movies by using AVC has been established,
which is used for a wide range of applications such as Blu-Ray
Discs.
[0008] However, there has recently been a growing need for encoding
at a higher compression rate, for example, compression of an image
having about 4000.times.2000 pixels, which is four times the number
of pixels included in a high-definition image, or distribution of
high-definition images in an environment with a limited
transmission capacity, such as the Internet. Therefore, in VCEG
(Video Coding Expert Group) under ITU-T, ongoing studies for
enhancing encoding efficiency have been performed.
[0009] Meanwhile, for the purpose of realizing higher encoding
efficiency than that of AVC, standardization of a coding format
called HEVC (High Efficiency Video Coding) is currently being
conducted by JCTVC (Joint Collaboration Team-Video Coding), which
is a standards group of ITU-T and Iso/IEC (see, for example, NPL
1).
[0010] In the HEVC coding format, coding units (CUs) are defined as
units of processing which are similar to macroblocks used in AVC.
Unlike a macroblock used in AVC, the size of a CU is not fixed to
16.times.16 pixels, and is specified in image compression
information in each sequence.
[0011] CUs are hierarchically configured from a largest coding unit
(LCU) to a smallest coding unit (SCU). That is, it may be generally
considered that an LCU corresponds to a macroblock used in AVC, and
a CU in a layer lower than the LCU corresponds to a sub-macroblock
used in AVC.
[0012] Meanwhile, there is a coding format in which an encoding
mode for encoding and outputting image data and a non-encoding mode
for outputting image data without encoding the image data are
provided, whether the encoding mode or the non-encoding mode is to
be used is selected in units of macroblocks, and the encoding mode
and the non-encoding mode can be used in combination within a
single picture (see, for example, PTL 1). Also in the AVC coding
format, an I_PCM mode for outputting image data without encoding
the image data is supported as mb_type (see, for example, PTL 2).
This used for ensuring real-time operation of arithmetic coding
processing in a case where a quantization parameter is set to be a
small value, such as QP=0, and in a case where the information
amount of encoded data is larger than that of an input image. Also,
it is possible to realize lossless coding by using I-PCM.
[0013] Also, a method for increasing internal arithmetic has been
suggested (see, for example, NPL 2). Accordingly, an internal
arithmetic error caused in processing such as an orthogonal
transform and motion compensation can be reduced, and encoding
efficiency can be enhanced.
[0014] Furthermore, a technique in which an FIR filter is provided
in a motion compensation loop has been suggested (see, for example,
NPL 3). In an encoding apparatus, by obtaining an FIR filter
coefficient using a Wiener filter so as to minimize an error with
respect to an input image, degradation in a reference image can be
minimized, and encoding efficiency of image compression information
to be output can be enhanced.
CITATION LIST
Patent Literature
[0015] PTL 1: Japanese Patent No. 3992303 [0016] PTL 2: Japanese
Patent No. 4240283
Non Patent Literature
[0017] NPL 1: "Test Model under Consideration", JCTVC-B205, Joint
Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and
ISO/IEC JTC1/SC29/WG112nd Meeting: Geneva, CH, 21-28 Jul., 2010
[0018] NPL 2: Takeshi Chujoh, Reiko Noda, "Internal bit depth
increase except frame memory", VCEG-AF07, ITU-Telecommunicatiois
Standardization Sector STUDY GROUP 16 Question 6 Video Coding
Experts Group (VCEG) 32nd Meeting: San Jose, USA, 20-21 Apr., 2007
[0019] NPL 3: Takeshi Chuoh, Goki Yasuda, Naofumi Wada, Takashi
Watanabe, Tomoo Yamakage, "Block-based Adaptive Loop Filter",
VCEG-AI18, ITU-Telecommunications Standardization Sector STUDY
GROUP 16 Question 6 Video Coding Experts Group (VCEG) 35th Meeting:
Berlin, Germany, 16-18 Jul., 2008
SUMMARY OF INVENTION
Technical Problem
[0020] However, in the case of a coding format in which CUs are
defined as in HEVC and various processing operations are performed
in units of CUs, it is considered that a macroblock used in AVC
corresponds to an LCU, but if I_PCM can be set only in units of
LCIs, unnecessary encoding processing increases because the unit of
processing is 128.times.128 pixels at maximum, and the efficiency
of encoding processing may decrease. For example, it may become
difficult to ensure real-time operation of CABAC.
[0021] Also, the coding formats suggested in NPL 2 and NPL 3 are
not included in the AVC coding format, and compatibility with the
I_PCM mode is not disclosed therein.
[0022] The present disclosure has been made in view of these
circumstances, and is directed to enhancing encoding efficiency
while suppressing a decrease in the efficiency of encoding
processing.
Solution to Problem
[0023] According to an aspect of the present disclosure, there is
provided an image processing apparatus. The image processing
apparatus includes an encoding mode setter that sets, in units of
coding units having a hierarchical structure, whether a
non-compression mode is to be selected as an encoding mode for
encoding image data, the non-compression mode being an encoding
mode in which the image data is output as encoded data; and an
encoder that encodes the image data in units of the coding units in
accordance with a mode set by the encoding mode setter.
[0024] The image processing apparatus may further include a shift
processing controller that performs control, on a coding unit for
which the non-compression mode has been set by the encoding mode
setter, to skip shift processing in which a bit precision for
encoding or decoding is increased; and a shift processor that
performs the shift processing on a coding unit of the image data,
the coding unit being controlled by the shift processing controller
so as to undergo the shift processing.
[0025] The image processing apparatus may further include a filter
processing controller that performs control, on a coding unit for
which the non-compression mode has been set by the encoding mode
setter, to skip filter processing in which filtering is performed
on a locally decoded image; a filter coefficient calculator that
calculates a filter coefficient for the filter processing by using
image data corresponding to a coding unit which is controlled by
the filter processing controller so as to undergo the filter
processing; and a filter processor that performs the filter
processing in units of blocks, which are units of the filter
processing, by using the filter coefficient calculated by the
filter coefficient calculator.
[0026] The filter processor may perform the filter processing on
only pixels which are controlled by the filter processing
controller so as to undergo the filter processing, the pixels being
included in a current block which is a target to be processed.
[0027] The image processing apparatus may further include a filter
identification information generator that generates filter
identification information in units of the blocks, the filter
identification information being identification information
indicating whether the filter processing is to be performed.
[0028] The filter processor may perform adaptive loop filtering on
the locally decoded image, the adaptive loop filtering being
adaptive filter processing using classification processing.
[0029] In a case where an amount of code of encoded data, which is
obtained by encoding the image data corresponding to a current
coding unit as a target of encoding processing, is smaller than or
equal to an amount of input data, which is a data amount of the
image data corresponding to the current coding unit, the encoding
mode setter may set an encoding mode of the current coding unit to
be the non-compression mode.
[0030] The image processing apparatus may further include an input
data amount calculator that calculates the amount of input data.
The encoding mode setter may compare, regarding the current coding
unit, the amount of input data calculated by the input data amount
calculator with the amount of code.
[0031] The image processing apparatus may further include an
identification information generator that generates identification
information in units of the coding units, the identification
information indicating whether the non-compression mode has been
set by the encoding mode setter.
[0032] According to an aspect of the present disclosure, there is
provided an image processing method for an image processing
apparatus. The image processing method includes setting, with an
encoding mode setter, in units of coding units having a
hierarchical structure, whether a non-compression mode is to be
selected as an encoding mode for encoding image data, the
non-compression mode being an encoding mode in which the image data
is output as encoded data; and encoding, with an encoder, the image
data in units of the coding units in accordance with a set
mode.
[0033] According to another aspect of the present disclosure, there
is provided an image processing apparatus. The image processing
apparatus includes an encoding mode determiner that determines, in
units of coding units having a hierarchical structure, whether a
non-compression mode has been selected as an encoding mode for
encoding image data, the non-compression mode being an encoding
mode in which the image data is output as encoded data; and a
decoder that decodes the encoding result in units of the coding
units in accordance with a mode determined by the encoding mode
determiner.
[0034] The image processing apparatus may further include a shift
processing controller that performs control, on a coding unit for
which the encoding mode determiner has determined that the
non-compression mode has been selected, to skip shift processing in
which a bit precision for encoding or decoding is increased; and a
shift processor that performs the shift processing on a coding unit
of the image data, the coding unit being controlled by the shift
processing controller so as to undergo the shrift processing.
[0035] The image processing apparatus may further include a filter
processing controller that performs control, on a coding unit for
which the encoding mode determiner has determined that the
non-compression mode has been selected, to skip filter processing
in which filtering is performed on a locally decoded image; and a
filter processor that performs the filter processing on the image
data in units of blocks, which are units of the filter processing.
The filter processor may perform the filter processing on only
pixels which have been controlled by the filter processing
controller so as to undergo the filter processing, the pixels being
included in a current block which is a target to be processed.
[0036] The filter processor may perform adaptive loop filtering on
the locally decoded image, the adaptive loop filtering being
adaptive filter processing using classification processing.
[0037] The filter processor may perform the filter processing, in a
case where filter identification information indicating whether the
filter processing has been performed indicates that the filter
processing has been performed on image data corresponding to the
current block which is a target to be processed, only when control
is performed by the filter processing controller so as to perform
the filter processing on all pixels included in the current
block.
[0038] The encoding mode determiner may determine whether the
non-compression mode has been selected, on the basis of
identification information indicating whether the non-compression
mode has been selected in units of the coding units.
[0039] According to another aspect of the present disclosure, there
is provided an image processing method for an image processing
apparatus. The image processing method includes determining, with
an encoding mode determiner, in units of coding units having a
hierarchical structure, whether a non-compression mode has been
selected as an encoding mode for encoding image data, the
non-compression mode being an encoding mode in which the image data
is output as encoded data; and decoding, with a decoder, the
encoded data in units of the coding units in accordance with a
determined mode.
[0040] According to an aspect of the present disclosure, whether a
non-compression mode is to be selected as an encoding mode for
encoding image data is set in units of coding units having a
hierarchical structure, the non-compression mode being an encoding
mode in which the image data is output as encoded data, and the
image data is encoded in units of the coding units in accordance
with a set mode.
[0041] According to another aspect of the present disclosure,
whether a non-compression mode has been selected as an encoding
mode for encoding image data is determined in units of coding units
having a hierarchical structure, the non-compression mode being an
encoding mode in which the image data is output as encoded data,
and the encoded data is decoded in units of the coding units in
accordance with a determined mode.
Advantageous Effects of Invention
[0042] According to the present disclosure, an image can be
processed. In particular, encoding efficiency can be enhanced while
suppressing a decrease in the efficiency of encoding
processing.
BRIEF DESCRIPTION OF DRAWINGS
[0043] FIG. 1 is a block diagram illustrating an image encoding
apparatus that outputs image compression information based on an
AVC coding format.
[0044] FIG. 2 is a block diagram illustrating an image decoding
apparatus that receives image compression information based on the
AVC coding format.
[0045] FIG. 3 is a diagram illustrating an example of the type of
macroblock.
[0046] FIG. 4 is a diagram describing an example configuration of
coding units.
[0047] FIG. 5 is a diagram describing a method for increasing an
amount of bits in internal arithmetic.
[0048] FIG. 6 is a diagram describing an adaptive loop filter.
[0049] FIG. 7 is a block diagram illustrating a main example
configuration of an image encoding apparatus.
[0050] FIG. 8 is a block diagram illustrating a main example
configuration of the lossless encoder, loop filter, and PCM encoder
in FIG. 7.
[0051] FIG. 9 is a block diagram illustrating a main example
configuration of the PCM deciding unit in FIG. 8.
[0052] FIG. 10 is a flowchart describing an example of the flow of
encoding processing.
[0053] FIG. 11 is a flowchart describing an example of the flow of
PCM encoding control processing.
[0054] FIG. 12 is a flowchart describing an example of the flow of
PCM encoding processing.
[0055] FIG. 13 is a flowchart describing an example of the flow of
reference image generation processing.
[0056] FIG. 14 is a flowchart describing an example of the flow of
loop filter processing.
[0057] FIG. 15 is a block diagram illustrating a main example
configuration of an image decoding apparatus.
[0058] FIG. 16 is a block diagram illustrating a main example
configuration of the lossless decoder, loop filter, and PCM decoder
in FIG. 15.
[0059] FIG. 17 is a flowchart describing an example of the flow of
decoding processing.
[0060] FIG. 18 is a flowchart continued from FIG. 17, describing
the example of the flow of decoding processing.
[0061] FIG. 19 is a flowchart describing an example of the flow of
loop filter processing.
[0062] FIG. 20 is a diagram describing an example of I_PCM
information.
[0063] FIG. 21 is a block diagram illustrating a main example
configuration of a personal computer.
[0064] FIG. 22 is a block diagram illustrating a main example
configuration of a television receiver.
[0065] FIG. 23 is a block diagram illustrating a main example
configuration of a mobile phone.
[0066] FIG. 24 is a block diagram illustrating a main example
configuration of a hard disk recorder.
[0067] FIG. 25 is a block diagram illustrating a main example
configuration of a camera.
DESCRIPTION OF EMBODIMENTS
[0068] Hereinafter, embodiments for carrying out the present
technology (hereinafter referred to as embodiments) will be
described. Note that the description will be given in the following
order.
[0069] 1. First embodiment (image encoding apparatus)
[0070] 2. Second embodiment (image decoding apparatus)
[0071] 3. Third embodiment (personal computer)
[0072] 4. Fourth embodiment (television receiver)
[0073] 5. Fifth embodiment (mobile phone)
[0074] 6. Sixth embodiment (hard disk recorder)
[0075] 7. Seventh embodiment (camera)
1. First Embodiment
Image Encoding Apparatus Compatible with AVC Coding Format
[0076] FIG. 1 illustrates the configuration of an image encoding
apparatus according to an embodiment, which encodes an image using
an H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC
(Advanced Video Coding)) coding format.
[0077] An image encoding apparatus 100 illustrated in FIG. 1 is an
apparatus that encodes an image using a coding format based on the
AVC standard and outputs the encoded image. As illustrated in FIG.
1, the image encoding apparatus 100 includes an A/D converter 101,
a screen rearrangement buffer 102, a computing unit 103, an
orthogonal transform unit 104, a quantizer 105, a lossless encoder
106, and an accumulation buffer 107. Also, the image encoding
apparatus 100 includes a dequantizer 108, an inverse orthogonal
transform unit 109, a computing unit 110, a deblocking filter 111,
a frame memory 112, a selector 113, an intra prediction unit 114, a
motion prediction/compensation unit 115, a selector 116, and a rate
controller 117.
[0078] The A/D converter 101 A/D-converts image data input thereto,
and outputs the image data to the screen rearrangement buffer 102
to store the image data therein. The screen rearrangement buffer
102 rearranges, in accordance with a GOP (Group of Picture)
structure, frame images stored therein which are arranged in a
display order so that the frame images are rearranged in an
encoding order. The screen rearrangement buffer 102 supplies the
rearranged frame images to the computing unit 103. Also, the screen
rearrangement buffer 102 supplies the rearranged frame images to
the intra prediction unit 114 and the motion
prediction/compensation unit 115.
[0079] The computing unit 103 subtracts, from an image read out
from the screen rearrangement buffer 102, a prediction image
supplied from the intra prediction unit 114 or the motion
prediction/compensation unit 115 via the selector 116, and outputs
difference information thereof to the orthogonal transform unit
104.
[0080] For example, in the case of an image on which intra coding
is to be performed, the computing unit 103 subtracts a prediction
image supplied from the intra prediction unit 114 from an image
read out from the screen rearrangement buffer 102. Also, for
example, in the case of an image on which inter coding is to be
performed, the computing unit 103 subtracts a prediction image
supplied from the motion prediction/compensation unit 115 from an
image read out from the screen rearrangement buffer 102.
[0081] The orthogonal transform unit 104 performs an orthogonal
transform, such as the discrete cosine transform or Karhunen-Loeve
transform, on the difference information supplied from the
computing unit 103, and supplies a transform coefficient thereof to
the quantizer 105.
[0082] The quantizer 105 quantizes the transform coefficient output
from the orthogonal transform unit 104. The quantizer 105 performs
quantization by setting a quantization parameter on the basis of
information about a target value of an amount of code supplied from
the rate controller 117. The quantizer 105 supplies the quantized
transform coefficient to the lossless encoder 106.
[0083] The lossless encoder 106 performs lossless coding, such as
variable-length coding or arithmetic coding, on the quantized
transform coefficient. Coefficient data has been quantized under
control performed by the rate controller 117, and thus the amount
of code thereof is equal to (or approximate to) a target value set
by the rate controller 117.
[0084] The lossless encoder 106 obtains information indicating
intra prediction and so forth from the intra prediction unit 114,
and obtains information indicating an inter prediction mode, motion
vector information, and so forth from the motion
prediction/compensation unit 115. Note that information indicating
intra prediction (intra-screen prediction) will also be referred to
as intra prediction mode information hereinafter. Also, information
indicating an information mode indicating inter prediction
(inter-screen prediction) will also be referred to as inter
prediction mode information.
[0085] The lossless encoder 106 encodes the quantized transform
coefficient, and also causes various pieces of information, such as
a filter coefficient, intra prediction mode information, inter
prediction mode information, and a quantization parameter, to be
part of header information of encoded data (multiplexes the various
pieces of information). The lossless encoder 106 supplies encoded
data which has been obtained through encoding to the accumulation
buffer 107 to store the encoded data therein.
[0086] For example, in the lossless encoder 106, lossless coding
processing such as variable-length coding or arithmetic coding is
performed. An example of variable-length coding includes CAVLC
(Context-Adaptive Variable Length Coding) defined in the H.264/AVC
format. An example of arithmetic coding includes CABAC
(Context-Adaptive Binary Arithmetic Coding).
[0087] The accumulation buffer 107 temporarily holds the encoded
data supplied from the lossless encoder 106, and outputs the
encoded data to, for example, a recording device or a transmission
channel in the subsequent stage (not illustrated) at a certain
timing, as an encoded image which has been encoded using the
H.264/AVC format.
[0088] In addition, the transform coefficient quantized by the
quantizer 105 is also supplied to the dequantizer 108. The
dequantizer 108 dequantizes the quantized transform coefficient
using a method corresponding to the quantization performed by the
quantizer 105. The dequantizer 108 supplies the transform
coefficient obtained thereby to the inverse orthogonal transform
unit 109.
[0089] The inverse orthogonal transform unit 109 performs inverse
orthogonal transform on the transform coefficient supplied thereto
using a method corresponding to the orthogonal transform processing
performed by the orthogonal transform unit 104. The output obtained
through the inverse orthogonal transform (recovered difference
information) is supplied to the computing unit 110.
[0090] The computing unit 110 adds a prediction image supplied from
the intra prediction unit 114 or the motion prediction/compensation
unit 115 via the selector 116 to the result of inverse orthogonal
transform supplied from the inverse orthogonal transform unit 109,
that is, recovered difference information, and obtains a locally
decoded image (decoded image).
[0091] For example, in a case where the difference information
corresponds to an image on which intra coding is to be performed,
the computing unit 110 adds a prediction image supplied from the
intra prediction unit 114 to the difference information. Also, for
example, in a case where the difference information corresponds to
an image on which inter coding is to be performed, the computing
unit 110 adds a prediction image supplied from the motion
prediction/compensation unit 115 to the difference information.
[0092] The result of addition is supplied to the deblocking filter
111 or the frame memory 112.
[0093] The deblocking filter 111 performs deblocking filter
processing as appropriate, thereby removing a block distortion of a
decoded image. The deblocking filter 111 supplies the result of
filter processing to the frame memory 112. Note that the decoded
image output from the computing unit 110 can be supplied to the
frame memory 112 without via the deblocking filter 111. That is,
deblocking filter processing by the deblocking filter 111 can be
skipped.
[0094] The frame memory 112 stores the decoded image supplied
thereto, and outputs the stored decoded image as a reference image
to the intra prediction unit 114 or the motion
prediction/compensation unit 115 via the selector 113 at a certain
timing.
[0095] For example, in the case of an image on which intra coding
is to be performed, the frame memory 112 supplies a reference image
to the intra prediction unit 114 via the selector 113. Also, for
example, in a case where inter coding is to be performed, the frame
memory 112 supplies a reference image to the motion
prediction/compensation unit 115 via the selector 113.
[0096] In a case where the reference image supplied from the frame
memory 112 is an image on which intra coding is to be performed,
the selector 113 supplies the reference image to the intra
prediction unit 114. On the other hand, in a case where the
reference image supplied from the frame memory 112 is an image on
which inter coding is to be performed, the selector 113 supplies
the reference image to the motion prediction/compensation unit
115.
[0097] The intra prediction unit 114 performs intra prediction
(intra-screen prediction) in which a prediction image is generated
using pixel values of a target picture to be processed supplied
from the frame memory 112 via the selector 113. The intra
prediction unit 114 performs intra prediction using a plurality of
prepared modes (intra prediction modes).
[0098] In the H.264 image information coding format, an intra
4.times.4 prediction mode, an intra 8.times.8 prediction mode, and
an intra 16.times.16 prediction mode are defined for luminance
signals. Also, regarding color-difference signals, a prediction
mode independent of that for luminance signals can be defined for
individual macroblocks. Regarding the intra 4.times.4 prediction
mode, one intra prediction mode is defined for each 4.times.4
luminance block. Regarding the intra 3.times.8 prediction mode, one
intra prediction mode is defined for each 8.times.8 luminance
block. Regarding the intra 16.times.16 prediction mode and
color-difference signals, one prediction mode is defined for one
macroblock.
[0099] The intra prediction unit 114 generates prediction images
using all candidate intra prediction modes, evaluates the cost
function values of the individual prediction images using an input
image supplied the screen rearrangement buffer 102, and selects an
optimal mode. After selecting an optimal intra prediction mode, the
intra prediction unit 114 supplies the prediction image which has
been generated using the optimal mode to the computing unit 103 and
the computing unit 110 via the selector 116.
[0100] Also, as described above, the intra prediction unit 114
supplies information, such as intra prediction mode information
indicating the adopted intra prediction mode, to the lossless
encoder 106 as appropriate.
[0101] The motion prediction/compensation unit 115 performs motion
prediction (inter prediction) on an image on which inter coding is
to be performed, using an input image supplied from the screen
rearrangement buffer 102 and a reference image supplied from the
frame memory 112 via the selector 113, performs motion compensation
processing in accordance with a detected motion vector, and
generates a prediction image (inter prediction image information).
The motion prediction/compensation unit 115 performs such inter
prediction using a plurality of prepared modes (inter prediction
modes).
[0102] The motion prediction/compensation unit 115 generates
prediction images using all candidate inter prediction modes,
evaluates the cost function values of the individual prediction
images, and selects an optimal mode. The motion prediction
compensation unit 115 supplies the generated prediction image to
the computing unit 103 and the computing unit 110 via the selector
116.
[0103] Also, the motion prediction/compensation unit 115 supplies
inter prediction mode information indicating the adopted inter
prediction mode and motion vector information indicating a
calculated motion vector to the lossless encoder 106.
[0104] In the case of an image on which intra coding is to be
performed, the selector 116 supplies the output of the intra
prediction unit 114 to the computing unit 103 and the computing
unit 110. In the case of an image on which inter coding is to be
performed, the selector 116 supplies the output of the motion
prediction/compensation unit 115 to the computing unit 103 and the
computing unit 110.
[0105] The rate controller 117 controls the rate of the
quantization operation performed by the quantizer 105 on the basis
of the compressed images accumulated in the accumulation buffer 107
so that overflow or underflow does not occur.
[Image Decoding Apparatus Compatible with AVC Coding Format]
[0106] FIG. 2 is a block diagram illustrating a main example
configuration of an image decoding apparatus that realizes image
compression using an orthogonal transform, such as the discrete
cosine transform or Karhunen-Loeve transform, and by motion
compensation. The image decoding apparatus 200 illustrated in FIG.
2 is a decoding apparatus corresponding to the image encoding
apparatus 100 illustrated in FIG. 1.
[0107] Encoded data which has been encoded by the image encoding
apparatus 100 is supplied to the image decoding apparatus 200
corresponding to the image encoding apparatus 100 via an arbitrary
path, for example, a transmission channel, a recording medium, or
the like, and is decoded.
[0108] As illustrated in FIG. 2, the image decoding apparatus 200
includes an accumulation buffer 201, a lossless decoder 202, a
dequantizer 203, an inverse orthogonal transform unit 204, a
computing unit 205, a deblocking filter 206, a screen rearrangement
buffer 207, and a D/A converter 208. Also, the image decoding
apparatus 200 includes a frame memory 209, a selector 210, an intra
prediction unit 211, a motion prediction/compensation unit 212, and
a selector 213.
[0109] The accumulation buffer 201 accumulates encoded data
transmitted thereto. The encoded data has been encoded by the image
encoding apparatus 100. The lossless decoder 202 decodes encoded
data read out from the accumulation buffer 201 at a certain timing,
using a format corresponding to the coding format used by the
lossless encoder 106 illustrated in FIG. 1.
[0110] Also, in a case where a target frame has been intra coded,
intra prediction mode information is stored in the header portion
of the encoded data. The lossless decoder 202 also decodes the
intra prediction mode information, and supplies the information to
the intra prediction unit 211. In contrast, in a case where a
target frame has been inter coded, motion vector information is
stored in the header portion of the encoded data. The lossless
decoder 202 also decodes the motion vector information, and
supplies the information to the motion prediction/compensation unit
212.
[0111] The dequantizer 203 dequantizes coefficient data (quantized
coefficient) which is obtained through decoding performed by the
lossless decoder 202, using a method corresponding to the
quantization method used by the quantizer 105 illustrated in FIG.
1. That is, the dequantizer 203 dequantizes the quantized
coefficient using a method similar to that used by the dequantizer
108 illustrated in FIG. 1.
[0112] The dequantizer 203 supplies the dequantized coefficient
data, that is, an orthogonal transform coefficient, to the inverse
orthogonal transform unit 204. The inverse orthogonal transform
unit 204 performs inverse orthogonal transform on the orthogonal
transform coefficient using a method corresponding to the
orthogonal transform method used by the orthogonal transform unit
104 illustrated in FIG. 1 (a method similar to the method used by
the inverse orthogonal transform unit 109 illustrated in FIG. 1)
and obtains decoded residual data corresponding to residual data
before orthogonal transform is performed by the image encoding
apparatus 100. For example, fourth-order inverse orthogonal
transform is performed.
[0113] The decoded residual data obtained through inverse
orthogonal transform is supplied to the computing unit 205. Also,
the computing unit 205 is supplied with a prediction image from the
intra prediction unit 211 or the motion prediction/compensation
unit 212 via the selector 213.
[0114] The computing unit 205 adds the decoded residual data and
the prediction image, thereby obtaining decoded image data
corresponding to the image data from which the prediction image has
not been subtracted by the computing unit 103 of the image encoding
apparatus 100. The computing unit 205 supplies the decoded image
data to the deblocking filter 206.
[0115] The deblocking filter 206 removes a block distortion of the
decoded image supplied thereto, and supplies the decoded image to
the screen rearrangement buffer 207.
[0116] The screen rearrangement buffer 207 rearranges images. That
is, the frames which have been rearranged in an encoding order by
the screen rearrangement buffer 102 illustrated in FIG. 1 are
rearranged in the original display order. The D/A converter 208
D/A-converts the image supplied from the screen rearrangement
buffer 207, and outputs the image to a display (not illustrates) to
display the image thereon.
[0117] The output of the deblocking filter 206 is also supplied to
the frame memory 209.
[0118] The frame memory 209, the selector 210, the intra prediction
unit 211, the motion prediction/compensation unit 212, and the
selector 213 respectively correspond to the frame memory 112, the
selector 113, the intra prediction unit 114, the motion
prediction/compensation unit 115, and the selector 116 of the image
encoding apparatus 100.
[0119] The selector 210 reads out an image on which inter
processing is to be performed and a reference image from the frame
memory 209, and supplies the images to the motion
prediction/compensation unit 212. Also, the selector 210 reads out
an image to be used for intra prediction from the frame memory 209,
and supplies the image to the intra prediction unit 211.
[0120] The intra prediction unit 211 is supplied with, for example,
information indicating an intra prediction mode which has been
obtained by decoding header information from the lossless decoder
202 as appropriate. The intra prediction unit 211 generates a
prediction image from the reference image obtained from the frame
memory 209 on the basis of the information, and supplies the
generated prediction image to the selector 213.
[0121] The motion prediction/compensation unit 212 obtains
information which has been obtained by decoding the header
information (prediction mode information, motion vector
information, reference frame information, flag, various parameters,
etc.) from the lossless decoder 202.
[0122] The motion prediction/compensation unit 212 generates a
prediction image from the reference image obtained from the frame
memory 209 on the basis of the information supplied from the
lossless decoder 202, and supplies the generated prediction image
to the selector 213.
[0123] The selector 213 selects the prediction image which has been
generated by the motion prediction/compensation unit 212 or the
intra prediction unit 211, and supplies the image to the computing
unit 205.
[Macroblock Type]
[0124] By the way, as disclosed in PTL 1, there is a coding format
in which an encoding mode for encoding and outputting image data
and a non-encoding mode for outputting image data without encoding
the image data are provided, whether the encoding mode or the
non-encoding mode is to be used is selected in units of
macroblocks, and the encoding mode and the non-encoding mode can be
used in combination within a single picture. As disclosed in PTL 2,
also in the AVC coding format, an I_PCM (Intra-block pulse code
modulation) mode (non-compression mode) for outputting image data
without encoding the image data is supported as one of the types of
macroblocks (mb_type), as illustrated in FIG. 3.
[0125] This is used for ensuring real-time operation of arithmetic
coding processing in a case where a quantization parameter is set
to be a small value, such as QP=0, and in a case where the
information amount of encoded data is larger than that of an input
image. Also, lossless coding can be realized by using the I-PCM
mode (non-compression mode).
[Cost Function]
[0126] Meanwhile, to achieve higher encoding efficiency in the AVC
coding format, it is important to select an appropriate prediction
mode.
[0127] An example of a selection method is a method which is loaded
in reference software of H.264/MPEG-4 AVC called JM (Joint Model),
which is released in
http://iphome.hhi.de/suchring/tml/index.htm.
[0128] According to JM, the following two mode determination
methods, that is, High Complexity Mode and Low Complexity Mode, can
be selected. In either of them, cost function values regarding
individual prediction modes are calculated, and a prediction mode
with the smallest cost function value is selected as an optimal
mode for a target block or macroblock.
[0129] The cost function regarding the High Complexity Mode is
expressed by the following equation (1)
Cost(Mode.epsilon..OMEGA.)=D+.lamda.*R (1)
[0130] Here, .OMEGA. represents a universal set of candidate modes
for encoding a target block or macroblock, and D represents the
differential energy between a decoded image and an input image in a
case where encoding is performed using the prediction mode. .lamda.
represents a Lagrange undetermined multiplier which is given as a
function of a quantization parameter. R represents a total amount
of code including an orthogonal transform coefficient in a case
where encoding is performed using the mode.
[0131] That is, to perform encoding using the High Complexity Mode,
it is necessary to once perform preliminary encoding processing
using all the candidate modes in order to calculate the parameters
D and R, which involves a larger amount of computation.
[0132] The cost function in the Low Complexity Mode is expressed by
the following equation (2)
Cost(Mode.epsilon..OMEGA.)=D+QP2Quant(QP)*HeaderBit (2)
[0133] Here, D represents the differential energy between a
prediction image and an input image, unlike in the High Complexity
Mode. QP2Quant (QP) is given as a function of a quantization
parameter QP, and HeaderBit represents an amount of code regarding
information belonging to the Header, such as a motion vector and
mode, not including an orthogonal transform coefficient.
[0134] That is, in the Low Complexity Mode, it is necessary to
perform prediction processing regarding individual candidate modes,
but a decoded image is not necessary, and thus it is not necessary
to perform encoding processing. Thus, the Low Complexity Mode can
be realized with an amount of computation smaller than that of the
High Complexity Mode.
[Coding Unit]
[0135] Next, description will be given of coding units, which are
defined in the HEVC coding format described in NPL 1.
[0136] Coding units (CUs) are also called coding tree blocks
(CTBs), are partial regions of an image of each picture which play
a role similar to macroblocks in AVC, and are coding units having a
hierarchical structure. The size of a macroblock is fixed to
16.times.16 pixels, whereas the size of a CU is not fixed and is
specified in image compression information in each sequence.
[0137] In particular, a CU having the largest size is called an LCU
(Largest Coding Unit), and a CU having the smallest size is called
an SCU (Smallest Coding Unit). For example, the sizes of these
regions are specified in a sequence parameter set (SPS) included in
image compression information. The individual regions are square
shaped, and the sizes thereof are limited to sizes expressed by a
power of 2.
[0138] FIG. 4 illustrates an example of coding units defined in
HEVC. In the example illustrated in FIG. 4, the size of LCU is 128,
and the maximum layer depth is 5. When the value of split_flag is
"1", the CU having a size of 2N.times.2N is split into CUs each
having a size of N.times.N in the immediately lower layer.
[0139] Furthermore, a CU is split into prediction units (PUs), each
being a region serving as a unit of processing for intra or inter
prediction (partial region of an image of each picture), or is
split into transform units (TUs), each being a region serving as a
unit of processing for orthogonal transform (partial region of an
image of each picture). At present, in HEVC, 16.times.16 and
32.times.32 orthogonal transform can be used in addition to
4.times.4 and 8.times.8 orthogonal transform.
[IBDI]
[0140] Meanwhile, NPL 2 suggests a method for increasing internal
arithmetic (IBDI (Internal bit depth increase except frame memory))
illustrated in FIG. 5. In this method, as illustrated in FIG. 5,
the bit depth of data is increased, for example, from 3 bits to 12
bits, in quantization processing, lossless coding processing,
dequantization processing, filter processing, prediction
processing, lossless decoding processing, and so forth performed by
an encoding apparatus and a decoding apparatus. Accordingly, an
internal arithmetic error in processing such as orthogonal
transform or motion compensation can be decreased, and encoding
efficiency can be enhanced.
[BALF]
[0141] Meanwhile, NPL 3 suggests a method in which an FIR filter is
provided in a motion compensation loop and loop filter processing
using the filter (BALF (Block-based Adaptive Loop Filter))) is
adaptively performed, as illustrated in FIG. 5. In an encoding
apparatus, the FIR filter coefficient is obtained using a Wiener
filter so as to minimize an error with respect to an input image,
and thereby degradation in a reference image can be minimized, and
encoding efficiency of image compression information to be output
can be enhanced.
[Efficiency of Encoding Processing]
[0142] Meanwhile, in the case of a coding format in which CUs are
defined and various processing operations are performed in units of
CUs, as in HEVC, it can be considered that a macroblock in AVC
corresponds to an LCU. However, since CUs have a hierarchical
structure as illustrated in FIG. 4, the size of the LCU in the top
layer is generally set to be larger than a macroblock in AVC, for
example, 128.times.128 pixels.
[0143] Therefore, in such a coding format, as in the case of AVC,
if the I_PCM mode is set in units of LCUs, the unit of processing
becomes larger than that in AVC, for example, 128.times.128
pixels.
[0144] The mode of intra prediction or inter prediction is
determined by calculating and comparing cost function values, as
described above. That is, prediction and encoding are performed
using all the modes, individual cost function values are
calculated, an optimal mode is selected, and encoded data is
generated using the optimal mode.
[0145] However, when, the I_PCM mode is adopted, the encoded data
generated using the optimal mode is discarded, and an input image
(non-encoded data) is adopted as an encoding result. Thus, when the
I_PCM mode is selected, all the processing operations for
generating the encoded data of the optimal mode are not necessary.
That is, if the unit of selection control of the I_PCM mode becomes
large, unnecessary processing operations further increase. That is,
as described above, if it is selected for each LCU whether or not
the I_PCM mode is to be adopted, the efficiency of encoding
processing may further decrease. Thus, for example, it may become
difficult to ensure real-time operation of CABAC.
[0146] Also, the above-described technologies such as IBDI and BALF
are not included in the AVC coding format. In a case where the
I_PCM mode is adopted, it is unknown how to control these
processing operations.
[0147] Accordingly, the present embodiment enables more detailed
control of selection of the I_PCM mode (non-compression mode), and
also enables enhancement of encoding efficiency while suppressing a
decrease in the efficiency of encoding processing. Also, the
present embodiment enables appropriate control of execution of IBDI
and BALF according to selection of the I_PCM mode, and also enables
further suppression of a decrease in the efficiency of encoding
processing.
[Image Encoding Apparatus]
[0148] FIG. 7 is a block diagram illustrating a main example
configuration of an image encoding apparatus.
[0149] The image encoding apparatus 300 illustrated in FIG. 7 is
basically similar to the image encoding apparatus 100 illustrated
in FIG. 1, and encodes image data. As illustrated in FIG. 7, the
image encoding apparatus 300 includes an A/D converter 301, a
screen rearrangement buffer 302, an adaptive shift-to-left unit
303, a computing unit 304, an orthogonal transform unit 305, a
quantizer 306, a lossless encoder 307, and an accumulation buffer
308. Also, the image encoding apparatus 300 includes a dequantizer
309, an inverse orthogonal transform unit 310, a computing unit
311, a loop filter 312, an adaptive shift-to-right unit 313, a
frame memory 314, an adaptive shift-to-left unit 315, a selector
316, an intra prediction unit 317, a motion prediction/compensation
unit 318, a selector 319, and a rate controller 320.
[0150] The image encoding apparatus 300 further includes a PCM
encoder 321.
[0151] The A/D converter 301 A/D-converts image data input thereto,
as in the case of the A/D converter 101. The A/D converter 301
supplies the converted image data (digital data) to the screen
rearrangement buffer 302 to store the image data therein. The
screen rearrangement buffer 302 rearranges, in accordance with a
GOP (Group of Picture) structure, frame images stored therein which
are arranged in a display order so that the frame images are
rearranged in an encoding order, as in the case of the screen
rearrangement buffer 102. The screen rearrangement buffer 302
supplies the rearranged frame images to the adaptive shift-to-left
unit 303.
[0152] In addition, the screen rearrangement buffer 302 supplies
the rearranged frame images also to the lossless encoder 307 and
the PCM encoder 321.
[0153] The adaptive shift-to-left unit 303 is controlled by the PCM
encoder 321, shifts image data which has been read out from the
screen rearrangement buffer 302 in the left direction, and
increases the bit depth thereof by a certain number of bits (for
example, 4 bits). For: example, the adaptive shift-to-left unit 303
increases the bit depth of the image data which has been read out
from the screen rearrangement buffer 302 from 8 bits to 12 bits. As
a result of increasing the bit depth in this way, the accuracy of
internal arithmetic in each of processing operations, such as
orthogonal transform processing, quantization processing, lossless
coding processing, prediction processing, and so forth can be
increased, and errors can be suppressed.
[0154] Note that the amount of shift to the left (the amount of
bits) is not specified, and may be fixed or variable. Also, the
shift-to-left processing may be skipped in accordance with control
performed by the PCM encoder 321.
[0155] The adaptive shift-to-left unit 303 supplies the image data
on which shift-to-left processing has been performed to the
computing unit 304 (in a case where the processing is skipped, the
image data output from the screen rearrangement buffer 302 is
supplied to the computing unit 304). In addition, the adaptive
shift-to-left unit 303 also supplies the image data to the intra
prediction unit 317 and the motion prediction/compensation unit
318.
[0156] The computing unit 304 subtracts, from the image supplied
from the adaptive shift-to-left unit 303, a prediction image
supplied from the intra prediction unit 317 or the motion
prediction/compensation unit 318 via the selector 319, as in the
case of the computing unit 103. The computing unit 304 outputs
difference information thereof to the orthogonal transform unit
305.
[0157] For example, in the case of an image on which intra coding
is to be performed, the computing unit 304 subtracts a prediction
image supplied from the intra prediction unit 317 from the image
supplied from the adaptive shift-to-left unit 303. Also, for
example, in the case of an image on which inter coding is to be
performed, the computing unit 304 subtracts a prediction image
supplied from the motion prediction/compensation unit 318 from the
image supplied from the adaptive shift-to-left unit 303.
[0158] The orthogonal transform unit 305 performs an orthogonal
transform, such as the discrete cosine transform or Karhunen-Loeve
transform, on the difference information supplied from the
computing unit 304, as in the case of the orthogonal transform unit
104. The method for the orthogonal transform is not specified. The
orthogonal transform unit 305 supplies a transform coefficient
thereof to the quantizer 306.
[0159] The quantizer 306 quantizes the transform coefficient
supplied from the orthogonal transform unit 305, as in the case of
the quantizer 105. The quantizer 306 performs quantization by
setting a quantization parameter on the basis of information about
a target value of an amount of code supplied from the rate
controller 320. The method for the quantization is not specified.
The quantizer 306 supplies the quantized transform coefficient to
the lossless encoder 307.
[0160] The lossless encoder 307 performs lossless coding, such as
variable-length coding or arithmetic coding, on the transform
coefficient quantized by the quantizer 306, as in the case of the
lossless encoder 106. Coefficient data has been quantized under
control performed by the rate controller 320, and thus the amount
of code thereof is equal to (or approximate to) a target value set
by the rate controller 320.
[0161] Note that, in a case where the I_PCM mode is selected by the
PCM encoder 321, the lossless encoder 307 regards an input image
(non-encoded data) supplied from the screen rearrangement buffer
302 as an encoding result (that is, encoding is actually
skipped).
[0162] Also, the lossless encoder 307 obtains information
indicating the mode of intra prediction and so forth from the intra
prediction unit 317, and obtains information indicating the mode of
inter prediction, motion vector information, and so forth from the
motion prediction/compensation unit 318. Furthermore, the lossless
encoder 307 obtains the filter coefficient used by the loop filter
312.
[0163] The lossless encoder 307 encodes the various pieces of
information, such as a filter coefficient, information indicating
the mode of intra prediction mode or inter prediction, and a
quantization parameter, as in the case of the lossless encoder 106,
and causes the various pieces of information to be part of header
information of encoded data (multiplexes the various pieces of
information). The loss less encoder 307 supplies encoded data which
has been obtained through encoding (including non-encoded data in
the case of the I_PCM mode) to the accumulation buffer 308 to store
the encoded data therein.
[0164] For example, in the lossless encoder 307, as in the case of
the lossless encoder 106, lossless coding processing such as
variable-length coding or arithmetic coding is performed. An
example of variable-length coding includes CAVLC (Context-Adaptive
Variable Length Coding) defined in the H.264/AVC format. An example
of arithmetic coding includes CABAC (Context-Adaptive Binary
Arithmetic Coding). Of course, the lossless encoder 307 may perform
encoding using a method other than these methods.
[0165] The accumulation buffer 308 temporarily holds the encoded
data supplied from the lossless encoder 307 (including non-encoded
data in the case of the I_PCM mode), as in the case of the
accumulation buffer 107. The accumulation buffer 308 outputs the
encoded data held therein to, for example, a recording device
(recording medium) or a transmission channel in the subsequent
stage (not illustrated) at a certain timing.
[0166] In addition, the transform coefficient quantized by the
quantizer 306 is also supplied to the dequantizer 309. The
dequantizer 309 dequantizes the quantized transform coefficient
using a method corresponding to the quantization performed by the
quantizer 306, as in the case of the dequantizer 108. The method
for the dequantization is not limited as long as the method
corresponds to the quantization processing performed by the
quantizer 306. The dequantizer 309 supplies the transform
coefficient obtained thereby to the inverse orthogonal transform
unit 310.
[0167] The inverse orthogonal transform unit 310 performs inverse
orthogonal transform on the transform coefficient supplied from the
dequantizer 309 using a method corresponding to the orthogonal
transform processing performed by the orthogonal transform unit
305, as in the case of the inverse orthogonal transform unit 109.
The method for the inverse orthogonal transform is not limited as
long as the method corresponds to the orthogonal transform
processing performed by the orthogonal transform unit 305. The
output obtained through the inverse orthogonal transform (recovered
difference information) is supplied to the computing unit 311.
[0168] The computing unit 311 adds a prediction image supplied from
the intra prediction unit 317 or the motion prediction/compensation
unit 318 via the selector 319 to the result of inverse orthogonal
transform supplied from the inverse orthogonal transform unit 310,
that is, recovered difference information, and obtains a locally
decoded image (decoded image), as in the case of the computing unit
110.
[0169] For example, in a case where the difference information
corresponds to an image on which intra coding is to be performed,
the computing unit 311 adds a prediction image supplied from the
intra prediction unit 317 to the difference information. Also, for
example, in a case where the difference information corresponds to
an image on which inter coding is to be performed, the computing
unit 311 adds a prediction image supplied from the motion
prediction/compensation unit 318 to the difference information.
[0170] The result of addition (decoded image) is supplied to the
loop filter 312 or the adaptive shift-to-right unit 313.
[0171] The loop filter 312 includes a deblocking filter, an
adaptive loop filter, or the like, and performs filter processing
on the decoded image supplied from the computing unit 311 as
appropriate. For example, the loop filter 312 performs deblocking
filter processing which is similar to that performed by the
deblocking filter 111 on the decoded image, thereby removing a
block distortion of the decoded image. Also, for example, the loop
filter 312 is controlled by the PCM encoder 321, and performs loop
filter processing on the result of the deblocking filter processing
(the decoded image from which a block distortion has been removed)
by using a Wiener filter, thereby improving the image quality. Note
that the adaptive loop filter processing may be skipped in
accordance with control performed by the PCM encoder 321.
[0172] Alternatively, the loop filter 312 may perform arbitrary
filter processing on the decoded image. Also, the loop filter 312
may supply a filter coefficient used for filter processing to the
lossless encoder 307 as necessary, so that the filter coefficient
is encoded.
[0173] The loop filter 312 supplies the result of filter processing
(the decoded image on which filter processing has been performed)
to the adaptive shift-to-right unit 313. Note that, as described
above, the decoded image output from the computing unit 311 may be
supplied to the adaptive shift-to-right unit 313 without via the
loop filter 312. That is, filter processing by the loop filter 312
may be skipped.
[0174] The adaptive shift-to-right unit 313 is controlled by the
PCM encoder 321, shifts the image data supplied from the computing
unit 311 or the loop filter 312 in the right direction, and
decreases the bit depth thereof by a certain number of bits (for
example, 4 bits). That is, the adaptive shift-to-right unit 313
shifts, to the right, the image data by the number of bits by which
the image data has been shifted to the left by the adaptive
shift-to-left unit 303, so as to change the bit depth of the image
data to the state before the image data is shifted to the left (the
state at the time when the image data is read out from the screen
rearrangement buffer 302).
[0175] For example, the adaptive shift-to-right unit 313 decreases
the bit depth of the image data supplied from the computing unit
311 or the loop filter 312 from 12 bits to 3 bits. As a result of
decreasing the bit depth in this way, the data amount of image data
stored in the frame memory can be decreased.
[0176] Note that the amount of shift to the right (the amount of
bits) is not specified as long as the amount matches the amount of
shift to the left in the adaptive shift-to-left unit 303. That is,
the amount may be fixed or variable. Also, the shift-to-right
processing may be skipped in accordance with control performed by
the PCM encoder 321.
[0177] The adaptive shift-to-right unit 313 supplies the image data
on which shift-to-right processing has been performed to the frame
memory 314 (in a case where the processing is skipped, the image
data output from the computing unit 311 or the loop filter 312 is
supplied to the frame memory 314).
[0178] The frame memory 314 stores the decoded image supplied
thereto, as in the case of the frame memory 112, and outputs the
stored decoded image to the adaptive shift-to-left unit 315 as a
reference image at a certain timing.
[0179] The adaptive shift-to-left unit 315 is a processing unit
similar to the adaptive shift-to-left unit 303, is controlled by
the PCM encoder 321, appropriately shifts the image data (reference
image) read out from the frame memory 314 in the left direction,
and increases the bit depth thereof by a certain number of bits
(for example, 4 bits).
[0180] For example, in a case where the mode is not the I_PCM mode,
the data of an input image is shifted to the left by the adaptive
shift-to-left unit 303. Thus, the adaptive shift-to-left unit 315
shifts the data of a reference image read out from the frame memory
314 to the left in accordance with control performed by the PCM
encoder 321, and increases the bit depth by the number of bits that
is the same as in the case of the adaptive shift-to-left unit 303
(for example, changes the bit depth from 8 bits to 12 bits).
[0181] Then, the adaptive shift-to-left unit 315 supplies the image
data on which shift-to-left processing has been performed to the
selector 316. As a result of increasing the bit depth in this way,
the bit depth of the reference image can be made the same as the
bit depth of the input image, and the reference image can be added
to the input image. Also, the accuracy of internal arithmetic such
as prediction processing can be increased, and errors can be
suppressed.
[0182] In contrast, for example, in the case of the I_PCM mode, the
data of an input image is not shifted to the left by the adaptive
shift-to-left unit 303. Thus, the adaptive shift-to-left unit 315
supplies the reference image read out from the frame memory 314 to
the selector 316 without increasing the bit depth, in accordance
with control performed by the PCM encoder 321.
[0183] In the case of intra prediction, the selector 316 supplies
the reference image supplied from the adaptive shift-to-left unit
315 to the intra prediction unit 317, as in the case of the
selector 113. Also, in the case of inter prediction, the selector
316 supplies the reference image supplied from the adaptive
shift-to-left unit 315 to the motion prediction/compensation unit
318, as in the case of the selector 113.
[0184] The intra prediction unit 317 performs intra prediction
(intra-screen prediction) in which a prediction image is generated
using the reference image supplied from the adaptive shift-to-left
unit 315 via the selector 316. The intra prediction unit 317
performs intra prediction using a plurality of prepared modes
(intra prediction modes). The intra prediction unit 317 is also
capable of performing intra prediction using an arbitrary mode
other than the modes defined in the AVC coding format.
[0185] The intra prediction unit 317 generates prediction images
using all candidate intra prediction modes, evaluates the cost
function values of the individual prediction images using an input
image supplied the adaptive shift-to-left unit 303, and selects an
optimal mode. After selecting an optimal intra prediction mode, the
intra prediction unit 317 supplies the prediction image which has
been generated using the optimal mode to the computing unit 304 and
the computing unit 311 via the selector 319.
[0186] Also, as described above, the intra prediction unit 317
supplies information, such as intra prediction mode information
indicating the adopted intra prediction mode, to the lossless
encoder 307 as appropriate, so that the information is encoded.
[0187] The motion prediction/compensation unit 318 performs motion
prediction (inter prediction) on an image on which inter coding is
to be performed, using an input image supplied from the adaptive
shift-to-left unit 303 and a reference image supplied from the
adaptive shift-to-left unit 315 via the selector 316, performs
motion compensation processing in accordance with a detected motion
vector, and generates a prediction image (inter prediction image
information). The motion prediction/compensation unit 318 performs
such inter prediction using a plurality of prepared modes (inter
prediction modes). The motion prediction/compensation unit 318 is
also capable of performing inter prediction using an arbitrary mode
other than the modes defined in the AVC coding format.
[0188] The motion prediction/compensation unit 318 generates
prediction images using all candidate inter prediction modes,
evaluates the cost function values of the individual prediction
images, and selects an optimal mode. After selecting the optimal
inter prediction mode, the motion prediction/compensation unit 318
supplies the prediction image generated using the optimal mode to
the computing unit 304 and the computing unit 311 via the selector
319.
[0189] Also, the motion prediction/compensation unit 318 supplies
inter prediction mode information indicating the adopted inter
prediction mode and motion vector information indicating a
calculated motion vector to the lossless encoder 307, so that the
information is encoded.
[0190] In the case of an image on which intra coding is to be
performed, the selector 319 supplies the output of the intra
prediction unit 317 to the computing unit 304 and the computing
unit 311, as in the case of the selector 116. In the case of an
image on which inter coding is to be performed, the selector 319
supplies the output of the motion prediction/compensation unit 313
to the computing unit 304 and the computing unit 311.
[0191] The rate controller 320 controls the rate of the
quantization operation performed by the quantizer 306 on the basis
of the amount of code of the encoded data accumulated in the
accumulation buffer 308 so that overflow or underflow does not
occur.
[0192] Also, the rate controller 320 supplies the amount of code
(the amount of generated code) of the encoded data accumulated in
the accumulation buffer 308 to the PC encoder 321.
[0193] The PCM encoder 321 compares the amount of code supplied
from the rate controller 320 with the data amount of the input
image supplied from the screen rearrangement buffer 302, and
selects whether or not the I_PCM mode is to be adopted. At this
time, the PCM encoder 321 performs selection in units of CUs, which
are smaller than LCUs. That is, the PCM encoder 321 controls
whether or not the I_PCM mode is to be adopted in more detail.
[0194] In accordance with the result of selection, the PCM encoder
321 controls the operations of the lossless encoder 307, the
adaptive shift-to-left unit 303, the adaptive shift-to-right unit
313, the adaptive shift-to-left unit 315, and the loop filter
312.
[Lossless Encoder, PCM Encoder, and Loop Filter]
[0195] FIG. 8 is a block diagram illustrating a main example
configuration of the lossless encoder 307, the PCM encoder 321, and
the loop filter 312 illustrated in FIG. 7.
[0196] As illustrated in FIG. 8, the lossless encoder 307 includes
a NAL (Network Abstraction Layer) encoder 331 and a CU encoder
332.
[0197] The NAL encoder 331 encodes a NAL, for example, an SPS
(Sequence Parameter Set), a PPS (Picture Parameter Set), or the
like, on the basis of a user instruction, specifications, or the
like input via a user interface (not illustrated). The NAL encoder
331 supplies the encoded NAL (NAL data) to the accumulation buffer
308, so that the NA data is added to CU data, which is an encoded
VCL (Video Coding Layer) supplied from the CU encoder 332 to the
accumulation buffer 308.
[0198] The CU encoder 332 is controlled by the PCM encoder 321 (on
the basis of an On/Off control signal supplied from the PCM encoder
321) and encodes a VCL. For example, in a case where the I_PCM mode
is not selected by the PCM encoder 321 (in a case where a control
signal representing "On" is supplied from the PCM encoder 321), the
CU encoder 332 encodes quantized orthogonal transform coefficients
of individual CUs. The CU encoder 332 supplies the pieces of
encoded data (CU data) of the individual CUs to the accumulation
buffer 308.
[0199] Also, for example, in a case where the I_PCM mode is
selected by the PCM encoder 321 (in a case where a control signal
representing "Off" is supplied from the PCM encoder 321), the CU
encoder 332 supplies input pixel values which are supplied from the
screen rearrangement buffer 302 to the accumulation buffer 308, as
an encoding result (CU data).
[0200] In addition, the CU encoder 332 also encodes a flag
(I_PCM_flag) which indicates whether or not the mode of encoding is
the I_PCM mode and which is supplied from the PCM encoder 321, and
supplies the encoded flag as CU data to the accumulation buffer
308. Furthermore, the CU encoder 332 encodes information about
filter processing, such as an adaptive filter flag and a filter
coefficient, supplied from the loop filter 312, and supplies the
encoded information as CU data to the accumulation buffer 308.
[0201] The method for encoding used by the CU encoder 332 is not
specified (for example, CABAC, CAVLC, or the like). The NAL data
and CU data supplied to the accumulation buffer 308 are combined
together and accumulated therein.
[0202] Note that the PCM encoder 321 actually controls whether or
not the I_PCM mode is to be selected, by using the amount of code
of encoded data which is generated by encoding, with the CU encoder
332, a quantized orthogonal transform coefficient.
[0203] Thus, for example, in a case where the I_PCM mode is not
selected, the encoded data which has been supplied to the
accumulation buffer 308 is adopted as an encoding result of the
quantized orthogonal transform coefficient of the target CU. Thus,
it is only necessary for the CU encoder 332 to encode additional
information, such as an I_PCM_flag.
[0204] In contrast, for example, in a case where the I_PCM mode is
selected, the CU encoder 332 supplies the input pixel values of the
target CU which are supplied from the screen rearrangement buffer
302 to the accumulation buffer 308, as an encoding result (CU
data). Thus, in this case, the encoded data of the target CU which
has been supplied (encoded data generated by encoding the quantized
orthogonal transform coefficient) is discarded. That is, all the
processing operations regarding generation of the encoded data are
redundant.
[0205] As illustrated in FIG. 8, the PCM encoder 321 includes an
I_PCM_flag generator 341 and a PCM deciding unit 342.
[0206] The I_PCM_flag generator 341 generates an I_PCM_flag in
accordance with a decision made by the PCM deciding unit 342, and
decides the value thereof. The I_PCM_flag generator 341 supplies
the generated I_PCM_flag to the CU encoder 332 of the lossless
encoder 307. For example, in a case where the PCM deciding unit 342
selects the I_PCM mode, the I_PCM_flag generator 341 sets the value
of the I_PCM_flag to be a value indicating that the I_PCM mode is
selected (for example, "1"), and supplies the I_PCM_flag to the CU
encoder 332. Also, for example, for example, in a case where the
PCM deciding unit 342 does not select the I_PCM mode, the
I_PCM_flag generator 341 sets the value of the I_PCM_flag to be a
value indicating that the I_PCM mode is not selected (for example,
"0"), and supplies the I_PCM_flag to the CU encoder 332.
[0207] The PCM deciding unit 342 decides whether or not the mode of
encoding is to be the I_PCM mode. The PCM deciding unit 342 obtains
the amount of data of the input pixel values supplied from the
screen rearrangement buffer 302, compares the amount of data with
the amount of generated code supplied from the rate controller 320,
and decides whether or not the I_PCM mode is to be selected on the
basis of the comparison result. The PCM deciding unit 342 supplies
an On/Off control signal representing the selection result to the
CU encoder 332 and the I_PCM_flag generator 341, thereby
controlling an operation in accordance with the selection
result.
[0208] For example, in a case where the amount of data of the input
pixel values is larger than the amount of generated code, the PCM
deciding unit 342 does not select the I_PCM mode. In this case, the
PCM deciding unit 342 supplies a control signal representing "On"
to the CU encoder 332, and causes the CU encoder 332 to encode the
quantized orthogonal transform coefficient. Also, the PCM deciding
unit 342 supplies a control signal representing "On" to the
I_PCM_flag generator 341, and causes the I_PCM_flag generator 341
to generate an I_PCM_flag having a value indicating that the I_PCM
mode is not selected (for example, "0").
[0209] In contrast, for example, in a case where the amount of data
of the input pixel values is smaller than or equal to the amount of
generated code, the PCM deciding unit 342 selects the I_PCM mode.
In this case, the PCM deciding unit 342 supplies a control signal
representing "Off" to the CU encoder 332, and causes the CU encoder
332 to output the input, pixel values as an encoding result (CU
data). Also, the PCM deciding unit 342 supplies a control signal
representing "Off" to the I_PCM_flag generator 341, and causes the
I_PCM_flag generator 341 to generate an I_PCM_flag having a value
indicating that the I_PCM mode is selected (for example "1").
[0210] The PCM deciding unit 342 is capable of deciding whether or
not the PCM mode is to be selected, in units of CUs of all the
sizes (in an arbitrary layer) which are set in a sequence parameter
set, as well as LCUs. Accordingly, for example, processing of
suppressing the generation of many bits with a low QP by using the
I_PCM mode can be executed in units of smaller CUs. Thus, the
amount of code (including the data amount of non-encoded data in
the I_PCM mode) can be controlled in more detail, and redundant
processing which occurs in the I_PCM mode can be reduced.
[0211] Also, the PCM deciding unit 342 supplies an On/Off control
signal representing a selection result to the adaptive
shift-to-left unit 303, the adaptive shift-to-right unit 313, and
the adaptive shift-to-left unit 315, thereby controlling IBDI in
accordance with the selection result. That is, in a case where the
mode of a target CU is the I_PCM mode, the PCM deciding unit 342
performs control so that processing of increasing and decreasing
the bit precision is not performed by an adaptive shift device.
[0212] For example, in a case where the I_PCM mode is not selected,
the PCM deciding unit 342 supplies a control signal representing
"On" to the adaptive shift-to-left unit 303, the adaptive
shift-to-right unit 313, and the adaptive shift-to-left unit 315,
and causes shift-to-left processing and shift-to-right processing
to be performed, so that the bit precision in internal processing
is increased.
[0213] In contrast, for example, in a case where the I_PCM mode is
selected, the PCM deciding unit 342 supplies a control signal
representing "Off" to the adaptive shift-to-left unit 303, the
adaptive shift-to-right unit 313, and the adaptive shift-to-left
unit 315, and causes shift-to-left processing and shift-to-right
processing to be skipped, so that the bit precision in internal
processing is not increased.
[0214] In the I_PCM mode, input image pixel values are transmitted
to image compression information, and thus no arithmetic errors
occur. Increasing the bit arithmetic precision therefor is
redundant processing. The PCM deciding unit 342 is capable of
eliminating such redundant processing by performing processing in
the above-described manner.
[0215] Furthermore, the PCM deciding unit 342 supplies the On/Off
control signal representing the selection result to the loop filter
312, thereby controlling adaptive loop filter processing (BALF) in
accordance with the selection result. That is, in a case where the
mode of a target CU is the I_PCM mode, the PCM deciding unit 342
performs control so that adaptive loop filter processing is not
performed by the loop filter 312.
[0216] For example, in a case where the I_PCM mode is not selected,
the PCM deciding unit 342 supplies a control signal representing
"On" to the loop filter 312, so that adaptive loop filter
processing is performed. In contrast, for example, in a case where
the I_PCM mode is selected, the PCM deciding unit 342 supplies a
control signal representing "Off" to the loop filter 312, so that
adaptive loop filter processing is skipped.
[0217] In the I_PCM mode, input image pixel values are transmitted
to image compression information, and thus degradation does not
occur. Performing adaptive loop filter processing thereon is
redundant. The PCM deciding unit 342 is capable of eliminating such
redundant processing by performing processing in the
above-described manner.
[0218] As illustrated in FIG. 8, the loop filter 312 includes a
deblocking filter 351, a pixel sorting unit 352, a filter
coefficient calculator 353, and a filtering unit 354.
[0219] As in the case of the deblocking filter 111, the deblocking
filter 351 performs deblocking filter processing on a decoded image
(before-deblocking-filter pixel values) supplied from the computing
unit 311, thereby removing a block distortion.
[0220] Even if a target CU to be processed is processed using the
I_PCM mode, it is not always that the CU adjacent to the target CU
has been processed using the I_PCM mode. Thus, even if the target
CU is processed using the I_PCM mode, a block distortion may occur.
Thus, deblocking filter processing is performed regardless of
whether or not the mode of the target CU is the I_PCM mode.
[0221] The deblocking filter 351 supplies the result of filter
processing (after-deblocking-filter pixel values) to the pixel
sorting unit 352.
[0222] The pixel sorting unit 352 sorts individual results of
filter processing (after-deblocking-filter pixel values) as pixel
values on which adaptive loop filter processing is to be performed
or as pixel values on which adaptive loop filter processing is not
be to performed, in accordance with the values of On/Off control
signals supplied from the PCM deciding unit 342.
[0223] For example, in a case where a control signal representing
"On" is supplied from the PCM deciding unit 342, the pixel sorting
unit 352 sorts the after-deblocking-filter pixel values of the
corresponding CU as pixel values on which adaptive loop filter
processing is to be performed. In contrast, for example, in a case
where a control signal representing "Off" is supplied from the PCM
deciding unit 342, the pixel sorting unit 352 sorts the
after-deblocking-filter pixel values of the corresponding CU as
pixel values on which adaptive loop filter processing is not to be
performed.
[0224] The pixel sorting unit 352 supplies the sorted pixel values
of individual pixels (after-deblocking-filter pixel values) to the
filter coefficient calculator 353.
[0225] The filter coefficient calculator 353 calculates, using a
Wiener filter, a filter coefficient (FIR filter coefficient) of an
adaptive loop filter for the pixel values on which adaptive loop
filter processing is to be performed among the
after-deblocking-filter pixel values supplied thereto, so as to
minimize the error with respect to an input image. That is, the
filter coefficient calculator 353 calculates a filter coefficient
by excluding pixels which are to be processed using the I_PCM
mode.
[0226] The filter coefficient calculator 353 supplies the
after-deblocking-filter pixel values and the calculated filter
coefficient to the filtering unit 354.
[0227] The filtering unit 354 performs, using the filter
coefficient supplied thereto, adaptive loop filter processing on
the pixel values which have been sorted as pixel values on which
adaptive loop filter processing is to be performed. The filtering
unit 354 supplies the pixel values on which filter processing has
been performed and the pixel values which have been sorted as pixel
values on which adaptive loop filter processing is not to be
performed, which are after-adaptive-filter pixel values, to the
adaptive shift-to-right unit 313.
[0228] Also, the filtering unit 354 generates an adaptive filter
flag (on/off_flag), which is filter identification information
indicating whether or not filter processing has been performed, for
each of certain blocks which are set independently of CUs. The
method for setting the value of the adaptive filter flag is not
specified.
[0229] For example, in a case where adaptive loop filter processing
has been performed on some or all of the pixels in a target block
to be processed (current block), the adaptive filter flag may be
set to have a value indicating that filter processing has been
performed (for example, "1"). Also, for example, in a case where
adaptive loop filter processing has not been performed on all the
pixels in the block, the adaptive filter flag may be set to have a
value indicating that filter processing has not been performed (for
example, "0"). The value of the adaptive filter flag may be set on
the basis of another standard.
[0230] The filtering unit 354 supplies the generated adaptive
filter flag to the CU encoder 332 of the lossless encoder 307, so
that the adaptive filter flag is encoded and is provided to the
decoding side. Note that, in a case where the value of the adaptive
filter flag is a value indicating that filter processing has not
been performed (for example, "0", providing the adaptive filter
flag to the decoding side may be skipped (the adaptive filter flag
is not provided to the decoding side).
[0231] For example, in a case where the value of the adaptive
filter flag is a value indicating that filter processing has not
been performed (for example, "0") and where the coding format used
by the lossless encoder 307 (CU encoder 332) is VLC, the filtering
unit 354 skips supply of the adaptive filter flag (does not provide
the adaptive filter flag to the decoding side). Also, for example,
in a case where the value of the adaptive filter flag is a value
indicating that filter processing has not been performed (for
example, "0") and where the coding format used by the lossless
encoder 307 (CU encoder 332) is CABAC, the filtering unit 354
supplies the adaptive filter flag to the CU encoder 332 of the
lossless encoder 307 (provides the adaptive filter flag to the
decoding side).
[0232] This is because, in the case of VLC, if the amount of input
information is small, it is possible to realize higher encoding
efficiency, but in the case of CABAC, if the same information is
continuously input, the probability at the time of performing
arithmetic coding is biased, and higher encoding efficiency can be
realized.
[0233] Furthermore, the filtering unit 354 supplies the filter
coefficient used for the adaptive loop filter processing to the CU
encoder 332 of the lossless encoder 307, so that the filter
coefficient is encoded and provided to the decoding side.
[PCM Deciding Unit]
[0234] FIG. 9 is a block diagram illustrating a main example
configuration of the PCM deciding unit 342 illustrated in FIG.
8.
[0235] As illustrated in FIG. 9, the PCM deciding unit 342 includes
an input data amount calculator 361, a PCM determining unit 362, an
encoding controller 363, an adaptive shift controller 364, and a
filter controller 365.
[0236] The input data amount calculator 361 calculates, for a
target CU, the amount of input data which is the amount of data of
input pixel values supplied from the screen rearrangement buffer
302, and supplies the calculated amount of input data to the PCM
determining unit 362.
[0237] The PCM determining unit 362 obtains the amount of generated
code (generated bits) supplied from the rate controller 320,
compares the amount of generated code with the amount of input data
supplied from the input data amount calculator 361, and determines
whether or not the I_PCM mode is to be selected for the CU on the
basis of the comparison result. That is, the PCM determining unit
362 determines, for each CU in an arbitrary layer, whether or not
the I_PCM mode is to be selected. The PCM determining unit 362
supplies the determination result to the encoding controller 363,
the adaptive shift controller 364, and the filter controller
365.
[0238] On the basis of the determination result supplied from the
PCM determining unit 362 (identification information indicating
whether or not the I_PCM mode is selected), the encoding controller
363 supplies an On/Off control signal to the CU encoder 332 and the
I_PCM_flag generator 341.
[0239] Thus, the encoding controller 363 is capable of controlling
the mode of encoding in units of CUs in an arbitrary layer.
Accordingly, the encoding controller 363 is capable of controlling
the amount of code (including the data amount of non-encoded data
in the I_PCM mode) in more detail, and is also capable of reducing
redundant processing when the I_PCM mode is selected.
[0240] On the basis of the determination result supplied from the
PCM determining unit 362 (information indicating whether or not the
I_PCM mode is selected), the adaptive shift controller 364 supplies
an On/Off control signal to the adaptive shift-to-left unit 303,
the adaptive shift-to-right unit 313, and the adaptive
shift-to-left unit 315.
[0241] Thus, the adaptive shift controller 364 is capable of
performing control so that the bit depth is not increased in
internal arithmetic when the I_PCM mode is selected. Accordingly,
the adaptive shift controller 364 is capable of reducing redundant
processing.
[0242] On the basis of the determination result supplied from the
PCM determining unit 362 (information indicating whether or not the
I_PCM mode is selected), the filter controller 365 supplies an
On/Off control signal to the pixel sorting unit 352.
[0243] Thus, the filter controller 365 is capable of performing
control so that adaptive loop filter processing is not performed
when the I_PCM mode is selected. Accordingly, the filter controller
365 is capable of reducing redundant processing.
[0244] As described above, the image encoding apparatus 300 is
capable of reducing redundant processing and suppressing a decrease
in the efficiency of encoding processing. Also, the image encoding
apparatus 300 is capable of selecting the I_PCM mode
(non-compression mode) in more detail (in units of smaller data
units), and enhancing encoding efficiency. Therefore, the image
encoding apparatus 300 is capable of enhancing encoding efficiency
which suppressing a decrease in the efficiency of encoding
processing.
[Flow of Encoding Processing]
[0245] Next, description will be given of the flow of individual
processing operations performed by the above-described image
encoding apparatus 300. First, an example of the flow of encoding
processing will be described with reference to the flowchart in
FIG. 10.
[0246] In step S301, the A/D converter 301 A/D-converts an input
image. In step S302, the screen rearrangement buffer 302 stores the
A/D-converted image therein and rearranges pictures from a display
order to an encoding order.
[0247] In step S303, the adaptive shift-to-left unit 303 adaptively
performs shift to the left on the input image on the basis of
control performed by the PCM encoder 321. In step S304, the
adaptive shift-to-left unit 315 adaptively performs shift to the
left on a reference image.
[0248] In step S305, the intra prediction unit 317 performs intra
prediction processing in the intra prediction mode by using the
reference image which is shifted to the left in step S304. In step
S306, the motion prediction/compensation unit 318 performs inter
motion prediction processing, in which motion prediction or motion
compensation is preformed in the inter prediction mode, by using
the reference image which is shifted to the left in step S304.
[0249] Note that, actually, the processing of shifting the bit
depth of the reference image to the left may be performed when the
reference image is read out from the frame memory 314 in intra
prediction processing or inter motion prediction processing.
[0250] In step S307, the selector 319 determines an optimal mode on
the basis of the individual cost functions which have been output
from the intra prediction unit 317 and the motion
prediction/compensation unit 3138. That is, the selector 319
selects any one of a prediction image generated by the intra
prediction unit 317 and a prediction image generated by the motion
prediction/compensation unit 318.
[0251] Also, selection information indicating which of the
prediction images has been selected is supplied to, among the intra
prediction unit 317 and the motion prediction/compensation unit
318, the one corresponding to the selected prediction image. In a
case where the prediction image of the optimal intra prediction
mode is selected, the intra prediction unit 317 supplies intra
prediction mode information indicating the optimal intra prediction
mode and so forth to the lossless encoder 307. In a case where the
prediction image of the optimal inter prediction mode is selected,
the motion prediction/compensation unit 318 outputs information
indicating the optimal inter prediction mode, and if necessary,
information based on the optimal inter prediction mode, to the
lossless encoder 307. Examples of the information based on the
optimal inter prediction mode include motion vector information,
flag information, and reference frame information.
[0252] In step S308, the computing unit 304 computes the difference
between the image in which the bit depth has been shifted to the
left through the processing in step S303 and the prediction image
selected through the processing in step S307. The prediction image
is supplied to the computing unit 304 via the selector 319 from the
motion prediction/compensation unit 318 in a case where inter
prediction is to be performed, and from the intra prediction unit
317 in a case where intra prediction is to be performed.
[0253] The amount of difference data is smaller than that of the
original image data. Thus, the amount of data can be reduced
compared to a case where an image itself is encoded.
[0254] In step S309, the orthogonal transform unit 305 performs an
orthogonal transform on the difference information generated
through the processing in step S308. Specifically, an orthogonal
transform such as the discrete cosine transform or Karhunen-Loeve
transform is performed, and a transform coefficient is output.
[0255] In step S310, the quantizer 306 quantizes the orthogonal
transform coefficient obtained through the processing in step
S309.
[0256] In step S311, the lossless encoder 307 encodes the transform
coefficient quantized through the processing in step S310. That is,
lossless coding such as variable-length coding or arithmetic coding
is performed on the difference image.
[0257] The lossless encoder 307 encodes a quantization parameter
calculated in step S310 and adds the quantization parameter to
encoded data. Also, the lossless encoder 307 encodes information
about the mode of the prediction image selected through the
processing in step S307, and adds the information to the encoded
data which is obtained by encoding the difference image. That is,
the lossless encoder 307 also encodes the optimal intra prediction
mode information supplied from the intra prediction unit 317 or
information based on the optimal inter prediction mode supplied
from the motion prediction/compensation unit 318, and adds the
information to the encoded data.
[0258] Furthermore, the lossless encoder 307 encodes the filter
coefficient and flag information obtained from the loop filter 312
and adds them to the encoded data. Furthermore, the lossless
encoder 307 encodes NAL data.
[0259] In step S312, the accumulation buffer 308 accumulates the
encoded data output from the lossless encoder 307. The encoded data
accumulated in the accumulation buffer 308 is read out as
appropriate, and is transmitted to the decoding side via a
transmission channel or a recording medium.
[0260] In step S313, the rate controller 320 calculates the amount
of code (the amount of generated code) of the encoded data
accumulated in the accumulation buffer 308 through the processing
in step S312, and controls the rate of the quantization operation
performed by the quantizer 306 on the basis of the amount of code
so that overflow or underflow does not occur. Also, the rate
controller 320 supplies the amount of generated code to the PCM
encoder 321.
[0261] In step S314, the PCM encoder 321 performs PCM encoding
control processing by using the amount of generated code calculated
in step S313. In step S315, the lossless encoder 307 performs PCM
encoding processing in accordance with control performed by the PCM
encoder 321.
[0262] In step S316, the dequantizer 309 to the frame memory 314
perform reference image generation processing in which the
difference information quantized through the processing in step
S310 is locally decoded to generate a reference image.
[0263] After the processing in step S316 ends, the encoding
processing ends. The encoding processing is repeatedly performed
on, for example, individual CUs.
[PCM Encoding Control Processing]
[0264] Next, an example of the flow of the PCM encoding control
processing which is performed in step S314 in FIG. 10 will be
described with reference to the flowchart in FIG. 11.
[0265] After the PCM encoding control processing starts, in step
S331, the PCM determining unit 362 of the PCM deciding unit 342
obtains, from the rate controller 320, the amount of generated code
of the encoded data of the quantized orthogonal transform
coefficient of the target CU.
[0266] In step S332, the input data amount calculator 361
calculates the amount of input data of the input pixel values of
the target CU.
[0267] In step S333, the PCM determining unit 362 compares the
amount of code obtained in step S331 with the amount of input data
calculated in step S332, and determines whether or not encoding is
to be performed using the I_PCM mode.
[0268] In step S334, the I_PCM_flag generator 341 generates an
I_PCM_flag on the basis of an On/Off control signal which
represents the determination result generated in step S333 and
which is supplied from the encoding controller 363.
[0269] In step S335, the encoding controller 363 supplies an On/Off
control signal representing the determination result generated in
step S333 to the CU encoder 332, thereby controlling encoding of CU
data.
[0270] In step S336, the adaptive shift controller 364 supplies an
On/Off control signal representing the determination result
generated in step S333 to the adaptive shift-to-left unit 303, the
adaptive shift-to-right unit 313, and the adaptive shift-to-left
unit 315, thereby controlling adaptive shift processing.
[0271] in step S337, the encoding controller 363 supplies an On/Off
control signal representing the determination result generated in
step S333 to the pixel sorting unit 352 of the loop filter 312,
thereby controlling adaptive loop filter processing.
[0272] After the processing in step S337 ends, the PCM deciding
unit 342 ends the PCM encoding control processing, the processing
returns to step S314 in FIG. 10, and the processing is performed
from step S315.
[Flow of PCM Encoding Processing]
[0273] Next, an example of the flow of the PCM encoding processing
which is performed in step S315 in FIG. 10 will be described with
reference to the flowchart in FIG. 12
[0274] After the PCM encoding processing starts, in step S351, the
CU encoder 332 determines whether or not encoding is to be
performed using the I_PCM mode. In a case where control is
performed in the above-described PCM encoding control processing so
that encoding is to be performed using the I_PCM mode, the CU
encoder 332 causes the processing to proceed to step S352. In step
S352, the CU encoder 332 selects the input pixel values of the
target CU as an encoding result. The CU encoder 332 causes the CU
data of the target CU in the accumulation buffer 308 to be
discarded, and causes the input pixel values to be accumulated in
the accumulation buffer 308.
[0275] After step S352 ends, the CU encoder 332 causes the
processing to proceed to step S353. On the other hand, if it is
determined in step S351 that encoding is not to be performed using
the I_PCM mode, the CU encoder 332 causes the processing to proceed
to step S353.
[0276] In step S353, the CU encoder 332 encodes the I_PCM_flag
which has been generated in the above-described PCM encoding
control processing, and accumulates the I_PCM_flag in the
accumulation buffer 308.
[0277] After step S353 ends, the CUT encoder 332 ends the PCM
encoding processing, the processing returns to step S315 in FIG.
10, and the processing is performed from step S316.
[Flow of Reference Image Generation Processing]
[0278] Next, an example of the flow of the reference image
generation processing which is performed in step S316 in FIG. 10
will be described with reference to the flowchart in FIG. 13.
[0279] After the reference image generation processing starts, the
adaptive shift-to-left unit 315 determines in step S371 whether or
not the I_PCM mode is selected on the basis of control performed by
the adaptive shift controller 364. If the I_PCM mode is selected,
the bit depth is not increased in internal arithmetic, and thus
processing is performed again starting from prediction processing.
That is, if it is determined that the I_PCM mode is selected, the
adaptive shift-to-left unit 315 causes the processing to proceed to
step S372 without performing shift-to-left processing on a
reference image.
[0280] In step S372, the intra prediction unit 317 performs intra
prediction processing using a reference image in which the bit
depth is not shifted to the left. In step S373, the motion
prediction/compensation unit 318 performs inter motion prediction
processing using the reference image in which the bit depth is not
shifted to the left.
[0281] In step S374, the selector 319 decides an optimal mode on
the basis of the individual cost function values output from the
intra prediction unit 317 and the motion prediction/compensation
unit 318. That is, the selector 319 selects any one of a prediction
image generated by the intra prediction unit 317 and a prediction
image generated by the motion prediction/compensation unit 318.
[0282] Also, selection information indicating which of the
prediction images has been selected is supplied to, among the intra
prediction unit 317 and the motion prediction/compensation unit
318, the one corresponding to the selected prediction image. In a
case where the prediction image of the optimal intra prediction
mode is selected, the intra prediction unit 317 supplies intra
prediction mode information indicating the optimal intra prediction
mode and so forth to the lossless encoder 307. In a case where the
prediction image of the optimal inter prediction mode is selected,
the motion prediction/compensation unit 318 supplies information
indicating the optimal inter prediction mode, and if necessary,
information based on the optimal inter prediction mode, to the
lossless encoder 307. Examples of the information based on the
optimal inter prediction mode include motion vector information,
flag information, and reference frame information.
[0283] In a case where the I_PCM mode is selected, the bit depth in
internal arithmetic is not increased. That is, shift-to-left
processing performed on the reference image by the adaptive
shift-to-left unit 303 is skipped. In step S375, the computing unit
304 computes the difference between the input image in which the
bit depth is not shifted to the left and the prediction image
selected through the processing in step S374. The prediction image
is supplied to the computing unit 304 via the selector 319 from the
motion prediction/compensation unit 318 in a case where inter
prediction is performed and from the intra prediction unit 317 in a
case where intra prediction is performed.
[0284] In step S376, the orthogonal transform unit 305 performs an
orthogonal transform on the difference information generated
through the processing in step S375. Specifically, an orthogonal
transform such as the discrete cosine transform or Karhunen-Loeve
transform is performed, and a transform coefficient is output. In
step S377, the quantizer 306 quantizes the orthogonal transform
coefficient obtained through the processing in step S376.
[0285] After the orthogonal transform coefficient has been
quantized, the quantizer 306 causes the processing to proceed to
step S378, where a reference image is generated using the
orthogonal transform coefficient (data on which shift-to-left
processing has not been performed) quantized in step S377.
[0286] In contrast, if it is determined in step S371 that the I_PCM
mode is not selected, the adaptive shift-to-left unit 315 skips the
processing operations in step S372 to step S377, and causes the
processing to proceed to step S378. That is, in a case where the
I_PCM mode is not selected, a reference image is generated using
the orthogonal transform coefficient (data on which shift-to-left
processing has been performed) quantized in step S310 in FIG.
10.
[0287] In step S378, the dequantizer 309 dequantizes the quantized
orthogonal transform coefficient (also referred to as quantized
coefficient) in accordance with the characteristic corresponding to
the characteristic of the quantizer 306. In step S379, the inverse
orthogonal transform unit 310 performs inverse orthogonal transform
on the orthogonal transform coefficient obtained through the
processing in step S378, in accordance with the characteristic
corresponding to the characteristic of the orthogonal transform
unit 305.
[0288] In step S380, the computing unit 311 adds the prediction
image to the locally decoded difference information, thereby
generating a locally decoded image (an image corresponding to the
input to the computing unit 304). For example, in a case where the
I_PCM mode is selected, the computing unit 311 adds a prediction
image on which shift-to-left processing has not been performed to
difference information on which shift-to-left processing has not
been performed, so as to generate a decoded image on which
shift-to-left processing has not been performed. Also, for example,
in a case where the I_PCM mode is not selected, the computing unit
311 adds a prediction image on which shift-to-left processing has
been performed to difference information on which shift-to-left
processing has been performed, so as to generate a decoded image on
which shift-to-left processing has been performed.
[0289] In step S381, the loop filter 312 performs loop filter
processing on the locally decoded image obtained through the
processing in step S380 on the basis of control performed by the
filter controller 365, and performs loop filter processing
including deblocking filter processing, adaptive loop filter
processing, and so forth as appropriate.
[0290] In step S382, the adaptive-shift-to-right unit 313
determines whether or not the I_PCM mode is selected, on the basis
of control performed by the adaptive shift controller 364. If it is
determined that the I_PCM mode is not selected, the adaptive
shift-to-right unit 313 causes the processing to proceed to step
S383
[0291] In a case where the I_PCM mode is not selected, the decoded
image has undergone an increase in the bit depth in internal
arithmetic. Thus, in step S383, the adaptive shift-to-right unit
313 adaptively performs shift to right on the bit depth of the
result of filter processing (decoded image) obtained through the
loop filter processing in step S381. After the processing in step
S383 ends, the adaptive shift-to-right unit 313 causes the
processing to proceed to step S384.
[0292] If it is determined in step S382 that the I_PCM mode is
selected, the adaptive shift-to-right unit 313 causes the
processing to proceed to step S384 without performing
shift-to-right processing.
[0293] In step S384, the frame memory 314 stores the decoded image.
After the processing in step S384 ends, the frame memory 314 ends
the reference image generation processing, the processing returns
to step S316 in FIG. 10, and the encoding processing ends.
[Flow of Loop Filter Processing]
[0294] Next, an example of the flow of the loop filter processing
which is performed in step S381 in FIG. 13 will be described with
reference to the flowchart in FIG. 14.
[0295] After the loop filter processing starts, in step S401, the
deblocking filter 351 of the loop filter 312 performs deblocking
filter processing on the decoded image (before-deblocking-filter
pixel values) supplied from the computing unit 311.
[0296] In step S402, the pixel sorting unit 352 sorts the
individual pixels of the decoded image on the basis of whether or
not the mode is the I_PCM mode, under control performed by the
filter controller 365 of the PCM deciding unit 342.
[0297] In step S403, the filter coefficient calculator 353
calculates a filter coefficient for the pixels which have been
sorted to undergo filter processing (target pixels to be
processed). In step S404, the filtering unit 354 performs adaptive
filter processing on the target pixels to be processed, using the
filter coefficient calculated in step S403.
[0298] In step S405, the filtering unit 354 sets an adaptive filter
flag for the target block to be processed, and supplies the
adaptive filter flag and the filter coefficient to the CU encoder
332, so that the adaptive filter flag and the filter coefficient
are encoded.
[0299] After the processing in step S405 ends, the loop filter 312
ends the loop filter processing, the processing returns to step
S381 in FIG. 13, and the processing is performed from step
S382.
[0300] By performing the individual processing operations in the
above-described manner, the image encoding apparatus 300 is capable
of enhancing encoding efficiency while suppressing a decease in the
efficiency of encoding processing.
2. Second Embodiment
Image Decoding Apparatus
[0301] FIG. 15 is a block diagram illustrating a main example
configuration of an image decoding apparatus. The image decoding
apparatus 500 illustrated in FIG. 15 is an apparatus which is
basically similar to the image decoding apparatus 200 illustrated
in FIG. 2, and decodes encoded data which is generated by encoding
image data.
[0302] The image decoding apparatus 500 illustrated in FIG. 15 is a
decoding apparatus corresponding to the image encoding apparatus
300 illustrated in FIG. 7. Encoded data which has been encoded by
the image encoding apparatus 300 is supplied to the image decoding
apparatus 500 via an arbitrary path, for example, a transmission
channel, a recording medium, or the like, and is decoded.
[0303] As illustrated in FIG. 15, the image decoding apparatus 500
includes an accumulation buffer 501, a lossless decoder 502, a
dequantizer 503, an inverse orthogonal transform unit 504, a
computing unit 505, a loop filter 506, an adaptive shift-to-right
unit 507, a screen rearrangement buffer 508, and a D/A converter
509. Also, the image decoding apparatus 500 includes a frame memory
510, an adaptive shift-to-left unit 511, a selector 512, an intra
prediction unit 513, a motion prediction/compensation unit 514, and
a selector 515.
[0304] The image decoding apparatus 500 further includes a PCM
decoder 516.
[0305] The accumulation buffer 501 accumulates encoded data
transmitted thereto, as in the case of the accumulation buffer 201.
The encoded data has been encoded by the image encoding apparatus
300.
[0306] The lossless decoder 502 reads out encoded data from the
accumulation buffer 501 at a certain timing, and decodes the
encoded data using a format corresponding to the coding format used
by the lossless encoder 307 illustrated in FIG. 15. At this time,
the lossless decoder 502 supplies the I_PCM_flag included in the
encoded data to the PCM decoder 516, and causes the PCM decoder 516
to determine whether or not the mode is the I_PCM mode
(non-compression mode).
[0307] In a case where the mode is the I_PCM mode, the CUT data
obtained from the accumulation buffer 501 is non-encoded data.
Thus, the lossless decoder 502 supplies the CU data to the
dequantizer 503 in accordance with control performed by the PCM
decoder 516.
[0308] In a case where the mode is not the I_PCM mode, the CU data
obtained from the accumulation buffer 501 is encoded data. Thus,
the lossless decoder 502 decodes the CU data in accordance with
control performed by the PCM decoder 516, and supplies a decoding
result to the dequantizer 503.
[0309] Note that, for example, in a case where the target CU has
been intra-coded, intra prediction mode information is stored in
the header portion of the encoded data. The lossless decoder 502
also decodes the intra prediction mode information, and supplies
the information to the intra prediction unit 513. On the other
hand, in a case where the target CU has been inter-coded, motion
vector information and inter prediction mode information are stored
in the header portion of the encoded data. The lossless decoder 502
also decodes the motion vector information and the inter prediction
mode information, and supplies the information to the motion
prediction/compensation unit 514.
[0310] In a case where the mode is not the I_PCM mode, the
dequantizer 503 dequantizes the coefficient data (quantized
coefficient) supplied from the lossless decoder 502 using a method
corresponding to the quantization method used by the quantizer 306
illustrated in FIG. 7, as in the case of the dequantizer 203. That
is, the dequantizer 503 dequantizes the quantized coefficient using
a method similar to that used by the dequantizer 309 illustrated in
FIG. 7. The dequantizer 503 supplies the dequantized coefficient
data, that is, the orthogonal transform coefficient, to the inverse
orthogonal transform unit 504.
[0311] Also, in a case where the mode is the I_PCM mode, the
dequantizer 503 supplies the CU data supplied from the lossless
decoder 502 (image data which is not encoded) to the inverse
orthogonal transform unit 504.
[0312] In a case where the mode is not the I_PCM mode, the inverse
orthogonal transform unit 504 performs inverse orthogonal transform
on the orthogonal transform coefficient using a method
corresponding to the orthogonal transform method used by the
orthogonal transform unit 305 illustrated in FIG. 7 (a method
similar to that used by the inverse orthogonal transform unit 310
illustrated in FIG. 7), as in the case of the inverse orthogonal
transform unit 204. With the inverse orthogonal transform
processing, the inverse orthogonal transform unit 504 obtains
decoded residual data corresponding to the residual data before
orthogonal transform is performed in the image encoding apparatus
300. For example, fourth-order inverse orthogonal transform is
performed. The inverse orthogonal transform unit 504 supplies the
decoded residual data obtained through inverse orthogonal transform
to the computing unit 505.
[0313] Also, in a case where the mode is the I_PCM mode, the
inverse orthogonal transform unit 504 supplies the CU data supplied
from the dequantizer 503 (image data which is not encoded) to the
computing unit 505.
[0314] Also, a prediction image is supplied to the computing unit
505 from the intra prediction unit 513 or the motion
prediction/compensation unit 514 via the selector 515.
[0315] In a case where the mode is not the I_PCM mode, the
computing unit 505 adds the decoded residual data and the
prediction image, and thereby obtains decoded image data
corresponding to the image data before the prediction image is
subtracted by the computing unit 304 of the image encoding
apparatus 300, as in the case of the computing unit 205. The
computing unit 505 supplies the decoded image data to the loop
filter 506.
[0316] In a case where the mode is the I_PCM mode, the computing
unit 505 supplies the CU data supplied from the inverse orthogonal
transform unit 504 (image data which is not encoded) to the loop
filter 506. In this case, the CU data is not residual information,
and thus addition to the prediction image is not necessary.
[0317] The loop filter 506 performs loop filter processing
including deblocking filter processing, adaptive loop filter
processing, or the like, on the decoded image supplied from the
computing unit 505 as appropriate, under control performed by the
PCM decoder 516.
[0318] More specifically, the loop filter 506 performs deblocking
filter processing similar to that performed by the deblocking
filter 206 on the decoded image, thereby removing a block
distortion of the decoded image. Also, the loop filter 506 performs
loop filter processing using a Wiener filter on the result of the
deblocking filter processing (decoded image from which a block
distortion has been removed) in accordance with control performed
by the PCM decoder 516, thereby improving image quality.
[0319] Alternatively, the lop filter 506 may perform arbitrary
filter processing on the decoded image. Alternatively, the loop
filter 506 may perform filter processing by using the filter
coefficient supplied from the image encoding apparatus 300
illustrated in FIG. 7.
[0320] The loop filter 506 supplies the result of filter processing
(the decoded image on which filter processing has been performed)
to the adaptive shift-to-right unit 507.
[0321] The adaptive shift-to-right unit 507 is a processing unit
similar to the adaptive shift-to-right unit 313 (FIG. 7), and is
controlled by the PCM decoder 516. In a case where the mode is not
the I_PCM mode, the adaptive shift-to-right unit 507 shifts the
decoded image data supplied from the loop filter 506 in the right
direction, so as to reduce the bit depth by a certain number of
bits (for example, 4 bits), for example, the bit depth is decreased
from 12 bits to 8 bits. That is, the adaptive shift-to-right unit
507 shifts the decoded image data to the right by the number of
bits by which the image data has been shifted to the left in the
image encoding apparatus 300, so as to change the bit depth of the
image data to the state where shift to the left has not been
performed (the state at the time when the image data is read out
from the screen rearrangement buffer 302 (FIG. 7)).
[0322] The adaptive shift-to-right unit 507 supplies the image data
on which shift-to-right processing has been performed to the screen
rearrangement buffer 508 and the frame memory 510.
[0323] Note that the amount of shift to the right (the amount of
bits) is not specified as long as the amount is the same as the
amount of shift to the left in the image encoding apparatus 300
(adaptive shift-to-let unit 303). That is, the amount may be fixed
or variable. For example, the amount of shift may be predetermined
(shared in advance between the image encoding apparatus 300 and the
image decoding apparatus 500), or the image decoding apparatus 500
may be allowed to calculate the amount of shift in the image
encoding apparatus 300, or information indicating the amount of
shift may be provided from the image encoding apparatus 300.
[0324] Alternatively, the shift-to-right processing may be skipped
in accordance with control performed by the PCM decoder 516. For
example, in the case of the I_PCM mode, image data does not undergo
shift-to-left processing in the image encoding apparatus 300.
Therefore, in this case, the adaptive shift-to-right unit 507 is
controlled by the PCM decoder 516, and supplies the decoded image
data supplied from the loop filter 506 to the screen rearrangement
buffer 508 and the frame memory 510 without shifting the decoded
image data in the right direction.
[0325] The screen rearrangement buffer 508 rearranges images, as in
the case of the screen rearrangement buffer 207. That is, the
frames which have been rearranged in an encoding order by the
screen rearrangement buffer 302 illustrated in FIG. 7 are
rearranged in the original display order. The screen rearrangement
buffer 508 supplies the decoded image data of individual frames to
the D/A converter 509 in the original display order.
[0326] The D/A converter 509 D/A-converts the frame images supplied
from the screen rearrangement buffer 508, and outputs the frame
images to a display (not illustrated) so that the frame images are
displayed, as in the case of the D/A convertor 208.
[0327] The frame memory 510 stores the decoded image supplied from
the adaptive shift-to-right unit 507, and supplies the stored
decoded image as a reference image to the adaptive shift-to-left
unit 511 at a certain timing.
[0328] The adaptive shift-to-left unit 511 is a processing unit
similar to the adaptive shift-to-left unit 315, is controlled by
the PCM decoder 516, shifts the image data (reference image) read
out from the frame memory 510 in the left direction as appropriate,
and increases the bit depth thereof by a certain number of bits
(for example, 4 bits).
[0329] For example, in a case where the mode is not the I_PCM mode,
the decoded image data supplied from the inverse orthogonal
transform unit 504 to the computing unit 505 is image data which
has been shifted to the left in the image encoding apparatus 300
(for example, the bit depth is 12 bits). Thus, the adaptive
shift-to-left unit 511 increases the bit depth of the reference
image which has been read out from the frame memory 510 by a
certain number of bits (for example, increases the bit depth from 8
bits to 12 bits) in accordance with control performed by the PCM
decoder 516.
[0330] Then, the adaptive shift-to-left unit 511 supplies the image
data on which shift-to-left processing has been performed to the
selector 512. As a result of increasing the bit depth in this way,
the bit depth of the reference image can be made the same as the
bit depth of the de code image, so that the reference image can be
added to the decoded image. Also, the accuracy of internal
arithmetic such as prediction processing can be increased, and
errors can be suppressed.
[0331] On the other hand, for example, in case where the mode is
the I_PCM mode, the decoded image data supplied from the inverse
orthogonal transform unit 504 to the computing unit 505 is image
data on which shift-to-left processing has not been performed in
the image encoding apparatus 300 (for example, the bit depth is 8
bits). Thus, the adaptive shift-to-left unit 511 supplies the
reference image read out from the frame memory 510 to the selector
512 without increasing the bit depth in accordance with control
performed by the PCM decoder 156.
[0332] In the case of intra prediction, the selector 512 supplies
the reference image supplied from the adaptive shift-to-left unit
511 to the intra prediction unit 513. On the other hand, in the
case of inter prediction, the selector 512 supplies the reference
image supplied from the adaptive shift-to-left unit 511 to the
motion prediction/compensation unit 514.
[0333] The intra prediction unit 513 is supplied with information
indicating the intra prediction mode, which has been obtained by
decoding header information, and so forth from the lossless decoder
502 as appropriate. The intra prediction unit 513 performs intra
prediction using the reference image obtained from the frame memory
510 using the intra prediction mode used by the intra prediction
unit 317, and generates a prediction image. That is, the intra
prediction unit 513 is also capable of performing intra prediction
using an arbitrary mode other than the modes defined in the AVC
coding format, like the intra prediction unit 317.
[0334] The intra prediction unit 513 supplies the generated
prediction image to the selector 515.
[0335] The motion prediction/compensation unit 514 is supplied with
information obtained by decoding header information (prediction
mode information, motion vector information, reference frame
information, flag, various parameters, etc.) from the lossless
decoder 502.
[0336] The motion predict ion/compensation unit 514 performs inter
prediction using the reference image obtained from the frame memory
510 using the inter prediction mode used by the motion
prediction/compensation unit 318, thereby generating a prediction
image. That is, the motion prediction/compensation unit 514 is also
capable of performing intra prediction using an arbitrary mode
other than the modes defined in the AVC coding format, like the
motion prediction/compensation unit 318.
[0337] The motion prediction/compensation unit 514 supplies the
generated prediction image to the selector 515.
[0338] The selector 515 selects the prediction image generated by
the motion prediction/compensation unit 514 or the intra prediction
unit 513, and supplies the prediction image to the computing unit
505, as in the case of the selector 213.
[0339] The PCM decoder 516 controls the lossless decoder 502, the
loop filter 506, the adaptive shift-to-right unit 507, and the
adaptive shift-to-left unit 511 on the basis of the I_PCM_flag
supplied from the lossless decoder 502.
[Lossless Decoder, PCM Decoder, and Loop Filter]
[0340] FIG. 16 is a block diagram illustrating a main example
configuration of the lossless decoder 502, the PCM decoder 516, and
the loop filter 506 illustrated in FIG. 15.
[0341] As illustrated in FIG. 16, the lossless decoder 502 includes
a NAL decoder 531 and a CU decoder 532. The NAL decoder 531 decodes
encoded NAL data supplied from the accumulation buffer 501, and
supplies the decoded NAL data to the CU decoder 532.
[0342] The CU decoder 532 decodes encoded CU data which is supplied
from the accumulation buffer 501, on the basis of the NAL data
supplied from the NAL decoder 531.
[0343] Also, upon obtaining an I_PCM_flag by decoding the CU data,
the CU decoder 532 supplies the I_PCM_flag to the PCM decoder 516.
The CU decoder 532 decodes the encoded CU data in accordance with
control performed by the PCM decoder 156 based on the I_PCM_flag
(on the basis of an On/Off control signal supplied from the PCM
encoder 321).
[0344] For example, in a case where the I_PCM_flag has a value
indicating that the mode is not the I_PCM mode, and where a control
signal representing "On" is obtained from the PCM decoder 516, the
CU decoder 532 decodes the encoded CU data supplied from the
accumulation buffer 501 on the basis of the NAL data supplied from
the NAL decoder 531, and supplies a quantized orthogonal transform
coefficient to the dequantizer 503.
[0345] On the other hand, in a case where the I_PCM_flag has a
value indicating that the mode is the I_PCM mode, and where a
control signal representing "Off" is obtained from the PCM decoder
516, the CU decoder 532 supplies the CU data supplied from the
accumulation buffer 501 (image data which is not encoded) to the
dequantizer 503.
[0346] Note that the CU decoder 532 supplies information obtained
by decoding the CU data, such as a filter coefficient and an
adaptive filter flag, to the loop filter 506.
[0347] As illustrated in FIG. 8, the PCM decoder 516 includes an
I_PCM_flag buffer 541 and a PCM controller 542.
[0348] The I_PCM_flag buffer 541 stores the I_PCM_flag supplied
from the CU decoder 532 of the lossless decoder 502, and supplies
the I_PCM_flag to the PCM controller 542 at a certain timing.
[0349] The PCM controller 542 determines whether or not the I_PCM
mode was selected at the time of encoding in the image encoding
apparatus 300, on the basis of the value of the I_PCM_flag obtained
from the I_PCM_flag buffer 541. On the basis of the determination
result, the PCM controller 542 supplies an On/Off control signal to
the CU decoder 532, and controls the operation thereof.
[0350] For example, in a case where it is determined that the I_PCM
mode was not selected, the PCM controller 542 supplies a control
signal representing "On" to the CU decoder 532, causes the CU
decoder 532 to decode the encoded CU data, and causes the CU
decoder 532 to supply a quantized orthogonal transform coefficient
obtained through the decoding to the dequantizer 503.
[0351] On the other hand, for example, in a case where it is
determined that the I_PCM mode was selected, the PCM controller 542
supplies a control signal representing "Off" to the CU decoder 532,
and causes the CU decoder 532 to supply CU data, which is
non-encoded data, to the dequantizer 503 as output pixel
values.
[0352] With the above-described control, the CU decoder 532 is
capable of appropriately decoding encoded data supplied from the
image encoding apparatus 300. That is, the CU decoder 532 is
capable of decoding encoded data more appropriately even if whether
or not the mode is the I_PCM mode is controlled in units of CUs,
which are smaller than LCUs.
[0353] Also, the PCM controller 542 supplies an On/Off control
signal to the adaptive shift-to-right unit 507 and the adaptive
shift-to-left unit 511 on the basis of the determination result
about the I_PCM mode, and controls the operation thereof.
[0354] For example, in a case where it is determined that the I_PCM
mode was not selected, the PCM controller 542 supplies a control
signal representing "n" to the adaptive shift-to-right unit 507 and
the adaptive shift-to-left unit 511, and causes shift-to-left
processing and shift-to-right processing to be performed, so that
the bit precision in internal processing is increased.
[0355] On the other hand, in a case where it is determined that the
I_PCM mode was selected, the PCM controller 542 supplies a control
signal representing "Off" to the adaptive shift-to-right unit 507
and the adaptive shift-to-left unit 511, and causes shift-to-left
processing and shift-to-right processing to be skipped, so that the
bit precision in internal processing is not increased.
[0356] With the above-described control, the PCM controller 542 is
capable of appropriately eliminating redundant processing even if
whether or not the mode is the I_PCM mode is controlled in units of
CUs, which are smaller than LCUs.
[0357] Furthermore, the PCM controller 542 supplies an On/ff
control signal to the loop filter 506 on the basis of the
determination result about the I_PCM mode, and controls the
operation thereof. For example, in a case where it is determined
that the I_PCM mode was not selected, the PCM controller 542
supplies a control signal representing "On" to the loop filter 506.
On the other hand, for example, in a case where it is determined
that the I_PCM mode was selected, the PCM controller 542 supplies a
control signal representing "Off" to the loop filter 506.
[0358] With the above-described control, the PCM controller 542 is
capable of eliminating redundant processing more appropriately even
if whether or not the mode is the I_PCM mode is controlled in units
of CUs, which are smaller than LCUs.
[0359] As illustrated in FIG. 8, the loop filter 506 includes a
deblocking filter 551, a pixel sorting unit 552, and a filtering
unit 553.
[0360] The deblocking filter 551 performs deblocking filter
processing on the decoded image supplied from the computing unit
505 (before-deblocking-filter pixel values), thereby removing a
block distortion, as in the case of the deblocking filter 206.
[0361] As in the case of the image encoding apparatus 300,
deblocking filter processing is performed regardless of whether or
not the mode for the target CU is the I_PCM mode. The deblocking
filter 551 supplies the result of filter processing
(after-deblocking-filter pixel values) to the pixel sorting unit
552.
[0362] In accordance with the value of the On/Off control signal
supplied from the PCM controller 542, the pixel sorting unit 552
sorts individual results of filter processing
(after-deblocking-filter pixel values) as pixel values on which
adaptive loop filter processing is to be performed and pixel values
on which adaptive loop filter processing is not to be
performed.
[0363] For example, in a case where a control signal representing
"On" is supplied from the PCM controller 542, the pixel sorting
unit 552 sorts the after-deblocking-filter pixel values of the CU
as pixel values on which adaptive loop filter processing is to be
performed. In contrast, for example, in a case where a control
signal representing "Off" is supplied from the PCM controller 542,
the pixel sorting unit 552 sorts the after-deblocking-filter pixel
values of the CU as pixel values on which adaptive loop filter
processing is not to be performed.
[0364] The pixel sorting unit 552 supplies the sorted pixel values
of individual pixels (after-deblocking-filter pixel values) to the
filtering unit 553.
[0365] The filtering unit 553 performs adaptive loop filter
processing on the pixel values which have been sorted as pixel
values on which adaptive loop filter processing is to be performed,
on the basis of the adaptive loop filter flag supplied from the CU
decoder 532.
[0366] Adaptive loop filter processing is performed in units of
certain blocks which are set independently of CUs. In a case where
the value of the adaptive loop filter flag of the target block is a
value indicating that filter processing has not been performed on
the encoding side (for example, "0"), the filtering unit 553 skips
adaptive loop filter processing for the target block, and supplies
the after-deblocking-filter pixel values supplied thereto to the
adaptive shift-to-right unit 507 as after-adaptive-filter pixel
values.
[0367] By the way, the method for setting an adaptive loop filter
flag depends on the specification and so forth of the image
encoding apparatus 300. Thus, depending on the method for setting
an adaptive loop filter flag by the image encoding apparatus 300,
even if the value of the adaptive loop filter flag of the target
block is a value indicating that filter processing has been
performed on the encoding side (for example, "1"), there is a
possibility that a pixel of the I_PCM mode is included in the
target block.
[0368] Thus, in a case where there is a pixel value which has been
sorted as a pixel value on which adaptive loop filter processing is
not to be performed in the target block, the filtering unit 553
skips adaptive loop filter processing for the target block even if
the value of the adaptive loop filter flag of the target block is a
value indicating that filter processing has been performed on the
encoding side (for example, "1"), and supplies the
after-deblocking-filter pixel values supplied thereto to the
adaptive shift-to-right unit 507 as after-adaptive-filter pixel
values.
[0369] That is, the filtering unit 553 performs adaptive loop
filter processing on the target block only in a case where the
value of the adaptive loop filter flag of the target block is a
value indicating that filter processing has been performed on the
encoding side (for example, "1") and where all the pixels in the
target block have been sorted as pixels on which adaptive loop
filter processing is to be performed.
[0370] Note that, in the case of performing adaptive loop filter
processing, the filtering unit 553 performs adaptive loop filter
processing using the filter coefficient supplied from the CU
decoder 32 (the filter coefficient used in the adaptive loop filter
processing on the encoding side).
[0371] The filtering unit 553 supplies the pixel values on which
adaptive loop filter processing has been performed to the adaptive
shift-to-right unit 507 as after-adaptive-filter pixel values.
[0372] With the above-described control, the PCM controller 542 is
capable of appropriately eliminating redundant processing even if
whether or not the mode is the I_PCM mode (non-compression mode) is
controlled in units of CUs, which are smaller than LCUs.
[0373] As described above, with the processing operations performed
by the individual processing units, the image decoding apparatus
500 is capable of realizing enhanced encoding efficiency while
suppressing a decrease in the efficiency of encoding
processing.
[Flow of Decoding Processing]
[0374] Next, the flow of individual processing operations performed
by the above-described image decoding apparatus 500 will be
described. First, an example of the flow of decoding processing
will be described with reference to the flowcharts in FIG. 17 and
FIG. 18
[0375] After decoding processing starts, in step S501, the
accumulation buffer 501 accumulates an image (encoded data)
transmitted thereto. In step S502, the CU decoder 532 obtains an
adaptive filter flag from the encoded data accumulated in step
S501. In step S503, the CU decoder 532 obtains an I_PCM_flag from
the encoded data accumulated in step S501. The I_PCM_flag buffer
541 stores the I_PCM_flag.
[0376] In step S504, the PCM controller 542 determines whether or
not the encoding mode of the encoded data (CU data) accumulated in
step S501 is the I_PCM mode (that is, non-encoded data), on the
basis of the value of the I_PCM_flag stored in the I_PCM_flag
buffer 541.
[0377] If it is determined that the encoding mode is not the I_PCM
mode, the PCM controller 542 causes the processing to proceed to
step S505. In step S505, the lossless decoder 502 performs lossless
decoding processing, decodes the encoded data (CU data) accumulated
in step S501, and obtains a quantized orthogonal transform
coefficient, filter coefficient, and so forth.
[0378] In step S506, the dequantizer 503 dequantizes the quantized
orthogonal transform coefficient, which is obtained through the
processing in step S505. In step S507, the inverse orthogonal
transform unit 504 performs inverse orthogonal transform on the
orthogonal transform coefficient which is dequantized through the
processing in step S506, and generates decoded image data.
[0379] In step S508, the adaptive shift-to-left unit 511 obtains
the reference image corresponding to the target CU from the frame
memory 510, and performs shift-to-left processing on the reference
image in accordance with control performed by the PCM decoder
516.
[0380] In step S509, the intra prediction unit 513 or the motion
prediction/compensation unit 514 performs prediction processing
using the reference image on which shift-to-left processing is
performed in step S508, thereby generating a prediction image.
[0381] In step S510, the computing unit 505 adds the prediction
image generated by the intra prediction unit 513 or the motion
prediction/compensation unit 514 in step S509, to the difference
information obtained through the processing in step S507.
[0382] In step S511, the loop filter 506 performs loop filter
processing on the addition result obtained in step S510.
[0383] In step S512, the adaptive shift-to-right unit 507 performs
shift-to-right processing on the result of loop filter processing
obtained in step S511, in accordance with control performed by the
PCM decoder 516.
[0384] In step S513, the frame memory 510 stores the decoded image
data as a reference image.
[0385] In step S514, the screen rearrangement buffer 508 rearranges
the frames of the decoded image data. That is, the frames of the
decoded image data which have been rearranged in an encoding order
by the screen rearrangement buffer 302 (FIG. 7) of the image
encoding apparatus 300 are rearranged in the original display
order.
[0386] In step S515, the D/A converter 509 D/A-converts the decoded
image data in which the frames are rearranged in step S514. The
decoded image data is output to a display (not illustrated) and the
image thereof is displayed.
[0387] Also, if it is determined in step S504 that the encoding
mode is the I_PCM mode, the PCM controller 542 causes the
processing to proceed to step S521 in FIG. 18. In step S521, the
lossless decoder 502 performs lossless decoding processing, and
regards the encoded data (non-compressed data) accumulated in step
S501 as an encoding result (output pixel values).
[0388] In step S522, the loop filter 506 performs loop filter
processing on the output pixel values obtained in step S521. After
the processing in step S522 ends, the loop filter 506 returns the
processing to step S513 in FIG. 17, so that the subsequent step is
performed.
[Flow of Loop Filter Processing]
[0389] Next, an example of the flow of the loop filter processing
which is performed in step S511 in FIG. 17 and step S522 in FIG. 18
will be described with reference to the flowchart in FIG. 19.
[0390] After the loop filter processing starts, in step S541, the
deblocking filter 551 performs deblocking filter processing on the
before-deblocking-filter pixel values which are obtained in step
S510 or step S521.
[0391] In step S542, the filtering unit 553 obtains a filter
coefficient. In step S543, the pixel sorting unit 552 determines
whether or not adaptive loop filtering is to be performed, on the
basis of the value of the adaptive loop filter flag. In a case
where it is determined that adaptive loop filter processing is to
be performed, the pixel sorting unit 552 causes the processing to
proceed to step S544.
[0392] In step S544, the pixel sorting unit 552 sorts the
after-deblocking-filter pixel values in accordance with whether or
not the mode is the I_PCM mode, in accordance with control
performed by the PCM controller 542.
[0393] In step S545, the filtering unit 545 performs adaptive loop
filter processing on the after-deblocking-filter pixel values which
have been sorted to undergo adaptive loop filter processing, by
using the filter coefficient obtained in step S542. After the
processing in step S545 ends, the filtering unit 545 ends the loop
filter processing, the processing returns to step S511 in FIG. 17
or step S522 in FIG. 18, and the processing proceeds to step S512
in FIG. 17 or step S513 in FIG. 17.
[0394] If it is determined in step S543 in FIG. 19 that adaptive
loop filter processing is not to be performed, the pixel sorting
unit 552 ends the loop filter processing, the processing returns to
step S511 in FIG. 17 or step S522 in FIG. 18, and the processing
proceeds to step S512 in FIG. 17 or step S513 in FIG. 17.
[0395] As a result of performing the individual processing
operations in the above-described manner, the image decoding
apparatus 500 is capable of realizing enhanced encoding efficiency
while suppressing a decrease in the efficiency of encoding
processing.
[Position of I_PCM Information]
[0396] As described above, selection control of the I_PCM mode can
be performed in units of CUs, so that encoding using the I_PCM mode
can be performed on only some of CUs in an LCU, as illustrated in
part A of FIG. 20, for example. Part A of FIG. 20 illustrates the
structure of CUs in one LCU. The LCU illustrated in part A of FIG.
20 is constituted by seven CUs, that is, CU0 to CU6. The individual
numerals indicate the processing order in the LCU. In the example
illustrated in part A of FIG. 20, the shaded CUs (CU1, CU3, and
CU6) are to be encoded using the I_PCM mode.
[0397] In this case, as illustrated in part B of FIG. 20, I_PCM
information, which is information regarding I_PCM inside the LCU,
may be added at the top of the data of LCU in a code stream.
[0398] The I_PCM information includes, for example, flag
information indicating whether or not the LCU includes a CU which
is to be encoded using the I_PCM mode. In the case of the example
illustrated in FIG. 20, CU1, CU3, and CU6 are CUs to be encoded
using the I_PCM mode, and thus the flag information is set to be a
value indicating that the LCU includes a CU which is to be encoded
using the I_PCM mode (for example, "1").
[0399] In this way, the image encoding apparatus 300 adds
information about I_PCM inside the LCU at the top of the LCU, and
thereby the image decoding apparatus 500 is capable of easily
determining whether or not the LCU includes a CU which is to be
encoded using the I_PCM mode, before decoding the LCU.
[0400] Also, the I_PCM information includes, for example,
information indicating a CU which is to be encoded using the I_PCM
mode included in the LCU. In the case of the example illustrated in
FIG. 20, CU1, CU3, and CU6 are CUs which are to be encoded using
the I_PCM mode, and thus the I_PCM information includes information
indicating these CUs (CU1, CU3, and CU6).
[0401] In this way, the image encoding apparatus 300 adds
information about I_PCM inside the LCU at the top of the LCU, and
thereby the image decoding apparatus 500 is capable of easily
determining which CU in the LCU has been encoded using the I_PCM
mode (non-compression mode), before decoding the LCU.
[0402] Note that, in the description given above, pieces of
information other than image data, such as an I_PCM_flag, an
adaptive filter flag, a filter coefficient, and I_PCM information,
are provided from the image encoding apparatus 300 to the image
decoding apparatus 500 as necessary, but these pieces of
information may be added at an arbitrary position of encoded data.
For example, the pieces of information may be added at the top of a
CU or LCU, or may be added to a slice header, or may be stored in a
sequence parameter set (SPS), a picture parameter set (PPS), or the
like. Alternatively, for example, the pieces of information may be
stored in a parameter set (for example, a header of a sequence or
picture) of an SEI (Supplemental Enhancement Information) or the
like.
[0403] Furthermore, the pieces of information may be transmitted to
the decoding side separately from encoded data. In that case, it is
necessary to clarify (allow the decoding side to determine) the
correspondence between the pieces of information and encoded data,
but the method therefor is not specified. For example, table
information indicating the correspondence may be separately
created, or link information indicating the other side may be
embedded in the data of each side.
[0404] Alternatively, the pieces of information may be shared in
advance by the image encoding apparatus 300 and the image decoding
apparatus 500. In that case, transmission of the pieces of
information may be omitted.
3. Third Embodiment
Personal Computer
[0405] The above-described series of processing operations may be
executed by hardware or may be executed by software. In this case,
for example, the hardware or software may be constituted as the
personal computer illustrated in FIG. 21.
[0406] In FIG. 21, a CPU (Central Processing Unit) 601 of a
personal computer 600 executes various processing operations in
accordance with a program stored in a ROM (Read Only Memory) 602 or
a program loaded from a storage unit 613 to a RAM (Random Access
Memory) 603. Also, data which is necessary for the CPU 601 to
execute various processing operations is stored in the RAM 603 as
appropriate.
[0407] The CPU 601, the ROM 602, and the RAM 603 are connected to
one another via a bus 604. Also, an input/output interface 610 is
connected to the bus 604.
[0408] An input unit 611 including a keyboard, a mouse, or the
like; an output unit 612 including a display such as a CRT (Cathode
Ray Tube) or an LCD (Liquid Crystal Display), and a speaker or the
like; the storage unit 613 including a hard disk or the like; and a
communication unit 614 including a modem or the like are connected
to the input/output interface 610. The communication unit 614
performs communication processing via a network, including the
Internet.
[0409] Also, a drive 615 is connected to the input/output interface
610 as necessary, a removable medium 621 such as a magnetic disk,
an optical disc, a magneto-optical disc, or a semiconductor memory
is loaded thereto as appropriate, and a computer program read out
therefrom is installed in the storage unit 613 as necessary.
[0410] In the case of causing software to execute the
above-described series of processing operations, a program
constituting the software is installed via a network or a recording
medium.
[0411] The recording medium is constituted by, for example, as
illustrated in FIG. 21, the removable medium 621 which is provided
separately from the main body of the apparatus to distribute a
program to a user, which contains the program recorded thereon, and
which is constituted by a magnetic disk (including a flexible
disk), an optical disc (including a CD-ROM (Compact Disc-Read Only
Memory) and a DVD (Digital Versatile Disc)), a magneto-optical disc
(including an MD (Mini Disc)), or a semiconductor memory.
Alternatively, the recording medium is constituted by the ROM 602
containing the program recorded thereon or a hard disk included in
the storage unit 613, which is provided to a user in the state of
being incorporated in the main body of the apparatus in
advance.
[0412] The program executed by the computer may be a program in
which processing operations are performed in time series in
accordance with the order described in this description, or may be
a program in which processing operations are performed in parallel
or at necessary timings, for example, when the processing
operations are called.
[0413] Also, in this description, the steps describing a program
recorded on a recording medium may be processing operations which
are performed in time series in accordance with the described
order, or may be processing operations which are executed in
parallel or individually.
[0414] Also, in this description, a "system" is an entire apparatus
constituted by a plurality of devices.
[0415] Also, the configuration described above as a single device
(or processing unit) may be divided into a plurality of devices (or
processing units). In contrast, the configuration described above
as a plurality of devices (or processing units) may be combined
into a single device (or processing unit). Also, a configuration
other than that described above may of course be added to the
configuration of the individual devices (or processing units).
Furthermore, as long as the configuration and operation of the
entire system are substantially the same, part of the configuration
of a certain device (or processing unit) may be included in the
configuration of another device (or another processing unit). That
is, the embodiments of the present technology are not limited to
the above-described embodiments, and various modifications can be
made without deviating from the gist of the present technology.
[0416] For example, each of the lossless encoder 307, the loop
filter 312, and the PCM encoder 321 illustrated in FIG. 8 may be
configured as an independent device. Also, each of the NAL encoder
331, the CU encoder 332, the I_PCM_flag generator 341, the PCM
deciding unit 342, the deblocking filter 351, the pixel sorting
unit 352, the filter coefficient calculator 353, and the filtering
unit 354 illustrated in FIG. 8 may be configured as an independent
device.
[0417] Furthermore, each of the input data amount calculator 361,
the PCM determining unit 362, the encoding controller 363, the
adaptive shift controller 364, and the filter controller
illustrated in FIG. 9 may be configured as an independent
device.
[0418] Alternatively, these processing units may be arbitrarily
combined together to constitute an independent device. Of course,
these processing units may be combined with an arbitrary processing
unit illustrated in FIG. 7 to FIG. 9, or may be combined with a
processing unit which is not illustrated.
[0419] This is the same in the image decoding apparatus 500. For
example, each of the lossless decoder 502, the loop filter 506, and
the PCM decoder 516 illustrated in FIG. 15 may be configured as an
independent device. Also, each of the NAL decoder 531, the CU
decoder 532, the I_PCM_flag buffer 541, the PCM controller 542, the
deblocking filter 551, the pixel sorting unit 552, and the
filtering unit 553 illustrated in FIG. 16 may be configured as an
independent device.
[0420] Furthermore, these processing units may be arbitrarily
combined together to constitute an independent device. Of course,
these processing units may be combined with an arbitrary processing
unit illustrated in FIG. 15 and FIG. 16, or may be combined with a
processing unit which is not illustrated.
[0421] Also, for example, the above-described image encoding
apparatus and image decoding apparatus can be applied to an
arbitrary electronic apparatus. Hereinafter, examples thereof will
be described.
4. Fourth Embodiment
Television Receiver
[0422] FIG. 22 is a block diagram illustrating a main example
configuration of a television receiver that includes the image
decoding apparatus 500.
[0423] The television receiver 1000 illustrated in FIG. 22 includes
a terrestrial tuner 1013, a video decoder 1015, a video signal
processing circuit 1018, a graphic generation circuit 1019, a panel
drive circuit 1020, and a display panel 1021.
[0424] The terrestrial tuner 1013 receives a broadcast wave signal
of terrestrial analog broadcasting via an antenna, decodes the
signal, obtains a video signal, and supplies the video signal to be
video decoder 1015. The video decoder 1015 performs decoding
processing on the video signal supplied from the terrestrial tuner
1013, and supplies a digital component signal obtained thereby to
the video signal processing circuit 1018.
[0425] The video signal processing circuit 1018 performs certain
processing, such as noise reduction, on the video data supplied
from the video decoder 1015, and supplies the obtained video data
to the graphic generation circuit 1019.
[0426] The graphic generation circuit 1019 generates, for example,
video data of a program which is to be displayed on the display
panel 1021, or generates image data by performing processing based
on an application supplied via a network, and supplies the
generated video data or image data to the panel drive circuit 1020.
Also, the graphic generation circuit 1019 performs, as appropriate,
processing of generating video data (graphic) for displaying a
screen which is used by a user to select an item or the like, and
supplying video data which is obtained by superposing the generated
video data on video data of a program to the panel drive circuit
1020.
[0427] The panel drive circuit 1020 drives the display panel 1021
on the basis of the data supplied from the graphic generation
circuit 1019, and causes the display panel 1021 to display video of
a program or the above-described various screens.
[0428] The display panel 1021 is constituted by an LCD (Liquid
Crystal Display) or the like, and displays video of a program and
so forth in accordance with control performed by the panel drive
circuit 1020.
[0429] Also, the television receiver 1000 includes an audio A/D
(Analog/Digital) converter circuit 1014, an audio signal processing
circuit 1022, an echo cancellation/audio synthesis circuit 1023, an
audio amplifier circuit 1024, and a speaker 1025.
[0430] The terrestrial tuner 1013 demodulates a received broadcast
wave signal, and thereby obtains an audio signal as well as a video
signal. The terrestrial tuner 1013 supplies the obtained audio
signal to the audio A/D converter circuit 1014.
[0431] The audio A/D converter circuit 1014 performs A/D conversion
processing on the audio signal supplied from the terrestrial tuner
1013, and supplies a digital audio signal obtained thereby to the
audio signal processing circuit 1022.
[0432] The audio signal processing circuit 1022 performs certain
processing, such as noise reduction, on the audio data supplied
from the audio A/D converter circuit 1014, and supplies audio data
obtained thereby to the echo cancellation/audio synthesis circuit
1023.
[0433] The echo cancellation/audio synthesis circuit 1023 supplies
the audio data supplied from the audio signal processing circuit
1022 to the audio amplifier circuit 1024.
[0434] The audio amplifier circuit 1024 performs D/A conversion
processing and amplification processing on the audio data supplied
from the echo cancellation/audio synthesis circuit 1023, adjusts
the audio data so as to have a certain volume, and causes audio to
be output from the speaker 1025.
[0435] Furthermore, the television receiver 1000 includes a digital
tuner 1016 and an MPEG decoder 1017.
[0436] The digital tuner 1016 receives a broadcast wave signal of
digital broadcasting (terrestrial digital broadcasting, BS
(Broadcastings Satellite)/CS (Communications Satellite) digital
broadcasting) via an antenna, demodulates the signal, obtains an
MPEG-TS (Moving Picture Experts Group-Transport Stream), and
supplies it to the MPEG decoder 1017.
[0437] The MPEG decoder 1017 descrambles the MPEG-TS supplied from
the digital tuner 1016, and extracts a stream including the data of
a program as a target to be reproduced (target to be viewed and
listened to). The MPEG decoder 1017 decodes the audio packets
constituting the extracted stream and supplies the audio data
obtained thereby to the audio signal processing circuit 1022, and
also decodes the video packets constituting the stream and supplies
video data obtained thereby to the video signal processing circuit
1018. Also, the MPEG decoder 1017 supplies EPG (Electronic Program
Guide) data extracted from the MPEG-TS to a CPU 1032 via a path
which is not illustrated.
[0438] The television receiver 1000 includes the above-described
image decoding apparatus 500 serving as the MPEG decoder 1017 that
decodes video packets in this way. Note that the MPEG-TS
transmitted from a broadcast station or the like has been encoded
by the image encoding apparatus 300.
[0439] As in the case of the image decoding apparatus 500, the MPEG
decoder 1017 appropriately decodes encoded data for which the
selection of the I_PCM mode is controlled in units of CUs, which
are smaller than LCUs. Thus, the MPEG decoder 1017 is capable of
realizing a decrease in redundant processing for encoding and a
decrease in redundant information included in the encoded data.
Accordingly, the MPEG decoder 1017 is capable of realizing enhanced
encoding efficiency while suppressing a decrease in the efficiency
of encoding processing.
[0440] As in the case of the video data supplied from the video
decoder 1015, the video data supplied from the MPEG decoder 1017
undergoes certain processing in the video signal processing circuit
1018, video data or the like generated in the graphic generation
circuit 1019 is superposed thereon as appropriate, the video data
is supplied to the display panel 1021 via the panel drive circuit
1020, and the image thereof is displayed.
[0441] As in the case of the audio data supplied from the audio A/D
converter circuit 1014, the audio data supplied from the MPEG
decoder 1017 undergoes certain processing in the audio signal
processing circuit 1022, is supplied to the audio amplifier circuit
1024 via the echo cancellation/audio synthesis circuit 1023, and
undergoes D/A conversion processing and amplification processing.
As a result, audio adjusted to have a certain volume is output from
the speaker 1025.
[0442] Also, the television receiver 1000 includes a microphone
1026 and an A/D converter circuit 1027.
[0443] The A/D converter circuit 1027 receives a signal of user's
voice captured by the microphone 1026 that is provided in the
television receiver 1000 for voice conversation, performs A/D
conversion processing on the received audio signal, and supplies
obtained digital audio data to the echo cancellation/audio
synthesis circuit 1023.
[0444] In a case where audio data of a user (user A) of the
television receiver 1000 is supplied from the A/D converter circuit
1027, the echo cancellation/audio synthesis circuit 1023 performs
echo cancellation on the audio data of the user A, and causes audio
data which is obtained through synthesis with other audio data to
be output from the speaker 1025 via the audio amplifier circuit
1024.
[0445] Furthermore, the television receiver 1000 includes an audio
codec 1028, an internal bus 1029, an SDRAM (Synchronous Dynamic
Random Access Memory) 1030, a flash memory 1031, the CPU 1032, a
USB (Universal Serial Bus) I/F 1033, and a network I/F 1034.
[0446] The A/D converter circuit 1027 receives a signal of user's
voice captured by the microphone 1026 that is provided in the
television receiver 1000 for voice conversation, performs A/D
conversion processing on the received audio signal, and supplies
obtained digital audio data to the audio codec 1028.
[0447] The audio codec 1028 converts the audio data supplied from
the A/D converter circuit 1027 into data of a certain format for
transmitting it via a network, and supplies the audio data to the
network I/F 1034 via the internal bus 1029.
[0448] The network I/F 1034 is connected to a network via a cable
attached to a network terminal 1035. The network I/F 1034 transmits
audio data supplied from the audio codec 1028 to another apparatus
connected to the network, for example. Also, the network I/F 1034
receives, via the network terminal 1035, audio data transmitted
from another apparatus connected via the network, for example, and
supplies the audio data to the audio codec 1028 via the internal
bus 1029.
[0449] The audio codec 1028 converts the audio data supplied from
the network I/F 1034 into data of a certain format, and supplies
the data to the echo cancellation/audio synthesis circuit 1023.
[0450] The echo cancellation/audio synthesis circuit 1023 performs
echo cancellation on the audio data supplied from the audio codec
1023, and causes audio data obtained through synthesis with other
audio data to be output from the speaker 1025 via the audio
amplifier circuit 1024.
[0451] The SDRAM 1030 stores various pieces of data necessary for
the CPU 1032 to perform processing.
[0452] The flash memory 1031 stores a program executed by the CPU
1032. The program stored in the flash memory 1031 is read out by
the CPU 1032 at a certain timing, for example, at the start-up of
the television receiver 1000. The flash memory 1031 also stores EPG
data obtained via digital broadcasting and data obtained from a
certain server via a network.
[0453] For example, the flash memory 1031 stores an MPEG-TS
including content data obtained from a certain server via a network
under control performed by the CPU 1032. The flash memory 1031
supplies the MPEG-TS to the MPEG decoder 1017 via the internal bus
1029, for example, under control performed by the CPU 1032.
[0454] The MPEG decoder 1017 processes the MPEG-TS, as in the case
of the MPEG-TS supplied from the digital tuner 1016. In this way,
the television receiver 1000 is capable of receiving content data
of video, audio, or the like via a network, decoding the data using
the MPEG decoder 1017, and causing the video to be displayed or the
audio to be output.
[0455] Also, the television receiver 1000 includes a light receiver
1037 that receives an infrared signal transmitted from a remote
control 1051.
[0456] The light receiver 1037 receives an infrared ray from the
remote control 1051, and outputs a control code representing the
detail of a user operation obtained through demodulation to the CPU
1032.
[0457] The CPU 1032 executes a program stored in the flash memory
1031, and controls the entire operation of the television receiver
1000 in accordance with a control code or the like supplied from
the light receiver 1037. The CPU 1032 is connected to the
individual units of the television receiver 1000 via paths which
are not illustrated.
[0458] The USE I/F 1033 transmits data to/receives data from an
external apparatus of the television receiver 1000, the apparatus
being connected via a USB cable attached to a USB terminal 1036.
The network I/F 1034 connects to a network via a cable attached to
the network terminal 1035, and transmits/receives data other than
audio data to/from various apparatuses connected to the
network.
[0459] The television receiver 1000 includes the image decoding
apparatus 500 serving as the MPEG decoder 1017, thereby being
capable of realizing enhancement of encoding efficiency of content
data while suppressing a decrease in the efficiency of encoding
processing at the time of generating the content data, which is
obtained via a broadcast wave signal received via an antenna or a
network.
5. Fifth Embodiment
Mobile Phone
[0460] FIG. 23 is a block diagram illustrating a main example
configuration of a mobile phone that includes the image encoding
apparatus 300 and the image decoding apparatus 500.
[0461] The mobile phone 1100 illustrated in FIG. 23 includes a main
controller 1150 configured to collectively control individual
units, a power supply circuit unit 1151, an operation input
controller 1152, an image encoder 1153, a camera I/F unit 1154, an
LCD controller 1155, an image decoder 1156, a
multiplexer/demultiplexer unit 1157, a recording/reproducing unit
1162, a modulation/demodulation circuit unit 1158, and an audio
codec 1159. These are connected to one another via a bus 1160.
[0462] Also, the mobile phone 1100 includes an operation key 1119,
a CCD (Charge Coupled Devices) camera 1116, a liquid crystal
display 1118, a storage unit 1123, a transmission/reception circuit
unit 1163, an antenna 1114, a microphone 1121, and a speaker
1117.
[0463] When a call ends or a power key is turned on through a user
operation, the power supply circuit unit 1151 supplies power from a
battery pack to the individual units, thereby bringing the mobile
phone 1100 into an operable state.
[0464] The mobile phone 1100 performs various operations, such as
transmission/reception of an audio signal, transmission/reception
of an electronic mail or image data, image capturing, or data
recording, in various modes, such as an audio call mode or a data
communication mode, on the basis of control performed by the main
controller 1150, which includes a CPU, a ROM, a RAM, and so
forth.
[0465] For example, in the audio call mode, the mobile phone 1100
converts an audio signal collected by the microphone 1121 into
digital audio data using the audio codec 1159, performs spectrum
spread processing thereon using the modulation/demodulation circuit
unit 1158, and performs digital-to-analog conversion processing and
frequency conversion processing using the transmission/reception
circuit unit 1163. The mobile phone 1100 transmits a signal to be
transmitted obtained though the conversion processing to a base
station (not illustrated) via the antenna 1114. The signal to be
transmitted (audio signal) transmitted to the base station is
supplied to a mobile phone of the other end of a call via a public
telephone line network.
[0466] Also, for example, in the audio call mode, the mobile phone
1100 amplifies a reception signal received by the antenna 1114
using the transmission/reception circuit unit 1163, further
performs frequency conversion processing and analog-to-digital
conversion processing, performs spectrum inverse spread processing
using the modulation/demodulation circuit unit 1158, and converts
the signal into an analog audio signal using the audio codec 1159.
The mobile phone 1100 outputs the analog audio signal obtained
through the conversion from the speaker 1117.
[0467] Furthermore, for example, in the case of transmitting an
electronic mail in the data communication mode, the mobile phone
1100 accepts, in the operation input controller 1152, the text data
of the electronic mail input through an operation of the operation
key 1119. The mobile phone 1100 processes the text data using the
main controller 1150, and causes the text data to be displayed as
an image on the liquid crystal display 1118 via the LCD controller
1155.
[0468] Also, the mobile phone 1100 generates, in the main
controller 1150, electronic mail data on the basis of the text data
or a user instruction accepted by the operation input controller
1152. The mobile phone 1100 performs spectrum spread processing on
the electronic mail data using the modulation/demodulation circuit
unit 1158, and performs digital-to-analog conversion processing and
frequency conversion processing using the transmission/reception
circuit unit 1163. The mobile phone 1100 transmits a signal to be
transmitted obtained through the conversion processing to a base
station (not illustrated) via the antenna 1114. The signal to be
transmitted (electronic mail) transmitted to the base station is
supplied to a certain destination via a network and a mail server
or the like.
[0469] Also, for example, in the case of receiving an electronic
mail in the data communication mode, the mobile phone 1100 receives
a signal transmitted from the base station via the antenna 1114
using the transmission/reception circuit unit 1163, amplifies the
signal, and further performs frequency conversion processing and
analog-to-digital conversion processing. The mobile phone 1100
performs spectrum inverse spread processing on the received signal
using the modulation/demodulation circuit unit 1158 to restore
original electronic mail data. The mobile phone 1100 displays the
restored electronic mail data on the liquid crystal display 1118
via the LCD controller 1155.
[0470] Additionally, the mobile phone 1100 is also capable of
causing the received electronic mail data to be recorded (stored)
in the storage unit 1123 via the recording/reproducing unit
1162.
[0471] The storage unit 1123 is an arbitrary rewritable storage
medium. The storage unit 1123 may be, for example, a semiconductor
memory, such as a RAM or a built-in flash memory, a hard disk, or a
removable medium, such as a magnetic disk, a magneto-optical disc,
an optical disc, a USB memory, or a memory card. Of course, other
types of media may be used.
[0472] Furthermore, for example, in the case of transmitting image
data in the data communication mode, the mobile phone 1100
generates image data through image capturing using the CCD camera
1116. The CCD camera 1116 includes optical devices, such as a lens
and a diaphragm, and a CCD serving as a photoelectric conversion
element, captures an image of a subject, converts the intensity of
received light into an electric signal, and generates image data of
the image of the subject. The CCD camera 1116 encodes the image
data using the image encoder 1153 via the camera I/F unit 1154,
thereby converting the image data into encoded image data.
[0473] The mobile phone 1100 includes the above-described image
encoding apparatus 300 serving as the image encoder 1153 that
performs the above-described processing. As in the case of the
image encoding apparatus 300, the image encoder 1153 controls the
selection of the I_PCM mode in units of CUs, which are smaller than
LCUs. That is, the image encoder 1153 is capable of further
reducing redundant processing for encoding, and is also capable of
further reducing redundant information included in encoded data.
Accordingly, the image encoder 1153 is capable of enhancing
encoding efficiency while suppressing a decrease in the efficiency
of encoding processing.
[0474] In addition, at the same time, the mobile phone 1100
performs, in the audio codec 1159, analog-to-digital conversion on
audio collected by the microphone 1121 during image capturing by
the CCD camera 1116, and also encodes it.
[0475] The mobile phone 1100 multiplexes, in the
multiplexer/demultiplexer unit 1157, the encoded image data
supplied from the image encoder 1153 and the digital audio data
supplied from the audio codec 1159 using a certain method. The
mobile phone 1100 performs spectrum spread processing on the
multiplexed data obtained as a result using the
modulation/demodulation circuit unit 1158, and performs
digital-to-analog conversion processing and frequency conversion
processing using the transmission/reception circuit unit 1163. The
mobile phone 1100 transmits a signal to be transmitted obtained
through the conversion processing to a base station (not
illustrated) via the antenna 1114. The signal to be transmitted
(image data) which has been transmitted to the base station is
supplied to the other end of communication via a network or the
like.
[0476] Note that, in the case of not transmitting image data, the
mobile phone 1100 is capable of causing the image data generated by
the CCD camera 1116 to be displayed on the liquid crystal display
1118 via the LCD controller 1155, not via the image encoder
1153.
[0477] Also, for example, in the case of receiving data of a moving
image file that is linked to a simple web page or the like in the
data communication mode, the mobile phone 1100 receives a signal
transmitted from a base station via the antenna 1114 using the
transmission/reception circuit unit 1163, amplifies the signal, and
further performs frequency conversion processing and
analog-to-digital conversion processing thereon. The mobile phone
1100 performs spectrum inverse spread processing on the received
signal to restore original multiplexed data using the
modulation/demodulation circuit unit 1158. The mobile phone 1100
demultiplexes the multiplexed data into encoded image data and
audio data using the multiplexer/demuitiplexer unit 1157.
[0478] The mobile phone 1100 decodes the encoded image data using
the image decoder 1156 to generate reproduced moving image data,
and causes the data to be displayed on the liquid crystal display
1118 via the LCD controller 1155. Accordingly, for example, the
moving image data included in the moving image file linked to the
simple web page is displayed on the liquid crystal display
1118.
[0479] The mobile phone 1100 includes the above-described image
decoding apparatus 500 serving as the image decoder 1156 for
performing such processing. That is, as in the case of the image
decoding apparatus 500, the image decoder 1156 appropriately
decodes encoded data for which the selection of the I_PCM mode is
controlled in units of CUs, which are smaller than LCUs. Thus, the
image decoder 1156 is capable of realizing a decrease in redundant
processing for encoding and a decrease in redundant information
included in the encoded data. Accordingly, the image decoder 1156
is capable of realizing enhanced encoding efficiency while
suppressing a decrease in the efficiency of encoding
processing.
[0480] At this time, the mobile phone 1100 converts digital audio
data into an analog audio signal using the audio codec 1159, and
causes the signal to be output from the speaker 1117. Accordingly,
for example, audio data included the moving image file linked to
the simple web page is reproduced.
[0481] Note that, as in the case of an electronic mail, the mobile
phone 1100 is also capable of causing the received data linked to
the simple web page or the like to be recorded (stored) in the
storage unit 1123 via the recording/reproducing unit 1162.
[0482] Also, the mobile phone 1100 is capable of analyzing a
two-dimensional code obtained by the CCD camera 1116 through image
capturing and obtaining information recorded in the two-dimensional
code using the main controller 1150.
[0483] Furthermore, the mobile phone 1100 is capable of
communicating with an external apparatus through an infrared ray
using an infrared communication unit 1181.
[0484] By including the image encoding device 300 serving as the
image encoder 1153, the mobile phone 1100 is capable of enhancing
encoding efficiency while suppressing a decrease in the efficiency
of encoding processing, for example, when encoding and transmitting
image data generated by the CCD camera 1116.
[0485] Also, by including the image decoding apparatus 500 serving
as the image decoder 1156, the mobile phone 1100 is capable of
realizing enhanced encoding efficiency of data while suppressing a
decrease in the efficiency of encoding processing at the time of
generating the data (encoded data) of a moving image file linked to
a simple web page or the like.
[0486] Note that, although a description has been given above that
the mobile phone 1100 includes the CCD camera 1116, an image sensor
using a CMOS (Complementary Metal Oxide Semiconductor) (CMOS image
sensor) may be used instead of the CCD camera 1116. In this case,
too, the mobile phone 1100 is capable of capturing an image of a
subject and generating image data of the image of the subject, as
in the case of using the CCD camera 1116.
[0487] Also, although a description has been given above of the
mobile phone 1100, the image encoding apparatus 300 and the image
decoding apparatus 500 can be applied to any apparatus having an
image capturing function and a communication function similar to
those of the mobile phone 1100, such as a PDA (Personal Digital.
Assistants), a smart phone, a UMPC (Ultra Mobile Personal
Computer), a net book, or a notebook personal computer, as in the
case of the mobile phone 1100.
6. Sixth Embodiment
Hard Disk Recorder
[0488] FIG. 24 is a block diagram illustrating a main example
configuration of a hard disk recorder that includes the image
encoding apparatus 300 and the image decoding apparatus 500.
[0489] The hard disk recorder (HDD recorder) 1200 illustrated in
FIG. 24 is an apparatus that stores, in a hard disk built therein,
audio data and video data of a broadcast program included in a
broadcast wave signal (television signal) that is transmitted by a
satellite, an antenna on the ground, or the like and that is
received by a tuner, and that provides the stored data to a user at
a timing corresponding to an instruction provided by the user.
[0490] The hard disk recorder 1200 is capable of, for example,
extracting audio data and video data from a broadcast wave signal,
appropriately decoding them, and causing them to be stored in the
hard disk built therein. Also, the hard disk recorder 1200 is
capable of, for example, obtaining audio data and video data from
another apparatus via a network, appropriately decoding them, and
causing them to be stored in the hard disk built therein.
[0491] Furthermore, the hard disk recorder 1200 is capable of, for
example, decoding audio data and video data recorded on the hard
disk built therein, supplying them to a monitor 1260, causing the
image thereof to be displayed on the screen of the monitor 1260,
and causing the audio thereof to be output from the speaker of the
monitor 1260. Also, the hard disk recorder 1200 is capable of, for
example, decoding audio data and video data extracted from a
broadcast wave signal obtained via a tuner or audio data and video
data obtained from another apparatus via a network, supplying them
to the monitor 1260, causing the image thereof to be displayed on
the screen of the monitor 1260, and causing the audio thereof to be
output from the speaker of the monitor 1260.
[0492] Of course, another operation can be performed.
[0493] As illustrated in FIG. 24, the hard disk recorder 1200
includes a receiving unit 1221, a demodulating unit 1222, a
demultiplexer 1223, an audio decode 1224, a video decoder 1225, and
a recorder controller 1226. The hard disk recorder 1200 further
includes an EPG data memory 1227, a program memory 1228, a working
memory 1229, a display converter 1230, an OSD (On Screen Display)
controller 1231, a display controller 1232, a recording/reproducing
unit 1233, a D/A converter 1234, and a communication unit 1235.
[0494] Also, the display converter 1230 includes a video encoder
1241. The recording/reproducing unit 1233 includes an encoder 1251
and a decoder 1252.
[0495] The receiving unit 1221 receives an infrared signal from a
remote control (not illustrated), converts the signal into an
electric signal, and outputs the electric signal to the recorder
controller 1226. The recorder controller 1226 is constituted by,
for example, a microprocessor or the like, and executes various
processing operations in accordance with a program stored in the
program memory 1228. At this time, the recorder controller 1226
uses the working memory 1229 as necessary.
[0496] The communication unit 1235 is connected to a network, and
performs communication processing with another apparatus via the
network. For example, the communication unit 1235 is controlled by
the recorder controller 1226, communicates with a tuner (not
illustrated), and outputs a channel selection control signal mainly
to the tuner.
[0497] The demodulating unit 1222 demodulates the signal supplied
from the tuner and outputs the signal to the demultiplexer 1223.
The demultiplexer 1223 demultiplexes the data supplied from the
demodulating unit 1222 into audio data, video data, and EPG data,
and outputs them to the audio decoder 1224, the video decoder 1225,
and the recorder controller 1226, respectively.
[0498] The audio decoder 1224 decodes the audio data input thereto,
and outputs the audio data to the recording/reproducing unit 1233.
The video decoder 1225 decodes the video data input thereto, and
outputs the video data to the display converter 1230. The recorder
controller 1226 sup-lies the EPG data input thereto to the EPG data
memory 1227 so as to store it therein.
[0499] The display converter 1230 encodes, with the video encoder
1241, the video data supplied from the video decoder 1225 or the
recorder controller 1226 into video data of an NTSC (National
Television Standards Committee) format, for example, and outputs
the video data to the recording/reproducing unit 1233. Also, the
display converter 1230 converts the size of the screen of the video
data supplied from the video decoder 1225 or the recorder
controller 1226 into the size corresponding to the size of the
monitor 1260, converts the video data into video data of the NTSC
format with the video encoder 1241, converts the video data into an
analog signal, and outputs the analog signal to the display
controller 1232.
[0500] The display controller 1232 superposes the OSD signal output
from the OSD (On Screen Display) controller 1231 on the video
signal input from the display converter 1230 under control
performed by the recorder controller 1226, outputs it to the
display of the monitor 1260, and causes it to be displayed
thereon.
[0501] Also, the monitor 1260 is supplied with the audio data which
has been output from the audio decoder 1224 and which has been
converted into an analog signal by the D/A converter 1234. The
monitor 1260 outputs this audio signal from the speaker built
therein.
[0502] The recording/reproducing unit 1233 includes a hard disk
serving as a storage medium for holding video data, audio data, and
the like recorded thereon.
[0503] The recording/reproducing unit 1233 encodes, with the
encoder 1251, the audio data supplied from the audio decoder 1224,
for example. Also, the recording/reproducing unit 1233 encodes,
with the encoder 1251, the video data supplied from the video
encoder 1241 of the display converter 1230.
[0504] The recording/reproducing unit 1233 combines, with a
multiplexer, the encoded data of the audio data and the encoded
data of the video data. The recording/reproducing unit 1233
performs channel coding on the composite data to amplify it, and
writes the data on the hard disk via a recording head.
[0505] The recording/reproducing unit 1233 reproduces the data
recorded on the hard disk via a reproducing head, amplifies the
data, and demultiplexes the data into audio data and video data
using a demultiplexer. The recording/reproducing unit 1233 decodes,
with the decoder 1252, the audio data and the video data. The
recording/reproducing unit 1233 DA-converts the decoded audio data
and outputs the audio data to the speaker of the monitor 1260.
Also, the recording/reproducing unit 1233 D/A-converts the decoded
video data and outputs the video data to the display of the monitor
1260.
[0506] The recorder controller 1226 reads out the latest EPG data
from the EPG data memory 1227 on the basis of a user instruction
represented by an infrared signal which is supplied from the remote
control and which is received via the receiving unit 1221, and
supplies the EPG data to the OSD controller 1231. The OSD
controller 1231 generates image data corresponding to the input EPG
data, and outputs the image data to the display controller 1232.
The display controller 1232 outputs the video data input from the
OSD controller 1231 to the display of the monitor 1260, and causes
the video data to be displayed thereon. Accordingly, an EPG
(electronic program guide) is displayed on the display of the
monitor 1260.
[0507] Also, the hard disk recorder 1200 is capable of obtaining
various pieces of data, such as video data, audio data, or EPG
data, supplied from another apparatus via a network, such as the
Internet.
[0508] The communication unit 1235 is controlled by the recorder
controller 1226, obtains encoded data of video data, audio data,
EPG data, and so forth transmitted from another apparatus via a
network, and supplies the encoded data to the recorder controller
1226. For example, the recorder controller 1226 supplies the
obtained encoded data of video data and audio data to the
recording/reproducing unit 1233, and causes the hard disk to store
the encoded data. At this time, the recorder controller 1226 and
the recording/reproducing unit 1233 may perform processing, such as
re-encoding, as necessary.
[0509] Also, the recorder controller 1226 decodes the obtained
encoded data of video data and audio data, and supplies the
obtained video data to the display converter 1230. The display
converter 1230 processes the video data supplied from the recorder
controller 1226, like the video data supplied from the video
decoder 1225, supplies the video data to the monitor 1260 via the
display controller 1232, and causes the image thereof to be
displayed thereon.
[0510] Also, in accordance with display of the image, the recorder
controller 1226 may supply decoded audio data to the monitor 1260
via the D/A converter 1234 and cause the audio to be output from
the speaker.
[0511] Furthermore, the recorder controller 1226 decodes the
obtained encoded data of the EPG data, and supplies the decoded EPG
data to the EPG data memory 1227.
[0512] The hard disk recorder 1200 described above includes the
image decoding apparatus 500 serving as the video decoder 1225, the
decoder 1252, and the decoder included in the recorder controller
1226. That is, the video decoder 1225, the decoder 1252, and the
decoder included in the recorder controller 1226 appropriately
decodes encoded data for which the selection of the I_PCM mode is
controlled in units of CUs, which are smaller than LCUs, as in the
case of the image decoding apparatus 500. Accordingly, the video
decoder 1225, the decoder 1252, and the decoder included in the
recorder controller 1226 is capable of realizing a decrease in
redundant processing for encoding and a decrease in redundant
information included in the encoded data. Accordingly, the video
decoder 1225, the decoder 1252, and the decoder included in the
recorder controller 1226 are capable of realizing enhanced encoding
efficiency while suppressing a decrease in the efficiency of
encoding processing.
[0513] Therefore, the hard disk recorder 1200 is capable of
realizing enhanced encoding efficiency of data while suppressing a
decrease in the efficiency of encoding processing when generating
video data (encoded data) received by the tuner or the
communication unit 1235 or video data (encoded data) reproduced by
the recording/reproducing unit 1233.
[0514] Also, the hard disk recorder 1200 includes the image
encoding apparatus 300 serving as the encoder 1251. Thus, the
encoder 1251 controls the selection of the I_PCM mode in units of
CUs, which are smaller than LCUs, as in the case of the image
encoding apparatus 300. That is, the encoder 1251 is capable of
further reducing redundant processing for encoding and is also
capable of reducing redundant information included in encoded data.
Accordingly, the encoder 1251 is capable of enhancing encoding
efficiency while suppressing a decrease in the efficiency of
encoding processing.
[0515] Therefore, the hard disk recorder 1200 is capable of
enhancing encoding efficiency while suppressing a decrease in the
efficiency of encoding processing, for example, when generating
encoded data to be recorded on a hard disk.
[0516] Note that, although a description has been given above of
the hard disk recorder 1200 that records video data and audio data
on a hard disk, any types of recording media may of course be used.
For example, the image encoding apparatus 300 and the image
decoding apparatus 500 can be applied to a recorder that uses a
recording medium other than a hard disk, for example, a flash
memory, an optical disc, or video tape, as in the case of the
above-described hard disk recorder 1200.
7. Seventh Embodiment
Camera
[0517] FIG. 25 is a block diagram illustrating a main example
configuration of a camera that includes the image encoding
apparatus 300 and the image decoding apparatus 500.
[0518] The camera 1300 illustrated in FIG. 25 captures an image of
a subject, causes an LCD 1316 to display the image of the subject,
and records it as image data on a recording medium 1333.
[0519] A lens block 1311 causes light (that is, an image of a
subject) to enter a CCD/CMOS 1312. The CCD/CMOS 1312 is an image
sensor including a CCD or CMOS, converts the intensity of received
light into an electric signal, and supplies the electric signal to
a camera signal processor 1313.
[0520] The camera signal processor 1313 converts the electric
signal supplied from the CCD/CMOS 1312 in to color-difference
signals of Y, Cr, and Cb, and supplies them to an image signal
processor 1314. The image signal processor 1314 performs certain
image processing on an image signal supplied from the camera signal
processor 1313 and encodes, with an encoder 1341, the image signal
under control performed by a controller 1321. The image signal
processor 1314 supplies encoded data which is generated by encoding
the image signal to a decoder 1315. Furthermore, the image signal
processor 1314 obtains data to be displayed generated by an on
screen display (OSD) 1320, and supplies the data to the decoder
1315.
[0521] In the foregoing process, the camera signal processor 1313
uses a DRAM (Dynamic Random Access Memory) 1318 connected via a bus
1317 as appropriate, and causes the DRAM 1318 to hold image data,
encoded data obtained by encoding the image data, or the like as
necessary.
[0522] The decoder 1315 decodes encoded data supplied from the
image signal processor 1314, and supplies the image data obtained
thereby (decoded image data) to the LCD 1316. Also, the decoder
1315 supplies the data to be displayed supplied from the image
signal processor 1314 to the LCD 1316. The LCD 1316 combines the
image of the decoded image data supplied from the decoder 1315 and
the image of the data to be displayed as appropriate, and displays
the composite image.
[0523] The on screen display 1320 outputs data to be displayed,
such as a menu screen made up of symbols, characters, or figures,
and icons, to the image signal processor 1314 via the bus 1317
under control performed by the controller 1321.
[0524] The controller 321 executes various processing operations on
the basis of a signal representing the detail of an instruction
provided from a user using an operation unit 1322, and controls the
image signal processor 1314, the DRAM 1318, an external interface
1319, the on screen display 1320, a medium drive 1323, and so forth
via the bus 1317. Programs, data, and so forth which are necessary
for the controller 1321 to execute various processing operations
are stored in a flash ROM 1324.
[0525] For example, the controller 1321 is capable of encoding
image data stored in the DRAM 1318 and decoding encoded data stored
in the DRAM 1318 on behalf of the image signal processor 1314 or
the decoder 1315. At this time, the controller 1321 may perform
encoding/decoding processing using a format similar to an
encoding/decoding format of the image signal processor 1314 or the
decoder 1315, or may perform encoding/decoding processing using a
format incompatible with the image signal processor 1314 or the
decoder 1315.
[0526] Also, for example, in a case where an instruction to start
printing an image is provided from the operation unit 1322, the
controller 1321 reads out image data from the DRAM 1318 and
supplies it to a printer 1334 connected to the external interface
1319 via the bus 1317 to print it.
[0527] Furthermore, for example, in a case where an instruction to
record an image is provided from the operation unit 1322, the
controller 1321 reads out encoded data from the DRAM 1313 and
supplies it to the recording medium 1333 loaded in the medium drive
1323 via the bus 1317 to store it.
[0528] The recording medium 1333 is, for example, an arbitrary
readable and writable removable medium, such as a magnetic disk, a
magneto-optical disc, an optical disc, or a semiconductor memory.
Of course, the recording medium 1333 may be a removable medium of
any type, and may be a tape device, disc, or memory card. Of
course, the recording medium 1333 may be a noncontact IC card or
the like.
[0529] Also, the medium drive 1323 and the recording medium 1333
may be integrated together, and may be constituted by, for example,
a non-trans-portable storage medium, such as a built-in hard disk
drive or an SS (Solid State Drive).
[0530] The external interface 1319 is constituted by, for example,
a USB input/output terminal or the like, and is connected to the
printer 1334 in the case of printing an image. Also, a drive 1331
is connected to the external interface 1319 as necessary, a
removable medium 1332, such as a magnetic disk, an optical disc, or
a magneto-optical disc, is loaded thereto as appropriate, and a
computer program read out therefrom is installed into the flash ROM
1324 as necessary.
[0531] Furthermore, the external interface 1319 includes a network
interface connected to a certain network, such as a LAN or the
Internet. The controller 1321 is capable of, for example, reading
out encoded data from the DRAM 1318 and supplying it from the
external interface 1319 to another apparatus connected via a
network, in accordance with an instruction provided from the
operation unit 1322. Also, the controller 1321 is capable of
obtaining, via the external interface 1319, encoded data or image
data supplied from another apparatus via a network, and causing the
DRAM 1318 to hold it or supplying it to the image signal processor
1314.
[0532] The camera 1300 described above includes the image decoding
apparatus 500 serving as the decoder 1315. That is, the decoder
1315 appropriately decodes encoded data for which the selection of
the I_PCM mode is controller in units of CUs, which are smaller
than LCUs, as in the case of the image decoding apparatus 500.
Accordingly, the decoder 1315 is capable of realizing a decrease in
redundant processing for encoding and a decrease in redundant
information included in encoded data. Accordingly, the decoder 1315
is capable of realizing enhanced encoding efficiency while
suppressing a decrease in the efficiency of encoding
processing.
[0533] Therefore, the camera 1300 is capable of, for example,
realizing enhanced encoding efficiency of data while suppressing a
decrease in the efficiency of encoding processing when generating
image data generated by the CCD/CMOS 1312, encoded data of video
data read out from the DRAM 1318 or the recording medium 1333, or
encoded data of video data obtained via a network.
[0534] Also, the camera 1300 includes the image encoding apparatus
300 serving as the encoder 1341. The encoder 1341 controls the
selection of the I_PCM mode in units of CUs, which are smaller than
LCUs, as in the case of the image encoding apparatus 300. That is,
the encoder 341 is capable of further reducing redundant processing
for encoding and is also capable of further reducing redundant
information included in encoded data. Accordingly, the encoder 1341
is capable of enhancing encoding efficiency while suppressing a
decrease in the efficiency of encoding processing.
[0535] Therefore, the camera 1300 is capable of enhancing encoding
efficiency while suppressing a decrease in the efficiency of
encoding processing, for example, when generating encoded data to
be recorded on the DRAM 1318 or the recording medium 1333 or
encoded data to be provided to another apparatus.
[0536] In addition, the decoding method of the image decoding
apparatus 500 may be applied to decoding processing performed by
the controller 1321. Likewise, the encoding method of the image
encoding apparatus 300 may be applied to encoding processing
performed by the controller 1321.
[0537] Also, the image data captured by the camera 1300 may be a
moving image or a still image.
[0538] Of course, the image encoding apparatus 300 and the image
decoding apparatus 500 can be applied to an apparatus or a system
other than the above-described apparatuses.
[0539] The present technology can be applied to an image encoding
apparatus and an image decoding apparatus which are used to, for
example, receive image information (bit stream) compressed through
an orthogonal transform such as the discrete cosine transform and
by motion compensation via a network medium such as satellite
broadcasting, cable TV, the Internet, or a mobile phone, or to
process the image information on a storage medium such as an
optical or magnetic disk or a flash memory, as in MPEG, H.26x, or
the like.
[0540] In addition, the present technology may also provide the
following configurations.
[0541] (1) An image processing apparatus including:
[0542] an encoding mode setter that sets, in units of coding units
having a hierarchical structure, whether a non-compression mode is
to be selected as an encoding mode for encoding image data, the
non-compression mode being an encoding mode in which the image data
is output as encoded data; and
[0543] an encoder that encodes the image data in units of the
coding units in accordance with a mode set by the encoding mode
setter.
[0544] (2) The image processing apparatus according to (1), further
including:
[0545] a shift processing controller that performs control, on a
coding unit for which the non-compression mode has been set by the
encoding mode setter, to skip shift processing in which a bit
precision for encoding or decoding is increased; and
[0546] a shift processor that performs the shift processing on a
coding unit of the image data, the coding unit being controlled by
the shift processing controller so as to undergo the shift
processing.
[0547] (3) The image processing apparatus according to (1) or (2),
further including:
[0548] a filter processing controller that performs control, on a
coding unit for which the non-compression mode has been set by the
encoding mode setter, to skip filter processing in which filtering
is performed on a locally decoded image;
[0549] a filter coefficient calculator that calculates a filter
coefficient for the filter processing by using image data
corresponding to a coding unit which is controlled by the filter
processing controller so as to undergo the filter processing;
and
[0550] a filter processor that performs the filter processing in
units of blocks, which are units of the filter processing, by using
the filter coefficient calculated by the filter coefficient
calculator.
[0551] (4) The image processing apparatus according to (3), wherein
the filter processor performs the filter processing on only pixels
which are controlled by the filter processing controller so as to
undergo the filter processing, the pixels being included in a
current block which is a target to be processed.
[0552] (5) The image processing apparatus according to (3) or (4),
further including:
[0553] a filter identification information generator that generates
filter identification information in units of the blocks, the
filter identification information being flag information indicating
whether the filter processing is to be performed.
[0554] (6) The image processing apparatus according to any of (3)
to (5), wherein the filter processor performs adaptive loop
filtering on the locally decoded image, the adaptive loop filtering
being adaptive filter processing using classification
processing.
[0555] (7) The image processing apparatus according to any of (1)
to (6), wherein, in a case where an amount of code of encoded data,
which is obtained by encoding the image data corresponding to a
current coding unit as a target of encoding processing, is smaller
than or equal to an amount of input data, which is a data amount of
the image data corresponding to the current coding unit, the
encoding mode setter sets an encoding mode of the current coding
unit to be the non-compression mode.
[0556] (8) The image processing apparatus according to (7), further
including:
[0557] an input data amount calculator that calculates the amount
of input data,
[0558] wherein the encoding mode setter compares, regarding the
current coding unit, the amount of input data calculated by the
input data amount calculator with the amount of code.
[0559] (9) The image processing apparatus according to any of (1)
to (8), further including:
[0560] an identification information generator that generates
identification information in units of the coding units, the
identification information indicating whether the non-compression
mode has been set by the encoding mode setter.
[0561] (10) An image processing method for an image processing
apparatus, including:
[0562] setting, with an encoding mode setter, in units of coding
units having a hierarchical structure, whether a non-compression
mode is to be selected as an encoding mode for encoding image data,
the non-compression mode being an encoding mode in which the image
data is output as encoded data; and
[0563] encoding, with an encoder, the image data in units of the
coding units in accordance with a set mode.
[0564] (11) An image processing apparatus including:
[0565] an encoding mode determiner that determines, in units of
coding units having a hierarchical structure, whether a
non-compression mode has been selected as an encoding mode for
encoding image data, the non-compression mode being an encoding
mode in which the image data is output as encoded data; and
[0566] a decoder that decodes the encoded data in units of the
coding units in accordance with a mode determined by the encoding
mode determiner.
[0567] (12) The image processing apparatus according to (11),
further including:
[0568] a shift processing controller that performs control, on a
coding unit for which the encoding mode determiner has determined
that the non-compression mode has been selected, to skip shift
processing in which a bit precision for encoding or decoding is
increased; and
[0569] a shift processor that performs the shift processing on a
coding unit of the image data, the coding unit being controlled by
the shift processing controller so as to undergo the shift
processing.
[0570] (13) The image processing apparatus according to (11) or
(12), further including:
[0571] a filter processing controller that performs control, on a
coding unit for which the encoding mode determiner has determined
that the non-compression mode has been selected, to skip filter
processing in which filtering is performed on a locally decoded
image; and
[0572] a filter processor that performs the filter processing on
the image data in units of blocks, which are units of the filter
processing,
[0573] wherein the filter processor performs the filter processing
on only pixels which have been controlled by the filter processing
controller so as to undergo the filter processing, the pixels being
included in a current block which is a target to be processed.
[0574] (14) The image processing apparatus according to (13),
wherein the filter processor performs adaptive loop filtering on
the locally decoded image, the adaptive loop filtering being
adaptive filter processing using classification processing.
[0575] (15) The image processing apparatus according to (13) or
(14), wherein the filter processor performs the filter processing,
in a case where filter identification information indicating
whether or not the filter processing has been performed indicates
that the filter processing has been performed on image data
corresponding to the current block which is a target to be
processed, only when control is performed by the filter processing
controller so as to perform the filter processing on all pixels
included in the current block.
[0576] (16) The image processing apparatus according to any of (11)
to (15), wherein the encoding mode determiner determines whether
the non-compression mode has been selected, on the basis of
identification information indicating whether the non-compression
mode has been selected in units of the coding units.
[0577] (17) An image processing method for an image processing
apparatus, including:
[0578] determining, with an encoding mode determiner, in units of
coding units having a hierarchical structure, whether a
non-compression mode has been selected as an encoding mode for
encoding image data, the non-compression mode being an encoding
mode in which the image data is output as encoded data; and
[0579] decoding, with a decoder, the encoded data in units of the
coding units in accordance with a determined mode.
REFERENCE SIGNS LIST
[0580] 300 image encoding apparatus [0581] 303 adaptive
shift-to-left unit [0582] 307 lossless encoder [0583] 312 loop
filter [0584] 313 adaptive shift-to-right unit [0585] 315 adaptive
shift-to-left unit [0586] 320 rate controller [0587] 321 PCM
encoder [0588] 331 NAL encoder [0589] 332 CU encoder [0590] 341
I_PCM_flag generator [0591] 342 PCM deciding unit [0592] 351
deblocking filter [0593] 352 pixel sorting unit [0594] 353 filter
coefficient calculator [0595] 354 filtering unit [0596] 361 input
data amount calculator [0597] 362 PCM determining unit [0598] 363
encoding controller [0599] 364 adaptive shift controller [0600] 365
filter contoller [0601] 500 image decoding apparatus [0602] 502
lossless decoder [0603] 506 loop filter [0604] 507 adaptive
shift-to-right unit [0605] 511 adaptive shift-to-left unit [0606]
516 PCM decoder [0607] 531 NAL decoder [0608] 532 CU decoder [0609]
541 I_PCM_flag buffer [0610] 542 PCM controller [0611] 551
deblocking filter [0612] 552 pixel sorting unit [0613] 553
filtering unit
* * * * *
References