U.S. patent application number 15/351366 was filed with the patent office on 2017-03-02 for methods and systems for inter-layer image prediction signaling.
The applicant listed for this patent is Sharp Laboratories of America, Inc.. Invention is credited to Christopher A. Segall.
Application Number | 20170064306 15/351366 |
Document ID | / |
Family ID | 39641283 |
Filed Date | 2017-03-02 |
United States Patent
Application |
20170064306 |
Kind Code |
A1 |
Segall; Christopher A. |
March 2, 2017 |
Methods and Systems for Inter-Layer Image Prediction Signaling
Abstract
Embodiments of the present invention comprise systems and
methods for predicting high dynamic range (HDR) image blocks with
block-specific prediction data, where the systems and methods may
comprise low dynamic range (LDR) image data and HDR image data for
a target image block, where a scaled, offset LDR image block may be
combined with HDR residual image block to form an HDR image block
corresponding to the target image block.
Inventors: |
Segall; Christopher A.;
(Vancouver, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Sharp Laboratories of America, Inc. |
Camas |
WA |
US |
|
|
Family ID: |
39641283 |
Appl. No.: |
15/351366 |
Filed: |
November 14, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14605924 |
Jan 26, 2015 |
9497387 |
|
|
15351366 |
|
|
|
|
14161449 |
Jan 22, 2014 |
8953677 |
|
|
14605924 |
|
|
|
|
11626368 |
Jan 23, 2007 |
8665942 |
|
|
14161449 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 19/61 20141101;
G06T 5/50 20130101; H04N 19/134 20141101; H04N 19/46 20141101; H04N
19/36 20141101; H04N 19/187 20141101; G06T 5/009 20130101; H04N
5/2355 20130101; H04N 19/184 20141101; G06T 5/008 20130101; H04N
19/105 20141101; H04N 19/137 20141101; H04N 19/503 20141101; H04N
19/176 20141101; H04N 19/186 20141101; G06T 2207/20208 20130101;
H04N 19/44 20141101 |
International
Class: |
H04N 19/137 20060101
H04N019/137; H04N 19/186 20060101 H04N019/186; H04N 19/61 20060101
H04N019/61; H04N 19/36 20060101 H04N019/36; H04N 19/184 20060101
H04N019/184; H04N 19/176 20060101 H04N019/176; H04N 19/44 20060101
H04N019/44 |
Claims
1. A method for predicting a high dynamic range image block with
differentially-coded prediction data, the method comprising:
receiving high dynamic range (HDR) image data for a first image
block, the first image block HDR image data comprising a first
prediction data for the first image block; receiving high dynamic
range (HDR) image data for a second image block, the second image
block HDR image data comprising a prediction difference data
related to the first prediction data; combining the first
prediction data and the prediction difference data to determine
second prediction data for the second image block; and determining
a first HDR residual image block based on the received HDR image
data for the first image block and a decoded low dynamic range
(LDR) image, wherein the LDR image is a scaled decoded LDR image,
and wherein the scaled decoded LDR image is an offset scaled
decoded LDR image.
Description
CROSS-REFERENCE To RELA FED APPLICATIONS
[0001] This application is a divisional of U.S. Allowed patent
application Ser. No. 14/605,924, filed Jan. 26, 2015, which is a
continuation of U.S. patent application Ser. No. 14/161,449, filed
Jan. 22, 2014, which is now U.S. Pat. No. 8,953,677 issued Feb. 10,
2015, which is a divisional of U.S. patent application Ser. No.
11/626,368, filed Jan. 23, 2007, which is now U.S. Pat. No.
8,665,942 which issued on Mar. 4, 2014, the disclosures of which
are hereby incorporated by reference in their entirety.
FIELD OF THE INVENTION
[0002] Embodiments of the present invention comprise methods and
systems for inter-layer image prediction signaling.
SUMMARY
[0003] Some embodiments of the present invention comprise methods
and systems for prediction of images comprising multiple dynamic
range layers. Some embodiments comprise methods and systems for
communicating prediction variables between an encoder and a decoder
or transcoder.
[0004] The foregoing and other objectives, features, and advantages
of the invention will be more readily understood upon consideration
of the following detailed description of the invention taken in
conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE SEVERAL DRAWINGS
[0005] FIG. 1A is a chart showing an exemplary embodiment of the
present invention comprising prediction with a scaled and offset
LDR image element;
[0006] FIG. 1B is a chart showing an exemplary embodiment of the
present invention comprising scaling and offsetting decoded image
elements for HDR prediction;
[0007] FIG. 2 is a chart showing an exemplary embodiment of the
present invention comprising conversion to an alternative color
space;
[0008] FIG. 3 is a chart showing an exemplary embodiment of the
present invention comprising scaling an LDR image element according
to HDR bitstream data;
[0009] FIG. 4 is a chart showing an exemplary embodiment of the
present invention comprising scaling and applying an offset to an
LDR image element according to HDR bitstream data;
[0010] FIG. 5 is a chart showing an exemplary embodiment of the
present invention comprising scaling LDR transform coefficients for
HDR prediction;
[0011] FIG. 6 is a chart showing an exemplary embodiment of the
present invention comprising applying an offset to LDR transform
coefficients for HDR prediction;
[0012] FIG. 7 is a chart showing an exemplary embodiment of the
present invention comprising scaling LDR transform coefficients and
applying an offset to LDR transform coefficients for HDR
prediction;
[0013] FIG. 8 is a chart showing an exemplary embodiment of the
present invention comprising scaling and applying an offset to
color-transformed image elements for HDR prediction; and
[0014] FIG. 9 is a chart showing an exemplary embodiment of the
present invention comprising separate scaling and offset operations
for luminance and chrominance elements.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0015] Embodiments of the present invention will be best understood
by reference to the drawings, wherein like parts are designated by
like numerals throughout. The figures listed above are expressly
incorporated as part of this detailed description.
[0016] It will be readily understood that the components of the
present invention, as generally described and illustrated in the
figures herein, could be arranged and designed in a wide variety of
different configurations. Thus, the following more detailed
description of the embodiments of the methods and systems of the
present invention is not intended to limit the scope of the
invention but it is merely representative of the presently
preferred embodiments of the invention.
[0017] Elements of embodiments of the present invention may be
embodied in hardware, firmware and/or software. While exemplary
embodiments revealed herein may only describe one of these forms,
it is to be understood that one skilled in the art would be able to
effectuate these elements in any of these forms while resting
within the scope of the present invention.
[0018] Some embodiments of the present invention comprise systems
and methods for using the low dynamic range video sequence to
predict the high dynamic range version of the image data. This may
be referred to as inter-layer prediction in this application. Some
embodiments of the present invention comprise a spatially-varying
inter-layer prediction mechanism for HDR video coding. Some
embodiments of the present invention comprise an inter-layer
prediction mechanism for HDR video coding that operates in the
color spaces utilized for video compression and transmission. Some
embodiments utilize gamma corrected color spaces. Exemplary
embodiments may utilize xvYCC and YCbCr color spaces. Some
embodiments of the present invention comprise an inter-layer
prediction mechanism for HDR video coding that may be disabled
spatially. Some embodiments of the present invention comprise an
inter-layer prediction mechanism for HDR video coding that is
multiplication free. Some embodiments of the present invention
comprise an inter-layer prediction mechanism for HDR video coding
that can be utilized in a single-loop decoder. Some embodiments may
also be incorporated into multi-loop designs.
[0019] Embodiments of the present invention comprise an inter-layer
prediction technique for high-dynamic range video coding. Some
aspects of some embodiments comprise elements described in U.S.
patent application Ser. No. 11/362,571 filed on Feb. 24, 2006,
which is hereby incorporated herein by reference. Some embodiments
of the present invention comprise a method for projecting decoded
low dynamic range data to the high dynamic range coding space. This
process may be referred to as inter-layer prediction.
[0020] An analogous process to inter-layer prediction for high
dynamic range video coding is inter-layer prediction for bit-depth
scalability. In the problem for bit-depth scalability, the
baselayer of a video bit-stream contains a representation of the
video sequence at a reduced bit-depth. For example, the baselayer
may contain an eight-bit representation of the sequence, while the
enhancement layer of the bit-stream may contain a ten-bit
representation. In some scenarios, more than two layers may be
used. In some scenarios, an eight-bit version may represent the
eight most significant bits of the higher bit-depth sequence. The
higher bit-depth version is therefore predicted by multiplying (or
equivalently scaling) the decoded lower bit-depth data by the
higher bit-depth. In this specific example, the eight-bit data
would be decoded and subsequently scaled by a factor of four to
predict the ten-bit data. This scaling may be done in either the
intensity or transform domain, depending on the application.
[0021] High dynamic range video coding can be a more general case
of bit-depth scalability. The baselayer and enhancement layer may
contain data represented with different bit-depths. However, the
baselayer may not be constrained to represent the most significant
bits of the enhancement layer data. In some embodiments of the
present invention, the baselayer data may contain a lower bit-depth
representation of the high dynamic range sequence, and this lower
bit-depth may not always correspond to the most significant bits of
the corresponding higher bit-depth representation.
[0022] Some embodiments of the present invention may be described
with reference to FIG. 1A. In these embodiments, a high dynamic
range (HDR) image is received 100. A corresponding low dynamic
range (LDR) image may also be received 101 or created from the HDR
image. The LDR image may be created through a tone scale operation,
a conversion function or some other method. The LDR image may then
be predicted, transformed, quantized and encoded 102 as is well
known in the art. In a few exemplary embodiments the LDR image may
be transformed using a discrete cosine transform (DCT), a wavelet
transform or by other common transformation methods. The
prediction, transformation, quantization and encoding processes may
then be substantially reversed 103 to provide a decoded LDR image
as would be decoded at a typical decoder. Typically, a
de-quantization process is lossy and therefore does not produce an
exact copy of the originally encoded image. Other processes may
also affect the reproduction of the original LDR image. Regardless,
the decoded LDR image may be processed by one or more of the
following methods: color conversion, scaling 104 and offsetting
105. The decoded, processed LDR image may now be used to create 106
a residual HDR image. This may be performed by subtracting the
decoded, processed LDR image from the original HDR image. Other
methods may also be used.
[0023] The residual HDR image may then be transformed, quantized
and encoded 107 or otherwise prepared for transmission to a
destination. This step may comprise embedding the encoded residual
HDR image into an HDR or enhancement layer bitstream. Information
related to the color conversion, scaling and offset operations may
also be encoded and embedded 108 in the HDR or enhancement
bitstream. The HDR/enhancement layer bitstream may then be
transmitted 109 to a destination. An LDR/baselayer bitstream may
also be transmitted 110 to the destination. The LDR/baselayer
bitstream may also comprise a transformed, quantized and encoded
LDR image.
[0024] A decoder receiving the LDR/baselayer bitstream may then
decode the LDR/baselayer image. A decoder receiving the
LDR/baselayer bitstream and the HDR/enhancement layer bitstream may
decode both the LDR/baselayer image and the HDR/enhancement layer
image. Embodiments of the present invention comprise methods and
systems for encoding and decoding images in this framework and
similar scenarios.
[0025] Some embodiments of the present invention may be described
with reference to FIG. 1B. In these embodiments, a baselayer
decoder may receive baselayer data, such as from a baselayer
bitstream 2. The baselayer decoder may decode 6 a baselayer block
or other image element and represent it in the spatial domain. Some
embodiments may comprise full decoding of the block, including a
prediction process followed by residual refinement. Some
embodiments may comprise reconstruction of the residual only. In
some embodiments, the spatial information in the baselayer may be
utilized to predict the high dynamic range signal. Some embodiments
may comprise scaling 7 the baselayer information. Some embodiments
may also comprise adding an offset 8 to the baselayer information.
Some embodiments may comprise both scaling 7 and adding an offset
8. Once scaling 7 and/or adding an offset 8 are performed on the
decoded baselayer information, that scaled, offset information may
be used to predict 9 an enhancement layer, such as a higher dynamic
range (HDR) layer. In some embodiments, scaling 7 and offset 8 data
may be extracted from an enhancement layer 4 bitstream. In some
embodiments, subsequent refinement may be decoded from the
enhancement layer bit-stream 4.
[0026] Some embodiments of the present invention may be described
with reference to FIG. 2. In these embodiments, a decoder may
receive baselayer data 10 from which a block or other image element
may be decoded 12 into spatial image data. This spatial image data
may then be converted 13 to an alternative color space. This
converted data may then be scaled 14 and/or offset 15. Scaling and
offset operations may be performed according to instructions and/or
data received from an enhancement bitstream 11. This converted,
scaled and/offset data may then be converted 16 back to the coding
color space. Once converted back to the coding color space, the
scaled and/or offset data may be used to predict 17 an enhancement
layer, such as a higher dynamic range (HDR) layer.
[0027] Some embodiments of the present invention may be described
with reference to FIG. 3. In these embodiments, an LDR/baselayer
image is received 30 and corresponding HDR/enhancement layer data
is also received 31. An LDR/baselayer block or image element is
then decoded 32 from the LDR/baselayer image. The decoded
LDR/baselayer image element is then scaled 33. This scaling may be
performed according to data embedded in the HDR/enhancement layer
data. Scaling of individual image elements may be related to or a
function of image characteristics comprising spatial location,
luminance data, chrominance data and other data. The scaled,
decoded LDR/baselayer image may then be used to predict 34 a
corresponding HDR block or image element. In some embodiments, the
scaled, decoded LDR/baselayer image element may be added to a
corresponding decoded residual image element to form an
HDR/enhancement layer image element.
[0028] Some embodiments of the present invention may be described
with reference to FIG. 4. In these embodiments, an LDR/baselayer
image is received 40 and corresponding HDR/enhancement layer data
is also received 41. An LDR/baselayer block or image element is
then decoded 42 from the LDR/baselayer image. The decoded
LDR/baselayer image element is then scaled 43. This scaling may be
performed according to data embedded in the HDR/enhancement layer
data. Scaling of individual image elements may be related to or a
function of image characteristics comprising spatial location,
luminance data, chrominance data and other data. An offset may then
be added 44 to the scaled LDR image element. Offset data may be
carried in the corresponding HDR/enhancement layer data. Offset
data may vary between image elements and may be dependent on image
characteristics comprising spatial location, luminance data,
chrominance data and other data.
[0029] The scaled, offset and decoded LDR/baselayer image may then
be used to predict 45 a corresponding HDR block or image element.
In some embodiments, the scaled, offset and decoded LDR/baselayer
image element may be added to a corresponding decoded residual
image element to form an HDR/enhancement layer image element.
[0030] Some embodiments of the present invention may be described
with reference to FIG. 5. In these embodiments, an LDR/baselayer
image comprising LDR transform coefficients is received 50 and
corresponding HDR/enhancement layer data is also received 51. The
LDR/baselayer image transform coefficients may then be scaled 52.
This scaling may be performed according to data embedded in the
HDR/enhancement layer data. Scaling of LDR transform coefficients
may be related to or a function of image characteristics comprising
spatial location, luminance data, chrominance data and other data.
The scaled LDR/baselayer transform coefficients may then be used to
predict 53 transform coefficients for a corresponding HDR block or
image element.
[0031] Some embodiments of the present invention may be described
with reference to FIG. 6. In these embodiments, an LDR/baselayer
image comprising LDR transform coefficients is received 60 and
corresponding HDR/enhancement layer data is also received 61. The
LDR/baselayer image transform coefficients may then be offset 62.
Offset data may be carried in the corresponding HDR/enhancement
layer data 61. Offset data may vary between image elements and may
be dependent on image characteristics comprising spatial location,
luminance data, chrominance data and other data. The offset
LDR/baselayer transform coefficients may then be used to predict 63
transform coefficients for a corresponding HDR block or image
element.
[0032] Some embodiments of the present invention may be described
with reference to FIG. 7. In these embodiments, an LDR/baselayer
image comprising LDR transform coefficients is received 70 and
corresponding HDR/enhancement layer data is also received 71. The
LDR/baselayer image transform coefficients may then be scaled 72.
This scaling may be performed according to data embedded in the
HDR/enhancement layer data. Scaling of LDR transform coefficients
may be related to or a function of image characteristics comprising
spatial location, luminance data, chrominance data and other data.
The scaled LDR/baselayer image transform coefficients may then be
offset 73. Offset data may be carried in the corresponding
HDR/enhancement layer data 71. Offset data may vary between image
elements and may be dependent on image characteristics comprising
spatial location, luminance data, chrominance data and other data.
The scaled, offset LDR/baselayer transform coefficients may then be
used to predict 74 transform coefficients for a corresponding HDR
block or image element.
[0033] Some embodiments of the present invention may be described
with reference to FIG. 8. In these embodiments, an LDR/baselayer
image is received 80 and corresponding HDR/enhancement layer data
is also received 81. An LDR/baselayer block or image element is
then decoded 82 from the LDR/baselayer image. The decoded
LDR/baselayer image element may then be converted 83 or transformed
to an alternative color format or color space. While in this
alternative color space, the LDR image element may be scaled 84.
This scaling may be performed according to data embedded in the
HDR/enhancement layer data. Scaling of individual image elements
may be related to or a function of image characteristics comprising
spatial location, luminance data, chrominance data and other data.
Also, while in the alternative color space, an offset may then be
added 85 to the scaled, color-converted LDR image element. Offset
data may be carried in the corresponding HDR/enhancement layer
data. Offset data may vary between image elements and may be
dependent on image characteristics comprising spatial location,
luminance data, chrominance data and other data.
[0034] The scaled and/or offset and color-converted LDR/baselayer
image may then be converted back 86 to the coding color space. This
scaled and/or offset, coding-color-space LDR/baselayer image may
then be used to predict 87 a corresponding HDR block or image
element.
[0035] Some embodiments of the present invention may be described
with reference to FIG. 9. In these embodiments, an LDR/baselayer
image is received 90 and corresponding HDR/enhancement layer data
is also received 91. An LDR/baselayer block or image element may
then be decoded 92 from the LDR/baselayer image. In these
embodiments, the decoded LDR/baselayer image may comprise separable
luminance and chrominance values. In some embodiments, luminance
values may be scaled 93 in relation to their spatial position in
the image. Other factors may also affect the luminance value
scaling operation. In some embodiments, these luminance values may
be offset 94. The offset operation may also be related to the
spatial position of the luminance value. In some embodiments, the
chrominance values of the decoded LDR/baselayer image may be scaled
95. This chrominance scaling may also be related to the spatial
position of the chrominance value. In some embodiments, chrominance
values may also be offset 96. The chrominance value offset may be
related to a luminance offset, a chrominance value or scaling
factor and/or a spatial position of the chrominance value. Other
factors may also affect the chrominance offset.
[0036] Once the luminance and chrominance values are scaled and/or
offset, they may be used to predict 97 a corresponding
HDR/enhancement layer image element.
[0037] In some embodiments, the inter-layer prediction process may
be controlled at a fine granularity. As a specific example, the
scaling and offset factors may vary on a 4.times.4 block basis.
That is, for every 4.times.4 block in the image, an encoder may
signal the appropriate scaling and offset factor. Additionally, an
encoder may enable and disable inter-layer prediction on a block by
block basis. This allows, for example, the high dynamic range image
to be predicted from the low dynamic range image in a portion of
the frame while predicted with alternative mechanisms in other
spatial regions. Specifically, intra-frame and inter-frame
prediction mechanisms may be utilized in these other spatial
regions.
Exemplary Scaling Embodiments
[0038] Some embodiments of the present invention comprise
inter-layer prediction methods that are multiplication free. In
these embodiments, the baselayer data may be decoded and the
decoded samples may be processed with a sequence of binary shifts
and adds. In some embodiments, this may be accomplished with a
process described by equation 1.
HDR ( x , y ) = .A-inverted. i a i * LDR ( x , y ) i ( 1 )
##EQU00001##
where HDR and LDR are, respectively, the high dynamic range and low
dynamic range version of the image sequence, x and y denote the
spatial location within the image frame, and a.sub.i is a binary
indicator that belongs to the set {-1,0,1}. Some embodiments may
select i={0,1,2,3}.
Alternative Exemplary Scaling Embodiments
[0039] Some inter-layer prediction embodiments comprise an offset
in the inter-layer prediction process. Some embodiments may
comprise a process described in equation 2.
HDR ( x , y ) = .A-inverted. i a i * LDR ( x , y ) i + Offset ( x ,
y ) ( 2 ) ##EQU00002##
where Offset(x,y) is the offset value. In some embodiments, the
offset value may be signaled with the scaling values.
Alternatively, it may be signaled as part of a residual refinement
process.
Spatial Adaptivity
[0040] In some embodiments, control of the prediction process may
be enabled at fine granularity. For example, when the baselayer
video codec employs a block based structure, the inter-layer
prediction process may vary the scaling and offset parameters on a
similar block grid. In some embodiments, this may be achieved by
sending scaling and/or offset information from the encoder to the
decoder within an enhancement bit-stream.
[0041] In some signaling embodiments, the scaling factors may be
transmitted differentially. That is, the scale factor may be
predicted from previously received scale factors. Then, a
correction may be transmitted in the bit-stream. Some embodiments
may predict the scale factor from the upper or left-most neighbor
to the current block. Alternatively, some embodiments may predict
the scale factor as the minimum value of the upper or left-most
neighbor.
[0042] In addition, in some embodiments, the encoder may signal the
correction value as a function of the upper and left-most
neighbors. For example, the encoder and decoder may utilize a
specific context or state for signaling when the neighbors have the
same scale factor. An alternative state may be utilized when the
neighbors have different scale factors.
High Level Syntax
[0043] Some embodiments of the present invention comprise high
dynamic range video coding where the scale factor is the same
throughout an image region. To accommodate these cases, high level
information may also be transmitted from the encoder to the
decoder. This high level information can disable the transmission
of scaling and/or offset parameters on a block-by-block or
region-by-region basis. For the case that transmission of the
parameters is disabled, the high level information may comprise the
scaling and/or offset information to be utilized. In some
embodiments, this high level signaling will occur on a macroblock,
slice, picture or sequence basis.
Transform Domain Processing
[0044] In some embodiments of the present invention, the
inter-layer prediction process operates on intensity data. That is,
the information is decoded and converted to the spatial domain by
reversing any transform utilized for signaling. In alternative
prediction embodiments, the scaling and offset operations may be
directly applied in the transform domain. In these embodiments, the
transform coefficients may be de-quantized and then scaled by scale
factors. In some embodiments, transform coefficients may be
processed differently depending on their frequency characteristics.
For example, in some embodiments, the scaling operation may be
applied solely to the AC coefficients while the offset operation
may affect the DC component. In some embodiments, different scaling
and offset operations may be signaled for different coefficients or
coefficient types.
[0045] Some embodiments of the present invention may comprise a
video codec that may adaptively switch between transform domain and
spatial domain prediction mechanisms. In some embodiments, this
switch may be signaled on a sequence, frame or slice basis. In some
embodiments, this switch may operate at finer granularity, such as
a block or macro-block.
Color and Color Space Issues
[0046] An issue in scalable, high dynamic range video coding is the
management of color. In some embodiments of the present invention,
a color transform may be used prior to inter-layer prediction. This
addresses the fact that most color spaces utilized for video coding
are not iso-luminant. For example, a video codec typically
transmits data in the YCbCr color space with code word mappings
defined in International Telecommunication Union, "Parameter Values
for the HDTV standards for production and international programme
exchange," ITU-R BT.709-5, April, 2002.
[0047] Some embodiments of the present invention perform an
inter-layer prediction process in a color space closely related to
the coding color space. In some exemplary embodiments, the color
transform may be expressed in the following equations:
Y LDR = Y LDR ##EQU00003## b = Cb LDR Y LDR + Cr LDR + Cb LDR
##EQU00003.2## y = Y LDR Y LDR + Cr LDR + Cb LDR ##EQU00003.3##
where Y.sub.LDR, Cb.sub.LDR and Cr.sub.LDR are the luma and chroma
components in the low dynamic range image sequence, respectively.
Then, the scaling and offset process may be applied to Y.sub.LDR to
generate Y.sub.HDR. Finally, the inter-predicted region may be
computed with the following equations:
Y HDR = Y HDR ##EQU00004## Cb HDR = bY HDR y ##EQU00004.2## Cr HDR
= ( 1 - b - y ) Y HDR y ##EQU00004.3##
where Cb.sub.HDR and Cr.sub.HDR are predictions for the color
components in the high dynamic range layer.
[0048] In some embodiments wherein Y.sub.LDR, Cb.sub.LDR and
Cr.sub.LDR are not represented at the same resolution, the
components may be resampled. In some exemplary embodiments,
applications may down-sample the luma component when the chroma
components are stored at lower resolution. Alternatively, the
chroma components may be up-sampled to match the resolution of the
luma component.
Alternative Color and Color Space Issues
[0049] In some embodiments of the present invention, inter-layer
prediction may operate directly on the decoded data without
employing a color transform. In some exemplary embodiments, the
prediction process may be expressed by the following equations:
Y.sub.HDR(x, y)=Scale(x, y, c)*Y.sub.LDR(x, y)+Offset(x, y, c)
Cb.sub.HDR(x, y)=Scale(x, y, c)*Cb.sub.LDR (x, y)+Offset(x, y,
c)
Cr.sub.HDR(x, y)=Scale(x, y, c)*Cr.sub.LDR(x, y)+Offset(x, y,
c)
where the scaling and offset parameters are now a function of both
spatial location and chroma component. That is, the reconstructed
luma and chroma values are scaled with different scale factors.
[0050] In some exemplary inter-prediction processes, the luma and
chroma values may be scaled with the same scale factor but with
different offsets. This may be expressed with the following
equations:
Y.sub.HDR(x, y)=Scale(x, y)*Y.sub.LDR(x, y)+Offset(x, y, c)
Cb.sub.HDR(x, y)=Scale(x, y)*Cb.sub.LDR(x, y)+Offset(x, y, c)
Cr.sub.HDR(x, y)=Scale(x, y)*Cr.sub.LDR(x, y)+Offset(x, y, c)
In these embodiments, the scale factor may not depend on the chroma
component. In some embodiments, the encoder may transmit the
offsets within the enhancement layer bit-stream.
[0051] In other exemplary embodiments of the inter-prediction
process, the luma and chroma values may be scaled with the same
scale factor and the offset for the chroma values may be dependent
on the offset of the luma values as well as the decoded image data.
This relationship may be expressed in the following equations:
Y.sub.HDR(x, y)=Scale(x, y)*Y.sub.LDR(x, y)+Offset(x, y)
Cb.sub.HDR(x, y)=Scale(x, y)*Cb.sub.LDR(x, y)+f(Offset(x, y),
Cb.sub.LDR(x, y), Y.sub.LDR(x, y))
Cr.sub.HDR(x, y)=Scale(x, y)*Cr.sub.LDR(x, y)+f(Offset(x, y),
Cr.sub.LDR(x, y), Y.sub.LDR(x, y))
where f( ) denotes a mapping operation.
[0052] An exemplary mapping operation may be expressed as:
f ( Offset ( x , y ) , A LDR ( x , y ) , Y LDR ( x , y ) ) = Offset
( x , y ) A LDR ( x , y ) Y LDR ( x , y ) ##EQU00005##
where A.sub.LDR(x,y) denotes an arbitrary color component such as
Cb or Cr.
[0053] As mentioned before, the chroma and luma components may be
represented on different sampling grids. To address this problem,
the chroma and luma data may be resampled to the same resolution.
In some embodiments, a different mapping process may be employed.
In some exemplary embodiments, the mapping relationship may be
expressed as:
f ( Offset ( x , y ) , A LDR ( x , y ) , Y LDR ( x , y ) ) = Offset
( x , y ) Avg ( A LDR ( x , y ) ) Avg ( Y LDR ( x , y ) )
##EQU00006##
where Avg( ) denotes the mean operator. In another exemplary
embodiment, the mean may be replaced with a summation operation. In
other embodiments, non-linear operations such as the median, min
and max operations may be beneficial.
[0054] In some exemplary embodiments, the mean operator (or an
alternative operator) may be performed in a different domain than
that of the Offset variable. In some exemplary embodiments, the
mean operation may be computed in the transform domain by operating
solely on the DC coefficient. Similarly, in embodiments wherein the
spatial resolutions of the chroma and luma coefficients are not
matched, the mean operation may be computed by analyzing multiple
DC coefficients in the luma baselayer.
[0055] The terms and expressions which have been employed in the
foregoing specification are used therein as terms of description
and not of limitation, and there is no intention in the use of such
terms and expressions of excluding equivalence of the features
shown and described or portions thereof, it being recognized that
the scope of the invention is defined and limited only by the
claims which follow.
* * * * *