U.S. patent application number 14/363861 was filed with the patent office on 2014-11-06 for backwards-compatible delivery of digital cinema content with extended dynamic range.
This patent application is currently assigned to DOLBY LABORATORIES LICENSING CORPORATION. The applicant listed for this patent is DOLBY LABORATORIES LICENSING CORPORATION. Invention is credited to Walter C. Gish, Christopher J. Vogt.
Application Number | 20140327822 14/363861 |
Document ID | / |
Family ID | 47459142 |
Filed Date | 2014-11-06 |
United States Patent
Application |
20140327822 |
Kind Code |
A1 |
Gish; Walter C. ; et
al. |
November 6, 2014 |
Backwards-Compatible Delivery of Digital Cinema Content with
Extended Dynamic Range
Abstract
A digital cinema signal is encoded to produce a resulting coded
digital cinema bitstream. Decoding the resulting coded digital
cinema bitstream allows backwards-compatible delivery of digital
cinema content. A digital image or video signal is preprocessed to
produce two normalized digital image or video signals of differing
quality levels and forward and inverse mapping parameters, which
relate the normalized digital image or video signals. The
preprocessing can be used prior to the encoding of a digital cinema
signal to enable backwards-compatible delivery of digital cinema
content.
Inventors: |
Gish; Walter C.; (Oak Park,
CA) ; Vogt; Christopher J.; (Laguna Niguel,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
DOLBY LABORATORIES LICENSING CORPORATION |
San Francisco |
CA |
US |
|
|
Assignee: |
DOLBY LABORATORIES LICENSING
CORPORATION
San Francisco
CA
|
Family ID: |
47459142 |
Appl. No.: |
14/363861 |
Filed: |
December 6, 2012 |
PCT Filed: |
December 6, 2012 |
PCT NO: |
PCT/US2012/068275 |
371 Date: |
June 9, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61576141 |
Dec 15, 2011 |
|
|
|
Current U.S.
Class: |
348/453 |
Current CPC
Class: |
G06T 2207/10016
20130101; H04N 19/186 20141101; H04N 19/46 20141101; H04N 19/85
20141101; G06T 2207/10024 20130101; G06T 5/007 20130101; H04N 9/77
20130101; H04N 19/30 20141101; G06T 2207/20208 20130101; G06T
2200/16 20130101; H04N 9/68 20130101; G06T 2207/20048 20130101 |
Class at
Publication: |
348/453 |
International
Class: |
H04N 9/77 20060101
H04N009/77; G06T 5/00 20060101 G06T005/00; H04N 9/68 20060101
H04N009/68 |
Claims
1-69. (canceled)
70. A method of decoding a coded digital cinema bitstream (255),
wherein the coded digital cinema bitstream (255) comprises a coded
digital cinema image (105A, 105B) and mapping metadata (125A,
125B), the method comprising: decoding the coded digital cinema
bitstream (105A, 105B) with a decoder to generate a first
normalized digital cinema image (115A, 115) with a first dynamic
range and a first color gamut; and applying the mapping metadata to
the first normalized digital image to generate a second normalized
digital cinema image (130A, 130B) with a second dynamic range and a
second color gamut, wherein the mapping metadata comprise an
invertible mapping between the first and second normalized digital
images, wherein the invertible mapping comprises a lossless
invertible nonlinear transformation and an invertible matrix
transformation, wherein generating in an encoder the first
normalized digital image using the second normalized digital image
and the invertible mapping comprises: applying the nonlinear
transformation to the second normalized image to generate a first
result and then applying the matrix transformation to the first
result to generate the first normalized digital image, and
generating in the decoder the second normalized digital image using
the first normalized digital image and the invertible mapping
comprises: applying the inverse matrix transformation to the first
normalized image to generate a second result and then applying the
inverse of the nonlinear transformation to the second result to
generate the second normalized digital image.
71. The method of claim 70 wherein the first dynamic range is lower
than the second dynamic range.
72. The method of claim 70, wherein decoding the coded digital
cinema image (105B) generates the second digital cinema image
(115B) and the mapping metadata (125B) are applied to the second
digital cinema image to generate the first digital cinema image
(130B).
73. The method of claim 70, further comprising generating an output
digital cinema image with the second dynamic range and the second
gamut from the second normalized digital cinema image without using
any residual image data.
74. The method of claim 70, wherein the nonlinear transformation
comprises a six segment, cubic spline.
75. The method of claim 70, wherein the matrix transformation uses
a 3.times.3 matrix.
76. The method of claim 70, wherein the mapping metadata comprise a
three-dimensional lookup table.
77. The method of claim 70, wherein generating the second
normalized digital image comprises computing C i VDR = N i - 1 [ M
i , j - 1 C j SDR ] = N i - 1 [ j M i , j - 1 C j SDR ] ,
##EQU00003## wherein C.sub.i.sup.VDR denotes the i-th color
component of the second normalized digital image, C.sub.i.sup.SDR
denotes the i-th color component of the first normalized digital
image, N.sub.i.sup.-1[ ] denotes the inverse of the nonlinear
transformation N.sub.i[ ], and M.sup.-1 denotes the inverse of the
matrix transformation M.
78. A method to generate normalized digital images (230, 235, 330,
335) and mapping parameters (240, 245, 340, 345), the method
comprising: receiving an input image (205); generating (210) using
the input image a first digital image (215) having a first dynamic
range and a first color gamut, and a second digital image (220)
having a second dynamic range and a second color gamut, the second
dynamic range being higher than the first dynamic range and the
second color gamut being larger than the first color gamut;
producing a first normalized digital image (230) by pre-processing
the first digital image; producing a second normalized digital
image (235) by pre-processing the second digital image; and
producing mapping parameters (240, 245), wherein the mapping
parameters comprise an invertible mapping between the first and
second normalized digital images, wherein the invertible mapping
comprises an invertible nonlinear transformation and a lossless
invertible matrix transformation, wherein generating the first
normalized digital image using the second normalized digital image
and the invertible mapping comprises: applying the nonlinear
transformation to the second normalized image to generate a first
result and then applying the matrix transformation to the first
result to generate the first normalized digital image, and
generating the second normalized digital image using the first
normalized digital image and the invertible mapping comprises:
applying the inverse matrix transformation to the first normalized
image to generate a second result and then applying the inverse of
the nonlinear transformation to the second result to generate the
second normalized digital image.
79. The method of claim 78, wherein generating the second
normalized digital image using the first normalized digital image
comprises computing C i VDR = N i - 1 [ M i , j - 1 C j SDR ] = N i
- 1 [ j M i , j - 1 C j SDR ] , ##EQU00004## wherein
C.sub.i.sup.VDR denotes the i-th color component of the second
normalized digital image, C.sub.i.sup.SDR denotes the i-th color
component of the first normalized digital image, N.sub.i.sup.-1[ ]
denotes the inverse of the nonlinear transformation N.sub.i[ ], and
M.sup.-1 denotes the inverse of the matrix transformation M.
80. The method of claim 79, wherein generating the first normalized
digital image using the second normalized digital image comprises
computing
C.sub.i.sup.SDR=M.sub.i,jN.sub.j[C.sub.j.sup.VDR]=.SIGMA..sub.jM.sub.i,jN-
.sub.j[C.sub.j.sup.VDR].
81. The method of claim 78 wherein the first dynamic range is lower
than the second dynamic range.
82. The method of claim 78 wherein the second normalized digital
image (235) is visually identical to the second digital image
(220).
83. The method of claim 78, wherein the producing of the first
normalized digital image (230) comprises: performing a
normalization of the first digital image and the second digital
image to determine forward mapping parameters (245); and performing
forward mapping by applying the forward mapping parameters (245) to
the second digital image (220) to produce the first normalized
digital image (230).
84. The method of claim 83, wherein the producing of the second
normalized digital image (235) comprises: inverting the forward
mapping parameters (245) to obtain the inverse mapping parameters
(240); and performing inverse mapping by applying the inverse
mapping parameters (240) to the first normalized digital image
(230) to produce the second normalized digital image (235).
85. The method of claim 83, wherein the determining of the forward
mapping parameters (245) comprises: (a) setting an input matrix
equal to an identity matrix; (b) inverting the input matrix to
obtain an inverted matrix and applying the inverted matrix to the
first digital image (215), thus obtaining an intermediate first
digital image; (c) producing forward mapping parameters by
determining nonlinear transformation parameters corresponding to a
nonlinear transformation between the second digital image (220) and
the intermediate first digital image; (d) applying the nonlinear
transformation parameters to the second digital image (220), thus
obtaining an intermediate second digital image; (e) producing an
estimated matrix corresponding to a matrix transformation between
the intermediate second digital image and the first digital image
(215); (f) repeating steps (b) through (e), wherein the input
matrix of step (b) is set equal to the estimated matrix of step
(e); and (g) iterating step (f) until a desired result is
obtained.
86. The method of claim 83, wherein the determining of the forward
mapping parameters (245) comprises: (a) setting input nonlinear
transformation parameters to identity; (b) applying the input
nonlinear transformation parameters to the second digital image
(220) to obtain an intermediate second digital image; (c) producing
a matrix corresponding to a matrix transformation between the
intermediate second digital image and the first digital image
(215); (d) inverting the matrix and applying the inverted matrix to
the first digital image (215), thus obtaining an intermediate first
digital image; (e) producing forward mapping parameters (245) by
determining estimated nonlinear transformation parameters
corresponding to a nonlinear transformation between the second
digital image (220) and the intermediate first digital image; (f)
repeating steps (b) through (e), wherein the input nonlinear
transformation parameters of step (b) are set equal to the
estimated nonlinear transformation parameters of step (e); and (g)
iterating step (f) until a desired result is obtained.
87. The method of claim 78 further comprising: encoding the first
normalized digital image (230) to generate a first coded digital
cinema image (255); and combining the first coded digital image and
the mapping parameters to generate a coded digital cinema bit
stream.
88. The method of claim 78, further comprising: encoding the second
normalized digital image (335) to generate a second coded digital
cinema image (355); and combining the second coded digital image
and the mapping parameters to generate a coded digital cinema bit
stream.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of priority to U.S.
Provisional Patent Application Ser. No. 61/576,141, filed on Dec.
15, 2011, hereby incorporated by reference in its entirety. The
present application is also related to PCT Application
PCT/US2011/048861, entitled "Extending Image Dynamic Range", filed
on Aug. 23, 2011, which is incorporated herein by reference in its
entirety. The present application is also related to PCT
Application PCT/US2010/026953 entitled "Layered Compression of High
Dynamic Range, Visual Dynamic Range, and Wide Color Gamut Video,"
filed on Mar. 11, 2010, which is incorporated herein by reference
in its entirety.
TECHNICAL FIELD
[0002] The present disclosure relates to image processing for
digital cinema as well as preprocessing and coding of digital image
and/or video content. More particularly, an embodiment of the
present invention relates to backwards-compatible delivery of
digital cinema content with extended range and related
preprocessing and coding methods.
BACKGROUND
[0003] As used herein, the term `dynamic range` (DR) may relate to
a capability of the human visual system (HVS) to perceive a range
of intensity (e.g., luminance, luma) in an image, e.g., from
darkest darks to brightest brights. In this sense, DR relates to a
`scene-referred` intensity. DR may also relate to the ability of a
display device to adequately or approximately render an intensity
range of a particular breadth. In this sense, DR relates to a
`display-referred` intensity. Unless a particular sense is
explicitly specified to have particular significance at any point
in the description herein, it should be inferred that the term may
be used in either sense, e.g. interchangeably.
[0004] As used herein, the term high dynamic range (HDR) relates to
a DR breadth that spans the some 14-15 orders of magnitude of the
HVS. For example, well adapted humans with essentially normal
vision (e.g., in one or more of a statistical, biometric or
opthamological sense) have an intensity range that spans about 15
orders of magnitude. Adapted humans may perceive dim light sources
of a few photons. Yet, these same humans may perceive the near
painfully brilliant intensity of the noonday sun in desert, sea or
snow (or even glance into the sun, however briefly to prevent
damage). This span though is available to `adapted` humans, e.g.,
those whose HVS has a time period in which to reset and adjust.
[0005] In contrast, the DR over which a human may simultaneously
perceive an extensive breadth in intensity range may be somewhat
truncated, in relation to HDR. As used herein, the term `visual
dynamic range` (VDR) may relate to the DR that is simultaneously
perceivable by a HVS. As used herein, VDR may relate to a DR that
spans 5-6 orders of magnitude. Thus while perhaps somewhat narrower
in relation to true scene referred HDR, VDR nonetheless represents
a wide DR breadth.
[0006] Until fairly recently, displays have had a significantly
narrower DR than HDR or VDR. Television (TV) and computer monitor
apparatus that use typical cathode ray tube (CRT), liquid crystal
display (LCD) with constant fluorescent white back lighting or
plasma screen technology may be constrained in their DR rendering
capability to approximately three orders of magnitude. Such
conventional displays thus typify a low dynamic range (LDR) or
standard dynamic range (SDR), in relation to VDR and HDR. Digital
cinema systems exhibit some of the same limitations as other
display devices.
[0007] Advances in their underlying technology, however, will allow
future digital cinema systems to render image and video content
with significant improvements in various quality characteristics
over the same content, as rendered on today's digital cinema
systems. For example, future digital cinema systems may be capable
of a DR (e.g. VDR) that is higher than the SDR/LDR of conventional
digital cinema systems as well as a larger color gamut than the
color gamut of conventional digital cinema systems. The challenge
is providing digital cinema content which may be displayed on
conventional SDR, small color gamut systems at a standard quality
level as well as more advanced VDR, larger color gamut systems at a
correspondingly higher quality level.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] FIGS. 1A and 1B depict schematic diagrams of digital cinema
decoding architectures in accordance with exemplary embodiments of
the present disclosure.
[0009] FIG. 2 is an example of a pre-processing architecture of a
digital image or video prior to coding.
[0010] FIG. 3 is an alternative example of a pre-processing
architecture of a digital image or video prior to coding.
[0011] FIG. 4 depicts a schematic representation of an embodiment
of the preprocessor of FIGS. 2-3 in greater detail.
[0012] FIG. 5 depicts a typical non-linearity when clipping occurs
at both the low and high limits of intensity.
DESCRIPTION OF EXAMPLE EMBODIMENTS
[0013] In an example embodiment, a method of decoding a coded
digital cinema bitstream is provided, the method comprising:
providing a coded digital cinema bitstream; providing mapping
parameters; decoding the coded digital cinema bitstream, the
decoding producing a first decoded digital cinema image or video
with a first dynamic range and a first color gamut; and expanding
the first dynamic range and the first color gamut of the first
decoded digital cinema image or video by inverse mapping the first
decoded digital cinema image or video with the mapping parameters,
thus obtaining a second decoded digital cinema image or video with
a second dynamic range higher than the first dynamic range and a
second color gamut larger than the first color gamut.
[0014] In an example embodiment, a method of decoding a coded
digital cinema bitstream is provided, the method comprising:
providing a coded digital cinema bitstream; providing mapping
parameters; decoding the coded digital cinema bitstream, the
decoding producing a first decoded digital cinema image or video
with a first dynamic range and a first color gamut; and compressing
the first dynamic range and the first color gamut of the first
decoded digital cinema image or video by forward mapping the first
decoded digital cinema image or video with the mapping parameters,
thus obtaining a second decoded digital cinema image or video with
a second dynamic range lower than the first dynamic range and a
second color gamut smaller than the first color gamut.
[0015] In an example embodiment, a method of pre-processing a
digital image or video signal is provided, the method comprising:
processing the digital image or video signal by performing color
grading to produce a first digital image or video signal having a
first dynamic range and a first color gamut, and a second digital
image or video signal having a second dynamic range and a second
color gamut, the second dynamic range being higher than the first
dynamic range and the second color gamut being larger than the
first color gamut; producing a first normalized digital image or
video signal by pre-processing the first digital image or video
signal; producing a second normalized digital image or video signal
by pre-processing the second digital image or video signal, wherein
the first normalized digital image or video signal is obtainable
from the second normalized digital image or video signal through a
forward mapping and the second normalized digital image or video
signal is obtainable from the first normalized digital image or
video signal through an inverse mapping; and producing forward
mapping parameters and inverse mapping parameters.
[0016] In an example embodiment, a method of estimating nonlinear
forward mapping parameters for digital image or video signals is
presented, the method comprising: providing a first digital image
or video signal; providing a second digital image or video signal;
providing a matrix; inverting the matrix and applying the inverted
matrix to the first digital image or video signal, thus obtaining
an intermediate digital image or video signal; and producing
nonlinear transformation parameters corresponding to a nonlinear
transformation between the second digital image or video signal and
the intermediate digital image or video signal.
[0017] In an example embodiment, a method of estimating matrix
forward mapping parameters for digital image or video signals is
presented, the method comprising: at the providing a first digital
image or video signal; providing a second digital image or video
signal; providing a nonlinear transformation; applying the
nonlinear transformation to the second digital image or video
signal, thus obtaining an intermediate digital image or video
signal; and producing matrix transformation parameters
corresponding to a matrix transformation between the intermediate
digital image or video signal and the first digital image or video
signal.
[0018] In an example embodiment, a method of determining forward
mapping parameters for digital image or video signals is presented,
the method comprising: (a) setting an input matrix equal to an
identity matrix; (b) providing a first digital image or video
signal; (c) providing a second digital image or video signal; (d)
inverting the input matrix, thus obtaining an inverted matrix and
applying the inverted matrix to the first digital image or video
signal, thus obtaining an intermediate first digital image or video
signal; (e) producing nonlinear transformation parameters
corresponding to a nonlinear transformation between the second
digital image or video signal and the intermediate first digital
image or video signal; (f) applying the nonlinear transformation
parameters to the second digital image or video signal, thus
obtaining an intermediate second digital image or video signal; (g)
producing an estimated matrix corresponding to a matrix
transformation between the intermediate second digital image or
video signal and the first digital image or video signal; (h)
repeating steps (d) through (g), wherein the input matrix of step
(d) is set equal to the estimated matrix of step (g); and (i)
iterating step (h) until a desired result is obtained.
[0019] In an example embodiment, a method of determining forward
mapping parameters for digital image or video signals is presented,
the method comprising: (a) setting input nonlinear transformation
parameters to identity; (b) providing a first digital image or
video signal; (c) providing a second digital image or video signal;
(d) applying the input nonlinear transformation parameters to the
second digital image or video signal, thus obtaining an
intermediate second digital image or video signal; (e) producing a
matrix corresponding to a matrix transformation between the
intermediate second digital image or video signal and the first
digital image or video signal; (f) inverting the matrix and
applying the inverted matrix to the first digital image or video
signal, thus obtaining an intermediate first digital image or video
signal; (g) producing estimated nonlinear transformation parameters
corresponding to a nonlinear transformation between the second
digital image or video signal and the intermediate first digital
image or video signal; (h) repeating steps (d) through (g), wherein
the input nonlinear transformation parameters of step (d) are set
equal to the estimated nonlinear transformation parameters of step
(g); and (i) iterating step (h) until a desired result is
obtained.
[0020] In an example embodiment, a system configured to decode a
coded digital cinema bitstream is presented, the system comprising:
a decoder configured to decode the coded digital cinema bitstream
and produce a first decoded digital cinema image or video; and a
mapping applicator configured to expand a first dynamic range and a
first color gamut of the first decoded digital cinema image or
video by inverse mapping the first decoded digital cinema image or
video with mapping parameters, thus obtaining a second decoded
digital cinema image or video with a second dynamic range higher
than the first dynamic range and a second color gamut larger than
the first color gamut.
[0021] In an example embodiment, a system configured to decode a
coded digital cinema bitstream is provided, the system comprising:
a decoder configured to decode the coded digital cinema bitstream
and produce a first decoded digital cinema image or video; and a
mapping applicator configured to compress a first dynamic range and
a first color gamut of the first decoded digital cinema image or
video by forward mapping the first decoded digital cinema image or
video with mapping parameters, thus obtaining a second decoded
digital cinema image or video with a second dynamic range lower
than the first dynamic range and a second color gamut smaller than
the first color gamut.
[0022] In an example embodiment, a system configured to pre-process
a digital image or video signal is provided, the system comprising:
a color grading module configured to process the digital image or
video signal to produce a first digital image or video signal
having a first dynamic range and a first color gamut, and a second
digital image or video signal having a second dynamic range and a
second color gamut, the second dynamic range being higher than the
first dynamic range in the second color gamut being larger than the
first color gamut; and a preprocessor configured to produce a first
normalized digital image or video signal by preprocessing the first
digital image or video signal; configured to produce a second
normalized digital image or video signal by preprocessing the
second digital image or video signal, wherein the first normalized
digital image or video signal is obtainable from the second
normalized digital image or video signal through a forward mapping
and the second normalized digital image or video signal is
obtainable from the first normalized digital image or video signal
through an inverse mapping; and configured to produce forward
mapping parameters and inverse mapping parameters.
[0023] As used herein, the term digital cinema refers to the
projection of a theatrical motion picture through a digital cinema
projection system. As used herein, the term digital cinema signal
refers to a signal representing digital cinema information.
[0024] As used herein, the terms digital image or video signal
refer to digital content which may be, by way of example and not of
limitation, live action, rendered CGI (computer-generated imagery),
or from any source capable of producing a digital image or video
signal.
[0025] FIG. 1A depicts a schematic diagram of a digital cinema
decoding architecture in accordance with an embodiment of the
present disclosure.
[0026] A coded bitstream (105A) is input to a decoder (110A). In
the embodiment of the figure, the bitstream comprises a 12-bit
digital cinema signal with a 4:4:4 color representation. Typical
input bit rates are in the 125-250 Mbps range. The digital cinema
signal has a dynamic range, e.g. a 2000:1 dynamic range, and a
color gamut, e.g. a P3 color gamut. The decoder (110A) can be any
decoder able to operate on a digital cinema signal, e.g. a
JPEG-2000 decoder.
[0027] The decoder (110A) outputs a first digital cinema image or
video (115A) with the same dynamic range and color gamut of the
coded input bitstream (105A).
[0028] Inverse mapping is performed on the digital cinema image or
video (115A) by a mapping applicator (120A) to expand the dynamic
range and color gamut of the image or video. In accordance with an
embodiment of the disclosure, the inverse of a nonlinear
transformation N followed by a matrix transformation M, i.e.
(M.smallcircle.N).sup.-1 (where the .smallcircle. indicates the
transformation on the right is carried out prior to the
transformation on the left), is performed. By way of example, the
nonlinear transformation N can be a six segment, cubic spline,
while matrix M can be a 3.times.3 matrix. The nonlinear
transformation parameters Nj and matrix parameters Mij are sent to
the mapping applicator (120A) as mapping metadata (125A). In one
embodiment, the mapping applicator can be implemented as a
three-dimensional (3-D) lookup table. While 3-D lookup tables in
general are known to those skilled in the art, an embodiment
according to the present disclosure may use the 3-D lookup table to
produce VDR digital cinema video.
[0029] The mapping applicator (120A) outputs a second 12-bit, 4:4:4
digital cinema image or video (130A) with a higher dynamic range
and a larger color gamut than the dynamic range and color gamut of
the first digital cinema image or video (115A). For example, the
second digital cinema image or video (130A) can have a 10000:1
dynamic range, and a color gamut larger than P3.
[0030] Therefore, the decoding architecture of FIG. 1A provides a
dual-layered digital cinema content. In particular, the output
(135A) of the first layer provides a 12-bit, 4:4:4 digital cinema
signal (115A) with a first dynamic range and first color gamut,
while the output (140A) of the second layer provides a 12-bit,
4:4:4 digital cinema signal (130A) with a second dynamic range
higher than the first dynamic range and a second color gamut larger
than the first color gamut.
[0031] The architecture of FIG. 1A thus provides a `single
inventory`, backwards-compatible, approach for digital cinema
content, which can be used both with (a) digital cinema projectors
compatible with the first dynamic range and the first color gamut
and (b) digital cinema projectors compatible with the second
dynamic range and second color gamut. Output (135A) will be sent to
the first type of projectors, while output (140A) will be sent to
the second type of projectors. Alternatively, decoder (110A) and
mapping applicator (120A) may be located in a projector (150A) and
one of the outputs (135A, 140A) can be sent to a screen (160A).
[0032] The person skilled in the art will appreciate that the
decoding and mapping architecture of FIG. 1A is a residual-free
architecture, where no residual is employed to improve decoding of
the digital cinema bitstream (105A).
[0033] In one embodiment, the output (135A) represents a
conventional/LDR/SDR digital cinema version, while the output
(140A) represents a VDR/HDR digital cinema version. As mentioned
above, use of the LDR/SDR or VDR/HDR version will depend on the
kind of projector available to theatres.
[0034] FIG. 1B depicts a schematic diagram of a digital cinema
decoding architecture in accordance with an alternative embodiment
of the present disclosure. Such architecture is similar to the
embodiment of FIG. 1A with the following differences. Coded
bitstream (105B) of FIG. 1B is characterized by a higher dynamic
range and a larger color gamut than coded bitstream (105A) of FIG.
1A. First digital cinema video (115B) is sent to forward mapping
applicator (120B) to produce a second digital cinema video (130B)
with a lower dynamic range and a smaller color gamut than the first
digital cinema video (115B).
[0035] In one embodiment, the output (135B) represents a VDR/HDR
digital cinema version, while the output (140B) represents a
conventional/LDR/SDR digital cinema version. As mentioned above,
use of the LDR/SDR or VDR/HDR version will depend on the kind of
projector available to theatres.
[0036] While the dual-layer architectures of FIGS. 1A and 1B have
been described in terms of conventional/SDR layer vs. VDR/HDR
layer, the person skilled in the art will understand that other
layered forms and denominations are also possible, such as base
layer vs. enhancement layer, first enhancement layer vs. second
enhancement layer, and so on.
[0037] As noted above, FIG. 1A depicts details of a decoding
architecture of a digital cinema bitstream. FIG. 2 depicts instead
an example of a pre-processing architecture of a digital cinema
image or video prior to coding. In a particular embodiment of the
present disclosure, the pre-processing architecture of FIG. 2 can
be used in combination with the decoding architecture of FIG.
1A.
[0038] As depicted in FIG. 2, a captured digital cinema signal
(205) is input to a color grading module (210), which outputs a
first digital cinema signal (215) with a first dynamic range and
color gamut and a second digital cinema signal (220) with a second
dynamic range higher than the first dynamic range and a second
color gamut larger than the first color gamut. By way of example,
signal (215) can be an LDR/SDR signal, while signal (220) can be a
VDR/HDR signal.
[0039] The first digital cinema signal (215) and second digital
cinema signal (220) are then normalized in a preprocessor (225),
thus producing a first normalized digital cinema signal (230) and a
second normalized digital cinema signal (235), where the second
normalized digital cinema signal (235) is identical to the second
digital cinema signal (220). The processing through the
preprocessor (225) allows the first normalized digital cinema
signal (230) and the second normalized digital cinema signal (235)
to be related by invertible mapping. In other words, once
normalized, digital cinema signal (230) can be obtained from
digital cinema signal (235) through forward mapping, while digital
cinema signal (235) can be obtained from digital cinema signal
(230) through inverse mapping. Assuming, for example, that the
first signal (230) is indicated by SDR* (where * is to indicate a
normalized version of input (215)) and the second signal (235) is
indicated by VDR* (which is usually equal to the VDR input (220) to
numerical precision), the following relationship holds true:
SDR*=(M.smallcircle.N)VDR*, where M.smallcircle.N is the forward
mapping operator mentioned above. The preprocessor (225) also
produces inverse and/or forward mapping parameters (240) and/or
(245) to be sent, e.g., as metadata. Such parameters allow
obtaining signal (235) from signal (230) through inverse mapping or
signal (230) from signal (235) through forward mapping. The mapping
parameters obtained and the mapping performed are such that inverse
mapping the SDR* signal will exactly reproduce the VDR signal.
[0040] The first normalized digital cinema signal (230) is then
encoded in an encoder (250) and sent to a digital cinema system as
a coded bitstream (255). Encoder (250) can be, for example, an
encoder configured to process the first signal (e.g., an SDR*
signal (230)) and not the second signal (e.g., a VDR* signal
(235)). See, for example, encoder 312 in FIG. 9A of the above
mentioned PCT Application PCT/US2011/048861, incorporated herein by
reference in its entirety. In an alternative embodiment, the second
signal (e.g., a VDR* signal (335)) may be encoded to allow
obtaining an SDR* signal from a VDR* signal. This alternative
embodiment is depicted in FIG. 3. Such embodiment can be combined
with FIG. 1B in a manner similar to the combination of FIGS. 2 and
1A.
[0041] Normalization pre-processing as described in FIGS. 2 and 3
can be used to prepare image or video data for backwards-compatible
delivery in distribution systems such as digital cinema systems. As
noted above, compatibility between the first and the second digital
signals discussed above is obtained at the output of the
preprocessor (225/325), where an SDR* signal can be obtained from a
VDR* signal and vice versa. In other words, the embodiments of
FIGS. 2-3 allow different realizations (various levels of SDR) of a
master image or video (e.g., VDR) to be derived by transforming the
master image or video with a transformation that is invertible.
[0042] According to the embodiment described above, one such
transformation is the M.smallcircle.N transformation. In other
words, a non-linear transformation N followed by a linear matrix
transformation M are performed. Such transformation (where repeated
indices imply summation and where the higher and lower dynamic
range indicators are depicted as VDR and SDR by way of example) can
be seen as follows:
C i VDR = i - th color component of VDR image C i SDR = i - th
color component of SDR image C i SDR = M i , j N j [ C j VDR ] = j
M i , j N j [ C j VDR ] Equation ( 1 ) ##EQU00001##
[0043] When the N and M transformation are invertible, the VDR
image or video can be recovered from the SDR image or video:
C i VDR = N i - 1 [ M i , j - 1 C j SDR ] = N i - 1 [ j M i , j - 1
C j SDR ] Equation ( 2 ) ##EQU00002##
[0044] In practice, SDR and VDR images or videos are often created
in separate color grading passes. The SDR version typically
satisfies Equation (1) approximately, but not necessarily exactly.
The function of normalization is to determine a modified version of
the SDR, i.e. SDR*. SDR* and the original VDR satisfy Equation (1)
exactly and, furthermore, SDR* and SDR are typically
indistinguishable visually. SDR and SDR* can be visually
indistinguishable approximately 99% of the time, and in cases where
there are visible differences, such differences can be visible only
when the sequence is halted at a particular frame.
[0045] In an embodiment of the present disclosure, input bitstream
(105A) of FIG. 1A corresponds to the encoded bitstream (255) of
FIG. 2, while mapping metadata (125A) of FIG. 1A correspond to
inverse mapping parameters (240) of FIG. 2. Therefore, according to
such embodiment, the decoding architecture of FIG. 1A can be used
in conjunction with the encoding architecture of FIG. 2. In
particular, assuming that the inverse mapping of FIG. 1A is
performed by inverse transformation (M.smallcircle.N).sup.-1, the
mapping of FIG. 2 is performed by transformation M.smallcircle.N as
later explained in greater detail. Similarly, the encoding
architecture of FIG. 3 can be used with alternative decoding
architectures based on that depicted in FIG. 1B. Input bitstream
(105B) of FIG. 1B corresponds to the encoded bitstream (355) of
FIG. 3, while mapping metadata (125B) of FIG. 1B correspond to
forward mapping parameters (345) of FIG. 3.
[0046] Reference will now be made to FIG. 4, which depicts a
schematic representation of an embodiment of the preprocessor
(225/325) of FIGS. 2 and 3 in greater detail. First signal
(215/315) and second signal (220/320) are sent to a normalization
module (410) which determines the forward mapping parameters of
nonlinear transformation N.sub.j[ ] and the forward mapping
parameters M.sub.ij of matrix transformation M. An example of how
these parameters are obtained will be later explained in
detail.
[0047] Such forward mapping parameters (245/345) are input to a
forward mapping module (420) together with the second signal
(220/320) in order to obtain the first normalized digital cinema
signal (230/330) discussed with reference to FIGS. 2 and 3, e.g. an
SDR* signal. Forward mapping parameters (245/345) are also input to
an inversion module (430) to obtain inverse mapping parameters
(240/340). Such inverse mapping parameters (240/340) are input to
an inverse mapping module (440) to obtain the second normalized
digital cinema signal (235/335) of FIGS. 2 and 3.
[0048] The parameters for N and M can be first estimated from the
original data. By way of example, such parameters can be determined
iteratively using two routines "EstimateN" and "EstimateM" that
estimate N or M while the other is fixed: [0049] N=EstimateN[VDR,
SDR, M] [0050] M=EstimateM[VDR, SDR, N]
[0051] As mentioned above, N can be modeled as a piecewise
polynomial such as a piecewise cubic spline with typically 5-6
segments, while M can be a 3.times.3 matrix.
[0052] The routine EstimateN[ ] inverts the matrix M and applies
that to the SDR image or video. N is then the transformation
between VDR and M.sup.-1SDR. Similarly, the routine EstimateM[ ]
applies the non-linear transformation to VDR and then M is the
transformation between N[VDR] and SDR. Thus, there are two
estimation sequences, depending upon whether N or M is first
estimated: [0053] Sequence A: [0054] Set M.sup.0=I; (identity)
[0055] N.sup.0=EstimateN[VDR, SDR, M.sup.0]; [0056]
M.sup.1=EstimateM[VDR, SDR, N.sup.0]; [0057] N.sup.1=EstimateN[VDR,
SDR, M.sup.1]; [0058] M.sup.2=EstimateM[VDR, SDR, N.sup.1]; [0059]
iterate until a desired result is obtained, e.g. mathematical
convergence. [0060] Sequence B: [0061] Set N.sup.0=I; (identity)
[0062] M.degree.=EstimateM[VDR, SDR, N.sup.0]; [0063]
N.sup.1=EstimateN[VDR, SDR, M.sup.0]; [0064] M.sup.1=EstimateM[VDR,
SDR, N.sup.1]; [0065] N.sup.2=EstimateN[VDR, SDR, M.sup.1]; [0066]
iterate until a desired result is obtained, e.g. mathematical
convergence.
[0067] Other methods for determining N and M can also be used, such
as software optimization packages (e.g. MATLAB).
[0068] In most cases the parameters for N[ ] and M determined as
described above are sufficient. In some cases, the non-linearity
must be slightly modified due to the so-called "clipping" in the
SDR signal. FIG. 5 depicts a typical non-linearity when clipping
occurs at both the low and high limits of intensity. In order to
make N[ ] invertible, the clipping should be softened. FIG. 5
depicts varying degrees of softening. The greater the softening the
larger the differences between SDR and SDR* for the clipped pixels,
so the desire to maintain visual equivalence between SDR and SDR*
constrains this softening.
[0069] The methods and systems described in the present disclosure
may be implemented in hardware, software, firmware or any
combination thereof. Features described as blocks, modules or
components may be implemented together (e.g., in a logic device
such as an integrated logic device) or separately (e.g., as
separate connected logic devices). The software portion of the
methods of the present disclosure may comprise a computer-readable
medium which comprises instructions that, when executed, perform,
at least in part, the described methods. The computer-readable
medium may comprise, for example, a random access memory (RAM)
and/or a read-only memory (ROM). The instructions may be executed
by a processor (e.g., a digital signal processor (DSP), an
application specific integrated circuit (ASIC), or a field
programmable logic array (FPGA)).
[0070] All patents and publications mentioned in the specification
may be indicative of the levels of skill of those skilled in the
art to which the disclosure pertains. All references cited in this
disclosure are incorporated by reference to the same extent as if
each reference had been incorporated by reference in its entirety
individually.
[0071] It is to be understood that the disclosure is not limited to
particular methods or systems, which can, of course, vary. It is
also to be understood that the terminology used herein is for the
purpose of describing particular embodiments only, and is not
intended to be limiting. As used in this specification and the
appended claims, the singular forms "a," "an," and "the" include
plural referents unless the content clearly dictates otherwise. The
term "plurality" includes two or more referents unless the content
clearly dictates otherwise. Unless defined otherwise, all technical
and scientific terms used herein have the same meaning as commonly
understood by one of ordinary skill in the art to which the
disclosure pertains.
[0072] The examples set forth above are provided to give those of
ordinary skill in the art a complete disclosure and description of
how to make and use the embodiments of the backwards-compatible
delivery of digital cinema content with extended range and related
preprocessing and coding methods of the disclosure, and are not
intended to limit the scope of what the inventors regard as their
disclosure. Modifications of the above-described modes for carrying
out the disclosure may be used by persons of skill in the video
art, and are intended to be within the scope of the following
claims.
[0073] A number of embodiments of the disclosure have been
described. Nevertheless, it will be understood that various
modifications may be made without departing from the spirit and
scope of the present disclosure. Accordingly, other embodiments are
within the scope of the following claims.
* * * * *