U.S. patent application number 15/177203 was filed with the patent office on 2016-12-08 for method and apparatus of encoding or decoding coding units of a video content in a palette coding mode using an adaptive palette predictor.
This patent application is currently assigned to INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE. The applicant listed for this patent is INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE. Invention is credited to Yao-Jen CHANG, Ching-Chieh LIN, Chun-Lung LIN, Jih-Sheng TU.
Application Number | 20160360207 15/177203 |
Document ID | / |
Family ID | 56117547 |
Filed Date | 2016-12-08 |
United States Patent
Application |
20160360207 |
Kind Code |
A1 |
LIN; Ching-Chieh ; et
al. |
December 8, 2016 |
METHOD AND APPARATUS OF ENCODING OR DECODING CODING UNITS OF A
VIDEO CONTENT IN A PALETTE CODING MODE USING AN ADAPTIVE PALETTE
PREDICTOR
Abstract
A method of encoding or decoding coding units of a video content
in a palette coding mode using an adaptive palette predictor is
provided. The method includes adaptively determining a maximum size
of the adaptive palette predictor based on at least one of a
complexity of the video content and coding quality of the video
content; and encoding or decoding the coding units of the video
content in the palette coding mode using the adaptive palette
predictor while limiting the adaptive palette predictor that is
derived from all palette(s) of previously encoded or decoded coding
unit(s) of the video content within the maximum size determined in
the adaptively determining step. An apparatus of encoding or
decoding coding units of a video content in a palette coding mode
using an adaptive palette predictor is also provided.
Inventors: |
LIN; Ching-Chieh; (Taipei
City, TW) ; CHANG; Yao-Jen; (Hsinchu City, TW)
; LIN; Chun-Lung; (Taipei City, TW) ; TU;
Jih-Sheng; (Yilan City, TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE |
Zhudong Township |
|
TW |
|
|
Assignee: |
INDUSTRIAL TECHNOLOGY RESEARCH
INSTITUTE
Zhudong Township
TW
|
Family ID: |
56117547 |
Appl. No.: |
15/177203 |
Filed: |
June 8, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62172256 |
Jun 8, 2015 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 19/96 20141101;
H04N 19/154 20141101; H04N 19/102 20141101; H04N 19/182 20141101;
H04N 19/176 20141101; H04N 19/136 20141101; H04N 19/46 20141101;
H04N 19/186 20141101; H04N 19/124 20141101; H04N 19/61 20141101;
H04N 19/15 20141101; H04N 19/593 20141101; H04N 19/134
20141101 |
International
Class: |
H04N 19/15 20060101
H04N019/15; H04N 19/154 20060101 H04N019/154; H04N 19/176 20060101
H04N019/176; H04N 19/186 20060101 H04N019/186; H04N 19/182 20060101
H04N019/182; H04N 19/124 20060101 H04N019/124 |
Claims
1. A method of encoding or decoding coding units of a video content
in a palette coding mode using an adaptive palette predictor, the
method comprising: adaptively determining a maximum size of the
adaptive palette predictor based on at least one of a complexity of
the video content and coding quality of the video content; and
encoding or decoding the coding units of the video content in the
palette coding mode using the adaptive palette predictor while
limiting the adaptive palette predictor that is derived from all
palette(s) of previously encoded or decoded coding unit(s) of the
video content within the maximum size determined in the adaptively
determining step.
2. The method of claim 1, wherein a palette size of the coding
units is equal to or less than a palette maximum size, and the
adaptively determining step includes: generating a set of candidate
size(s) Ni (i=0, 1, . . . , V-1) of the adaptive palette predictor
based on at least V, M, and the palette maximum size, wherein V is
an integer and V.gtoreq.1, M.gtoreq.1, and Ni.gtoreq.the palette
maximum size and Ni is capped at an upper bound that is set based
on M and the palette maximum size if Ni is larger than the upper
bound; and adaptively selecting one candidate size from the set of
candidate size(s) Ni of the adaptive palette predictor, based on
the at least one of the complexity of the video content and the
coding quality of the video content, as a maximum size of the
adaptive palette predictor.
3. The method of claim 2, wherein when M is an integer, the upper
bound is set to be M* the palette maximum size.
4. The method of claim 2, wherein the step of generating the set of
candidate size(s) of the adaptive palette predictor is further
based on a delta palette predictor size.
5. The method of claim 4, wherein the step of generating the set of
candidate size(s) of the adaptive palette predictor is further
based on the palette size.
6. The method of claim 2, wherein a size of the coding unit is set
between a minimum coding unit size and a maximum coding unit size,
and the complexity of the video content is determined based on at
least one of the size of the coding unit of the video content, the
maximum coding unit size and the minimum coding unit size, and a
frame size of the video content.
7. The method of claim 6, wherein the complexity of the video
content is determined based on the size of the coding unit of the
video content, the maximum coding unit size and the minimum coding
unit size.
8. The method of claim 6, wherein the complexity of the video
content is the size of the coding unit of the video content or the
frame size of the video content, and the step of adaptively
selecting the one candidate size from the set of candidate size(s)
Ni of the adaptive palette predictor includes selecting from a
look-up table a size corresponding to the size of the coding unit
of the video content or the frame size of the video content.
9. The method of claim 2, wherein the coding quality is determined
based on at least one of a current quantization parameter of and a
maximum quantization parameter of a coding unit level, a slice
level, a frame level, a syntax (parameter set) parsing level, a
syntax (parameter set) generating level, a configuration files
parsing level, or a configuration files generating level of the
video content, and V.ltoreq.the maximum quantization parameter.
10. The method of claim 9, wherein the coding quality is the
current quantization parameter of the coding unit level, the slice
level, the frame level, the syntax (parameter set) parsing level,
the syntax (parameter set) generating level, the configuration
files parsing level, or the configuration files generating level of
the video content, and the step of adaptively selecting the one
candidate size from the set of candidate size(s) Ni of the adaptive
palette predictor includes selecting from a look-up table a size
corresponding to the current quantization parameter of the coding
unit level, the slice level, the frame level, the syntax (parameter
set) parsing level, the syntax (parameter set) generating level,
the configuration files parsing level, or the configuration files
generating level of the video content.
11. The method of claim 2, wherein the step of adaptively selecting
the one candidate size from the set of candidate size(s) Ni of the
adaptive palette predictor includes selecting from a look-up table
a size corresponding to either the complexity of the video content
or the coding quality of the video content.
12. The method of claim 2, wherein the step of generating the set
of candidate size(s) of the adaptive palette predictor is performed
at one of a configuration files generating level, a configuration
files parsing level, a syntax (parameter set) generating level, a
syntax (parameter set) parsing level, a frame level, a slice level,
and a coding unit level.
13. The method of claim 12, wherein the step of adaptively
selecting the one candidate size from the set of candidate size(s)
Ni of the adaptive palette predictor is performed at the same one
of the configuration files generating level, the configuration
files parsing level, the syntax (parameter set) generating level,
the syntax (parameter set) parsing level, the frame level, the
slice level, and the coding unit level.
14. The method of claim 12, wherein the step of adaptively
selecting the one candidate size from the set of candidate size(s)
Ni of the adaptive palette predictor is performed at another one of
the configuration files generating level, the configuration files
parsing level, the syntax (parameter set) generating level, the
syntax (parameter set) parsing level, the frame level, the slice
level, and the coding unit level.
15. The method of claim 1, wherein the step of encoding or decoding
the coding units of the video content in the palette coding mode
using the adaptive palette predictor while limiting the adaptive
palette predictor within the maximum size includes: encoding or
decoding a current coding unit of the video content in the palette
coding mode using the adaptive palette predictor that is derived
from all palette(s) of previously encoded or decoded coding unit(s)
of the video content; and updating the adaptive palette predictor
for a next coding unit from the adaptive palette predictor for the
current coding unit and the palette of the current coding unit
while limiting the adaptive palette predictor for the next coding
unit within the maximum size.
16. An apparatus for encoding or decoding coding units of a video
content in a palette coding mode using an adaptive palette
predictor, the apparatus comprising: a memory device storing the
adaptive palette predictor; an adaptive palette predictor size
determination unit configured to adaptively determine a maximum
size of the adaptive palette predictor based on at least one of a
complexity of the video content and coding quality of the video
content; and an encoder or decoder configured to encode or decode
the coding units of the video content in the palette coding mode
using the adaptive palette predictor stored in the memory device
while limiting the adaptive palette predictor that is derived from
all palette(s) of previously encoded or decoded coding unit(s) of
the video content within the maximum size provided by the adaptive
palette predictor size determination unit.
17. The apparatus of claim 16, wherein a palette size of the coding
units is equal to or less than a palette maximum size, and the
adaptive palette predictor size determination unit includes: a
candidate size generation unit configured to generate a set of
candidate size(s) Ni (i=0, 1, . . . , V-1) of the adaptive palette
predictor based on at least V, M, and the palette maximum size,
wherein V is an integer and V.gtoreq.1, M.gtoreq.1, and
Ni.gtoreq.the palette maximum size and Ni is capped at an upper
bound that is set based on M and the palette maximum size if Ni is
larger than the upper bound; and an adaptive palette predictor size
selection unit configured to adaptively select one candidate size
from the set of candidate size(s) Ni of the adaptive palette
predictor, based on the at least one of the complexity of the video
content and the coding quality of the video content, as the maximum
size of the adaptive palette predictor.
18. The apparatus of claim 17, wherein when M is an integer, the
upper bound is set to be M* the palette maximum size.
19. The apparatus of claim 17, wherein the candidate size
generation unit is configured to generate the set of candidate
size(s) of the adaptive palette predictor further based on a delta
palette predictor size.
20. The apparatus of claim 19, wherein the candidate size
generation unit is configured to generate the set of candidate
size(s) of the adaptive palette predictor further based on the
palette size.
21. The apparatus of claim 17, wherein a size of the coding unit is
set between a minimum coding unit size and a maximum coding unit
size, and the complexity of the video content is determined based
on at least one of the size of the coding unit of the video
content, the maximum coding unit size and the minimum coding unit
size, and a frame size of the video content.
22. The apparatus of claim 21, wherein the complexity of the video
content is determined based on the size of the coding unit of the
video content, the maximum coding unit size and the minimum coding
unit size.
23. The apparatus of claim 21, wherein the complexity of the video
content is the size of the coding unit of the video content or the
frame size of the video content, and the adaptive palette predictor
size selection unit is configured to adaptively select the one
candidate size from the set of candidate size(s) Ni of the adaptive
palette predictor by selecting from a look-up table a size
corresponding to the size of the coding unit of the video content
or the frame size of the video content.
24. The apparatus of claim 17, wherein the coding quality is
determined based on at least one of a current quantization
parameter of and a maximum quantization parameter of a coding unit
level, a slice level, a frame level, a syntax (parameter set)
parsing level, a syntax (parameter set) generating level, a
configuration files parsing level, or a configuration files
generating level of the video content, and V.ltoreq.the maximum
quantization parameter.
25. The apparatus of claim 24, wherein the coding quality is the
current quantization parameter of the coding unit level, the slice
level, the frame level, the syntax (parameter set) parsing level,
the syntax (parameter set) generating level, the configuration
files parsing level, or the configuration files generating level of
the video content, and the adaptive palette predictor size
selection unit is configured to adaptively select the one candidate
size from the set of candidate size(s) Ni of the adaptive palette
predictor by selecting from a look-up table a size corresponding to
the current quantization parameter of the coding unit level, the
slice level, the frame level, the syntax (parameter set) parsing
level, the syntax (parameter set) generating level, the
configuration files parsing level, or the configuration files
generating level of the video content.
26. The apparatus of claim 17, wherein the adaptive palette
predictor size selection unit is configured to adaptively select
the one candidate size from the set of candidate size(s) Ni of the
adaptive palette predictor by selecting from a look-up table a size
corresponding to either the complexity of the video content or the
coding quality of the video content.
27. The apparatus of claim 17, wherein the candidate size
generation unit is configured to generate the set of candidate
size(s) Ni of the adaptive palette predictor at one of a
configuration files generating level, a configuration files parsing
level, a syntax (parameter set) generating level, a syntax
(parameter set) parsing level, a frame level, a slice level, and a
coding unit level.
28. The apparatus of claim 27, wherein the adaptive palette
predictor size selection unit is configured to adaptively select
the one candidate size from the set of candidate size(s) Ni of the
adaptive palette predictor at the same one of the configuration
files generating level, the configuration files parsing level, the
syntax (parameter set) generating level, the syntax (parameter set)
parsing level, the frame level, the slice level, and the coding
unit level.
29. The apparatus of claim 27, wherein the adaptive palette
predictor size selection unit is configured to adaptively select
the one candidate size from the set of candidate size(s) Ni of the
adaptive palette predictor at another one of the configuration
files generating level, the configuration files parsing level, the
syntax (parameter set) generating level, the syntax (parameter set)
parsing level, the frame level, the slice level, and the coding
unit level.
30. The apparatus of claim 16, wherein the encoder or decoder is
further configured to encode or decode a current coding unit of the
video content in the palette coding mode using the adaptive palette
predictor that is derived from all palette(s) of previously encoded
or decoded coding unit(s) of the video content; and update the
adaptive palette predictor for a next coding unit from the adaptive
palette predictor for the current coding unit and the palette of
the current coding unit while limiting the adaptive palette
predictor for the next coding unit within the maximum size.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This nonprovisional application claims priority under 35
U.S.C. .sctn.119(e) on, and claims the benefit of, U.S. Provisional
Application No. 62/172,256, filed on Jun. 8, 2015, the entirety of
which is incorporated herein by reference.
TECHNICAL FIELD
[0002] The present invention relates to a method and apparatus of
encoding or decoding coding units of a video content in a palette
coding mode using an adaptive palette predictor with an adaptive
maximum size.
BACKGROUND
[0003] Technologies for coding non-camera-captured content videos
or screen content videos have received great interests lately due
to the rapid growth of application areas such as wireless displays,
remote computer desktop access, real-time screen sharing for
videoconferencing, cloud gaming, etc. Compared to camera-captured
content videos, which contain rich colors and complex patterns,
screen content videos contain a significant portion of
computer-rendered graphics and text with fewer number of colors and
the repetition of textual patterns.
[0004] For example, in a screen content image with text, a coding
block typically contains only the foreground text color and the
background color. Sometimes, the random patterns of text characters
and letters make it challenging for the current coding block to
find a matching block in the same or previously coded pictures. It
may also be challenging to utilize the directional local intra
prediction for efficient compression in this circumstance. Since
traditional intra and inter video coding tools were designed
primarily for camera-captured content videos, and screen content
videos have significantly different characterizes from
camera-captured content videos, these traditional intra and inter
video coding tools are less sufficient for screen content videos.
Therefore, it creates an urgent need for efficient coding of screen
content videos.
[0005] In response to the market demands, the ITU-T Video Coding
Expert Group and ISO/IEC Motion Picture Expert Group have jointly
launched a new standardization project, i.e., the High Efficiency
Video Coding (HEVC) extensions on screen content coding (SCC).
Several new video coding tools, including palette coding, have been
developed and adopted into HEVC SCC draft standard to efficiently
encode/decode screen content videos.
[0006] Palette coding is a major color-based prediction method.
Different from traditionally intra and inter prediction that mainly
removes redundancy between different coding units, palette coding
targets at the redundancy of repetitive pixel values/patterns
within the coding unit. In order to reduce the overhead of
transmitting the original value of the major colors, a palette
prediction was introduced in palette coding. In the current palette
coding mode, all pixels of a coding block are analyzed and
classified into a list of major colors, except for some rarely used
pixels that cannot be classified to any of the major colors, which
are classified into escape colors. Each major color is a
representative color which has high frequency of occurrence in the
coding block. For each palette coded coding unit (CU), a color
index table, i.e., a palette, is formed with each index entry
associated with one major color. All the pixels in the CU are
converted into corresponding indices, except the escape pixels with
the escape colors. FIG. 1 illustrates a simplified version of the
palette coding process.
[0007] Then, the encoder starts a checking process to check if each
of the index entries representing the major colors in the current
CU matches any of the major colors in the current palette
predictor. For each entry in the palette predictor, a flag (1:
used; 0: not used) is sent to signal whether or not this entry is
used in the current palette. If yes, this entry will be put in
front of the current palette. Therefore, the flags corresponding to
the entries of the current palette predictor are sent to signal
which one(s) of the major colors in the current palette predictor
is used in the current CU. For those entries in the current palette
but not in the palette predictor, the number of them and their
pixel (e.g., Y/Cb/Cr or R/G/B) values are signaled, and these
signaled new entries are put at the bottom of the current palette.
An example of the palette coding mode at the encoder side is
illustrated in FIG. 2. The current palette size is then calculated
as the number of reused palette entries plus the number of signaled
new palette entries.
[0008] At the decoder side, the decoder receives the flags
indicating which one(s) of the major colors in the current palette
predictor is used in the current CU. The decoder checks the current
palette predictor in order with the flags to determine which one(s)
of the major colors in the current palette predictor is used in the
current CU. The decoder also receives their pixel (e.g., Y/Cb/Cr or
R/G/B) values of the new palette entries not in the current palette
predictor. The decoder then generates a received palette for the CU
with index entries corresponding to the used major colors (with the
flag (1)) in the current palette predictor in front of the received
palette, followed by the index entries corresponding to new major
colors not in the current palette predictor. An example of the
palette coding mode at the decoder side is illustrated in FIG.
3.
[0009] After palette coding the current CU or decoding the current
palette-coded CU, the palette predictor is updated for the next CU
or palette-coded CU. This is done using the information of the
current palette. The entries (including the new entries) of the
current/received palette are put in front of the new palette
predictor, followed by those unused entries from the previous
palette predictor. The new palette predictor size is then
calculated as the size of the current palette plus the number of
unused palette entries. An example of the update of the palette
predictor is illustrated in FIG. 4.
[0010] However, in the current design, the maximum size of the
palette predictor can be assigned as an arbitrary positive number.
The lack of the limitation of the palette predictor maximum size
raises problems in implementation and the update process of the
palette predictor, because the decoder may need to prepare a buffer
with an unlimited size for hardware implementation of the palette
prediction, which is infeasible under the current technology. In
addition, the current palette coding mode only allows one fixed
maximum size of the palette predictor signaled at the SPS level
regardless of the complexity of the video or the coding quality.
The fixed palette predictor maximum size makes the palette coding
inefficient and ineffective, because it may not fit the need for
all different coding conditions and coding quality
requirements.
SUMMARY
[0011] Accordingly, it is an object of the present invention to
provide an adaptive maximum size of the palette predictor according
to the complexity of the video content and/or the coding quality of
the video content to improve the coding efficiency in a palette
coding mode.
[0012] According to a one embodiment of the present invention, a
method of encoding or decoding coding units of a video content in a
palette coding mode using an adaptive palette predictor is
provided. The method comprises: adaptively determining a maximum
size of the adaptive palette predictor based on at least one of a
complexity of the video content and coding quality of the video
content; and encoding or decoding the coding units of the video
content in the palette coding mode using the adaptive palette
predictor while limiting the adaptive palette predictor that is
derived from all palette(s) of previously encoded or decoded coding
unit(s) of the video content within the maximum size determined in
the adaptively determining step.
[0013] Furthermore, according to another embodiment of the present
invention, an apparatus for encoding or decoding coding units of a
video content in a palette coding mode using an adaptive palette
predictor is provided. The apparatus comprises: a memory device
storing the adaptive palette predictor; an adaptive palette
predictor size determination unit configured to adaptively
determine a maximum size of the adaptive palette predictor based on
at least one of a complexity of the video content and coding
quality of the video content; and an encoder or decoder configured
to encode or decode the coding units of the video content in the
palette coding mode using the adaptive palette predictor stored in
the memory device while limiting the adaptive palette predictor
that is derived from all palette(s) of previously encoded or
decoded coding unit(s) of the video content within the maximum size
provided by the adaptive palette predictor size determination
unit.
[0014] Further scope of applicability of the present invention will
become apparent from the detailed description given hereinafter.
However, it should be understood that the detailed description and
specific examples, while indicating preferred embodiments of the
invention, are given by way of illustration only, since various
changes and modifications within the spirit and scope of the
invention will become apparent to those skilled in the art from
this detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The present invention will become more fully understood from
the detailed description given hereinbelow and the accompanying
drawings which are given by way of illustration only, and thus are
not limitative of the present invention, and wherein:
[0016] FIG. 1 illustrates a simplified version of the palette
coding process;
[0017] FIG. 2 illustrates an example of the palette coding mode at
the encoder side;
[0018] FIG. 3 illustrates an example of the palette coding mode at
the decoder side;
[0019] FIG. 4 illustrates an example of the update of the palette
predictor;
[0020] FIG. 5 is a block diagram of an apparatus for encoding or
decoding coding units of a video content in a palette coding mode
using an adaptive palette predictor in accordance with an
embodiment of the present invention;
[0021] FIG. 6 is a block diagram of an adaptive palette predictor
size determination unit in accordance with an embodiment of the
present invention;
[0022] FIGS. 7A-1 through 7A-5, 7B and 7C-1 through 7C-2 illustrate
several embodiments of the candidate size generation unit as shown
in FIG. 6;
[0023] FIGS. 8A-8E illustrate several embodiments of the adaptive
palette predictor size selection unit as shown in FIG. 6; and
[0024] FIGS. 9A-9D illustrate some possible arrangements of the
candidate size generation process and the adaptive palette
predictor size selection process in the palette coding mode in
accordance with embodiments of the present invention.
DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS
[0025] The present invention will now be described in detail with
reference to the accompanying drawings, wherein the same reference
numerals will be used to identify the same or similar elements
throughout the several views. It should be noted that the drawings
should be viewed in the direction of orientation of the reference
numerals.
[0026] As mentioned, a palette predictor with a fixed palette
predictor maximum size may not fit the need for all different
coding conditions and coding quality requirements in the palette
coding mode. If the coding quality requirement is high, more major
colors are expected to be generated with fine classification.
Therefore, in order to have a good palette prediction, the maximum
size of the palette predictor should be set higher to contain more
major colors that have appeared in the previously decoded coding
units. On the other hand, when coding quality requirement is low,
less major colors are expected in the coding unit. Therefore, the
maximum size of the palette predictor should be set lower to reduce
the checking time of comparing index entries of the current palette
with the palette predictor and reduce the chance to transmit the
values of the unpredicted major colors, thereby improving the
coding efficiency. In addition, with regard to the video content
complexity, the major colors for the complex coding unit will
increase and require a larger sized palette predictor to improve
the hitting rate for coding efficiency. On the other hand, the
simple video content or coding unit may not need so many major
colors to represent. Therefore, the size of the palette predictors
can be reduced to eliminate the extra checking times and to reduce
the overhead of transmitting unused flags for the palette
predictor. Therefore, instead of only having one fixed maximum size
of the palette predictor, the maximum size of the palette predictor
should be adaptively changed according to the complexity of the
video content and the coding quality to enhance the coding
efficiency in the palette coding mode.
[0027] FIG. 5 is a block diagram of an apparatus for encoding or
decoding coding units of a video content in a palette coding mode
using an adaptive palette predictor in accordance with an
embodiment of the present invention. As illustrated in FIG. 5, the
apparatus 100 comprises a memory device 20 storing the adaptive
palette predictor 30; an adaptive palette predictor size
determination unit 60 configured to adaptively determine a maximum
size of the adaptive palette predictor 30 based on at least one of
a complexity of the video content and coding quality of the video
content; and an encoder or decoder 40 configured to encode or
decode the coding units of the video content in the palette coding
mode using the adaptive palette predictor 30 stored in the memory
device 20 while limiting the adaptive palette predictor that is
derived from all palette(s) of previously encoded or decoded coding
unit(s) of the video content within the maximum size provided by
the adaptive palette predictor size determination unit 60. In an
embodiment, the apparatus can be implemented by a processor with
firmware/software, logic circuits, and/or hardware encoder/decoder,
etc.
[0028] FIG. 6 is a block diagram of the adaptive palette predictor
size determination unit 60 in accordance with an embodiment of the
present invention. As shown in FIG. 6, the adaptive palette
predictor size determination unit 60 receives the following
parameters including the adjustment parameter V, the palette
maximum size (the palette size of the coding units is equal to or
less than the palette maximum size), M, and parameter(s)
representing the complexity of the video content and/or the coding
quality of the video content. In some embodiments, the adaptive
palette predictor size determination unit 60 may also the delta
palette predictor size and/or the palette size of the coding
units.
[0029] As shown in FIG. 6, the adaptive palette predictor size
determination unit 60 includes: a candidate size generation unit 70
configured to generate a set of candidate size(s) Ni (i=0, 1, . . .
, V-1) of the adaptive palette predictor 30 based on at least V, M,
and the palette maximum size, wherein V is an integer and
V.gtoreq.1, M.gtoreq.1, and Ni.gtoreq.the palette maximum size and
Ni is capped at an upper bound of the maximum size of the adaptive
palette predictor if Ni is larger than the upper bound; and an
adaptive palette predictor size selection unit 80 configured to
adaptively select one candidate size from the set of candidate
size(s) Ni of the adaptive palette predictor, based on the at least
one of the complexity of the video content and the coding quality
of the video content, as the maximum size of the adaptive palette
predictor. In an embodiment, the upper bound of the maximum size of
the adaptive palette predictor is set based on M and the palette
maximum size. In an embodiment, when M is an integer, the upper
bound is set to be M* the palette maximum size, and when M is not
an integer, the upper bound is set to be one of the following:
round (M* the palette maximum size), ceiling (M* the palette
maximum size), floor (M* the palette maximum size), round (M)* the
palette maximum size, ceiling (M)* the palette maximum size, and
floor (M)* the palette maximum size.
[0030] FIGS. 7A-1 through 7A-5, 7B and 7C-1 through 7C-2 illustrate
several embodiments of the candidate size generation unit 70 as
shown in FIG. 6.
[0031] In the embodiments shown in FIG. 7A-1 through 7A-5, each of
the candidate size generation units 70A-1, 70A-2, 70A-3, 70A-4, and
70A-5 is configured to generate the set of candidate size(s) of the
adaptive palette predictor 30 further based on the delta palette
predictor size and the palette size in addition to V, M, and the
palette maximum size.
[0032] In the embodiment shown in FIG. 7A-1, the candidate size
generation units 70A-1 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) ,
##EQU00001##
[0033] where
[0034] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
p=Palette Size+Delta Palette Predictor Size,
d = min ( B - Delta Palette Predictor Size , Delta Palette
Predictor Size ) V , ##EQU00002##
[0035] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0036] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0037] In the embodiment shown in FIG. 7A-2, the candidate size
generation units 70A-2 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) ,
##EQU00003##
[0038] where
[0039] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
p=Palette Size+Delta Palette Predictor Size,
d = min ( B - Delta Palette Predictor Size , Delta Palette
Predictor Size ) V , ##EQU00004##
[0040] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0041] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0042] In the embodiment shown in FIG. 7A-3, the candidate size
generation units 70A-3 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) ,
##EQU00005##
[0043] where
[0044] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
p=Palette Size+Delta Palette Predictor Size,
d = min ( B - Delta Palette Predictor Size , Delta Palette
Predictor Size ) V , ##EQU00006##
[0045] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0046] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0047] In the embodiment shown in FIG. 7A-4, the candidate size
generation units 70A-4 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) ,
##EQU00007##
[0048] where
[0049] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
p=Palette Size+Delta Palette Predictor Size,
d = Delta Palette Predictor Size V , ##EQU00008##
[0050] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0051] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0052] In the embodiment shown in FIG. 7A-5, the candidate size
generation units 70A-5 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N.sub.i=p=Palette Size+Delta Palette Predictor Size
[0053] where
[0054] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
[0055] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0056] 1.ltoreq.M'log.sub.2 (Transform Unit (TU) Maximum Size).
[0057] In other words, the set of candidate size(s) Ni is a fixed
output for i=0, 1, . . . , V-1.
[0058] In the embodiment shown in FIG. 7B, the candidate size
generation unit 70B is configured to generate the set of candidate
size(s) of the adaptive palette predictor 30 further based on the
delta palette predictor size in addition to V, M, and the palette
maximum size without further based on the palette size.
[0059] In the embodiment shown in FIG. 7B, the candidate size
generation units 70B generates a set of candidate size(s) Ni (i=0,
1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( i , V , M ) = ( M - 1 ) .times. Delta Palette Predictor
Size V .times. ( i - 1 ) + Palette Maximum Size , ##EQU00009##
[0060] where
[0061] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
[0062] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0063] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0064] In the embodiments shown in FIGS. 7C-1 and 7C-2, each of the
candidate size generation units 70C-1 and 70C-2 is configured to
generate the set of candidate size(s) of the adaptive palette
predictor 30 based on V, M, and the palette maximum size without
further based on the delta palette predictor size or the palette
size.
[0065] In the embodiment shown in FIG. 7C-1, the candidate size
generation units 70C-1 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
N i = g ( i , V , M ) = ( M - 1 ) .times. Palette Maximum Size V
.times. ( i + 1 ) + Palette Maximum Size , ##EQU00010##
[0066] where
[0067] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B,
B=M*Palette Maximum Size (i.e., the upper bound of the maximum size
of the adaptive palette predictor, and M is an integer in this
embodiment),
[0068] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0069] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0070] In the embodiment shown in FIG. 7C-2, the candidate size
generation units 70C-2 generates a set of candidate size(s) Ni
(i=0, 1, . . . , V-1) of the adaptive palette predictor 30 as
follows:
[0071] N.sub.i is determined based on a lookup table.
[0072] where
[0073] Palette Maximum Size.ltoreq.N.sub.i.ltoreq.B (i.e., the
upper bound of the maximum size of the adaptive palette predictor,
and M is an integer in this embodiment),
B=M*Palette Maximum Size,
[0074] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive integer,
and
[0075] 1.ltoreq.M.ltoreq.log.sub.2 (Transform Unit (TU) Maximum
Size).
[0076] Below is an example of the lookup table for V=4
TABLE-US-00001 Palette Predictor Maximum Size N.sub.3 128 N.sub.2
96 N.sub.1 86 N.sub.0 64
[0077] FIGS. 8A-8E illustrate several embodiments of the adaptive
palette predictor size selection unit 80 as shown in FIG. 6.
[0078] In the embodiments shown in FIGS. 8A and 8B, each of the
adaptive palette predictor size selection units 80A and 80B is
configured to adaptively select one candidate size from the set of
candidate size(s) Ni of the adaptive palette predictor 30, based on
the coding quality of the video content, as the maximum size of the
adaptive palette predictor 30.
[0079] In an embodiment, the coding quality is determined based on
at least one of a current quantization parameter of and a maximum
quantization parameter of a coding unit level, a slice level, a
frame level, a syntax (parameter set) parsing level (e.g.
SPS/PPS/VPS/SEI parsing level), a syntax (parameter set) generating
level (e.g. SPS/PPS/VPS/SEI generating level), a configuration
files parsing level, or a configuration files generating level of
the video content, and V.ltoreq.the maximum quantization parameter
(qp).
[0080] For example, in the embodiment shown in FIG. 8A, the
adaptive palette predictor size selection units 80A adaptively
selects one candidate size from the set of candidate size(s) Ni of
the adaptive palette predictor 30 as follows:
[0081] Palette Predictor Maximum Size=Ns,
[0082] where
[0083] qp: the qp in the configuration files generating level (in
the encoder), the configuration files parsing level (in the
encoder), the syntax (parameter set) generating level (in the
encoder), the syntax (parameter set) parsing level (in the
decoder), the frame level, the slice level or the coding unit
level,
r = Maximum qp + 1 V , s = Maximum qp - qp r , ##EQU00011##
and
[0084] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive
integer.
[0085] In an embodiment, the coding quality is the current
quantization parameter (qp) of the coding unit level, the slice
level, the frame level, the syntax (parameter set) parsing level
(e.g. SPS/PPS/VPS/SEI parsing level), the syntax (parameter set)
generating level (e.g. SPS/PPS/VPS/SEI parameter set generating
level), the configuration files parsing level, or the configuration
files generating level of the video content, and the adaptive
palette predictor size selection unit is configured to adaptively
select the one candidate size from the set of candidate size(s) Ni
of the adaptive palette predictor by selecting from a look-up table
a size corresponding to the current quantization parameter of the
coding unit level, the slice level, the frame level, the syntax
(parameter set) parsing level, the syntax (parameter set)
generating level, the configuration files parsing level, or the
configuration files generating level of the video content.
[0086] For example, in the embodiment shown in FIG. 8B, the
adaptive palette predictor size selection units 80B adaptively
selects one candidate size from the set of candidate size(s) Ni of
the adaptive palette predictor 30 as follows:
[0087] qp: the qp in the configuration files generating level (in
the encoder), the configuration files parsing level (in the
encoder), the syntax (parameter set) generating level (in the
encoder), the syntax (parameter set) parsing level (in the
decoder), the frame level, the slice level or the coding unit
level
[0088] Palette Predictor Maximum Size: Determined by Lookup Table
mapping between qp and the candidate size(s) Ni.
[0089] Below is an example of the lookup table for V=3 with a
mapping between different ranges of qp and the candidate sizes
N.sub.0-N.sub.2.
TABLE-US-00002 qp Palette Predictor Maximum Size 0 .ltoreq. qp
.ltoreq. 16 N.sub.2 (106) 17 .ltoreq. qp .ltoreq. 34 N.sub.1 (96)
35 .ltoreq. qp .ltoreq. 51 N.sub.0 (86)
[0090] In the embodiments shown in FIGS. 8C, 8D and 8E, each of the
adaptive palette predictor size selection units 80C, 80D and 80E is
configured to adaptively select one candidate size from the set of
candidate size(s) Ni of the adaptive palette predictor 30, based on
the complexity of the video content, as the maximum size of the
adaptive palette predictor 30.
[0091] In an embodiment, a size of the coding unit is set between a
minimum coding unit size and a maximum coding unit size, and the
complexity of the video content is determined based on at least one
of the size of the coding unit of the video content, the maximum
coding unit size and the minimum coding unit size, and a frame size
of the video content.
[0092] In an embodiment, the complexity of the video content is
determined based on the size of the coding unit of the video
content, the maximum coding unit size and the minimum coding unit
size.
[0093] For example, in the embodiment shown in FIG. 8C, the
adaptive palette predictor size selection units 80C adaptively
selects one candidate size from the set of candidate size(s) Ni of
the adaptive palette predictor 30 as follows:
[0094] Palette Predictor Maximum Size=Ns,
[0095] where
r = Log 2 ( CU MAX ) - Log 2 ( CU Min ) + 1 V ##EQU00012## s = Log
2 ( CU ) - Log 2 ( CU Min ) r ##EQU00012.2##
[0096] 1.ltoreq.V.ltoreq.Maximum qp, and V is a positive
integer.
[0097] In an embodiment, the complexity of the video content is the
size of the coding unit of the video content, and the adaptive
palette predictor size selection unit is configured to adaptively
select the one candidate size from the set of candidate size(s) Ni
of the adaptive palette predictor by selecting from a look-up table
a size corresponding to the size of the coding unit of the video
content.
[0098] For example, in the embodiment shown in FIG. 8D, the
adaptive palette predictor size selection units 80D adaptively
selects one candidate size from the set of candidate size(s) Ni of
the adaptive palette predictor 30 as follows:
[0099] Palette Predictor Maximum Size: Determined by Lookup Table
mapping between the CU size and the candidate size(s) Ni.
[0100] Below is an example of the lookup table for V=4 with a
mapping between different CU sizes and the candidate sizes
N.sub.0-N.sub.3.
TABLE-US-00003 CU size Palette Predictor Maximum Size 64 N.sub.3
(127) 32 N.sub.2 (64) 16 N.sub.1 (32) 8 N.sub.0 (16)
[0101] In an embodiment, the complexity of the video content is the
frame size of the video content, and the adaptive palette predictor
size selection unit is configured to adaptively select the one
candidate size from the set of candidate size(s) Ni of the adaptive
palette predictor by selecting from a look-up table a size
corresponding to the frame size of the video content.
[0102] For example, in the embodiment shown in FIG. 8E, the
adaptive palette predictor size selection units 80E adaptively
selects one candidate size from the set of candidate size(s) Ni of
the adaptive palette predictor 30 as follows:
[0103] Palette Predictor Maximum Size: Determined by Lookup Table
mapping between the CU size and the candidate size(s) Ni.
[0104] Below is an example of the lookup table for V=3 with a
mapping between different frame sizes and the candidate sizes
N.sub.0-N.sub.2.
TABLE-US-00004 Frame size Palette Predictor Maximum Size 8K N.sub.2
(511) 4K N.sub.1 (255) Full HD N.sub.0 (127)
[0105] In an embodiment, the encoder or decoder 40 is further
configured to encode or decode a current coding unit of the video
content in the palette coding mode using the adaptive palette
predictor 30 that is derived from all palette(s) of previously
encoded or decoded coding unit(s) of the video content; and update
the adaptive palette predictor 30 for a next coding unit from the
adaptive palette predictor 30 for the current coding unit and the
palette of the current coding unit while limiting the adaptive
palette predictor 30 for the next coding unit within the maximum
size.
[0106] For illustration purpose, the following examples use the
candidate size generation unit 70A-1 shown in FIG. 7A-1 and the
adaptive palette predictor size selection unit 80A shown in FIG. 8A
to illustrate how a set of candidate size(s) Ni (i=0, 1, . . . ,
V-1) of the adaptive palette predictor is generated and how the
maximum size of the adaptive palette predictor is selected from the
set of candidate size(s) Ni.
EXAMPLE 1
[0107] V=1, M=2, Palette Maximum Size=Palette Size=Delta Palette
Predictor Size=64; Maximum qp=52, qp=any value.ltoreq.Maximum
qp.
[0108] In this example, since V=1, in the candidate size generation
process, the candidate size generation unit 70A-1 will generate
only one candidate size, i.e., N.sub.0, of the adaptive palette
predictor 30.
r = Maximum qp + 1 V = 52 + 1 1 = 53 ##EQU00013## s = Maximum qp -
qp r = 52 - qp 53 = 0 ##EQU00013.2##
[0109] For the adaptive palette predictor size selection process,
since there is only one candidate size N.sub.0 (128), the adaptive
palette predictor size selection unit 80A will select N.sub.0 (128)
as the maximum size of the adaptive palette predictor 30.
[0110] In addition, the same result will be obtained as
follows:
N 0 = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) = 64 + 64
+ min ( 2 .times. 64 - 64 , 64 ) 1 .times. ( 0 - 1 2 + 1 ) = 128 +
64 .times. 0 = 128. ##EQU00014##
[0111] Palette Predictor Maximum Size=N.sub.S=N.sub.0.
[0112] In addition, since N.sub.0 (128) is not larger than B (i.e.,
the upper bound of the maximum size of the adaptive palette
predictor)=M* Palette Maximum Size (2*64)=128, N.sub.0 (128) will
be selected as the maximum size of the adaptive palette predictor
30.
EXAMPLE 2
[0113] V=3, M=2, Palette Maximum Size=Palette Size=64; Delta
Palette Predictor Size=32; Maximum qp=52, qp=15.
[0114] In this example, since V=3, in the candidate size generation
process, the candidate size generation unit 70A-1 will generate
three candidate sizes, i.e., N.sub.0, N.sub.1 and N.sub.2, of the
adaptive palette predictor 30.
N 0 = g ( p , d , i , V ) = p + d .times. ( i - V 2 + 1 ) = 64 + 32
+ min ( 2 .times. 64 - 32 , 32 ) 3 .times. ( 0 - 3 2 + 1 ) = 96 +
10 .times. ( - 1 ) = 86. ##EQU00015## N 1 = g ( p , d , i , V ) = p
+ d .times. ( i - V 2 + 1 ) = 64 + 32 + min ( 2 .times. 64 - 32 ,
32 ) 3 .times. ( 1 - 3 2 + 1 ) = 96 + 10 .times. ( 0 ) = 96.
##EQU00015.2## N 2 = g ( p , d , i , V ) = p + d .times. ( i - V 2
+ 1 ) = 64 + 32 + min ( 2 .times. 64 - 32 , 32 ) 3 .times. ( 2 - 3
2 + 1 ) = 96 + 10 .times. ( 1 ) = 106. ##EQU00015.3##
[0115] For the adaptive palette predictor size selection process,
the adaptive palette predictor size selection unit 80A will select
one of N.sub.0 (86), N.sub.1 (96) and N.sub.2 (106) as the maximum
size of the adaptive palette predictor 30 as follows:
r = Maximum qp + 1 V = 52 + 1 3 = 18 ##EQU00016## s = Maximum qp -
qp r = 52 - 15 18 = 2 ##EQU00016.2##
[0116] Palette Predictor Maximum Size=N.sub.S=N.sub.2 (106).
[0117] In addition, since N.sub.2 (106) is not larger than B (i.e.,
the upper bound of the maximum size of the adaptive palette
predictor)=M* Palette Maximum Size (2*64)=128, N.sub.2 (106) will
not be capped by B and will be selected as the maximum size of the
adaptive palette predictor 30.
[0118] In this example, the selecting process can also be
implemented by a lookup table (LUT) as follows when applying all
possible qp to the above equations:
TABLE-US-00005 qp Palette Predictor Maximum Size 0 .ltoreq. qp
.ltoreq. 16 N.sub.2 (106) 17 .ltoreq. qp .ltoreq. 34 N.sub.1 (96)
35 .ltoreq. qp .ltoreq. 52 N.sub.0 (86)
[0119] FIGS. 9A-9D illustrate some possible arrangements of the
candidate size generation process and the adaptive palette
predictor size selection process in the palette coding mode in
accordance with embodiments of the present invention.
[0120] In an embodiment, the candidate size generation process that
is performed by the candidate size generation unit to generate the
set of candidate size(s) of the adaptive palette predictor can be
performed at one of the following levels: the configuration files
parsing level for the encoder, and the configuration files
generating level for the encoder, the syntax (parameter set)
parsing level (e.g. SPS/PPS/VPS/SEI parsing level) for the decoder,
the syntax (parameter set) generating level (e.g. SPS/PPS/VPS/SEI
parameter set generating level) for the encoder, the frame level,
the slice level, and the coding unit level, and the adaptive
palette predictor size selection process that is performed by the
adaptive palette predictor size selection unit to adaptively select
one candidate size from the set of candidate size(s) of the
adaptive palette predictor can also be performed at one of the
above levels.
[0121] In an embodiment, both the candidate size generation process
and the adaptive palette predictor size selection process are
performed at the same level. For example, as embodied in FIG. 9A,
both the candidate size generation process and the adaptive palette
predictor size selection process are performed at the SPS parsing
level. As embodied in FIG. 9B, both the candidate size generation
process and the adaptive palette predictor size selection process
are performed at the frame parsing level. Similarly, both the
candidate size generation process and the adaptive palette
predictor size selection process can be performed at the slice
level, or at the coding unit level.
[0122] In another embodiment, the candidate size generation process
and the adaptive palette predictor size selection process are
performed at different levels. For example, as embodied in FIG. 9C,
the candidate size generation process is performed at the SPS
parsing level, and the adaptive palette predictor size selection
process is performed at the slice level. As embodied in FIG. 9D,
the candidate size generation process is performed at the SPS
parsing level, and the adaptive palette predictor size selection
process is performed at the coding unit level.
[0123] In brief, when the candidate size generation process is
performed in one of the following levels: the configuration files
parsing level for the encoder, and the configuration files
generating level for the encoder, the syntax (parameter set)
parsing level (e.g. SPS/PPS/VPS/SEI parsing level) for the decoder,
the syntax (parameter set) generating level (e.g. SPS/PPS/VPS/SEI
parameter set generating level) for the encoder, the frame level,
the slice level, and the coding unit level, the adaptive palette
predictor size selection process can be performed at the same level
as the candidate size generation process or another level
therebelow. For example, when the candidate size generation process
is performed in the SPS parsing level, the adaptive palette
predictor size selection process can be performed at the SPS
parsing level, the frame level, the slice level, or the coding unit
level; when the candidate size generation process is performed in
the SPS frame level, the adaptive palette predictor size selection
process can be performed at the frame level, the slice level, or
the coding unit level, but not at the SPS parsing level, because it
is higher than the frame level.
[0124] As illustrated above, since the coding units are
encoded/decoded in the palette coding mode using an adaptive
palette predictor with an adaptive maximum size based on the
complexity of the video content and/or the coding quality of the
video content, the coding efficiency will be improved/enhanced,
i.e., higher hitting rate on the palette predictor, lower checking
time for comparing index entries of the palette and the palette
predictor, smaller memory/buffer size, lower overhead for
transmitting the flags and/or unpredicted major color values.
[0125] The invention being thus described, it will be obvious that
the same may be varied in many ways. Such variations are not to be
regarded as a departure from the spirit and scope of the invention,
and all such modifications as would be obvious to one skilled in
the art are intended to be included within the scope of the
following claims.
* * * * *