U.S. patent application number 11/987639 was filed with the patent office on 2008-07-03 for entropy deficiency based image.
This patent application is currently assigned to Human Monitoring Ltd.. Invention is credited to Ira Dvir, Nitzan Rabinowitz.
Application Number | 20080159387 11/987639 |
Document ID | / |
Family ID | 39583950 |
Filed Date | 2008-07-03 |
United States Patent
Application |
20080159387 |
Kind Code |
A1 |
Dvir; Ira ; et al. |
July 3, 2008 |
Entropy deficiency based image
Abstract
A method for obtaining a quantization factor for image
compression by quantization of coefficients of a transformation of
an image or part thereof, comprising determining an entropy-related
sensitivity to quantization of at least a part of the image; and
determining a quantization factor based on the entropy-related
sensitivity.
Inventors: |
Dvir; Ira; (Rishon-LeZion,
IL) ; Rabinowitz; Nitzan; (Ramat-HaSharon,
IL) |
Correspondence
Address: |
Martin D. Moynihan;PRTSI, Inc.
P.O.Box 16446
Arlington
VA
22215
US
|
Assignee: |
Human Monitoring Ltd.
Givat HaShlosha
IL
|
Family ID: |
39583950 |
Appl. No.: |
11/987639 |
Filed: |
December 3, 2007 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60878062 |
Jan 3, 2007 |
|
|
|
60878063 |
Jan 3, 2007 |
|
|
|
Current U.S.
Class: |
375/240.03 ;
375/E7.139; 382/251 |
Current CPC
Class: |
H04N 19/17 20141101;
H04N 19/44 20141101; H04N 19/187 20141101; H04N 19/119 20141101;
H04N 19/126 20141101; H04N 19/70 20141101; H04N 19/194 20141101;
H04N 19/60 20141101; H04N 19/14 20141101; H04N 19/423 20141101;
H04N 19/61 20141101; H04N 19/176 20141101 |
Class at
Publication: |
375/240.03 ;
382/251; 375/E07.139 |
International
Class: |
H04B 1/66 20060101
H04B001/66 |
Claims
1. A method for obtaining a quantization factor for image
compression by quantization of coefficients of a transformation of
the image or part thereof, comprising: (a) determining an
entropy-related sensitivity to quantization of at least a part of
the image; and (b) determining a quantization factor based on the
entropy-related sensitivity.
2. A method according to claim 1, wherein determining a sensitivity
to quantization comprises determining a change in entropy of a
quantized at least a part of the image relative to the entropy of
the at least part of the image, or a derivation thereof.
3. A method according to claim 2, wherein the derivation comprises
the ratio between the entropy of a quantized at least a part of an
image and the entropy of the at least part of the image (ED).
4. A method according to claim 3, comprising determining a
threshold of ED above which the visual quality of a compressed at
least a part of an image is insensitive, at least approximately, to
ED.
5. A method according to claim 4, wherein the threshold is
independent of the image.
6. A method according to claim 1, comprising determining a
threshold of a complexity quantification of the at least a part of
an image above which the visual quality of a compressed at least a
part of an image is insensitive, at least approximately, to the
complexity quantification.
7. A method according to claim 6, wherein the threshold is
independent of the image.
8. A method according to claim 1, wherein determining a
quantization factor based on the entropy-related sensitivity
comprises a linear function of the entropy-related sensitivity.
9. A method according to claim 1, wherein determining a
quantization factor comprises determination according to a range of
quantization factors.
10. A method according to claim 9, wherein determination according
to a range of quantization factors comprises a linear function of
the rounded range of quantization factors.
11. A method according to claim 10, wherein the quantization factor
is adjusted according to on at least one of the image target
bit-rate, the image target compression ratio, the target visual
quality of the compressed image, or a combination thereof.
12. A method according to claim 9, wherein the range of
quantization factors is based on at least one of the image target
bit-rate, the image target compression ratio, the target visual
quality of the compressed image, or a combination thereof.
13. A method according to claim 1, wherein the at least part of an
image comprises is arbitrary.
14. A method according to claim 1, wherein the at least part of the
image comprises a difference between a decompression of a
previously compressed part of the image and the original part of
the image.
15. A method for evaluation of entropy-related sensitivity to
quantization of an image or part thereof to obtain a quantization
factor for image compression by quantization of coefficients of a
transformation of the image or part thereof, comprising: (a)
determining a function of entropy-related quantification with
respect to a complexity quantification of at least a part of the
image; and (b) determining a quantization factor based on the
function.
16. A method according to claim 15, wherein the entropy-related
quantification comprises, at least approximately, a ratio between
the entropy of a quantized at least a part of an image and the
entropy of the at least part of the image (ED).
17. A method according to claim 15, wherein a quantification of the
complexity comprises a standard deviation of the at least part of
the image.
18. A method according to claim 15, wherein the function comprises
an exponential function of the complexity quantification.
19. A method according to claim 18, wherein the function is, at
least approximately, independent of the image.
20. A method according to claim 18, wherein the function comprises
an approximation of an exponential function.
21. A method according to claim 15, wherein determining a
quantization factor comprises determination according to a range of
quantization factors.
22. A method according to claim 21, wherein the range of
quantization factors is based on at least one of the image target
bit-rate, the image target compression ratio, the target visual
quality of the compressed image, or a combination thereof.
23. A method according to claim 16, comprising determining,
according to at least one of the curvature or the slope of the
function, a threshold of ED above which the visual quality of a
compressed at least a part of an image is insensitive, at least
approximately, to ED.
24. A method according to claim 15, comprising determining,
according to at least one of the curvature or the slope of the
function, a threshold of a complexity quantification of the at
least a part of an image above which the visual quality of a
compressed at least a part of an image is insensitive, at least
approximately, to the complexity quantification.
25. Apparatus configured to carry out the method of claim 1.
Description
RELATED APPLICATIONS
[0001] The present application claims the benefit under 35 USC
119(e) of U.S. Provisional Patent Application No. 60/878,062 filed
on Jan. 3, 2007 entitled "A SYSTEM AND APPARATUS FOR ENTROPY
DEFICIENCY DCT BASED COMPRESSION-DECOMPRESSION OF IMAGES", and U.S.
Provisional Patent Application No. 60/878,063 filed on Jan. 3, 2007
entitled "A SYSTEM AND APPARATUS FOR COMPRESSING HIGH RESOLUTION
STILL IMAGES OVER LOWER RESOLUTION VIDEO ENCODERS, FOR HANDSETS,
CAMERAS, CAMCORDERS, AND CAMERA EQUIPPED PORTABLE MEDIA PLAYERS";
the present application relates also to U.S. patent application
Ser. No. 11/882,811 filed on Aug. 6, 2007 entitled "COMPRESSING
HIGH RESOLUTION IMAGES IN A LOW RESOLUTION VIDEO". The disclosures
of the above-mentioned applications are incorporated herein by
reference.
FIELD OF THE INVENTION
[0002] This invention relates to image compression. Some
embodiments relate to methods for image compression responsive to
the sensitivity to quantization of an image or part thereof.
BACKGROUND OF THE INVENTION
[0003] Image compression is an application frequently employed in
cameras, mobile handsets or personal computers, usually with
respect to storage or transmission capacity.
[0004] While a good compression ratio is desirable for reducing
storage space or communications rate, it is also desirable that the
compressed image provides a good, or at least sufficient, visual
quality. Generally there is a tradeoff between the compression
ratio and the visual quality of the compressed image.
[0005] Some contemporary image compression techniques are based on
quantization, i.e., a reduction of the range of values the image or
its transformation comprises, allowing an effective compression.
For example, JPEG and MPEG utilize a quantization of the
coefficients of a DCT transform, JPEG2000 utilizes a quantization
of wavelet transforms. Therefore, with missing or reduced elements
the quality of a compressed image is inferior, at least to a
certain extent, relative to the original image.
[0006] Beyond a human judgment, some metrics were devised to assess
the quality of a compressed image. These metrics typically relate
to a statistical deviation between the original image and the
reconstructed image, for example, the commonly used PSNR
(peak-signal-to-noise-ratio). Recently some more complicated
metrics were developed, for example VQM (visual quality measures)
and SSIM (structural similarity). Basically, all these metrics are
evaluated after the compression.
[0007] Some approaches for image compression that try to improve
the visual quality of the compressed image are known. For example,
using variable sizes of pixels (e.g. U.S. Pat. No. 5,107,345), or
using different levels of quantization to edges and surfaces (e.g.
U.S. Pat. No. 5,793,892), or scalable DCT based compression schemes
(e.g. U.S. Pat. No. 6,826,232, U.S. Pat. No. 7,020,342, U.S. Pat.
No. 6,853,318), the disclosures of all of which patents are
incorporated herein by reference.
SUMMARY OF THE INVENTION
[0008] A broad aspect of exemplary embodiments of the invention
relates to a method for image compression by quantization that
achieves high compression ratio while maintaining good visual
quality which, at least typically, may be better than other
contemporary methods utilized by JPEG or MPEG, AVC or a video
inter-intra compression.
[0009] In exemplary embodiments of the invention, the high
compression ratio is obtained by high quantization of regions with
high complexity without sacrificing substantial details, and on the
other hand, using low quantification for low complexity regions,
preserving the gradually varying shades (such as faces, sky, walls,
etc.).
[0010] In exemplary embodiments of the invention, the quantization
is based on measure of the responsiveness, or sensitivity, of a
group of pixels to quantization.
[0011] In exemplary embodiments of the invention, the
responsiveness to quantization of a group of pixels is defined as
the change in entropy of a quantized group of pixel relative to the
entropy of the original pixels, or a derivation thereof.
Preferably, without limitation, the group of pixels is quantized by
a small factor relative to the range of the pixels values. The
relative change in entropy due to such a small quantization
resembles a differential of the entropy with respect to
quantization.
[0012] It was found, quite unexpectedly, that a plot of the
responsiveness to quantization of a group of pixels with respect to
a complexity measure of the group of pixels (such as the standard
mean deviation) exhibits a characteristic distribution.
Additionally, using over a thousand examples, it was found that the
plot resembles and approximates an exponential function.
Additionally, it was found that the exponential function is
characterized by identical or similar parameters. Therefore, in
exemplary embodiments of the invention, the responsiveness of a
group of pixels can be determined from the function that expresses
the responsiveness with respect to the complexity of the group of
pixels.
[0013] In exemplary embodiments of the invention, the image
compression is achieved by quantization of the coefficients of a
transformation of the group of pixels, such as the coefficients of
a DCT transformation as typically used in temporal and spatial
compression, for example, in JPEG, MPEG and AVC.
[0014] In exemplary embodiments of the invention, the method for
determining the quantization factor by responsiveness to
quantization applies as well to a difference between a
reconstructed compressed group of pixels and the original one.
[0015] In the specifications and claims, unless otherwise
specified, and without limiting the generality, the term `pixel`
denotes a visual pixel, or a derived element of a pixel (such as a
difference between a reconstructed compressed group of pixels and
the original one).
[0016] An aspect of exemplary embodiments of the invention relates
to a method for image compression by quantization factors that are
based on the responsiveness to quantization of group of pixels.
Such group of pixels may be as typically used in the MPEG and AVC
standards of 16.times.16 or 8.times.8 or 16.times.8 or 8.times.16
or 4.times.8 or 8.times.4 or 4.times.4 pixels.
[0017] In exemplary embodiments of the invention, the quantization
is adapted to a desired bit-rate, or the image target compression
ratio, or the compressed image quality, or a combination
thereof.
[0018] In exemplary embodiments of the invention, the quantization
factor is determined, according to the responsiveness, within a
range of quantization factors. Optionally, the range of
quantization factors is determined subject to requirements and/or
constraints of an application or a usage of the compressed image.
For example, the target bit-rate, or the image target compression
ratio, or the compressed image quality, or a combination
thereof.
[0019] In exemplary embodiments of the invention, based on the
curvature and/or the slope of the function of the responsiveness
with respect to complexity of a group of pixels, at least two
characteristic regions are identified in the function.
[0020] In exemplary embodiments of the invention, in one region
corresponding to low complexity and/or where the curve is slightly
curved and/or diagonal, the quantization is determined according to
the responsiveness such that the pixels would compress by the
minimum allowable, at least approximately, quantization factor and
preserve, at least approximately, the visual quality of the low
complexity pixels.
[0021] In exemplary embodiments of the invention, in a second
region corresponding to high complexity and/or where the curve is
approximately asymptotic and/or approximately horizontal, the
quantization factor is practically insensitive to the complexity;
that is, the visual quality of the compressed image is, at least
approximately, not affected by using a larger quantization than the
one derived from the function.
[0022] In exemplary embodiments of the invention, a threshold value
where the insensitivity region begins can be determined.
Optionally, the threshold is constant, at least approximately, for
any image or part thereof, without impairing, at least
approximately, the visual quality of the image compressed by a
quantization factor derived from the function according to the
constant threshold.
[0023] In the discussions that follow, unless otherwise specified,
the terms `image partition`, or `partition` denote a group of
pixels of an image.
[0024] According to an aspect of some embodiments of the present
invention there is provided a method for obtaining a quantization
factor for image compression by quantization of coefficients of a
transformation of the image or part thereof, comprising:
[0025] (a) determining an entropy-related sensitivity to
quantization of at least a part of the image; and
[0026] (b) determining a quantization factor based on the
entropy-related sensitivity.
[0027] According to some embodiments of the invention, determining
a sensitivity to quantization comprises determining a change in
entropy of a quantized at least a part of the image relative to the
entropy of the at least part of the image, or a derivation
thereof.
[0028] According to some embodiments of the invention, the
derivation comprises the ratio between the entropy of a quantized
at least a part of an image and the entropy of the at least part of
the image (ED).
[0029] According to some embodiments of the invention, the method
comprises determining a threshold of ED above which the visual
quality of a compressed at least a part of an image is insensitive,
at least approximately, to ED.
[0030] According to some embodiments of the invention, the
threshold is independent of the image.
[0031] According to some embodiments of the invention, the method
comprises determining a threshold of a complexity quantification of
the at least a part of an image above which the visual quality of a
compressed at least a part of an image is insensitive, at least
approximately, to the complexity quantification. According to some
embodiments of the invention, the threshold is independent of the
image.
[0032] According to some embodiments of the invention, determining
a quantization factor based on the entropy-related sensitivity
comprises a linear function of the entropy-related sensitivity.
[0033] According to some embodiments of the invention, determining
a quantization factor comprises determination according to a range
of quantization factors.
[0034] According to some embodiments of the invention,
determination according to a range of quantization factors
comprises a linear function of the rounded range of quantization
factors.
[0035] According to some embodiments of the invention, the
quantization factor is adjusted according to on at least one of the
image target bit-rate, the image target compression ratio, the
target visual quality of the compressed image, or a combination
thereof.
[0036] According to some embodiments of the invention, the range of
quantization factors is based on at least one of the image target
bit-rate, the image target compression ratio, the target visual
quality of the compressed image, or a combination thereof.
[0037] According to some embodiments of the invention, the at least
part of an image comprises is arbitrary.
[0038] According to some embodiments of the invention, the at least
part of the image comprises a difference between a decompression of
a previously compressed part of the image and the original part of
the image.
[0039] According to an aspect of some embodiments of the present
invention there is provided a method for evaluation of
entropy-related sensitivity to quantization of an image or part
thereof to obtain a quantization factor for image compression by
quantization of coefficients of a transformation of the image or
part thereof, comprising:
[0040] (a) determining a function of entropy-related quantification
with respect to a complexity quantification of at least a part of
the image; and
[0041] (b) determining a quantization factor based on the
function.
[0042] According to some embodiments of the invention, the
entropy-related quantification comprises, at least approximately, a
ratio between the entropy of a quantized at least a part of an
image and the entropy of the at least part of the image (ED).
[0043] According to some embodiments of the invention, a
quantification of the complexity comprises a standard deviation of
the at least part of the image.
[0044] According to some embodiments of the invention, the function
comprises an exponential function of the complexity
quantification.
[0045] According to some embodiments of the invention, the function
is, at least approximately, independent of the image.
[0046] According to some embodiments of the invention, the function
comprises an approximation of an exponential function.
[0047] According to some embodiments of the invention, determining
a quantization factor comprises determination according to a range
of quantization factors.
[0048] According to some embodiments of the invention, the range of
quantization factors is based on at least one of the image target
bit-rate, the image target compression ratio, the target visual
quality of the compressed image, or a combination thereof.
[0049] According to some embodiments of the invention, the method
comprises determining, according to at least one of the curvature
or the slope of the function, a threshold of ED above which the
visual quality of a compressed at least a part of an image is
insensitive, at least approximately, to ED.
[0050] According to some embodiments of the invention, the method
comprises determining, according to at least one of the curvature
or the slope of the function, a threshold of a complexity
quantification of the at least a part of an image above which the
visual quality of a compressed at least a part of an image is
insensitive, at least approximately, to the complexity
quantification.
[0051] According to an aspect of some embodiments of the present
invention there is provided an apparatus configured to carry-out
the methods recited above.
BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Non-limiting examples of embodiments of the present
invention are described with reference to figures listed below. In
the drawings which follow, identical or equivalent or similar
structures, elements, or parts that appear in more than one drawing
are generally labeled with the same numeral in the drawings in
which they appear. Dimensions of components and features shown in
the figures are chosen for convenience and clarity of presentation
and are not necessarily shown to scale.
[0053] FIG. 1 illustrates a chart of pixels values of a partition
before and after quantization, in accordance with exemplary
embodiments of the invention;
[0054] FIG. 2A shows an image, in accordance with exemplary
embodiments of the invention;
[0055] FIG. 2B illustrates entropy deficiency of the partitions of
the image of FIG. 2A, in accordance with exemplary embodiments of
the invention;
[0056] FIG. 2C illustrates the distribution of the image
deficiencies of the partitions of FIG. 2B and a fitted exponential
curve with respect to the standard deviation, in accordance with
exemplary embodiments of the invention;
[0057] FIG. 3A shows an image, in accordance with exemplary
embodiments of the invention;
[0058] FIG. 3B illustrates the entropy deficiency of the partitions
of the image of FIG. 3A, in accordance with exemplary embodiments
of the invention;
[0059] FIG. 3C illustrates the distribution of the image
deficiencies of the partitions of FIG. 3B and a fitted exponential
curve, with respect to the standard deviation in accordance with
exemplary embodiments of the invention;
[0060] FIG. 4A shows an image, in accordance with exemplary
embodiments of the invention;
[0061] FIG. 4B illustrates the entropy deficiency of the partitions
of the image of FIG. 2A, in accordance with exemplary embodiments
of the invention;
[0062] FIG. 4C illustrates the distribution of the image
deficiencies of the partitions of FIG. 2B and a fitted exponential
curve, with respect to the standard deviation, in accordance with
exemplary embodiments of the invention;
[0063] FIG. 5 illustrates a chart with respect to complexities of
image partitions and a range of quantization factors with (a) a
graph of the entropy deficiencies of the partitions, and (b) a
graph of quantization factors, in accordance with exemplary
embodiments of the invention;
[0064] FIG. 6 is a flowchart that outlines a sequence of operations
for determining a quantization factor for image partitions and
their subsequent compression, in accordance with exemplary
embodiments of the invention.
DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
[0065] The discussion below is divided into sections with headers
which are intended for clarity and readability only.
Responsiveness to Quantization--Entropy Deficiency (ED)
[0066] In exemplary embodiments of the invention, the
responsiveness to quantization is defined as a change in the
entropy of a group of pixels quantized by a preferably (without
limiting) a small quantization, relative to the entropy of the
original pixels. Optionally, the responsiveness to quantization is
obtained by a convenient derivation of the latter definition,
namely, as the ratio of the entropy of a quantized group of pixels
to the entropy of the original pixels.
[0067] In exemplary embodiments of the invention, the
responsiveness is defined according to the following formula.
R=(E(qP)-E(qP))/E(P)=1-E(qP)/E(P) (1)
[0068] Wherein
[0069] R is the responsiveness according to the definition,
[0070] E is the entropy -.SIGMA.p.sub.i log.sub.2(1/p.sub.i), where
p.sub.i is the probability of each pixel in the group,
[0071] P is a group of pixels,
[0072] qP is the quantized group of pixels.
[0073] According to formula (1) the responsiveness is bounded by a
range between 1 (full responsiveness) and 0 (no
responsiveness).
[0074] In exemplary embodiments of the invention, a convenient
derivation of responsiveness for a group of pixels is derived
according to the following formula.
ED=1-R (2)
So that:
ED=E(qP)/E(P) (3)
[0075] Where ED is the responsiveness, denoted as Entropy
Deficiency. As such, the ED is in a range between 0 (full
responsiveness) and 1 (no responsiveness).
[0076] In exemplary embodiments of the invention, the complexity is
quantified as the entropy of the pixels, or as the standard
deviation of the pixels, or other complexity quantification such as
contrast (or non-uniformity) measures.
[0077] In the following discussion, without limiting the
generality, and unless otherwise specified, the responsiveness to
quantization refers to responsiveness to quantization where the
complexity is the entropy of a group of pixels. Such responsiveness
will be referred to as `entropy deficiency`.
[0078] FIG. 1 illustrates a chart of pixels values of a group of 64
pixels before quantization (102) and after quantization (104), in
accordance with exemplary embodiments of the invention. Originally
there were about 250 different values, and a quantization a factor
of 32 reduced the range to only 8 distinct values. The entropy of
the original image is 6.0, and after quantization the entropy is
3.0, yielding an entropy deficiency of: 3.0/6.0=0.50.
[0079] In exemplary embodiments of the invention, quantizing
comprises dividing the pixels by a uniform factor. Optionally and
alternatively, the factor is different for different values range
and/or relations between values (e.g. smaller factor for edge
pixels).
[0080] In exemplary embodiments of the invention, the quantization
factor for calculating ED according to formula (3) is a small value
relative to the range of pixels values in the group. Preferably,
but not limited to, the quantization factor is 4.
Entropy Deficiency Curve (Function)
[0081] It was unexpectedly found that the entropy deficiency of a
group of pixels exhibits a characteristic distribution with respect
to the complexity of the group of pixels, such as standard
deviation or entropy or other quantification of the pixel
complexity. The distribution aggregates in a pattern resembling a
curve with an initial steep increase followed by a gradual decline,
resembling a negative exponential.
[0082] The entropy deficiency of over a thousand different image
partitions were plotted and/or evaluated with respect to the
corresponding complexity, and a fitted curve was found to exhibit
an exponential function, as follows:
ED=A+B.times.Exp(-C.times.P) (4)
[0083] wherein
[0084] a) ED is the entropy deficiency,
[0085] b) A, B and C are constants, and
[0086] c) P comprises a complexity quantification of the group of
pixels (e.g. standard deviation).
[0087] In exemplary embodiments of the invention, for different
groups of pixels the constants may vary, yet they typically
aggregate in ranges of close values. Optionally, the ranges are as
follows:
[0088] A is approximately between 0.3 and 1.0,
[0089] B is approximately between -0.2 and -1.0, and
[0090] C is approximately between 0.2 and 0.3.
[0091] Furthermore, it was found that a curve according to formula
(4) where A is 0.7, B is -0.4 and C is 0.25 yields a sufficient fit
for a determination of quantization factor for an efficient
compression with high visual quality as described later on.
[0092] The distribution of entropy deficiencies and the
corresponding fitted curve for a few images are illustrated in
FIGS. 2, 3, and 4.
[0093] In accordance with exemplary embodiments of the invention,
FIG. 2A shows an image 202, and FIG. 2B illustrates the entropy
deficiencies 204 of partitions 206 of image 202. FIG. 2C
illustrates a distribution 208 of the entropy deficiencies 204 of
partitions 206 of image 202, together with a fitted exponential
curve 210, with respect to the standard deviation 212 of
corresponding partitions 206. Entropy deficiencies 204 of
partitions 206 of image 202 are depicted in a gray scale, together
with a reference scale 214. Coordinate 216 represents entropy
deficiencies of distribution 208 and graph 212.
[0094] In accordance with exemplary embodiments of the invention,
entropy deficiencies 204 of partitions 206 of image 202 were
derived according to formula (3). Regions of image 202 having a
constant or low variation, such the lower background (218a), the
white shoulder strip of the shirt (218b), or the sky (218c) have
low entropy deficiencies (220a, 220b and 220c, respectively),
whereas complex regions such as plants (222a) or grass (222b) or
illuminated hair (222c) have high entropy deficiency (224a, 224b
and 224c, respectively).
[0095] FIG. 2C illustrates how distribution 208 of the entropy
deficiencies 204 of partitions 206 resembles an exponential with
respect to the partitions complexity (standard deviation 210).
Fitted graph 212, which takes into account the dispersion of the
distribution, is an exponential according to formula (4).
[0096] In order to illustrates how the distribution of entropy
deficiencies and curve fitting are similar for various images and
partitions, different images with respective entropy deficiencies
and distributions and fitted curves are shown.
[0097] Similar to FIG. 2A, FIGS. 3A and 4A show other images
(302/402), and FIGS. 3B and 4B illustrate the entropy deficiency
(304/404) of partitions (306/406) of the images (302/402). FIGS. 3C
and 4C illustrate the distribution (308/408) of the entropy
deficiencies (304/404) of partitions (306/406), together with
fitted exponential curves (310/410), with respect to the standard
deviation (312/412). The image deficiencies (304/406) of partitions
(306/406) of the images (302/402) are depicted in gray scale, with
reference scales (314/414). The coordinates of the distributions
(308/408) and graphs (312/412) correspond to the entropy
deficiencies (304/404).
[0098] The additional images 302/402, and the accompanying entropy
deficiencies 304/404, and particularly the accompanying
distributions (308/408) and graphs (310/410) illustrate that the
shape of the distributions of the entropy deficiencies with respect
to complexity is a common property of different kinds of
images.
[0099] Consequently, based on hundreds of different images, in
accordance with exemplary embodiments of the invention formula (4)
can be used to determine, at least approximately, the entropy
deficiency of a group of pixels directly from the complexity of the
pixels.
[0100] Furthermore, formula (4) enables to determine a quantization
factor for compression of a group of pixels, as described
below.
Quantization According to Entropy Deficiency
[0101] In exemplary embodiments of the invention, a function (or
curve) according to formula (4) enables to determine an effective
quantization factor for compression of image partitions in terms of
compression ratio and/or image visual quality.
[0102] FIG. 5 and the following discussion describe some properties
of the function, and how they relate to, and enable the
determination of quantization factors for compression of a
partition, in accordance with exemplary embodiments of the
invention.
[0103] FIG. 5 illustrates a chart 500 with respect to complexities
512 of image partitions. Graph (curve) 510 depicts the entropy
deficiencies of the partitions according to formula (4), in
accordance with exemplary embodiments of the invention.
[0104] In exemplary embodiments of the invention, a range of
quantization factors 530 between a minimal value 532 (`MinQ`), and
a maximal value 534 (`MaxQ`) is provided. Range 530 is optionally
preset and/or determined by the compression application and/or the
intended use of the compression and/or the bit-rate and/or the
intended visual quality of the compressed partitions (`rate
control`).
[0105] In exemplary embodiments of the invention, MaxQ is derived
from and/or equals the image target bit-rate, or the image target
compression ratio, or the compressed image quality, or a
combination thereof. Optionally, MinQ is also related to the image
target bit-rate, or the image target compression ratio, or the
compressed image quality, or a combination thereof.
[0106] In order to determine a quantization factor (`Q`) for a
partition, in exemplary embodiments of the invention the entropy
deficiency is mapped (transformed) onto the quantization factors in
range 530. Optionally, the mapping is a linear mapping, as is
illustrated by line 560 that maps a complexity of 22.0 (562), or
its corresponding entropy deficiency 0.7 (566), to a quantization
factor Q of 0.86 (564). Optionally, the factor is rounded to an
integer. Optionally, the mapping comprises a linear mapping with
additional terms, as illustrated below.
[0107] In exemplary embodiments of the invention, a quantization
factor Q for compressing an image partition is determined by
mapping of the entropy deficiency of a partition on quantization
factors range 530 according to the following formula.
Q=Min Q-A+Round(((Max Q-Min Q)+B).times.ED) (5)
[0108] Where A and B are constants and ED is the entropy
deficiency, and wherein Q is bounded by MinQ; that is, if Q
evaluates to a value lower than MinQ, Q is set to MinQ.
[0109] In exemplary embodiments of the invention, at least A or B
is 3. Consequently, in exemplary embodiments of the invention, Q is
evaluated according to formula (5) with substitutions for A and B,
namely:
Q=Min Q-3+Round(((Max Q-Min Q)+3).times.ED) (6)
[0110] FIG. 5 graphically illustrates relations between the entropy
deficiency function and the quantization factor, in accordance with
exemplary embodiments of the invention. Graph 510 of the entropy
deficiencies of image partitions is plotted, according to formula
(4), with respect to corresponding complexities 512. Graph 540
depicts the mapped quantization factors Q for the corresponding
entropy deficiencies according to formula (6), where MinQ is 0 and
MaxQ is 20, with respect to a secondary coordinate 538 of Q. For
example, for a partition complexity of 3 (572), or entropy
deficiency is 0.46 (574), the corresponding quantization factor is
8 (576), as shown with dotted lines in chart 500.
[0111] Graph 510 of the entropy deficiency function may be divided
into two regions:
[0112] (a) A rising region 542 with partitions of low complexities
and corresponding low entropy deficiencies or large responsiveness
(formula (3)), where the partitions are sensitive to quantization
(sensitivity region). The quantization factor Q vary rapidly in the
sensitivity region 542, as shown in chart 500 where Q vary between
4 and 13 for entropy deficiencies between 0 and 11,
respectively.
[0113] (b) An asymptotic region (saturation region) 544 with
partitions of high complexities and corresponding high entropy
deficiencies and small responsiveness (formula (3)), where the
partitions are insensitive (or less sensitive) to quantization
(insensitivity region). The quantization factor Q is approximately
constant in the saturation region 544, as shown in chart 500 where
Q is 13 for entropy deficiencies between 11 and 50, or higher.
[0114] In exemplary embodiments of the invention, a partition in
the sensitivity region 542 is sensitive to quantization such that
increasing the quantization above Q will decrease the visual
quality of the compressed partition (e.g. blockiness or abrupt
changes), and decreasing the quantization will decrease the
compression ratio without gaining in visual quality. Yet, using a
quantization factor Q according to formulas (5) or (6) (or chart
500) will typically yield a sufficient compression (with respect to
application requirements) with a good visual quality (e.g.
preserving the variation of shades).
[0115] In exemplary embodiments of the invention, a partition in
the saturation region 544 is insensitive, at least approximately,
to the complexity of the partition, such that a partition
corresponding of a low complexity in the saturation region 544 may
be quantized by a (possibly larger) factor Q corresponding to a
higher complexity, yielding a better compression ratio without
affecting the visual quality of the compressed partition. As such,
region 544 may be referred to as the insensitive region.
[0116] The determination of a quantization factor responsive to
entropy deficiency as described above is new and unique, providing
a quantization adapted to the contents of the partition and
comprises two exceptional aspects. Firstly, partitions within the
sensitivity region (i.e. low complexity) are quantized such that
the mild variations are maintained in the compressed image, and
being of low complexity the low quantization does not overload the
volume of the compressed image. Secondly, partitions with low
complexity within the insensitivity region may be compressed with
high quantization providing high compression ratio without
affecting the visual quality of the compressed image. In other
words, the same quantization may be used for all partitions in the
insensitivity region without adverse visual affect in the
compressed image.
[0117] In exemplary embodiments of the invention, insensitivity
region 544 begins about a complexity measure of about 11.0 (550) or
about a corresponding entropy deficiency value of about 0.67 (552).
Optionally, threshold point 550 for the division of graph 540 to
sensitivity region 542 and insensitivity region 544, may vary. For
example, the determination of the threshold value 550 can be
determined according to the curvature and/or slope of graph 510 or
the corresponding entropy deficiency function (4). Optionally the
threshold value 550 depends on the constants used in formula (4).
Optionally or alternatively, the divisions may be affected to the
evaluation of complexity function used to determine the
responsiveness and/or the complexity function used in formula (4)
(e.g. not a standard deviation). For example, insensitivity region
544 may begin at a threshold value 550 of 15.0 or the corresponding
entropy deficiency value of 0.69. Typically, for given parameters
of function 4 and a given complexity quantification, threshold
value 550 is constant, at least approximately, for any image.
[0118] In exemplary embodiments of the invention, entropy
deficiency graph 510 (or function) may be approximated. Optionally,
the approximation is by a piece-wise linear approximation, for
example, linear sections 554, 556 and 558, such that determining
the quantization factor by the approximation will not affect visual
quality, or only negligibly affect the visual quality. Optionally,
graph 510 may be fitted with two linear sections. Optionally, other
curve approximations may be used, such as sigmoid or Heaviside step
functions, optionally yielding better approximation to graph 510
(or entropy deficiency function) relative to a linear
approximation.
[0119] In exemplary embodiments of the invention, using an
approximation for graph 510 can boost computation time for finding
the quantization factor Q. For example, using a linear
approximation, the factor Q may be determined by simple arithmetic
operation, avoiding more complex operations such as
exponentials.
[0120] In exemplary embodiments of the invention, the entropy
deficiency function (formula (4), graph 510) may be pre-calculated
into a table, which consequently can be used as a lookup table,
optionally with interpolations.
[0121] In exemplary embodiments of the invention, a plurality of
ranges of factors MinQ to MaxQ are preset and stored. Subsequently,
according to the bit-rate and/or intended quality of the compressed
partition an appropriate range is selected and used to determine
the quantization factors.
Adjusting the Quantization Factor
[0122] In exemplary embodiments of the invention, the quantization
factor Q obtained in saturation region 544 is lower than a factor
which will still maintain, at least approximately, the image
quality as by using Q; that is, a better compression ratio could be
achieved without sacrificing quality. Additionally, a quantization
factor Q obtained in the sensitivity region might be somewhat
larger for than desired for a desired visual quality.
[0123] In exemplary embodiments of the invention, the determination
of an adjusted quantization factor requires sub-dividing
sensitivity region 442. According to the slope and/or curvature of
sensitivity region 442, region 442 is divided into two regions: (a)
a steep semi-linear region 546 and (b) an inflection region 548.
The dividing point in terms of complexity or corresponding entropy
deficiency, such as complexity value 6 in chart 500, is optionally
determined about where the steep part begins to inflect, or the
slope begins to decrease or, when the curvature is increasing
beyond a certain value.
[0124] In exemplary embodiments of the invention, the adjustment of
the quantization factor pertains to the coefficients of a
transformation of a partition, and comprises the following
operations, wherein the order of the operations is not mandatory
where applicable.
[0125] a) Determining the number of non-zero coefficients of the
transformation. Optionally, coefficients of low values, such as
lower than the median of the non-zero coefficients values or lower
than the average of non-zero coefficients (e.g. low than 10% or
lower than 5% or 1%) are considered as zero.
[0126] b) Obtaining a quantization factor Q according to formula
(5) or (6) (or according to chart 500 described above), denoted as
Q.sub.0.
[0127] c) Quantizing (dividing) coefficients of the transformed
partition by Q.sub.0.
[0128] d) Determining the number of non-zero coefficients after
quantization by Q.sub.0. Optionally, coefficients of low values,
such as lower than the median of the non-zero coefficients values
and/or lower than the average of non-zero coefficients (e.g. low
than 10% or lower than 5% or 1%) are considered as zero.
[0129] e) Determining a ratio RQ between the number of non-zero
un-quantized coefficients and the number of non-zero quantized
coefficients;
[0130] f) Subtracting the ratio RQ from the image target bit-rate,
or the image target compression ratio, or a value derived therefrom
(e.g. with a proportionality factor), to obtain a difference
DQ;
[0131] g) Determining a new quantization factor Q, the
determination comprising:
[0132] i) If the partition entropy deficiency or complexity is
about inflection region 548 then Q equals Q.sub.0 (i.e. the
quantization factor does not change).
[0133] ii) If DQ>0 and the partition entropy deficiency or
complexity is in insensitivity region 544 then Q.sub.0 is increased
by |DQ| or by a value depending on |DQ|, obtaining a new Q.
[0134] Optionally, the increase is limited to a range of values.
Optionally, the increase is in a range between 1 and 10.
Optionally, the range is between 0 and 5. Optionally, the range is
between 1 and 4. Optionally, the increase is according to |DQ|, so
that the larger |DQ| the larger is the increase. For example, for a
range between 1 and 4, the increase is by |DQ|, but limited by
4.
[0135] iii) If DQ<0 and the partition entropy deficiency or
complexity is below inflection region 548 (i.e. in region 546) then
Q.sub.0 is decreased by |DQ| or by a value depending on |DQ|,
obtaining a new Q.
[0136] Optionally, the decrease is limited to a range of values.
Optionally, the decrease is in a bounded range between 0 and 6.
Optionally, the range is between 1 and 5. Optionally, the range is
between 1 and 2. Optionally, the decrease is according to |DQ|, so
that the larger |DQ| the larger is the decrease. For example, for a
range between 1 and 2, the increase is by |DQ|, but limited by
2.
[0137] In exemplary embodiments of the invention, the compression
according to the modified quantization factor Q is limited so that
it does not effect exceeding the image target bit-rate or the
target compression ratio of the image, or a combination
thereof.
[0138] In exemplary embodiments of the invention, when the
quantization factor is decreased a better visual quality is
achieved on the expense of some decrease in the compression ratio.
Yet, since region 456 belong to partitions of low complexities, the
reduction in compression ratio is, typically, insignificant.
Partitions and Blocks
[0139] In exemplary embodiments of the invention, the determination
of the responsiveness of a group of pixels to quantization (e.g.
entropy deficiency), and a subsequent compression by quantization,
can be performed on a collection of pixels with no geometrical
constraints.
[0140] In exemplary embodiments of the invention, the group of
pixels comprises a partition of an image. In exemplary embodiments
of the invention, a partition comprises a rectangular shape.
Optionally, a dimension of a partition is one of 2, 4, 8, 16, 32 or
64 pixels. Optionally, a partition dimension is larger than 64
pixels. Optionally or alternatively, a partition comprises a
non-rectangular shape. Optionally, the partition shape is according
to the values of the pixels and/or the complexity of the pixels
and/or geometry of features and/or computational considerations.
For example, the partition shape may be adapted to comprise pixels
of the same or similar complexity, or adapted to comprise a limited
range of values.
[0141] In exemplary embodiments of the invention, a partition
comprises one or more blocks. Optionally, a dimension of a block is
one of 2, 4, 8, 16 or 32 or 64 pixels. Optionally, a block
dimension is larger than 64 pixels. Optionally, a block comprises
one or more blocks. Optionally, a block comprises a non-rectangular
shape, for example, such as to comprise pixels of the same or
similar range of values and/or similar complexity.
[0142] In exemplary embodiments of the invention, a partition
comprises disjointed blocks, that is, the blocks are separated by
one or more pixels not belonging to the partition.
[0143] In exemplary embodiments of the invention, a partition is
divided into blocks based on the complexity of the partition. In
that manner, a better quantization (in terms of compression ratio
and/or visual quality) may be determined for each block separately
rather than the whole partition. Optionally, the division into
blocks is such that above a certain complexity the partition is
divided into a plurality of blocks having relative high and low
complexities, each optionally resulting in different quantization
factors.
[0144] For example, a standard deviation of a partition is found to
be 28.1 which may be considered as too complex. Therefore, the
partition is divided into two blocks having standard deviations of
4.4 and 26.9, respectively. Thus, the first block falls within the
sensitivity region (542 of FIG. 5) and quantized by a small factor
(9), while the second block is falls in the insensitivity region
(544 of FIG. 5) and quantized by a larger factor (13).
[0145] In exemplary embodiments of the invention, a partition
and/or a block dimension is according to a method of the image
compression. For example, when MPEG (e.g. h.264) is used the
partition dimensions comprise 16.times.16 pixels (`macro-block`),
or the dimensions comprise 8.times.8 pixels frequently used in DCT
transformation such as JPEG.
[0146] In exemplary embodiments of the invention, the quantization
is independent of the size and/or shape of a partition or block,
since only the collection of pixels is considered.
[0147] In exemplary embodiments of the invention, the methods
and/or embodiments described for a partition apply, at least
partially, to a block within a partition.
Image Compression and Decompression
[0148] In exemplary embodiments of the invention, an image, or part
thereof, such as a group of pixels, or a partition, or a block, is
quantized according to a quantization factor that is determined as
described above and, optionally, is subsequently compressed.
Optionally or alternatively, the image pixels are used to determine
the quantization factors as described, and the coefficients of a
transformation, such as DCT, of the respective pixels are quantized
(divided) by the factors. Optionally or alternatively, the
transformed pixels are used to determine the quantization factors
and are quantized accordingly. Optionally, the quantization is by a
modified factor, such as by limiting the value of the factor
according to the compression method.
[0149] In exemplary embodiments of the invention, the quantized
pixels or quantized coefficients are encoded, for example, the
entropy encoding or arithmetic encoding.
[0150] In exemplary embodiments of the invention, the image is a
gray-scale. Optionally, the image is a color image separated into
channels, such as RGB, YIQ, YUV, etc., and each channel is
quantized and/or compressed separately according to exemplary
methods and embodiments of the invention. Optionally, the image
comprises of pixels packing one or more colors such as RGB, or
luminance (brightness) and one or more color components (e.g.
YUC).
[0151] In exemplary embodiments of the invention, an image is
compressed in a video or pseudo-video sequence, wherein a video
frame comprises one or more image partitions or blocks. Optionally,
the frames are compressed according to intra- or inter-predictive
methods. Optionally or additionally, motion and/or temporal
compressions are used to compress the frames.
[0152] In exemplary embodiments of the invention, the quantization
factors obtained as discussed above can be used within the
framework of compression standards, such as the spatial compression
in JPEG, MPEG or H.264.
[0153] In exemplary embodiments of the invention, a compressed
image is decompressed using techniques of the art. Optionally or
alternatively, when the compression is non-standard, a matching
decoder can be devised.
Operation Outline
[0154] An exemplary procedure for obtaining a quantization factor
according to the entropy deficiency, or responsiveness to
quantization, and using the factor for compression is described
with respect to FIG. 6.
[0155] FIG. 6 is a flowchart that outlines a sequence of operations
for determining a quantization factor for image partitions and
their subsequent compression, in accordance with exemplary
embodiments of the invention.
[0156] According to the application requirements or system
constraints, a target bit-rate is set (602). Optionally, the
requirement is set by the bit-rate, or image target compression
ratio, or the compressed image quality, or a combination
thereof.
[0157] In order to determine quantization factor, the entropy
deficiency function is established (604), such as by formula (4).
Optionally, an approximation of the function is established, such
as by linear segments or a lookup table, in order to simplify and
speed up the determination of the quantization factors. Optionally,
the regions of the function are established, that is, the
sensitivity region 542, the insensitivity region 544 or the
inflection region 548.
[0158] The image is divided into partitions. Optionally, the
partitions are determined according to the application, such as
8.times.8 pixels for JPEG or 16.times.16 for video macro-blocks, as
discussed above.
[0159] A partition is obtained or selected in the image (606), and
the partition complexity or the entropy deficiency is determined
(608). According to the complexity and/or the entropy deficiency
the quantization factor is found (610).
[0160] Optionally, when the quantization pertains to coefficients
of a transformation, such as DCT, the quantization factor is
adjusted with respect to the bit-rate (612). Optionally, the
adjustment is with respect to the bit-rate, or image target
compression ratio, or the compressed image quality, or a
combination thereof.
[0161] Subsequently the partition or its transformation (e.g. DCT
coefficients), are quantized (614), that is, optionally, dividing
the pixels values or the coefficients by the quantization factor
(or a derivation thereof). The quantized values are compressed
(616) by a method of the art, such as by entropy encoding. Then the
next partition is obtained (616) and the sequence is repeated (618)
until the image, or the part of the image intended for compression,
is compressed.
Derived Image--Difference
[0162] In exemplary embodiments of the invention, image compression
comprises a multi-step spatial compression. For example, a
compressed partition is decompressed, resulting in a partition
which is different from the original. Diff may comprise a part of
the compressed partition so that during compression, the Diff is
optionally added to the decompressed partition to yield a better
visual quality, that is, with more details. In exemplary
embodiments of the invention, in order to improve the compression
ratio, Diff is compressed too, and as a part of decompression the
decompressed Diff is added to the decompressed partition. In
exemplary embodiments of the invention, Diff is quantized by a
factor determined according to formula (4), or optionally,
according to formula (3), similar to or as the original partition
is quantized and compressed.
[0163] In exemplary embodiments of the invention, additional Diff
partitions are obtained and quantized and compressed as described
above. For example, the first uncompressed Diff is added to the
first uncompressed partition and the combined partitions are
subtracted from the original partition to yield another Diff with
finer details. In this manner more Diff partitions may be
obtained.
[0164] In exemplary embodiments of the invention, Diff pixels are
optionally scaled and/or shifted and/or otherwise manipulated (e.g.
contrast enhancement) before the quantization, and reverse
operations are applied to the uncompressed Diff.
Apparatus
[0165] In exemplary embodiments of the invention, existing
equipment for image and/or video compression (coder) is used,
optionally with provisions to set the quantization factors for
particular image partitions.
[0166] In exemplary embodiments of the invention, the coder
comprises one or more software modules and/or libraries.
Optionally, the coder comprises hardware and/or firmware.
Optionally, the coder comprises a chip-set with internal or
external memory and/or one or more processors. Optionally, the
coder is part of a device.
[0167] In exemplary embodiments of the invention, the chipset is
used on mobile devices such as cameras or cellular phones or PDAs.
Optionally, the coder (or codec) is part of the device.
[0168] In exemplary embodiments of the invention, off-the-shelf or
proprietary tools for constructing the compressed image and/or the
video sequence are utilized. Optionally, the tools comprise SDK
(software development kit) using techniques such API or procedure
calls to compress and construct the image and/or video. Optionally
or additionally, hardware modules are used. Optionally, a
combination of software, hardware or firmware is used.
[0169] In exemplary embodiments of the invention, the coder is
linked to an imaging sensor. Optionally, the sensor transfers the
image to a memory. Optionally, the sensor may be tapped for the
image. Optionally, the sensor is a part of the chip-set.
[0170] In exemplary embodiments of the invention, processors may be
used in coding, such as a general purpose or image co-processor
(ICP), an application processor or a communications processor.
Optionally, the processor is a dedicated processor. Optionally, the
processor comprises a DSP.
[0171] In exemplary embodiments of the invention, the coder may
accept, in addition to the particular quantization factors,
operation parameters such as bit-rate, a range of allowed or
desired quantization factors, target compression level, or one or
more visual quality metrics. For example, by setting a
configuration file or using API (application programming interface)
to set the parameters.
[0172] In exemplary embodiments of the invention, the image or
video sequence may be stored in memory, optionally for subsequent
decoding or transfer to another device. Optionally, the image or
video sequence may be `streamed`, that is, transferred as it is
constructed.
[0173] In exemplary embodiments of the invention, the coder is
adapted for compression standards such as JPEG, or video standards
such as AVC (H.264), MPEG 1/2/4 or other formats. Optionally, a
coder for non-standard formats is used.
[0174] In exemplary embodiments of the invention, equipment such as
described above is used to decompress the image and/or video
(decoder).
General
[0175] In the description and claims of the present application,
each of the verbs "comprise", "include" and "have" as well as any
conjugates thereof, are used to indicate that the object or objects
of the verb are not necessarily a complete listing of members,
components, elements or parts of the subject or subjects of the
verb.
[0176] The present invention has been described using detailed
descriptions of embodiments thereof that are provided by way of
example and are not intended to necessarily limit the scope of the
invention. In particular, numerical values may be higher or lower
than ranges of numbers set forth above and still be within the
scope of the invention. The described embodiments comprise
different features, not all of which are required in all
embodiments of the invention. Some embodiments of the invention
utilize only some of the features or possible combinations of the
features. Alternatively and additionally, portions of the invention
described/depicted as a single unit may reside in two or more
separate physical entities which act in concert to perform the
described/depicted function. Alternatively and additionally,
portions of the invention described/depicted as two or more
separate physical entities may be integrated into a single physical
entity to perform the described/depicted function. Variations of
embodiments of the present invention that are described and
embodiments of the present invention comprising different
combinations of features noted in the described embodiments can be
combined in all possible combinations including, but not limited to
use of features described in the context of one embodiment in the
context of any other embodiment. The scope of the invention is
limited only by the following claims.
[0177] All publications and/or patents and/or product descriptions
cited in this document are fully incorporated herein by reference
to the same extent as if each had been individually incorporated
herein by reference or if they were reproduced in full herein.
* * * * *