U.S. patent application number 11/353850 was filed with the patent office on 2006-06-15 for methods of suppressing ringing artifact of decompressed images.
This patent application is currently assigned to Samsung Electronics Co., Ltd.. Invention is credited to Yeong-Taeg Kim.
Application Number | 20060125816 11/353850 |
Document ID | / |
Family ID | 30444338 |
Filed Date | 2006-06-15 |
United States Patent
Application |
20060125816 |
Kind Code |
A1 |
Kim; Yeong-Taeg |
June 15, 2006 |
Methods of suppressing ringing artifact of decompressed images
Abstract
A suppression method that provides adaptive (i.e., selective)
processing of an input picture to generate an enhanced output
picture with ringing-like areas of the input picture suppressed.
For each such window in the input picture, if the window is
detected as around a ringing-like area in the picture, then the
output pixel for the position of the window comprises the low-pass
filtered (i.e., smoothed) pixel in the input picture. However, if
the window is not detected as around a ringing-like area (i.e., the
window is around a non-ringing-like area) then the output pixel for
the position of the window comprises essentially the unchanged
window in the input picture. As a result no blurring is introduced
into the input picture in areas where ringing-like patterns are not
detected. Therefore, the output picture includes portions of the
input picture in which ringing-like patterns were not detected, and
portions of the input picture with suppressed ringing-like patterns
where detected. The output picture is an enhanced version of the
input picture with suppressed (i.e., smoothed) ringing-like
patterns.
Inventors: |
Kim; Yeong-Taeg; (Irvine,
CA) |
Correspondence
Address: |
MYERS DAWES ANDRAS & SHERMAN, LLP
19900 MACARTHUR BLVD.,
SUITE 1150
IRVINE
CA
92612
US
|
Assignee: |
Samsung Electronics Co.,
Ltd.
Suwon City
KR
|
Family ID: |
30444338 |
Appl. No.: |
11/353850 |
Filed: |
February 13, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10617068 |
Jul 10, 2003 |
|
|
|
11353850 |
Feb 13, 2006 |
|
|
|
Current U.S.
Class: |
345/204 ;
348/E5.077; 375/E7.135; 375/E7.161; 375/E7.176 |
Current CPC
Class: |
G06T 2207/20012
20130101; H04N 19/117 20141101; H04N 19/136 20141101; H04N 5/21
20130101; G09G 5/00 20130101; G06T 5/002 20130101; G06T 5/20
20130101; G09G 2320/02 20130101; H04N 19/176 20141101; G06T
2207/20192 20130101 |
Class at
Publication: |
345/204 |
International
Class: |
G09G 5/00 20060101
G09G005/00 |
Claims
1-13. (canceled)
14. A method for adaptive reduction of ringing artifacts in an
input image including pixels of image information, comprising the
steps of: (a) selecting a pixel window including a set of pixels
from the input image pixels; (b) detecting areas of ringing
artifacts in the pixel window based on the pixel information; (c)
determining local variance of each pixel in the window with respect
to neighboring pixels; (d) based on the local variances, detecting
if the location of the window is proximate a noisy area in the
input image; (e) processing the window pixels to generate processed
pixels including pixels with reduced ringing artifacts; (f)
selecting pixels with reduced ringing artifacts from the processed
pixels, based on the detected ringing artifact areas and the
detected window location information; and (g) generating an
enhanced output image comprising: (i) the selected pixels, and (ii)
the remaining window pixels.
15. The method of claim 14, wherein in step (b) detecting the areas
of ringing artifacts includes the steps of: detecting areas of
ringing artifacts in the pixel window as a function of gradation
level differences between one or more pixels therein.
16. The method of claim 14, wherein in step (b) detecting the areas
of ringing artifacts includes the steps of: for a pixel in the
window, determining the gradation level difference between that
pixel and that of neighboring pixels; and detecting if the
gradation level difference is within a selected threshold,
indicating ringing-like artifacts proximate the pixel position in
the window.
17. The method of claim 14, wherein in step (e) processing said
pixels includes the steps of performing low pass filtering of the
pixels to reduce ringing artifacts.
18. The method of claim 14, wherein in step (e) processing said
pixels includes the steps of performing smoothing on the pixels to
reduce ringing artifacts.
19. The method of claim 14 wherein the input image comprises a
decompressed image.
20. The method of claim 14, wherein in step (f) selecting pixels
with reduced ringing artifacts from the processed pixels, further
includes the steps of: (f) selecting pixels with reduced ringing
artifacts from the processed pixels in the detected ringing
artifact areas, based on the window location information.
21. The method of claim 14, wherein in step (f) selecting pixels
with reduced ringing artifacts from the processed pixels, further
includes the steps of: (f) selecting pixels with reduced ringing
artifacts from the processed pixels in the detected ringing
artifact areas, substantially in noisy picture locations.
22-30. (canceled)
Description
FIELD OF THE INVENTION
[0001] The present invention relates generally to video processing,
and more particularly to video image compression.
BACKGROUND OF THE INVENTION
[0002] The development of modern digital video technology has
enhanced the video quality for consumer electronics such as in DVD
players and digital TV (DTV) systems, as compared to analog TV
systems. However, significant video enhancement shortcomings remain
in digital video systems such as DTV systems, including contrast
enhancement, brightness enhancement and detail enhancement.
[0003] Post processing for decompressed pictures plays an important
role in the quality of any digital video system depending on
compression technology. This is because artifacts inherently arise
as a result of data quantization in data compression. Examples
include blocking artifacts, ringing artifacts, mosquito noise, etc.
When a digital image or video has been compressed to a low bit
rate, such artifacts are especially annoying.
[0004] Conventionally, a low-pass-filter has been used to remove
the ringing artifacts from the digital picture. However, the
low-pass-filter also removes the picture details which introduces
blurring artifacts. There is, therefore, a need for a method of
suppressing ringing artifacts that commonly arise in decompressed
pictures.
BRIEF SUMMARY OF THE INVENTION
[0005] The present invention addresses the above shortcomings. In
one embodiment, the present invention provides a systematic way of
detecting ringing-like areas in a picture by examining local
statistics. This allows adaptive suppression of ringing artifacts
without introducing blurring to the areas where no ringing
artifacts are detected.
[0006] A suppression method that provides adaptive (i.e.,
selective) processing of an input picture to generate an enhanced
output picture with ringing-like areas of the input picture
suppressed. Subsets of the input picture are analyzed as one or
more windows. For each such window in the input picture, if the
window is around a ringing-like area in the picture, then the
output picture comprises the low-pass filtered (i.e., smoothed)
window in the input picture. However, if the window is not around a
ringing-like area (i.e., the window is around a non-ringing-like
area) then the output picture comprises essentially the unchanged
window in the input picture. As a result no blurring is introduced
into the input picture in areas where ringing-like patterns are not
detected. Therefore, the output picture includes portions of the
input picture in which ringing-like patterns were not detected, and
portions of the input picture with suppressed ringing-like patterns
where detected. The output picture is an enhanced version of the
input picture with suppressed (i.e., smoothed) ringing-like
patterns.
[0007] In another embodiment, the present invention further
provides a device that adaptively reduces ringing artifacts in an
input image including pixels of image information. In one example,
such a device reduces ringing artifacts comprises: a
ringing-artifact detector that detects areas of ringing artifacts
in a pixel window based on the pixel information, the pixel window
including a set of pixels from the input image pixels; an image
processor that processes window pixels to generate pixels with
reduced ringing artifacts; and a combiner that selects the
processed pixels with reduced ringing artifacts in the detected
ringing-artifact areas, and generates an output image comprising:
(i) the selected processed pixels with reduced ringing artifacts,
and (ii) the remaining window pixels.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] These and other features, aspects and advantages of the
present invention will become understood with reference to the
following description, appended claims and accompanying figures
where:
[0009] FIG. 1 shows an example window in a digital picture that is
processed using an embodiment of an adaptive suppression method
according to the present invention;
[0010] FIG. 2 shows a flowchart of the steps of an embodiment of an
adaptive suppression method according to the present invention;
[0011] FIG. 3 shows an example curve representing an example
relationship between local variances .sigma.(y,x) in a signal
detection function .beta.(y,x); and
[0012] FIG. 4 shows a block diagram of an example architecture of
an adaptive suppression device according to the present
invention.
DETAILED DESCRIPTION OF THE INVENTION
Overview
[0013] As noted above, in one embodiment, the present invention
provides a method of detecting ringing-like areas in a picture by
examining local statistics in the picture. This allows adaptive
reduction (e.g., suppression) of ringing artifacts in decompressed
pictures, without introducing blurring to the areas where no
ringing artifacts are present, according to the present invention.
Referring to FIG. 1, in one example, the set {I} denotes a
decompressed picture (field or frame) 10, and I(y,x) denotes a
gradation value of the x.sup.th pixel 12 of the y.sup.th line of
the picture 10.
[0014] FIG. 2 shows a flowchart of the steps of an example
suppression method according to the present invention. The
suppression method includes the steps of:
[0015] Computing local variance/deviation .sigma.(y,x) for each
pixel 12 in the image 10 based on neighboring pixels in the NXM
window 14 (step 20);
[0016] Using the local variances .sigma.(y,x) in a signal detection
function .beta.(y,x) to detect if the window 14 is in a noisy area
(small local deviation) or in a good signal area (large local
deviation), in the input picture 10 (step 22);
[0017] Detecting ringing-like patterns in the window 14 using a
ringing-like pattern detection function g(y,x) (step 24);
[0018] Using detection of noisy/signal area and detection of
ringing-like-patterns in a ringing-like pattern area function
.gamma.(y,x) to determine if the window 14 is around ringing-like
pattern area in the picture 10 (step 26)
[0019] Performing low pass filtering on a copy of the input picture
10 (step 28); and
[0020] Selectively combining portions of {I} with portions of
I.sub.LPF(y,x) based on detected ringing-like pattern areas, to
generate enhanced signal J(y,x) where ringing-like areas are
essentially suppressed (step 30).
[0021] Such a method provides adaptive (i.e., selective) processing
of the input picture 10 to generate an enhanced output picture with
ringing-like areas of the input picture 10 suppressed. As such, in
the above example, for each input pixel I.sub.n(y,x). Based on such
N.times.M neighbor pixel window 14, if the window 14 is around a
ringing-like area in the picture 10, then the output pixel J(y,x)
comprises the low-pass filtered (i.e., smoothed) pixel of the input
picture. However, if the window 14 is not around a ringing-like
area (i.e., the window is around a non-ringing-like area) then the
output pixel J(y,x) comprises essentially the unchanged pixel of
the input picture which is J(y,x). As a result no blurring (due to
smoothing) is introduced into the input picture in areas where
ringing-like patterns are not detected. Therefore, the output
picture comprises portions of the input picture in which
ringing-like patterns were not detected, and portions of the input
picture with low pass filtering where ringing-like patterns were
detected. Note that this changes from pixel to pixel. The output
picture is an enhanced version of the input picture with suppressed
(i.e., smoothed) ringing-like patterns.
Example Implementation
[0022] In one example implementation described hereinbelow,
I.sub.LPF(y,x) denotes the gradation value of the x.sup.th pixel of
the y.sup.th line of the picture {I} after low-pass-filtering. A
local deviation .sigma.(y,x) for I(y,x) in the picture {I} is
computed based on the local/neighboring samples (pixels) of I(y,x).
This can be performed using a window centered at (y,x) and
computing the variance of the samples in the window. Although any
arbitrary window shape can be used, an N.times.M rectangular window
is used in the description herein for explanation purposes.
[0023] The deviation value .sigma.(y,x) can be computed as: .sigma.
.times. .times. ( y , x ) = i = 1 N .times. j = 1 M .times. ( w i ,
j - m .function. ( y , x ) ) 2 N M ( 1 ) ##EQU1## where w.sub.i,j
is the (i,j).sup.th element of the N.times.M rectangular window,
and m(y,x) is the mean of the samples in the N.times.M rectangular
window, wherein: m .function. ( y , x ) = i = 1 N .times. j = 1 M
.times. w i , j N M . ##EQU2##
[0024] Because the expression in relation (1) can be expensive to
compute, alternatively the following relation can be used to
determine the local deviation, wherein: .sigma. .times. .times. ( y
, x ) = i = 1 N .times. j = 1 M .times. .times. w i , j - m
.function. ( y , x ) N M . ( 2 ) ##EQU3##
[0025] Further, the denominator NM in relation (2) is a constant,
once the dimensions of the window have been determined. As such,
the following relation can be used to determine a relative quantity
of the local deviation as: .sigma. .times. .times. ( y , x ) = i =
1 N .times. j = 1 M .times. .times. w i , j - m .function. ( y , x
) . ( 3 ) ##EQU4##
[0026] A computation according to relation (3) is simpler in terms
of a hardware implementation than (1) or (2).
Detecting Noisy and Good-Signal Image Areas
[0027] Based on the local deviation .sigma.(y,x), a signal
detection function .beta.(y,x) based on .sigma.(y,x) is selected,
wherein the signal detection function .beta.(y,x) satisfies the
following three constraints:
[0028] As .sigma.(y,x) increases, .beta.(y,x) approaches 1,
[0029] As .sigma.(y,x) decreases to 0, .beta.(y,x) approaches 0,
and
[0030] .beta.(y,x) monotonically increases as .sigma.(y,x)
increases.
[0031] Other signal detection functions which satisfy the above
three constraints can also be utilized.
[0032] As described further below, the role of the signal detection
function .beta.(y,x) is to detect whether the widow is located in a
noisy area (i.e., small local deviation),in a signal area (i.e.,
large local deviation) or between. In this example, the value of
.beta.(y,x) is bounded as 0.ltoreq..beta.(y,x).ltoreq.1. In
general, when .beta.(y,x) is close to zero, it implies the window
is substantially in a noisy area, and when .beta.(y,x) is close to
one, it implies that the window is substantially in a signal area
(for simplicity, it is assumed that the noise level is smaller than
the signal level).
[0033] One example of the signal detection signal .beta.(y,x) can
be: .beta. .times. .times. ( y , x ) = { .times. 0 , .times.
.sigma. .times. .times. ( y , x ) .ltoreq. t 1 .times. .sigma.
.times. .times. ( y , x ) - t 1 t 2 - t 1 .times. t 1 .ltoreq.
.sigma. .times. .times. ( y , x ) .ltoreq. t 2 .times. 1 , .times.
.sigma. .times. .times. ( y , x ) .gtoreq. t 2 ( 5 ) ##EQU5##
[0034] wherein t.sub.1 and t.sub.2 are some pre-determined
constants. As shown by the example curve 31 in FIG. 3, depending on
the values of t.sub.1 and t.sub.2, the characteristics of
.beta.(y,x) can be defined in association with the value of
.sigma.(y,x). The ranges for t.sub.1 and t.sub.2 are dependent on
application.
Detecting Ringing-Like Patterns
[0035] In detecting a ringing-like pattern in the window, in one
example, a ringing like area is recognized as a weak (small) edge
area. The weak edge area is indicated by a gradation level
difference between a pixel and its neighboring pixels, that is
within a selected threshold. In one example, a relatively small
gradation level difference indicates that there might be weak
(small) edge at the pixel position. A large gradation level
difference implies that there might be a strong edge at the pixel
position. In real images, ringing does not exhibit a strong edge.
Therefore, to detect a ringing-like pattern in the window, a
pattern detection function g(y,x) is utilized, wherein g(y,x)
includes the components g.sub.x(y,x) and g.sub.y(y,x), such that: g
x .function. ( y , x ) = .DELTA. x min .DELTA. x max ( 6 )
##EQU6##
[0036] where:
.DELTA..sub.x.sup.max=max(|I(y,x-1)-I(y,x)|,|I(y,x+1)-I(y,x)|)
.DELTA..sub.x.sup.max=min(|I(y,x-1)-I(y,x)|,|I(y,x+1)-I(y,x)|), and
g y .function. ( y , x ) = .DELTA. y min .DELTA. y max ( 7 )
##EQU7##
[0037] where:
.DELTA..sub.y.sup.max=max(|I(y+1,x)-I(y,x)|,|I(y-1,x)-I(y,x)|)
.DELTA..sub.y.sup.min=min(|I(y+1,x)-I(y,x)|,|I(y-1,x)-I(y,x)|)
[0038] The component g.sub.x(y,x) indicates ringing pattern-like
features in the horizontal direction in the window and the
component g.sub.y(y,x) indicates ringing pattern-like features in
the vertical direction in the window. As such, the ringing-like
pattern detection function g(y,x) can be represented as:
g(y,x)=max(g.sub.x(y,x),g.sub.y(y,x)) (8) [0039] where
0.ltoreq.g(y,x).ltoreq.1.
[0040] For a given pixel I(y,x) the larger the gradation level
difference between that pixel and its neighboring pixels, then the
lower the ringing effect (i.e., g(y,x) decreases). When g(y,x) is
close to zero, it implies a substantially non-ringing-like pattern,
and when g(y,x) is close to one, it implies a substantially
ringing-like pattern.
Detecting Ringing-Like Areas
[0041] A combination of the signal detection function .beta.(y,x)
and the ringing-like pattern detection function g(y,x), provides a
ringing-like area detection function .gamma.(y,x) which satisfies
the constraints in Table 1 below: TABLE-US-00001 TABLE 1 g(y, x)
.gamma.(y, x) 0 .fwdarw. 1 .beta.(y, x) 0 0 1 .dwnarw. 1 0 0
[0042] The arrows in Table 1 indicate a corresponding value
changing from e.g., 0 to 1.
[0043] One example of the function .gamma.(y,x) satisfying said
constraints can be: .gamma.(y,x)=(1-.beta.(y,x))g(y,x). (9)
[0044] In one example, when .gamma.(y,x)=1, it implies that the
window is located around a ringing-like pattern area, whereas when
.gamma.(y,x)=0, it implies that the window is located around a
non-ringing-like pattern area.
Generating Enhanced Output Image
[0045] Based on the ringing-like area detection function
.gamma.(y,x), an enhanced output J(y,x) is provided as:
J(y,x)=(1-.gamma.(y,x))I(y,x)+.gamma.(y,x)I.sub.LPF(y,x) (10)
[0046] wherein J(y,x) combines the original sample values I(y,x)
and the low-pass-filtered values I.sub.LPF(y,x) depending on the
degree of ringing-like value .gamma.(y,x).
[0047] If a ringing-like area is detected (e.g., .gamma.(y,x)=1),
then: J(y,x)=I.sub.LPF(y,x)
[0048] which implies a smoothed output.
[0049] If a non-ringing-like area is detected (e.g.,
.gamma.(y,x)=0), then:
[0050] J(y,x)=I(y,x)
[0051] which implies that no change is made and thus no blurring is
introduced.
[0052] Relation (10) above represents a point-to-point process,
wherein the value of .gamma.(y,x) is estimated point-to-point (or,
pixel-to-pixel). Hence, relation (10) provides an adaptive
de-ringing method according to an embodiment of the present
invention.
[0053] Although in the preferred embodiment described above, both
the signal detection function .beta.(y,x) and the pattern-like
detection function g(y,x), are used in relations (9) and (10)
above, in other embodiments of the present invention, the
pattern-like detection function g(y,x) can be used independent of
the signal detection function .beta.(y,x). As such, in an
alternative example, relation (10) above for generating the output
picture J(y,x) can modified to be:
J(y,x)=(1-g(y,x))I(y,x)+g(y,x)I.sub.LPF (y,x) (10a) Adaptive
Suppression Device
[0054] FIG. 4 shows a block diagram of an example architecture of
an embodiment of an adaptive suppression device 40 according to the
present invention, implementing the above method. The device 40
includes: a low-pass filter (LPF) block 42; a local variance
compute block 44 that computes the local deviation .sigma.(y,x) for
a pixel in a window based on neighboring pixels; signal detection
function block 46 that uses the local variances .sigma.(y,x) to
detect location of the window in relation to noisy and signal areas
in the input picture; a ringing-like detection function block 48
that detects ringing-like patterns using the function .beta.(y,x);
a ringing-like area detection block 50 that uses detection of
noisy/signal area and detection of ringing-like-patterns in the
ringing-like pattern area function .gamma.(y,x) to determine if
window is around ringing-like pattern area in the picture; and a
combiner that selectively combines portions of {I} with portions of
I.sub.LPF(y,x) based on detected ringing-like pattern areas (as
described), to generate enhanced output signal J(y,x) representing
an output picture in which ringing-like areas are essentially
suppressed.
[0055] The example suppression device in FIG. 4 can be implemented
in many ways known to those skilled in the art, such as for
example, as program instructions for execution by a processor, as
logic circuits such as ASIC, etc. For example, instead of low pass
filtering, other methods of suppressing ringing like artifacts in
the input picture can be used according to the present invention.
Further, other methods for detecting other artifacts, and
suppression of such artifacts can also be utilized according to the
present invention. As such, while this invention is susceptible of
embodiments in many different forms, there are shown in the
drawings and described herein, preferred embodiments of the
invention with the understanding that the present disclosure is to
be considered as an exemplification of the principles of the
invention and is not intended to limit the broad aspects of the
invention to the embodiments illustrated. Therefore, the spirit and
scope of the appended claims should not be limited to the
description of the preferred versions contained herein.
* * * * *