U.S. patent application number 10/327146 was filed with the patent office on 2004-06-24 for edge-oriented interpolation method for deinterlacing with sub-pixel accuracy.
Invention is credited to Lin, Wen-Kuo, Lu, Chung-Yen.
Application Number | 20040120605 10/327146 |
Document ID | / |
Family ID | 32594180 |
Filed Date | 2004-06-24 |
United States Patent
Application |
20040120605 |
Kind Code |
A1 |
Lin, Wen-Kuo ; et
al. |
June 24, 2004 |
Edge-oriented interpolation method for deinterlacing with sub-pixel
accuracy
Abstract
An edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy. To interpolate a missing pixel of a first scan
line, first, a first pixel group of a second scan line and a second
pixel group of a third scan line in a first orientation are
provided, and a third pixel group of the second scan line and a
fourth pixel group of the third scan line in a second orientation
are provided. Then, a first sub-pixel of the second scan line is
calculated according to the first pixel group and the third pixel
group, and a second sub-pixel of the third scan line is calculated
according to the second pixel group and the fourth pixel group by
employing a linear interpolation method or an ideal interpolation
function based on the sampling theorem. Thereafter, the missing
pixel is interpolated according to the first sub-pixel and the
second sub-pixel.
Inventors: |
Lin, Wen-Kuo; (Taipei,
TW) ; Lu, Chung-Yen; (Taipei, TW) |
Correspondence
Address: |
BIRCH STEWART KOLASCH & BIRCH
PO BOX 747
FALLS CHURCH
VA
22040-0747
US
|
Family ID: |
32594180 |
Appl. No.: |
10/327146 |
Filed: |
December 24, 2002 |
Current U.S.
Class: |
382/300 |
Current CPC
Class: |
G06T 3/403 20130101 |
Class at
Publication: |
382/300 |
International
Class: |
G06K 009/32 |
Claims
What is claimed is:
1. An edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy, to interpolate a missing pixel of a first scan
line, comprising the steps of: providing a first pixel group of a
second scan line and a second pixel group of a third scan line in a
first orientation corresponding to the missing pixel; providing a
third pixel group of the second scan line and a fourth pixel group
of the third scan line in a second orientation corresponding to the
missing pixel; calculating a first sub-pixel of the second scan
line according to the first pixel group and the third pixel group;
calculating a second sub-pixel of the third scan line according to
the second pixel group and the fourth pixel group; and
interpolating the missing pixel according to the first sub-pixel
and the second sub-pixel.
2. The edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy as claimed in claim 1 further interpolating the
missing pixel according to the first pixel group, the third pixel
group, and the first sub-pixel of the second scan line, and the
second pixel group, the fourth pixel group, and the second
sub-pixel of the third scan line.
3. The edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy as claimed in claim 1 wherein each of the first
pixel group, the second pixel group, the third pixel group, and the
fourth pixel group contains at least one pixel.
4. The edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy as claimed in claim 1 wherein the first
sub-pixel and the second sub-pixel are calculated by employing a
linear interpolation method.
5. The edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy as claimed in claim 1 wherein the first
sub-pixel and the second sub-pixel are calculated by employing an
ideal interpolation function based on the sampling theorem.
6. The edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy as claimed in claim 1 wherein the second scan
line and the third scan line are the neighboring scan lines of the
first scan line.
Description
BACKGROUND OF THE INVENTION
[0001] 1. Field of the Invention
[0002] The present invention relates to an edge-oriented
interpolation method for deinterlacing, and particularly to an
edge-oriented interpolation method for deinterlacing taking into
account the sub-pixel information.
[0003] 2. Description of the Related Art
[0004] In the deinterlacing process, pixels in the missing scan
lines are very often reconstructed by intra-field interpolation.
That is, the information in the neighboring scan lines of the same
field is exploited for pixel value reconstruction, such as shown in
FIG. 1. The missing pixel X of scan line y is given by X=(b+e)/2,
where b and e are the reference pixels in the neighboring scan
lines (y-1 and y+1).
[0005] The above intra-field interpolation method is easy and
straightforward for practical implementation, but it does not
reflect the edge information in a video field, because only the
information in the vertical orientation is used. Consequently, the
reconstructed edges may have visually annoying staircases, such as
at highly contrast diagonal edges.
[0006] A better method is edge-oriented interpolation, exploiting
more orientation information to reconstruct the missing scan lines.
One related art for the edge-oriented intra-field interpolation is
shown in FIG. 2. In FIG. 2, the value of the missing pixel X is
interpolated with pixel pair {a,f}, {b,e} and {c,d} in three
orientations. That is, the value of X is interpolated by the pixel
pair that has a minimum difference as follows, 1 X = { ( a + f ) /
2 if Ua = min { Ua , Ub , Uc } ( b + e ) / 2 if Ub = min { Ua , Ub
, Uc } ( c + d ) / 2 if Uc = min { Ua , Ub , Uc }
[0007] where Ua=.vertline.a-f.vertline.,
Ub=.vertline.b-e.vertline., and Uc=.vertline.c-d.vertline.
corresponding to the pixel difference in the orientations of 135,
90 and 45 degrees respectively.
[0008] In the above case, the interpolation accuracy only covers
the edges oriented between 45 and 135 degrees. Therefore, to
improve accuracy in interpolating flatter edges, more pixel pairs
in neighboring scan lines can be used, as shown in FIG. 3. In FIG.
3, two more pixel pairs of {i,l} and {j,k} in the orientations of
154 and 26 degrees are employed to interpolate the missing pixel X.
Similarly, 2 X = { ( a + f ) / 2 if Ua = min { Ua , Ub , Uc , Ui ,
Uj } ( b + e ) / 2 if Ub = min { Ua , Ub , Uc , Ui , Uj } ( c + d )
/ 2 if Uc = min { Ua , Ub , Uc , Ui , Uj } ( i + l ) / 2 if Ui =
min { Ua , Ub , Uc , Ui , Uj } ( j + k ) / 2 if Uj = min { Ua , Ub
, Uc , Ui , Uj }
[0009] where
[0010] Ua=.vertline.a-f.vertline., Ub=.vertline.b-e.vertline.,
Uc=.vertline.c-d.vertline., Ui=.vertline.i-l.vertline., and
Uj=.vertline.j-k.vertline..
[0011] Theoretically, the more number of the pixel pairs in
different orientation used, the more accurate the interpolation
results for edges oriented at wider angles. However, the
interpolated results may converge to an orientation corresponding
to the pixel pair having a localized minimum difference if more
orientations are used. The interpolated results may lose
accuracy.
[0012] To prevent the problems discussed above, multiple pixel
pairs of the same orientation can be exploited to determine the
actual edge orientation. That is, only if the sum of the multiple
pixel pairs' difference in one orientation is the minimum, the
intra-field interpolation would be performed in that
orientation.
[0013] FIG. 4 illustrates an example of multi-orientation edge
interpolation in three orientations, such as in 45, 90 and 135
degrees. That is, the pixel pairs in 45 degrees are {b,k}, {c,d}
and {j,e}, 90 degrees, {a,d}, {b,e} and {c,f}, and in 135 degrees
{i,e}, {a,f} and {b,l}.
[0014] The missing pixel X is interpolated with the three pixel
pairs that have the total minimum difference as follows, 3 X = { (
a + f ) / 2 if Ua = min { Ua , Ub , Uc } ( b + e ) / 2 if Ub = min
{ Ua , Ub , Uc } ( c + d ) / 2 if Uc = min { Ua , Ub , Uc }
[0015] where
[0016]
Ua=.vertline.a-f.vertline.+.vertline.i-e.vertline.+.vertline.b-l.ve-
rtline.,
Ub=.vertline.b-e.vertline.+.vertline.a-d.vertline.+.vertline.c-f.-
vertline., and
[0017]
Uc=.vertline.c-d.vertline.+.vertline.b-k.vertline.+.vertline.j-e.ve-
rtline..
[0018] Since three neighboring pixel pairs of the same orientation
are used to determine the minimum difference, the orientations of
the object edge can be more accurately predicted than when using
only one pixel pair. However, it is still not robust when the
object edge does not pass along the center of pixels. In this
situation, the information of sub-pixels must be considered to
enhance accuracy.
SUMMARY OF THE INVENTION
[0019] It is therefore an object of the present invention to
provide an edge-oriented interpolation method for deinterlacing
taking into account the sub-pixel information.
[0020] To achieve the above object, the present invention provides
an edge-oriented interpolation method for deinterlacing with
sub-pixel accuracy. According to the embodiment of the invention, a
missing pixel of a first scan line is interpolated.
[0021] First, a first pixel group of a second scan line and a
second pixel group of a third scan line in a first orientation are
provided, and a third pixel group of the second scan line and a
fourth pixel group of the third scan line in a second orientation
are provided.
[0022] Then, a first sub-pixel of the second scan line is
calculated according to the first pixel group and the third pixel
group, and a second sub-pixel of the third scan line is calculated
according to the second pixel group and the fourth pixel group.
Thereafter, the missing pixel is interpolated according to the
first sub-pixel and the second sub-pixel.
[0023] Each of the first pixel group, the second pixel group, the
third pixel group, and the fourth pixel group may contain at least
one pixel, and the first sub-pixel and the second sub-pixel are
calculated by employing a linear interpolation method or an ideal
interpolation function based on the sampling theorem.
BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The aforementioned objects, features and advantages of this
invention will become apparent by referring to the following
detailed description of the preferred embodiment with reference to
the accompanying drawings, wherein:
[0025] FIG. 1 illustrates an example of intra-field interpolation
using only vertical orientation;
[0026] FIG. 2 illustrates an example of edge-oriented interpolation
with three orientations;
[0027] FIG. 3 illustrates another example of edge-oriented
interpolation with five orientations;
[0028] FIG. 4 illustrates an example of multi-orientation edge
interpolation in three orientations;
[0029] FIG. 5 is a flowchart illustrating the operations of the
edge-oriented interpolation method for deinterlacing with sub-pixel
accuracy according to the embodiment of the present invention;
and
[0030] FIG. 6 illustrates a preferred embodiment of the present
invention for the sub-pixel orientation prediction.
DETAILED DESCRIPTION OF THE INVENTION
[0031] The present invention improves the accuracy for the
orientation prediction by exploiting the sub-pixel information.
That is, pixel pairs in sub-pixel accuracy are obtained from the
original pixels to give a finer edge orientation prediction
scale.
[0032] FIG. 5 illustrates the operation of the edge-oriented
interpolation method for deinterlacing with sub-pixel accuracy
according to the embodiment of the present invention. According to
the embodiment of the invention, a missing pixel of a first scan
line is interpolated.
[0033] First, in step S51, a first pixel group of a second scan
line and a second pixel group of a third scan line in a first
orientation are provided, and in step S52, a third pixel group of
the second scan line and a fourth pixel group of the third scan
line in a second orientation are also provided. It should be noted
that, each of the first pixel group, the second pixel group, the
third pixel group, and the fourth pixel group may contain several
pixels.
[0034] Then, in step S53, a first sub-pixel of the second scan line
is calculated according to the first pixel group and the third
pixel group, and in step S54, a second sub-pixel of the third scan
line is calculated according to the second pixel group and the
fourth pixel group. Thereafter, in step S55, the missing pixel is
interpolated according to the first sub-pixel and the second
sub-pixel.
[0035] It should be noted that the first sub-pixel and the second
sub-pixel may be calculated by employing a linear interpolation
method or an ideal interpolation function based on the sampling
theorem. The ideal interpolation function based on the sampling
theorem is briefly introduced as follows.
[0036] The sampling process is commonly used in image process. The
ideal sampling process can be summarized by the well-known sampling
theorem stated as follows:
[0037] Assuming the signal X.sub.a(t) is bandlimited with bandwidth
B, i.e., let X.sub.a(F).ident.0 for .vertline.F.vertline..gtoreq.B.
If X.sub.a(t) is sampled at multiples of some basic sampling
interval T.sub.s, where 4 T S 1 2 B
[0038] to yield the sequence 5 { X a ( nT S ) } n = - .infin. +
.infin. ,
[0039] it is then possible to reconstruct the original signal
X.sub.a(t) from the sampled values by the reconstruction formula, 6
x a ( t ) = n = - .infin. + .infin. x a ( nT S ) sin c ( t - nT S )
,
[0040] where 7 sin c ( t ) = sin ( tF S ) ( tF S ) = sin ( t / T S
) ( t / T S ) ,
[0041] and called the ideal interpolation function. Since the first
sub-pixel and the second sub-pixel can be easily calculated with
the ideal interpolation function based on the ideal sampling
theorem by those skilled in the imaging process, the detailed
process is omitted here.
[0042] FIG. 6 illustrates a preferred embodiment of the present
invention for the sub-pixel orientation prediction. The black
points represent the sub-pixels interpolated using the original
pixels. For example, pixel ia is interpolated from pixel i and a
and given by ia=(i+a)/2. In this case, each pixel group contains
one pixel and the linear interpolation method is employed.
[0043] Consequently, except the three orientations of 45, 90 and
135 degrees given by the original pixel pairs, the interpolated
sub-pixels give four further orientations of 34, 63, 117 and 146
degrees.
[0044] Therefore, the three pixel pairs in
[0045] 34 degrees are {c,k}, {cj,kd} and {j,d},
[0046] 45 degrees are {bc,kd}, {c,d} and {cj,de},
[0047] 63 degrees are {b,d}, {bc,de} and {c,e},
[0048] 90 degrees are {ab,de}, {b,e} and {bc,ef},
[0049] 117 degrees are {a,e}, {ab,ef} and {b,f},
[0050] 135 degrees are {ia,ef}, {a,f} and {ab,fl}, and
[0051] 146 degrees are {i,f}, {ia,fl} and {a,l}.
[0052] The value of missing pixel X is then given by, 8 X = { ( a +
f ) / 2 if Ua = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj } ( b +
e ) / 2 if Ub = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj } ( c +
d ) / 2 if Uc = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj } ( ia +
fl ) / 2 if Uia = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj } ( ab
+ ef ) / 2 if Uab = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj } (
bc + de ) / 2 if Ubc = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj }
( cj + jd ) / 2 if Ucj = min { Ua , Ub , Uc , Uia , Uab , Ubc , Ucj
}
[0053] Where
[0054]
Ua=.vertline.ia-ef.vertline.+.vertline.a-f.vertline.+.vertline.ab-f-
l.vertline.,
Ub=.vertline.ab-de.vertline.+b-e.vertline.+.vertline.bc-ef.ve-
rtline.,
[0055]
Uc=.vertline.bc-kd.vertline.+.vertline.c-d.vertline.+cj-de.vertline-
.,
Uia=.vertline.i-f.vertline.+.vertline.ia-fl.vertline.+.vertline.a-l.ver-
tline.,
[0056]
Uab=.vertline.a-e.vertline.+.vertline.ab-ef.vertline.+.vertline.b-f-
.vertline.,
Ubc=.vertline.b-d.vertline.+.vertline.bc-de.vertline.+.vertlin-
e.c-e.vertline.,
[0057] and
Ucj=.vertline.c-k.vertline.+.vertline.cj-kd.vertline.+.vertline-
.j-d.vertline..
[0058] It should be noted that the present invention also can be
applied to the edge-oriented interpolation using one pixel pair in
one orientation discussed in FIG. 2 and 3, and the detailed process
to interpolate the missing pixel is similar thereto and is thus
omitted here.
[0059] As a result, using the edge-oriented interpolation method
for deinterlacing with sub-pixel accuracy according to the present
invention, the object edge orientation can be more precisely
predicted since pixel pairs with sub-pixel accuracy are used to
determine the orientation.
[0060] Although the present invention has been described in its
preferred embodiments, it is not intended to limit the invention to
the precise embodiments disclosed herein. Those who are skilled in
this technology can still make various alterations and
modifications without departing from the scope and spirit of this
invention. Therefore, the scope of the present invention shall be
defined and protected by the following claims and their
equivalents.
* * * * *