U.S. patent application number 12/309225 was filed with the patent office on 2009-11-26 for method and apparatus for processing a signal.
Invention is credited to Jeong Hyu Yang.
Application Number | 20090290643 12/309225 |
Document ID | / |
Family ID | 38923431 |
Filed Date | 2009-11-26 |
United States Patent
Application |
20090290643 |
Kind Code |
A1 |
Yang; Jeong Hyu |
November 26, 2009 |
Method and apparatus for processing a signal
Abstract
The present invention provides a signal processing method
including searching a correlated unit having a highest correlation
on a first domain for a current block, obtaining coding information
for the correlated unit, and predicting coding information for a
second domain of the current block using the obtained coding
information.
Inventors: |
Yang; Jeong Hyu; (Seoul,
KR) |
Correspondence
Address: |
HARNESS, DICKEY & PIERCE, P.L.C.
P.O. BOX 8910
RESTON
VA
20195
US
|
Family ID: |
38923431 |
Appl. No.: |
12/309225 |
Filed: |
July 12, 2007 |
PCT Filed: |
July 12, 2007 |
PCT NO: |
PCT/KR2007/003390 |
371 Date: |
January 12, 2009 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60830087 |
Jul 12, 2006 |
|
|
|
60847930 |
Sep 29, 2006 |
|
|
|
60853769 |
Oct 24, 2006 |
|
|
|
60854702 |
Oct 27, 2006 |
|
|
|
60886350 |
Jan 24, 2007 |
|
|
|
60909582 |
Apr 2, 2007 |
|
|
|
60907964 |
Apr 25, 2007 |
|
|
|
Current U.S.
Class: |
375/240.16 ;
375/240.12; 375/E7.123; 375/E7.211 |
Current CPC
Class: |
H04N 19/593 20141101;
H04N 19/46 20141101; H04N 19/51 20141101; H04N 19/597 20141101;
H04N 19/513 20141101; H04N 19/61 20141101; H04N 19/176 20141101;
H04N 19/103 20141101 |
Class at
Publication: |
375/240.16 ;
375/240.12; 375/E07.211; 375/E07.123 |
International
Class: |
H04N 7/50 20060101
H04N007/50; H04N 7/26 20060101 H04N007/26 |
Claims
1. A signal processing method comprising: searching a correlated
unit having a highest correlation on a first domain for a current
block; obtaining coding information for the correlated unit; and
predicting coding information for a second domain of the current
block using the obtained coding information.
2. The signal processing method of claim 1, wherein the correlation
on the first domain is determined by using information indicating
adjacency between the current block and the correlated unit on the
first domain, and wherein the information indicating the adjacency
includes a reference index, information indicating a display order,
information indicating a decoding order or view identification
information.
3. The signal processing method of claim 2, wherein the correlated
unit is a block, in a unit on the first domain, located at the same
position as the position of the current block.
4. The signal processing method of claim 2, wherein the correlated
unit is a block obtained by applying a global motion vector to a
unit on the first or second domain.
5. The signal processing method of claim 4, wherein the first and
second domains correspond to a temporal domain and a spatial
domain, respectively.
6. The signal processing method of claim 5, wherein the global
motion vector of the unit on the first domain is derived from an
anchor unit.
7. A signal processing method comprising: searching a correlated
unit on a first domain using a block neighboring on a current
block; obtaining coding information for the correlated unit; and
predicting coding information for a second domain of the current
block using the obtained coding information.
8. The signal processing method of claim 7, wherein the correlated
unit is a block, in a unit on the first domain, located at the same
position as the position of the current block.
9. The signal processing method of claim 7, wherein the correlated
unit is a block obtained by applying a global motion vector to a
unit on the first or second domain.
10. The signal processing method of claim 9, wherein the first and
second domains correspond to a temporal domain and a spatial
domain, respectively.
11. The signal processing method of claim 10, wherein the global
motion vector of the unit on the first domain is derived from an
anchor unit.
12. The signal processing method of claim 1, wherein the coding
information includes a motion vector, a reference index, a block
type, predictive direction information or residual information.
13. The signal processing method of claim 7, wherein the coding
information includes a motion vector, a reference index, a block
type, predictive direction information or residual information.
Description
TECHNICAL FIELD
[0001] The present invention relates to a signal processing method
and apparatus.
BACKGROUND ART
[0002] Compression coding means a series of signal processing
techniques for transmitting digitalized information via a
communication circuit or storing the digitalized information in a
form suitable for a storage medium. As targets of compression
coding, there are audio, video, characters, etc. In particular, a
technique for performing compression coding on video is called
video sequence compression. A video sequence is generally
characterized in having spatial redundancy or temporal redundancy.
Moreover, there is a multi-view video sequence belonging to one of
fields for 3-dimensional (3D) video processing that provides
multi-views to a user using at least one or more cameras.
DISCLOSURE OF THE INVENTION
Technical Problems
[0003] Since a multi-view video sequence has high correlation
between views, it is able to remove overlapped information via
spatial prediction between a plurality of views. So, various
compression techniques are needed to efficiently perform prediction
between a plurality of views.
[0004] If the spatial redundancy and the temporal redundancy are
not sufficiently removed, a compression rate in coding signals is
lowered. If the spatial redundancy and the temporal redundancy are
excessively removed, it is unable to generate information necessary
to perform coding signals. So, a reconstructing rate is
degraded.
[0005] Moreover, in a multi-view video signal, since inter-view
pictures mostly differ from each other in camera positions only,
correlation and redundancy of the inter-view pictures are very
high. If the redundancy of the inter-view pictures is not
sufficiently removed or excessively removed, a compression rate or
a reconstructing rate, however, is lowered.
TECHNICAL SOLUTION
[0006] An object of the present invention is to raise efficiency in
signal processing.
[0007] Another object of the present invention is to provide a
method of predicting coding information of a signal, by which the
signal can be efficiently processed.
[0008] A further object of the present invention is to provide a
method of predicting motion information, by which a video signal
can be efficiently coded.
ADVANTAGEOUS EFFECTS
[0009] In processing a signal, when a temporal-direction motion
vector is given to a current unit, the present invention is able to
predict a view-direction motion vector of the current unit using
the temporal-direction motion vector. Since the temporal-direction
motion vector refers to an image on a different time zone at the
same view, it is able to almost predict the temporal-direction
motion vector precisely unless dept of each object varies according
to elapse of time. More precise prediction is enabled by predicting
coding information for a current unit using coding information for
other units neighbor to the current unit, by which a corresponding
error transmission amount is reduced. So, it is able to perform
efficient coding. And, it is also able to perform efficient coding
by predicting the coding information for the current unit using
coding information for a view direction of a reference unit
associated with temporal directions of the neighbor units.
[0010] Moreover, even if motion information for a current unit is
not transmitted, it is able to calculate motion information very
similar to the motion information for the current unit. So, a
reconstructing rate can be enhanced.
DESCRIPTION OF DRAWINGS
[0011] FIG. 1 is a schematic block diagram of a video signal
decoder according to the present invention;
[0012] FIG. 2 is a diagram for a method of predicting coding
information for a current unit according to one embodiment of the
present invention;
[0013] FIG. 3 is a diagram for a method of predicting a motion
vector according to one embodiment of the present invention;
[0014] FIG. 4 is a diagram for a method of predicting a motion
vector of a current block in correspondence to whether a reference
picture of a neighbor block neighbor to the current block is in a
view direction or a temporal direction according to one embodiment
of the present invention;
[0015] FIG. 5 is a diagram for a method of predicting coding
information for a current block using inter-view correlation
according to another embodiment of the present invention;
[0016] FIG. 6 is a diagram for a method of predicting coding
information for a current block according to one embodiment of the
present invention;
[0017] FIG. 7 is a table for various examples to select either a
forward reference block or a backward reference block to predict
partition information for a current block if both of the forward
and backward reference blocks in a temporal direction refer to a
block in a view direction according to one embodiment of the
present invention;
[0018] FIG. 8 is a table for various examples to select either a
forward reference block or a backward reference block to predict
prediction direction information for a current block if both of the
forward and backward reference blocks in a temporal direction refer
to a block in a view direction according to one embodiment of the
present invention;
[0019] FIG. 9 is a flowchart of a method of predicting coding
information for a view direction of a current block according to
one embodiment of the present invention;
[0020] FIG. 10 is a flowchart of a method of predicting coding
information for a current block according to one embodiment of the
present invention; and
[0021] FIG. 11 is a flowchart of a method of predicting coding
information for a view direction of a current block according to
another embodiment of the present invention.
BEST MODE
[0022] To achieve these and other advantages and in accordance with
the purpose of the present invention, as embodied and broadly
described, a signal processing method includes searching a
correlated unit having a highest correlation on a first domain for
a current block, obtaining coding information for the correlated
unit, and predicting coding information for a second domain of the
current block using the obtained coding information.
[0023] To further achieve these and other advantages and in
accordance with the purpose of the present invention, a signal
processing method includes searching a correlated unit on a first
domain using a block neighbor to a current block, obtaining coding
information for the correlated unit, and predicting coding
information for a second domain of the current block using the
obtained coding information.
[0024] To further achieve these and other advantages and in
accordance with the purpose of the present invention, a signal
processing method includes extracting motion skip information for a
current block, extracting motion information for a co-located block
of the current block, obtaining motion information for a reference
block existing on a domain different that of the extracted motion
information, and deriving motion information for the current block
based on the obtained motion information for the reference
block.
[0025] To further achieve these and other advantages and in
accordance with the purpose of the present invention, a signal
processing unit includes obtaining motion information between a
current unit and a neighbor unit on a first domain, obtaining a
correlated block corresponding to a current block included in the
current unit based on the obtained motion information, obtaining
motion information of the correlated unit on a second domain, and
deriving motion information corresponding to the second domain of
the current unit using the obtained motion information for the
second domain.
MODE FOR INVENTION
[0026] Reference will now be made in detail to the preferred
embodiments of the present invention, examples of which are
illustrated in the accompanying drawings. It is to be understood
that both the foregoing general description and the following
detailed description are exemplary and explanatory and are intended
to provide further explanation of the invention as claimed.
[0027] FIG. 1 is a schematic block diagram of a video signal
decoder according to the present invention.
[0028] Referring to FIG. 1, the decoder mainly includes a parser
unit 100, an entropy decoding unit 200, an inverse
quantization/inverse transform unit 300, an intra-prediction unit
400, a deblocking filter unit 500, a decoded picture buffer unit
600, an inter-prediction unit 700, and the like. And, the
inter-prediction unit 700 includes an IC offset prediction unit
710, an illumination compensation unit 720, a motion compensation
unit 730, and the like.
[0029] The parser unit 100 performs parsing by NAL unit to decode a
received video sequence. Generally, at least one or more sequence
parameter sets and at least one or more picture parameter sets are
transferred to a decoder before a slice header and slice data are
decoded. In this case, various kinds of configuration information
can be included in a NAL header area or an extended area of a NAL
header.
[0030] Since MVC is an additional technique for the conventional
AVC technique, it would be more efficient to add various kinds of
configuration informations just in case of an MVC stream rather
than add them unconditionally. For instance, flag information for
discriminating a presence or non-presence of the MVC stream can be
added to the NAL header area or the extended area of the NAL
header. Only if an inputted bit stream corresponds to a multi-view
video coded bit stream according to the flag information, it is
able to add configuration informations for the multi-view
video.
[0031] For instance, the configuration informations can include
temporal level information, view level information, inter-view
picture group identification information, view identification
information, and the like.
[0032] The parsed stream is entropy-decoded by the entropy decoding
unit 200. A coefficient of each macroblock, a motion vector and the
like are then extracted. The inverse quantization/inverse transform
unit 300 obtains a transformed coefficient generated from
multiplying a received quantized value by a predetermined constant
and then reconstructs a pixel value by inverse-transforming the
coefficient. The intra-prediction unit 400 performs intra-screen
prediction from a decoded sample within a current picture using the
reconstructed pixel value. Meanwhile, the deblocking filter unit
500 applies the reconstructed pixel value to each coded macroblock
to reduce block distortion.
[0033] A filter improves an image quality of a decoded picture by
smoothening an edge of block. Selection of a filtering process
depends on a boundary strength and a gradient of an image sample
around the boundary. Pictures having undergone the filtering are
outputted or stored in the decoded picture buffer unit 600.
[0034] The decoded picture buffer unit 600 plays a role in storing
or opening the previously coded pictures to perform inter-picture
prediction. In this case, `frame_num` and POC (picture order count)
of each picture is used to store the corresponding picture in the
decoded picture buffer unit 600 or open the corresponding picture.
In MVC, since some of the previously coded pictures may differ from
a current picture in view, it is able to use view information for
identifying a view of picture together with the `frame_num` and POC
in order to utilize those pictures as reference pictures. And, the
inter-prediction unit 700 can use the reference pictures managed in
the above manner.
[0035] The inter-prediction unit 700 performs inter-picture
prediction using the reference picture stored in the decoded
picture buffer unit 600. An inter-coded macroblock can be divided
into macroblock partitions. And, each of the macroblock partitions
can be predicted from one or two reference pictures. The
inter-prediction unit 700 includes the IC offset prediction unit
710, the illumination compensation unit 720, and the motion
compensation unit 730.
[0036] In case that an inputted bit stream corresponds to a
multi-view sequence, since a plurality of view sequences are
obtained from different camera, respectively, there occurs an
illumination difference due to internal and external factors of the
cameras. To prevent the illumination difference, the illumination
compensation unit 720 performs illumination compensation. In
performing the illumination compensation, it is able to use flag
information indicating whether to perform the illumination
compensation on a predetermined layer of a video signal.
[0037] For instance, it is able to perform the illumination
compensation using flag information for indicating whether to
perform the illumination compensation on a corresponding slice or a
corresponding macroblock. In performing the illumination
compensation using the flag information, it is applicable to
various macroblock types (e.g., inter 16.times.16 mode, B-skip
mode, direct mode, etc.).
[0038] The motion compensation unit 730 compensates motion of a
current block using informations transmitted by the entropy
decoding unit 200. The motion compensation unit 730 extracts motion
vectors of blocks neighbor to a current block from a video signal
and obtains a predicted motion vector value of the current block.
The motion compensation unit 730 then compensates for a motion of
the current block using the obtained predicted motion vector value
and an offset vector extracted from the video signal. This motion
compensation can be performed using a single reference picture or a
plurality of pictures. In multi-view video coding, if a current
picture refers to pictures at another view, it is able to perform
the motion compensation using information for a reference picture
list for inter-view prediction stored in the decoded picture buffer
unit 600. And, it is also able to perform the motion compensation
using view information for identifying a view of the corresponding
picture.
[0039] A direct prediction mode (direct mode) is a coding mode for
predicting motion information for a current block from motion
information for a coded block. This method improves compression
efficiency because the number of bits necessary to encode motion
information can be saved. For instance, a temporal direct mode
predicts motion information for a current block using motion
information correlation in a temporal direction. The temporal
direct mode is effective when a speed of a motion in a sequence
that includes different motions. A spatial direct mode predicts
motion information for a current block using motion information
correlation in a spatial direction. The spatial direct mode is
effective when a speed of a motion varies in a sequence including
identical motions.
[0040] The inter-predicted pictures and the intra-predicted
pictures through the above process are selected according to a
prediction mode to reconstruct a current picture.
[0041] Various embodiments for providing an efficient signal
processing method are explained in the following description.
[0042] FIG. 2 is a diagram for a method of predicting coding
information for a current unit according to one embodiment of the
present invention.
[0043] First of all, a unit used in the present invention can
include the meanings of block, sub-block, macroblock, slice,
picture, frame, picture group, sequence and the like in being
applied to a video signal. So, the meaning of the unit should be
construed as that of each application example. Moreover, in
applying the unit to another signal as well as a video signal, the
unit can be construed as a different meaning suitable for the
corresponding signal.
[0044] In a multi-view video sequence, each unit can exist in a
first or second domain. For instance, the first domain corresponds
to a temporal domain and the second domain corresponds to a spatial
domain. In this case, the respective units can exist on axes in a
temporal direction or a spatial direction (for example, view
direction, etc). So, some units can obtain prediction information
in the temporal direction only or other units can obtain prediction
information in the spatial direction only. Also, some units can
obtain prediction information in both the temporal direction and
the spatial direction. In case of predicting coding information in
the temporal direction, if a reference unit exists on a different
time zone at the same view, each of the units has a
temporal-direction motion vector. And if a reference unit exists on
a different view at the same time, each of the units has a
spatial-direction motion vector. This vector may be called a
disparity vector. In the following description, a motion vector
includes a concept of the temporal-direction motion vector and a
concept of the spatial-direction motion vector.
[0045] In the present invention, coding information can include
information for a motion vector, a reference index, a block type, a
prediction direction, and the like.
[0046] According to one embodiment of the present invention, if a
temporal-direction motion vector is given to a current unit {circle
around (1)}, it is able to predict a view-direction motion vector
of the current unit {circle around (1)} using the
temporal-direction motion vector. For instance, it is able to find
a unit M{circle around (3)} indicated by the temporal-direction
motion vector from a unit {circle around (1)} (e.g., reference
unit) having highest correlation with the current unit {circle
around (1)}. In this case, if the unit MO has a view-direction
motion vector mv{circle around (3)}_f, it is able to predict it as
a view-direction motion vector of the current unit {circle around
(1)}. Since the temporal-direction motion vector indicates to an
image on a different time zone of the same view, it is able to
almost accurately predict the view-direction motion vector unless
depths of objects considerably vary according to lapse of time.
[0047] Likewise, if a view-direction motion vector is given to a
current unit {circle around (1)}, it is able to predict a
temporal-direction motion vector of the current unit {circle around
(1)} using the view-direction motion vector. For instance, it is
able to find a unit M.RTM. indicated by the view-direction motion
vector from a unit {circle around (4)} (e.g., reference unit)
having highest correlation with the current unit {circle around
(1)}. In this case, if the unit M{circle around (4)} has a
temporal-direction motion vector mv{circle around (4)}'_b, it is
able to predict it as a temporal-direction motion vector of the
current unit {circle around (1)}. Since the view-direction motion
vector indicates to an image on the same time zone of a different
view, a temporal motion of a specific object will be almost similar
for each camera. So, it is able to almost accurately predict the
temporal-direction motion vector.
[0048] Moreover, in the above embodiment, in case that
bi-directional prediction is carried out according to a block type
of a current unit, it is apparent that each of the above-explained
cases is applicable to both forward and backward directions.
[0049] In each of the above embodiments, a start position of the
unit (correlated unit {circle around (3)} or correlated unit
{circle around (4)}) indicated by the motion vector may not
coincide with a start position of a 4.times.4 block generated from
equally dividing a macroblock, to which each unit belongs), by
4.times.4. If so, it is able to predict motion information in a
manner of finding a closest 4.times.4 lock and then bringing motion
information for the block or in a manner of averaging motion
informations for blocks overlapped with the correlated unit {circle
around (3)} or the correlated unit {circle around (4)}. Thus, using
this method, it is able to predict motion information for each unit
on the first or second domain.
[0050] According to another embodiment of the present invention, it
is able to predict motion information for a current unit using
information for units neighbor to the current unit.
[0051] For instance, it is assumed that two neighbor units A and B
among a plurality of units neighbor to a current unit {circle
around (1)} have temporal-direction motion vectors and that a unit
C has a view-direction motion vector. In this case, if it is
attempted to predict a temporal-direction motion vector of the
current unit {circle around (1)}, the temporal-direction motion
vectors of the neighbor units A and B are usable. If the method
according to the former embodiment is used, it is able to predict a
temporal-direction motion vector of the neighbor unit C using the
view-direction motion vector of the neighbor unit C. And, it is
able to predict a temporal-direction motion vector of the current
unit {circle around (1)} using the three temporal-direction motion
vectors including the predicted temporal-direction motion vector of
the neighbor unit C. Thus, the same method of applying the motion
vector on the first domain is applicable to the second domain.
[0052] For instance, in case of attempting to predict a
view-direction motion vector of a current unit {circle around (1)},
it is able to directly use a view-direction motion vector of a
neighbor unit C. If the method of the former embodiment is used, it
is able to predict each view-direction motion vector using
temporal-direction motion vectors of the neighbor units A and B.
And, it is able to more accurately predict a view-direction motion
vector of the current unit {circle around (1)}.
[0053] Moreover, it is able to predict a temporal-direction motion
vector and a view-direction motion vector of the current unit
{circle around (1)} by applying the above method to all
combinations that the neighbor units of the current unit {circle
around (1)} have temporal-direction motion vectors and
view-direction motion vectors.
[0054] In order for blocks of each unit used as a reference unit to
have both temporal-direction motion vectors and view-direction
motion vectors in applying the above-explained method, the
following method is usable. It is assumed that a unit, which will
have prediction information in a temporal direction only, is named
a unit-T and that a unit, which will have prediction information in
a view direction only, is named a unit-V. In this case, the unit-T
needs to have a temporal-direction motion vector only and the
unit-V needs to have a temporal-direction motion vector only. The
unit-T or the unit-V may be intra-coded. If a current block is
intra-coded in the unit-V, it is able to predict a view-direction
motion vector of the current block using view-direction motion
vectors of neighbor blocks.
[0055] Likewise, if a current block of the unit-T is intra-coded,
it is able to predict a temporal-direction motion vector using a
temporal-direction motion vector of a neighbor block. And, it is
able to set a temporal-direction motion vector of the current block
to the predicted temporal-direction motion vector. Assuming that a
unit, which will have prediction information from both time and
view directions, is named a unit-T/V, it will have a single time or
view direction motion vector for each block only.
[0056] Besides, it is able to set a motion vector of a current
block to a motion vector predicted along a reference unit in the
aforesaid manner. In case that intra-coding is performed on the
unit-T/V, it is able to predict a motion vector of a current block
using temporal direction motion vectors and view-direction motion
vectors of neighbor blocks. Thus, for the unit to which
intra-coding is applied, the method of performing prediction
through neighbor units can be carried out in a manner of taking a
median of three neighbor units or an average of nine neighbor
4.times.4 units.
[0057] According to another embodiment of the present invention, it
is able to use an approximate unit in the vicinity of a portion
indicated by a motion vector of a current unit as a correlated unit
of a current unit. In this case, if is able to perform more
accurate prediction using a position difference vector between the
correlated unit and the approximate unit.
[0058] According to resolution of a motion vector, a position of an
initial correlated unit is found by resolution of 1/n pixel (n=1,
2, 4, 8, etc.). In this case, it may not be correctly mapped to a
unit in a normalized lattice structure. Namely, if n>1, pixels
of the correlated unit may be located at a position of a decimal
point pixel of a unit in a lattice structure. According to
information obtained from a correlated unit, mapping can be carried
out in a manner of approximating to a larger lattice structure by
moving from a position of an initially mapped correlated unit. For
instance, in case of attempting to predict a reference index, an
8.times.8 block position in a normalized lattice structure of a
correlated unit can be matched. In this case, for four 8.times.8
blocks of a current block, reference index information for each
8.times.8 block of a corresponding macroblock is usable. And,
motion information can be brought for each 4.times.4 block.
[0059] Since motion information is provided by 4.times.4 block
unit, in case of attempting to obtain this information, it is able
to match normalized 4.times.4 lattice block positions in correlated
unit. In this case, it is able to obtain motion information from
the corresponding 4.times.4 block. If mapping is carried out to
16.times.16 macroblock position in a normalized lattice structure
of a correlated unit, all information for a corresponding
macroblock can be brought. For instance, it is able to bring
information for a block type, a reference index, a motion vector,
and the like.
[0060] There are various embodiments in a method of approximation
to a position in a neighbor normalized structure from the
above-explained initial mapping position.
[0061] In a normalized structure, a lattice structure such as a
pixel scale, a 4.times.4 block scale, an 8.times.8 block scale, a
16.times.16 block scale and the like can be taken into
consideration. For instance, there is a method of approximation to
a left upper end pixel of a unit including an initial position
point to which a left upper corner pixel of a current unit is
mapped. In case of existence on a corner, approximation can be
carried out to a left upper end pixel position of a unit at one
side. In this case, it is able to select a unit at a left upper
end, a right upper end, a left lower end or a right lower end.
Alternatively, mapping can be carried out to a position point
nearest to an initially mapped position. For instance, in case of
attempting to perform mapping by 8.times.8 block scale, mapping to
a nearest position point can be performed by measuring a distance
from a left upper end pixel position of each 8.times.8 block.
[0062] According to another embodiment of the present invention, it
is able to predict coding information for a second domain of a
current unit using coding information for a second domain of a
first domain unit. In this case, block information as coding
information can be obtained together with motion information. And,
a preset table is usable to derive coding information. For
instance, a skip mode means that information for a different unit
coded prior to a current unit is utilized as information for the
current unit. In applying the skip mode, information existing on
different domains are usable. Details of the corresponding
embodiments are explained as follows.
[0063] For first example, it can be assumed that relative motion
relations between objects (or backgrounds) within two different
view sequences at time Tb are maintained almost similar at time Tc
close enough. In this case, there exists a high degree of
correlation between view direction coding information at the time
Tb and view direction coding information at the time Tc. High
coding efficiency can be obtained using motion information for a
corresponding block on a different time zone at the same view
intactly. And, it is able to use motion skip information indicating
whether this scheme is used. If a motion skip mode is applied
according to the motion skip information, it is able to predict
such motion information as bock type, motion vector, reference
index and the like from a corresponding block of a current block.
So, it is able to reduce a bit amount required from coding the
motion information. For instance, if motion_skip_flag is 0, the
motion skip mode is not applied. If motion_skip_flag is 1, the
motion skip mode is applied. The motion skip information may be
located at a macroblock layer. For instance, the motion skip
information is located in an expanded area of a macroblock layer to
inform whether motion information is brought from a bit stream in a
decoder side.
[0064] For second example, like the above example, the same scheme
is usable in a manner of switching first and second domains. In
particular, it is highly probable that an object (or background)
within a view Vb at a time Tb and an object (or background) within
a view Va neighbor to the view Vb at the time Tb may have motion
informations similar to each other. In this case, if motion
information for a corresponding block at another view on the same
time zone is brought to use intact, high coding efficiency is
available. And, motion skip information indicating whether this
scheme is used is usable.
[0065] For the above-explained two kinds of motion skip modes, a
method of finding a correlated block to a current unit is similarly
applicable. In the following description, for convenience, the
first example is named an inter-view motion skip and the second
example is named a temporal motion skip.
[0066] A method of finding a correlated unit to a current unit is
explained as follows.
[0067] First of all, as the correlated unit, it is able to use a
unit located at the same position as the position of the current
unit among units existing on a domain different from that of the
current unit. For instance, in case of inter-view motion skip mode,
a block, located at a same position as the position of the current
block, existing at a different view on a same time zone is usable
as a correlated block. In case of temporal motion skip mode, a
block, located at a same position as the position of the current
block, existing at a same view on a different time zone is usable
as a correlated block. Alternatively, it is able to find the
correlated block using motion information for neighbor blocks to a
current block. This has been explained in the foregoing
description. Thus, once the correlated block is found, it is able
to predict motion information for the current block using the
motion information for each correlated block. If the correlated
block dies not have motion information, the motion skip mode may
not be applicable. The above-explained method of finding the
correlated unit can be intactly extended as a method of finding a
correlated unit for a smaller or larger unit.
[0068] Various methods of the embodiments are available by
combining the aforesaid two kinds of skip mode schemes.
[0069] In a first embodiment, flags can be separately managed to
identify which one of the two kinds of the skip modes is used. For
instance, an inter-view skip mode can be represented as inter_view
motion_skip_flag and a temporal skip mode can be represented as
temporal motion skip flag.
[0070] It is able to represent a presence or non-presence of
application using two flag bits on syntax. This is to select a
better one after both of the two kinds of the skip modes have been
applied. Alternatively, one of the two kinds of the skip modes is
usable per slice only.
[0071] For instance, in case of an inter-view motion skip mode,
neighbor units to a current unit should refer to a unit existing on
a different time zone at a same view. If any one of the neighbor
units fails to meet this condition, the inter-view motion skip mode
may not be applicable.
[0072] Alternatively, after a reference unit in a view direction
has been found from motion information for neighbor units, if the
reference unit has motion information in a temporal direction, the
motion information in the temporal direction is usable. And, the
temporal motion skip mode is applicable in a similar manner.
Namely, neighbor units to a current unit should refer to a unit
existing at a different view on a same time zone. If any one of the
neighbor units fails to meet this condition, the temporal motion
skip mode may not be applicable. Alternatively, after a reference
unit in a temporal direction has been found from motion information
for neighbor units, if the reference unit has motion information in
a view direction, the motion information in the view direction is
usable.
[0073] For another instance, in case of deciding one motion skip
mode by slice unit and then using the decided motion skip mode, it
is able to insert information, which indicates how a motion skip
mode will be applied, in a slice header. For example, a syntax
element `motion_skip_enable` is usable. If `motion_skip_enable` is
1, an inter-view motion skip mode is applied. If
`motion_skip_enable` is 0, a temporal motion skip mode is applied.
And, it is able to represent whether a motion skip mode is applied
to a corresponding macroblock though `motion_skip_flag` for each
macroblock. A position of this flag may correspond to an extended
area of a macroblock layer.
[0074] In a second embodiment, if a reference unit predicted from
neighbor units to a current unit exists on a view axis, a temporal
motion skip mode is applied. If the predicted reference unit exists
on a temporal axis, an inter-view motion skip mode is applicable.
This can be known to a decoder side. So, one `motion_skip_flag` is
enough. In this case, a mode can be decided according to a method
of predicting motion information from neighbor units. For instance,
after a decoder side has parsed motion_skip_flag, if a motion skip
mode is applied, it is able to predict motion information from
neighbor units. If the predicted reference unit is a unit at a
different view on a same time zone, it is able to apply a temporal
motion skip mode. If the predicted reference unit is a unit
existing at a same view on a different time zone, it is able to
apply an inter-view motion skip mode. By the same method in a
spatial direct prediction mode as a method of obtaining motion
information to find a correlated unit from neighbor units, it is
able to derive a reference index and a motion vector.
[0075] In a third embodiment, it is able to use one of the two
kinds of motion skip modes only. In this case, one
`motion_skip_flag` is necessary only. For instance, in applying an
inter-view skip mode, neighbor units to a current unit should refer
to a unit at a same view on a different time zone. If any of the
neighbor units fails to meet this condition, the inter-view motion
skip mode may not be applicable. After a reference unit in a view
direction has been found from motion information for neighbor
units, if the reference unit has motion information in a temporal
direction, it is able to use the motion information in the temporal
direction. This scheme can be similarly applied to the case of
applying the temporal motion skip mode.
[0076] In case of an inter-view motion skip mode, if motion
information for a correlated unit is not the motion information for
a view direction, the algorithm may not be applied. And, it is also
able to derive a motion vector using a proportional formula
according to a time difference between a current unit and a
correlated unit or a time difference between a correlated unit and
a reference unit of the correlated unit. Likewise, in case of a
temporal motion skip mode, if motion information for a correlated
block is not the motion information in a temporal direction, the
above algorithm may not be applied. Alternatively, it is able to
derive a motion vector using a proportional formula according to an
inter-view geometrical or topological distance between a current
unit and a correlated unit or between a correlated unit and a
reference unit of the correlated unit. For instance, let's assume
that real cameras are located on a straight line in order of V0, V1
and V2. It is assumed that a current unit is located at V1. It is
assumed that a correlated unit is located at V0. And, it is assumed
that a reference unit of the correlated unit is located at V2. In
this case, a size of each component of a motion vector of the
correlated block is set to 1/2 to use. If a distance between real
cameras is given, it is able to derive a motion vector by
considering an actual distance. Moreover, even if cameras are not
located on a straight line, a geometrical or topological distance
is usable.
[0077] A discrimination between views can be known through view
identification information. And, a unit discrimination on a
temporal axis can be known through POC (picture order count), frame
number or the like. In a multi-view video sequence, it is unable to
find a correct unit between different views using a reference index
only. If a rearrangement process of reference units takes place
differently between views, a unit indicated by a reference index at
one view and a unit indicated by a reference index having an
identical value at a different view may not be correspondent to
each other. For instance, if a unit indicated by a reference index
of a macroblock M{circle around (1)} existing at Vb and Tb
corresponds to a unit {circle around (3)} exiting at Vb and Tc, an
identical reference index at a view Va may correspond to a unit
{circle around (5)} existing at Va and Ta. So, in brining motion
information for a correlated unit, at least one of picture order
count (POC), frame number, and/or view identification information
may be needed instead of a reference index. It is able to finally
find an identical reference by comparing at least one of picture
order counts (POC), frame numbers and/or view identification
informations of reference units on a reference list of a current
unit.
[0078] Moreover, the motion vector derived in the above manner is
used as a predicted value and an actual motion vector is found. So,
the corresponding difference is coded to use. In this case, it is
able to refer to a more accurate position.
[0079] FIG. 3 is a diagram for a method of predicting a motion
vector according to one embodiment of the present invention.
[0080] An encoder predicts motion information for a current block
using motion information for a block neighbor to the current block
and then transfers a difference between an actual motion vector and
a predicted motion vector.
[0081] Likewise, a decoder determines whether a reference picture
number of a picture referred to by a current macroblock is
identical to a reference picture number of a picture referred to by
a neighbor block and then acquires a predicted motion vector value.
For instance, if there exists one block having the same reference
picture number of the current macroblock among the neighbor blocks,
a motion vector of the neighbor block is used as it is. IN other
cases, a median value of motion vectors of the neighbor blocks is
used.
[0082] In multi-view video coding, a reference picture can exist on
a view axis as well as on a temporal axis. Due to this
characteristic, if a reference picture number of a current
macroblock differs from that of a neighbor block, it is highly
probable that corresponding motion vectors may not have
correlations with each other. If so, accuracy of a predicted motion
vector value is considerably lowered. So, a new motion vector
predicting method using inter-view correlation according to one
embodiment of the present invention is proposed.
[0083] For instance, a motion vector generated between views may be
dependent on depth of each object. If depth of sequence has not
considerable spatial variation and if a motion of the sequence
according to a variation on a temporal axis is not severe, depth at
a position of each macroblock will not considerably vary. In this
case, depth may mean information capable of indicating an
inter-view disparity. Moreover, since influences of global motion
vectors basically exits between cameras, if a global vector is
sufficiently larger than a depth variation despite that a depth
slightly varies, using the global motion vector may be more
efficient than using a temporal direction motion vector.
[0084] In this case, a global motion vector may means a motion
vector applicable to a predetermined area in common.
[0085] For instance, if a motion vector corresponds to a partial
area (e.g., macroblock, block, pixel, etc.), a global motion vector
(or global disparity vector) is the motion vector corresponding to
a whole area including the partial area. For instance, the whole
area may correspond to a single slice, a single picture or a whole
sequence. And, the whole area may correspond to at least one object
within a picture, a background or a predetermined area. The global
motion vector may be a value by a pixel unit, a 1/4 pixel unit, a
4.times.4 unit, an 8.times.8 unit or a macroblock unit.
[0086] According to one embodiment of the present invention, it is
able to predict a motion vector of a current block using inter-view
motion information of co-located block. In this case, the
co-located block may mean a block neighbor to a current block
existing in a same picture, or a block, located at a same position
as the position of the current block, included in a different
picture. For instance, the co-located block may be a spatial
co-located block in case of a different picture at a different view
or a temporal co-located block in case of a different picture at a
same view.
[0087] In multi-view video coding structure, random access can be
realized in a manner of providing pictures to be predicted in a
view direction with a predetermined temporal interval only. Thus,
if two pictures for predicting motion information in a view
direction only are decoded, it is able to apply a new motion vector
predicting method to pictures temporally existing between the two
pictures. For instance, it is able to obtain a view-direction
motion vector from a picture for view-direction prediction
only.
[0088] And, the obtained vector can be stored by a 4.times.4 block
unit. If an illumination difference is considerable in case of
performing view-direction prediction only, intra-prediction coding
frequently takes place. In this case, a motion vector can be set to
0. Yet, if coding by intra-prediction is frequently performed due
to the considerable illumination difference, many macroblocks
having unknown information for a view-direction motion vector are
generated. To compensate this, in case of intra-prediction, it is
able to calculate a virtual inter-view motion vector using a motion
vector of a neighbor block. And, a motion vector of a block coded
by the intra-prediction can be set to the virtual inter-view motion
vector.
[0089] After inter-view motion information has been obtained from
the two decoded pictures, it is able to code hierarchical
pictures-B existing between the two pictures.
[0090] In this case, the two decoded pictures may be anchor
pictures. In this case, the anchor picture means an encoded picture
that refers to slices in a frame on a same time zone only. For
instance, the anchor pictures means the encoded picture that refers
to the slice at a different view but does not refer to a slice at a
current view.
[0091] If neighbor blocks do not have inter-view motion vectors, it
is able to predict an inter-view motion vector of each partition
using an average of inter-view motion vectors of a co-located
4.times.4 block in the decoded picture.
[0092] And, the same manner can be applied to the case that a
neighbor block has both an inter-view motion vector and a temporal
motion vector. For instance, a motion vector of a current block can
be predicted using Formula 1 and Formula 2.
i = 0 M j = 0 N ( mv 0 , x [ i ] [ j ] + mv 1 , x [ i ] [ j ] ) +
MN 2 MN [ Formula 1 ] i = 0 M j = 0 N ( mv 0 , y [ i ] [ j ] + mv 1
, y [ i ] [ j ] ) + MN 2 MN [ Formula 2 ] ##EQU00001##
[0093] In this case, `M` indicates a horizontal block size of each
partition and `N` indicates a vertical block size of each
partition. And, `mv.sub.0` and `mv.sub.1` indicate motion vectors
of forward and backward reference pictures among the decoded
pictures, respectively. For instance, each of `mv.sub.0` and
`mv.sub.1` may indicate an inter-view motion vector of an anchor
picture or a global motion vector of an anchor picture.
[0094] Although neighbor blocks have inter-view reference pictures,
respectively, it is able to obtain a new predicted motion vector
value from a median or average of the predicted inter-view motion
vectors.
[0095] In case that predictive coding or predictive intra-coding in
a temporal direction is carried out on a current block, it is
unable to know inter-view motion information for the current block.
So, it is able to predict an inter-view motion vector by the
following methods according to another embodiment of the present
invention.
[0096] In the first method, co-located blocks of pre-reference
picture and post-reference picture of a current picture in a
temporal or view direction are usable. For instance, inter-view
motion vectors of 4.times.4 block are averaged to use.
[0097] In the second method, it is able to use an inter-view motion
vector of a block indicated by a temporal-direction motion vector.
This is to find an identical object and use a motion of the object.
So, although a motion is large, even if depth of the object is not
changed, it is able to predict an inter-view motion vector more
accurately. In this case, since a position of a corresponding
block, which is found using the temporal-direction motion vector,
does not coincide with a 4.times.4 block unit exactly, motion
information of nearest 4.times.4 block is used or motion
information, to which a weight of a ratio of an area overlapped
with the corresponding block is applied, is usable. Detailed
embodiments for the methods are explained with reference to FIG. 4
as follows.
[0098] FIG. 4 is a diagram for a method of predicting a motion
vector of a current block in correspondence to whether a reference
picture of a neighbor block neighbor to the current block is in a
view direction or a temporal direction according to one embodiment
of the present invention.
[0099] In case that every reference picture number of neighbor
blocks differs from a reference picture number of a current block,
0 or other predicting method are available. For instance, FIG. 4(a)
shows that every neighbor block refers to a view direction picture
and a current block refers to a temporal direction picture. In this
case, a predicted motion vector value of the current block can be
set to 0. FIG. 4(b) shows that every neighbor block refers to a
temporal direction picture and a current block refers to a view
direction picture. In this case, for a predicted motion vector
value of the current block, the motion vector predicting method
explained in FIG. 2 or FIG. 3 is applicable. Alternatively, a
predicted motion vector value of the current block can be set to
0.
[0100] There is another case that two of neighbor blocks refer to a
picture in a same direction of a current block. For instance, FIG.
4(c) shows that when a current block refers to a temporal direction
picture, two neighbor blocks refer to temporal direction pictures
and the rest one refers to a view direction picture. In this case,
motion vectors of the two neighbor blocks referring to the temporal
direction pictures are averaged to be used as a predicted motion
vector value of the current block. FIG. 4(d) shows that when a
current block refers to a view direction picture, two neighbor
blocks refer to view direction pictures and the rest one refers to
a temporal direction picture. In this case, motion vectors of the
two neighbor blocks referring to the view direction pictures are
averaged to be used as a predicted motion vector value of the
current block. The motion vector predicting method explained in
FIG. 2 or FIG. 3 is applicable to the neighbor block referring to
the temporal direction picture. A median value or a weighted
average value is obtained from the correspondingly obtained motion
vector and the motion vectors of the two neighbor blocks to be used
as a predicted motion vector value of the current block.
[0101] Moreover, the predicting method explained in FIG. 2 or FIG.
3 is applicable to FIGS. 4(a) to 4(d) by a block unit.
[0102] FIG. 5 is a diagram for a method of predicting coding
information for a current block using inter-view correlation
according to another embodiment of the present invention.
[0103] In a method of predicting a motion vector of a current
block, by finding a corresponding block existing at a view
different from that of the current block, it is able to predict
coding information for the current block using coding information
for the corresponding block. First of all, a method of finding the
corresponding block existing at the view different from that of the
current block is explained as follows.
[0104] For instance, a corresponding block may be a block indicated
by a view direction motion vector of a current block. In this case,
a view direction motion vector may means a vector representing
inter-view disparity or a global motion vector. The meaning of the
global motion vector is explained in the foregoing description of
FIG. 3. And, the global motion vector may represent a corresponding
macroblock position of neighboring view on the same temporal
instant. Referring to FIG. 5, pictures A and B exist in time Ta,
pictures C and D exist in time Tcurr, and pictures E and F exist in
time Tb. In this case, the pictures A, B, E and F in the time Ta or
Tb may be anchor pictures and the pictures C and D in the time
Tcurr may be non-anchor pictures. The pictures A, C and E exist at
a same view Vn and the pictures B, D and F exist at a same view Vm.
The picture C is the picture to be decoded. And, a corresponding
macroblock (MB) of the picture D is the block indicated by a view
direction global motion vector GDVcurr of a current block (current
MB). The global motion vector can be obtained by a macroblock unit
between the current picture and a picture at a view neighbor to
that of the current picture. In this case, information for the
neighbor view can be known by the information indicating inter-view
reference relation.
[0105] The information indicating inter-view reference relation
(view dependency) means the information indicating what kind
structure is used to predict interview sequences.
[0106] This can be obtained from a data area of a video signal. For
instance, the information can be obtained from a sequence parameter
set area. And, the inter-view reference information can be obtained
using the number of reference pictures and view information for the
reference pictures.
[0107] For instance, the number of total views is obtained. And,
view information discriminating each vie can be then obtained based
on the number of the total views. Moreover, the number of reference
pictures can be obtained for a reference direction of each view. It
is able to obtain view information for each of the reference
pictures according to the number of he reference pictures. In this
manner, the inter-view reference information can be obtained. And,
the inter-view reference information can be obtained according to a
case of anchor picture and a case of non-anchor picture. This can
be observed using anchor picture identification information
indicating whether a coded slice in a current NAL is an anchor
picture or not.
[0108] A method of obtaining the global motion vector may vary
according to the anchor picture identification information. For
instance, in case that a current picture is an anchor picture, it
is able to obtain the global motion vector from a received bit
stream. If a current picture is a non-anchor picture, the global
motion vector can be derived from that of the anchor picture.
[0109] In this case, information indicating a temporal distance is
usable together with the global motion vector of the anchor
picture. For instance, referring to FIG. 5, assuming that a global
motion vector of a picture A is named GDVa and that a global motion
vector of a picture E is named DGVb, a global motion vector of a
current picture C (non-anchor picture) can be obtained using the
global motion vectors of the pictures A and E (anchor pictures) and
temporal distance information. For instance, the temporal distance
information may include POC (picture order count) that indicates a
picture output sequence. So, the global motion vector of the
current picture can be derived using Formula 3.
GDV cur = GDV A + [ T cur - T A T B - T A .times. ( GDV B - GDV A )
] [ Formula 3 ] ##EQU00002##
[0110] Thus, a block indicated by the derived global motion vector
of the current picture can be regarded as a corresponding block to
predict coding information for the current block.
[0111] All motion information and mode information for the
corresponding block can be used to predict coding information for
the current block. The coding information can include such various
information necessary for coding the current block as motion
information, information for illumination compensation, weighted
prediction information and the like. In case that a motion skip
mode is applied to a current macroblock, instead of coding motion
information for the current block, motion information for a
previously coded picture at a different view can be used intact as
motion information for the current block. In this case, the motion
skip mode includes a case of obtaining motion information for a
current bock by depending on motion information for a corresponding
block at a neighbor view.
[0112] For instance, if a motion skip mode is applied to a current
macroblock, all motion information for the corresponding block,
e.g., macroblock type, reference index, motion vector, etc. can be
used intact as motion information for the current macroblock. Yet,
the motion skip mode may not be applicable to the following cases.
For instance, a current picture is the picture at a reference view
compatible with conventional codec or a current picture is an
anchor picture. Besides, the motion skip mode is applicable to a
case that a corresponding block exists at a neighbor view and is
coded in an inter-prediction mode. In case that the motion skip
mode is applied, motion information for a reference picture List0
is preferentially used according to the inter-view reference
information. And, motion information for a reference picture List1
is usable if necessary.
[0113] FIG. 6 is a diagram for a method of predicting coding
information for a current block according to one embodiment of the
present invention.
[0114] Coding information can be more accurately predicted using
coding information correlation in a view direction.
[0115] And, coding information for a current picture will have
great correlation with coding informations obtained from
pre-reference picture and post-reference picture in temporal
direction. So, more accurate prediction is available using theses
characteristics. Detailed embodiments thereof are explained with
reference to FIG. 6 as follows.
[0116] In FIG. 6, a horizontal axis indicates a temporal axis ( . .
. , T.sub.1, T.sub.2, T.sub.3, . . . ). For convenience of
explanation, a direction of a picture in a previous time is named a
forward direction and a direction of a picture in a post time is
named a backward direction, with reference to a current picture.
Likewise, a vertical axis indicates a view axis ( . . . , V.sub.1,
V.sub.2, V.sub.3, . . . ). For convenience of explanation, a
direction of a picture at a previous view V.sub.1 is named a
forward direction and a direction of a picture at a post view
V.sub.3 is named a backward direction, with reference to a current
picture.
[0117] According to one embodiment of the present invention, it is
able to predict view-direction motion informations using motion
informations obtained from previous and post pictures of a current
picture in a temporal direction. This method can use mode
information as well. For instance, by defining a motion information
prediction mode, it is able to inform whether the motion
information prediction mode shall be applied. In predicting
view-direction motion informations of a current block, it is able
to use a co-located block located at a same position as the
position of the current block among the pictures ahead of or behind
the current picture in a temporal direction. After a temporal
direction motion vector has been predicted from neighbor blocks to
the current block, it is able to use a block indicated by the
predicted motion vector.
[0118] For instance, in case that the motion information prediction
mode is applied, it is able to intactly use motion information for
a block located at a same position as the position of the current
block (or at a position indicated by a temporal direction motion
vector) in a pre-picture R1 or a post-picture R2 of the current
picture in the temporal direction. For instance, predictive
direction information, partition information, and a motion vector
value can be used intact. In this case, flag information indicating
that this mode will be applied and a block texture offset value can
be coded. According to another embodiment, predictive direction
information and partition information can be used intact. And,
motion vector values mv1, mv2, mv3 and mv4 of the co-located block
are usable as predicted motion vector values mvpL0 and mvpL1 of the
current block. So, in case that the motion information prediction
mode is applied, flag information indicating that the corresponding
mode will be applied, a motion vector offset value, and a block
texture offset value can be coded.
[0119] According to another embodiment, the predictive direction
information and the partition information for the current block in
the former embodiment may not use the predictive direction
information and the partition information for the co-located block
(or the block at the position indicated by the temporal direction
motion vector) as they are. For instance, a motion vector value of
the co-located block is used as a predicted motion vector value of
the current block, while predictive direction information and
partition information for the current block can be derived from
blocks neighbor to the current block. In this case, both of the
predictive direction information and the partition information can
be coded.
[0120] According to a further embodiment, if a current picture is
able to have a single reference picture in a view direction only, a
method of predicting coding information for the current picture is
explained. For instance, if forward prediction in a view direction
is available for a current picture or if forward prediction in a
view direction is available despite a plurality of reference
pictures, a method of predicting coding information for a current
picture is explained. In particular, with reference to a current
picture, it is assumed that there are forward and backward
reference pictures Ra and R2 in temporal direction. Coding
information for a block at a correspondent position of the current
block will be looked into. And, it will be looked into whether
blocks at a correspondent position of the reference picture have a
view-direction reference picture or a temporal-direction reference
picture.
[0121] If all of the blocks at the correspondent position of the
reference picture have the temporal-direction reference picture
only, the current block may not use coding information for the
blocks at the correspondent position of the reference picture. If
one of the blocks at the correspondent position of the reference
picture has a view-direction reference picture only, it is able to
predict view-direction coding information for the current block
using coding information for the corresponding block.
[0122] If both of the forward and backward reference pictures R1
and R2 in temporal direction refer to a picture in view direction,
coding information for the current block can be predicted from one
of the two reference pictures only. Using the coding information
for one of the two reference pictures can be decided according to
preset engagement. For instance, after partition information for
the two reference pictures has been recognized, the partition
information for the reference picture having a smaller partition
can be used. In particular, the partition information for the
current block can be predicted from the partition information for
the reference picture of which partition has a size smaller than
that of the other.
[0123] Detailed embodiment is explained with reference to FIG. 7 as
follows.
[0124] FIG. 7 is a table for various examples to select either a
forward reference block or a backward reference block to predict
partition information for a current block if both of the forward
and backward reference blocks in a temporal direction refer to a
block in a view direction according to one embodiment of the
present invention.
[0125] If one of partition informations of a plurality of reference
blocks is selected to predict partition information for a current
block, it is able to use partition information for the reference
block having a smaller partition among the partition informations
of a plurality of the reference blocks. If the partition
informations of the forward and backward reference blocks in
temporal direction have the same partition information, it is able
to predict the partition information for the current block as the
same information. Yet, in case they have different information, a
selection can be made.
[0126] For instance, if partition information for a forward
reference block in temporal direction indicates a direct mode and
if partition information for a backward reference block is
16.times.16, 16.times.8, 8.times.16 or 8.times.8, partition
information for a current block can be predicted as 16.times.16,
16.times.8, 8.times.16 or 8.times.8. If partition information for a
forward reference block in temporal direction indicates 16.times.8
and if partition information for a backward reference block
indicates 8.times.8, it is able to use a subdivided partition
8.times.8 as a predicted partition information value of a current
block. If partition information for a forward reference block in
temporal direction indicates 8.times.16 and if partition
information for a backward reference block indicates 16.times.8, it
is able to use a subdivided partition 8.times.8 as a predicted
partition information value of a current block.
[0127] The examples for the partition information in FIG. 7 may not
be applicable to the case of not attempting to use coding
information for a block at a correspondent position.
[0128] In this case, view-direction coding information for a
current block can be predicted using view-direction coding
information for a corresponding block only. For instance, if
partition information for a reference block and partition
information for a current block are matched, a view-direction
motion vector of a corresponding block can be used intact. In this
case, if both forward and backward reference blocks exist, it is
able to find a predicted motion vector value in a view direction by
averaging view-direction motion vectors of corresponding partitions
in both directions or applying a weight at a ratio of distance on a
temporal axis. Alternatively, information in one direction can be
used according to a given engagement.
[0129] If partition information for a reference block and partition
information for a current block are not matched, motion vectors
within blocks are averaged to use. For instance, when partition
information for a current block is 16.times.16 block, if a
reference block is predicted as 16.times.8, an average of motion
vectors of the two 16.times.8 reference blocks can be used as a
predicted motion vector value of the 16.times.16 current block. In
case that reference blocks exist in both forward and backward
directions, as mentioned in the foregoing description of the
method, after a predicted motion vector value has been found by
averaging motion vectors of partitions within the reference block
in each of the directions, motion vectors in a corresponding view
direction of both of the directions are averaged or a weight is
applied by a ratio of a distance on a temporal axis to find a final
predicted motion vector value.
[0130] Alternatively, a vector in one direction is usable according
to a given engagement.
[0131] Even if forward and backward reference pictures R1 and R2 in
temporal direction refer to pictures in view direction, like the
above example, either the reference picture R1 or the reference
picture R2 is usable. For instance, coding information for a
current block is predicted using either the forward reference
picture R1 or the backward reference picture R2.
[0132] And, flag information can be used to indicate whether both
of the two reference pictures R1 and R2, the forward reference
picture or the backward reference picture is used.
[0133] According to another embodiment, when a current picture is
able to have a plurality of reference pictures in view direction, a
method of predicting coding information for the current picture is
explained. For instance, when forward prediction, backward
prediction and bi-directional prediction in view direction are
available, a method of predicting coding information for a current
picture is explained. In this case, as mentioned in the foregoing
description of the former embodiments, the same manner is
applicable.
[0134] According to another embodiment, if both forward and
backward reference pictures R1 and R2 in temporal direction refer
to pictures in view direction, coding information for a current
block can be predicted from coding information for one of the two
reference pictures only. Using the coding information for one of
the two reference pictures can be decided according to preset
information. For instance, in case of predicting predictive
direction information of the coding information, predictive
direction information for the two reference pictures is obtained
and predictive direction information for the reference picture
capable of including them is then usable. Detailed embodiment is
explained with reference to FIG. 8.
[0135] FIG. 8 is a table for various examples to select either a
forward reference block or a backward reference block to predict
prediction direction information for a current block if both of the
forward and backward reference blocks in a temporal direction refer
to a block in a view direction according to one embodiment of the
present invention.
[0136] In case that one of predictive direction informations of a
plurality of reference blocks is selected to predict predictive
direction information for a current block, predictive direction
information for the reference picture capable of including all of
them among the predictive direction informations of a plurality of
the reference blocks can be used. If predictive direction
informations for forward and backward reference blocks in temporal
direction have the same predictive direction information,
predictive direction information for a current block can be
predicted as the same information. Yet, in case of having different
information, a selection can be made. For instance, if predictive
direction information for a forward reference block in temporal
direction indicates a forward direction in view direction and if
predictive direction information for a backward reference block in
temporal direction indicates a backward direction in view
direction, predictive direction information for a current block can
be predicted bi-directionally. If predictive direction information
for a forward reference block in temporal direction indicates
bi-directions in view direction and if predictive direction
information for a backward reference block in temporal direction
indicates a forward or backward direction in view direction,
bi-directions capable of including them can be predicted as
predictive direction information for a current block.
[0137] FIG. 9 is a flowchart of a method of predicting coding
information for a view direction of a current block according to
one embodiment of the present invention.
[0138] First of all, inter-view coding information prediction mode
information can be obtained from a video signal (S910). If
inter-view coding information prediction is applied by the
prediction mode information, it is checked whether a reference
block in temporal direction with reference to a current block has
coding information for a view direction (S920). In this case, the
reference block can be a co-located block located at a same
position as the position of the current block. Alternatively, the
reference block may be a block indicated by a motion vector derived
from neighbor blocks to the current block. In this case, if the
reference block does not have the coding information for the view
direction, the current block can predict the coding information for
the current block using coding information for the blocks neighbor
to the current block (S930). Yet, if the reference block has the
coding information for the view direction, various methods are
applicable. For instance, it is checked how many reference blocks
exist with the coding information for the view direction among the
whole reference blocks (S940). In case that a plurality of the
reference blocks have the coding information for the view direction
among the whole reference blocks, it is able to predict coding
information for the current block using the coding information for
the reference blocks. For instance, it is able to use partition
information or predictive direction information for the reference
block as partition information or predictive direction information
for the current block according to a preset engagement (S950). In
case that one of the reference blocks has the coding information
for the view direction, it is checked whether the reference block
has coding information for a temporal direction (S960). As a result
of the check, if the reference block does not have the coding
information for the temporal direction, it is able to use the
coding information for the view direction of the reference block as
a predicted value of the coding information for the current block
(S970). If this information is not usable, it is able to obtain a
predicted coding information value of the current block using
coding information for blocks neighbor to the current block
(S930).
[0139] FIG. 10 is a flowchart of a method of predicting coding
information for a current block according to one embodiment of the
present invention.
[0140] In multi-view video coding, it is able to use coding
information for a block neighbor to a current block to predict
coding information for the current block. For instance, the coding
information can include motion information, predictive direction
information, partition information, illumination compensation
information and the like. Yet, the neighbor blocks can refer to a
picture at a same view on a different time zone or a picture at a
different view on a same time zone. And, the neighbor blocks can
refer to a picture at a different view on a different time zone as
well. So, a method of using coding information for a neighbor block
selectively according to whether a picture referred to by a current
block is in a view direction or a temporal direction may be more
effective. Since correlation between coding information in view
direction and coding information in temporal direction is low, it
is able to differently design a coding information predicting
method in case of performing inter-view prediction. Detailed
embodiments are explained with reference to FIG. 10.
[0141] In FIG. 10, a horizontal axis indicates a temporal axis ( .
. . , T.sub.1, T.sub.2, T.sub.3, . . . ). For convenience of
explanation, a direction of a picture in a previous time T.sub.1 is
named a forward direction and a direction of a picture in a post
time T.sub.3 is named a backward direction, with reference to a
current picture (T.sub.2). Likewise, a vertical axis indicates a
view axis ( . . . , V.sub.1, V.sub.2, V.sub.3, . . . ). For
convenience of explanation, a direction of a picture at a previous
view V.sub.1 is named a forward direction and a direction of a
picture at a post view V.sub.3 is named a backward direction, with
reference to a current picture.
[0142] According to one embodiment of the present invention, coding
information for blocks A, B, C and D neighbor to a current block is
usable to predict coding information for a view direction of the
current block. For instance, in case that the neighbor block refers
to a picture in a view direction, it is able to use coding
information for the neighbor block as a predicted coding
information value of the current block. In case that the neighbor
block refers to a picture in a temporal direction, it is able to
set the coding information for the neighbor block to 0. In case
that predictive intra-coding is carried out on the neighbor block,
the coding information for the neighbor block is not used or set to
0.
[0143] According to another embodiment of the present invention, in
case that a block B neighbor to a current block refers to a picture
R3 in a temporal direction, it is able to predict coding
information in a view direction of the current block using coding
information for the reference picture R3 in the temporal direction.
For instance, a block at a position indicated by a motion vector of
a block B neighbor to a current block is firstly found. In this
case, like the reference picture R3 shown in FIG. 10, the block at
the position indicated by the motion vector of the neighbor block B
may not accurately coincide with one of the blocks within a
normalized lattice structure of the reference picture R3. Namely,
it may be overlapped with a plurality of blocks. In this case,
various embodiments are applicable to predict coding information
for a view direction of the current block.
[0144] According to one embodiment, it can be checked whether there
exists a block that refers to a picture in a view direction among
blocks overlapped with the block at the position indicated by the
motion vector of the neighbor block B. For instance, if there does
not exist a single block that refers to a picture in a view
direction among blocks overlapped with the block at the position
indicated by the motion vector of the neighbor block B (e.g., a
case of a block predicted in a temporal direction, a case of an
intra-predicted block), the overlapped blocks may not be used. As
the overlapped blocks are entirely removed, if it is unable to
predict a motion vector in a view direction of the current block,
the neighbor block B is not used to predict the motion vector of
the current block or set to 0.
[0145] Yet, if there exists at least one block that refers to a
picture in a view direction among the overlapped blocks, it is able
to use coding information for a block having a largest overlapped
area. For instance, it is able to use a motion vector value for a
view direction of the block having the largest overlapped area as a
predicted motion vector value for a view direction of the current
block. Alternatively, if at least tow areas are most widely
overlapped with the same area, it is able to find an average or
median of the largest overlapped blocks.
[0146] Alternatively, it is able to find a value generated from
applying a weight to each overlapped area. Alternatively, a method
of simply averaging the respective overlapped areas is usable. If
bi-directional prediction coding is carried out on a block used for
the overlapped area, the above-explained method is applicable in
using coding information in each view direction.
[0147] Since multi-view video coded pictures enable bi-directional
prediction in temporal direction, the following detailed algorithm
is applicable to each case.
[0148] First of all, in referring to a picture in a temporal
direction, if a predictive direction is either a forward direction
or a backward direction, the above-explained method can be intactly
applied. Yet, in case of bi-direction prediction in a temporal
direction, it is able to confirm whether a block found using a
motion vector in a temporal direction refers to a picture in a view
direction.
[0149] In case that one of tow directions refers to a picture in a
view direction only, it is able to use coding information for a
block that refers to a picture in the view direction only. Yet, if
the bi-directional reference blocks refer to the picture in the
view direction, it is able to use coding information for the
bi-directional reference blocks. In this case, in using the coding
information for the bi-directional reference blocks, the above
explanation is applicable intactly. For instance, it is able to use
coding information for a block that refers to a picture in a view
direction among blocks overlapped with a block at a position
indicated by a motion vector of a block neighbor to a current
block. And, details using methods can be applied in the same manner
of the above explanation.
[0150] Various embodiments can be provided as the case that the
bi-directional reference blocks refer to the picture in the view
direction. For instance, the reference blocks may correspond to
forward prediction, forward prediction or bi-directional prediction
in view direction. Likewise, the above explanation is intactly
applicable to each case.
[0151] In case that coding informations for the bi-directional
reference blocks are obtained, the coding informations are averaged
to be used as view-direction coding information for a block
neighbor to the current block. In this case, if a temporally
forward reference block differs from a temporally backward
reference block in a temporal distance, a weight is applicable.
[0152] After coding information in a view direction has been
predicted for each block neighbor to a current block by the
above-explained method, coding information for the current block
can be predicted. Alternatively, in the process for predicting
coding information for a current block by a related art method, it
is able to predict coding information in a view direction by the
above-explained method.
[0153] FIG. 11 is a flowchart of a method of predicting coding
information for a view direction of a current block according to
another embodiment of the present invention.
[0154] First of all, it is able to check whether a block neighbor
to a current block refers to a picture in a view direction (S1110).
In this case, it is able to check the neighbor blocks in a
predetermined order. For instance, a block (A) left to the current
block is firstly checked, a block (B) above the current block is
checked, a block (c) right to the upper block (B) is checked, and a
block (D) left to the upper block (B) is then checked.
Alternatively, after the upper block (B) has been firstly checked,
the left block (A) of the current block, the right block (C) of the
upper block (B), and the left block (D) of the upper block (B) can
be checked in sequence. If the neighbor block refers to a picture
in a view direction according to the checking process, it is able
to predict coding information for the current block using coding
information for the view direction of the neighbor block (S1120).
Yet, if the neighbor block does not refer to the picture in the
view direction, e.g., if the neighbor block is intra-coded, the
coding information for the neighbor block is not used or set to 0
(S1130).
[0155] According to another example of the case that the neighbor
block does not refer to a picture in a view direction, if the
neighbor block refers to a picture in a temporal direction, it is
able to search a reference block in the temporal direction (S1140).
And, it is able to check a part where the searched reference block
is overlapped with blocks of the reference picture in the temporal
direction. In this case, it is able to check whether there exists a
block that refers to a picture in a view direction among the
overlapped blocks (S1150). If the overlapped block does not refer
to the picture in the view direction, coding information for the
overlapped block is not used or set to 0 (S1160). Yet, if the
overlapped block refers to the picture in the view direction, it is
able to predict coding information for the current block using the
coding information for the view direction of the overlapped block
(S1170). In this case, if there exists at least one block that
refers to the picture in the view direction among the overlapped
blocks, it is able to use coding information for a block having a
largest overlapped area. For instance, it is able to use a motion
vector value for a view direction of the block having the largest
overlapped area as a predicted motion vector value for the view
direction of the current block. If at least two areas are most
widely overlapped with the same area, it is able to find an average
or median of the most widely overlapped blocks.
[0156] Alternatively, a method of applying a weight to each
overlapped area is applicable. If bi-directional prediction coding
in view direction is carried out on a block used in the overlapped
area, the method explained in FIG. 10 is applicable to using coding
information in each view direction.
[0157] As mentioned in the foregoing description, a signal
processing apparatus according to the present invention is provided
to a multimedia broadcast transmitting/receiving device for DMB
(digital multimedia broadcasting) and the like and then used to
decode a video signal, a data signal and the like. And, the
multimedia broadcast transmitting/receiving device can include a
mobile communication terminal.
[0158] A signal processing method according to the present
invention can be stored in a computer-readable recording medium by
being prepared as a program for execution in computer. And,
multimedia data having a data structure according to the present
invention can be stored in a computer-readable recording medium.
Moreover, the computer-readable recording media include all kinds
of storage devices for storing data readable by a computer system.
The computer-readable recording media include ROM, RAM, CD-ROM,
magnetic tape, floppy disc, optical data storage device, and the
like and also include a thing implemented with a carrier wave form
(e.g., transport via Internet). And, a bit stream generated by the
encoding method is stored in a computer-readable recording medium
or transported via wire/wireless communication network.
[0159] While the present invention has been described and
illustrated herein with reference to the preferred embodiments
thereof, it will be apparent to those skilled in the art that
various modifications and variations can be made therein without
departing from the spirit and scope of the invention. Thus, it is
intended that the present invention covers the modifications and
variations of this invention that come within the scope of the
appended claims and their equivalents.
INDUSTRIAL APPLICABILITY
[0160] Accordingly, the technique for compression encoding of video
signal data considers spatial duplication, temporal duplication,
scalable duplication, and inter-view duplication. In the
compression encoding process, it is able to perform compression
coding by considering mutual duplication existing between views.
The technique for the compression coding considering inter-view
duplication is just one embodiment of the present invention. And,
the technological idea of the present invention is applicable to
temporal duplication, scalable duplication, etc.
* * * * *