U.S. patent application number 13/001779 was filed with the patent office on 2011-06-23 for method and device for image encoding/decoding using arbitrary pixels in a sub-block.
This patent application is currently assigned to SK TELECOM CO., LTD.. Invention is credited to Byenngwoo Jeon, Dongwon Kim, Haekwang Kim, Hayoon Kim, Yunglyul Lee, Jooheee Moon, Mincheol Park.
Application Number | 20110150087 13/001779 |
Document ID | / |
Family ID | 41797639 |
Filed Date | 2011-06-23 |
United States Patent
Application |
20110150087 |
Kind Code |
A1 |
Kim; Hayoon ; et
al. |
June 23, 2011 |
METHOD AND DEVICE FOR IMAGE ENCODING/DECODING USING ARBITRARY
PIXELS IN A SUB-BLOCK
Abstract
A video encoding/decoding method and apparatus using arbitrary
pixels in each sub-block are disclosed. The present disclosure
comprises an extractor, a first predictor, a first subtractor, a
first transformer, a first quantizer, a first inverse quantizer, a
first inverse transformer, a first adder, a second predictor, a
second subtractor, a combiner for combining arbitrary residual
pixels, a second transformer, a second quantizer, and an encoder
whereby arbitrary pixels may be extracted from the respective
sub-blocks of a current block and then encoded and decoded into
reconstructed arbitrary pixels, which are used to predict and
encode the remaining pixels of the respective sub-blocks. According
to the disclosed encoding and decoding of a video, an intra
prediction of pixels in a block to currently encode or decode is
improved in accuracy to increase the video coding efficiency.
Inventors: |
Kim; Hayoon; (Gyeonggi-do,
KR) ; Moon; Jooheee; (Seoul, KR) ; Lee;
Yunglyul; (Seoul, KR) ; Kim; Haekwang; (Seoul,
KR) ; Jeon; Byenngwoo; (Gyeonggi-do, KR) ;
Kim; Dongwon; (Seoul, KR) ; Park; Mincheol;
(Gyeonggi-do, KR) |
Assignee: |
SK TELECOM CO., LTD.
Seoul
KR
|
Family ID: |
41797639 |
Appl. No.: |
13/001779 |
Filed: |
September 1, 2009 |
PCT Filed: |
September 1, 2009 |
PCT NO: |
PCT/KR09/04918 |
371 Date: |
February 4, 2011 |
Current U.S.
Class: |
375/240.12 ;
375/E7.214 |
Current CPC
Class: |
H04N 19/182 20141101;
H04N 19/119 20141101; H04N 19/176 20141101; H04N 19/11 20141101;
H04N 19/593 20141101; H04N 19/184 20141101; H04N 19/61 20141101;
H04N 19/59 20141101; H04N 19/159 20141101 |
Class at
Publication: |
375/240.12 ;
375/E07.214 |
International
Class: |
H04N 7/50 20060101
H04N007/50 |
Foreign Application Data
Date |
Code |
Application Number |
Sep 8, 2008 |
KR |
10-2008-0088088 |
Claims
1. An apparatus for encoding a video comprising: an extractor for
extracting arbitrary pixels from each of sub-blocks of a current
block of the video and generating an arbitrary pixel block; a first
predictor for predicting the arbitrary pixel block and generating a
predicted arbitrary pixel block; a first subtractor for subtracting
the predicted arbitrary pixel block from the arbitrary pixel block
and generating an arbitrary pixel residual block; a first
transformer for transforming the arbitrary pixel residual block; a
first quantizer for performing quantization with respect to the
transformed arbitrary pixel residual block; a first inverse
quantizer for performing inverse quantization with respect to the
quantized arbitrary pixel residual block; a first inverse
transformer for performing an inverse transform with respect to the
inverse quantized arbitrary pixel residual block; a first adder for
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; a second predictor for predicting the
remaining pixels in the respective sub-blocks by using adjacent
pixels to the current block and each of reconstructed arbitrary
pixels in the reconstructed arbitrary pixel block and generating
predicted blocks; a second subtractor for subtracting the predicted
blocks from the current block and generating residual blocks; a
combiner for combining each of inverse transformed arbitrary
residual pixels in the inverse transformed arbitrary pixel residual
block to the corresponding positions of the residual block; a
second transformer for transforming the residual blocks having the
respective inverse transformed arbitrary residual pixels combined;
a second quantizer for performing quantization with respect to the
transformed residual blocks; and an encoder for encoding the
quantized residual blocks and generating a bitstream.
2. The apparatus of claim 1, wherein the second transformer uses
basic vectors determined according to the sequence of a first
one-dimensional transform direction decided according to a
prediction direction of the first predictor.
3. The apparatus of claim 2, wherein the second transformer, when
applying a horizontal one-dimensional transform first and then a
vertical one-dimensional transform, performs the horizontal
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third rows
and a modified basic vector in their fourth rows, and performs the
vertical one-dimensional transform with respect to the respective
4.times.4 blocks using the modified basic vector in their first
columns and the modified basic vector in their second to fourth
columns.
4. The apparatus of claim 2, wherein the second transformer, when
applying a vertical one-dimensional transform first and then a
horizontal one-dimensional transform, performs the vertical
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third
columns and a modified basic vector in their fourth columns, and
performs the horizontal one-dimensional transform with respect to
the respective 4.times.4 blocks using the modified basic vector in
their first rows and the modified basic vector in their second to
fourth rows.
5. The apparatus of claim 1, further comprising: a second inverse
quantizer for performing inverse quantization with respect to the
quantized residual block; a second inverse transformer for
performing inverse transform with respect to the inverse quantized
residual block; a second adder for adding the inverse transformed
residual block to the predicted block and generating a
reconstructed current block; and a combiner for combining the
respective reconstructed arbitrary pixels in the reconstructed
arbitrary pixel block to the corresponding positions of the
reconstructed current block.
6. The apparatus of claim 5, wherein the second inverse transformer
uses basic vectors determined according to the sequence of a first
one-dimensional inverse transform direction.
7. The apparatus of claim 6, wherein the second inverse transformer
decides the sequence of the first one-dimensional inverse transform
direction in the reverse order of the sequence of the first
one-dimensional transform direction decided at the second
transformer.
8. The apparatus of claim 6, wherein the second inverse
transformer, when applying a horizontal one-dimensional inverse
transform first and then a vertical one-dimensional inverse
transform, performs the horizontal one-dimensional inverse
transform with respect to respective 4.times.4 blocks using a
modified basic vector in their first rows and an existing basic
vector in their second to fourth rows, and performs the vertical
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third columns and the modified basic vector in their fourth
columns.
9. The apparatus of claim 6, wherein the second inverse
transformer, when applying a vertical one-dimensional inverse
transform first and then a horizontal one-dimensional inverse
transform, performs the vertical one-dimensional inverse transform
with respect to respective 4.times.4 blocks using a modified basic
vector in their first columns and an existing basic vector in their
second to fourth columns, and performs the horizontal
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third rows and the modified basic vector in their fourth rows.
10. An apparatus for encoding a video comprising: an extractor for
extracting arbitrary pixels from each of sub-blocks of a current
block of the video and generating an arbitrary pixel block; a first
predictor for predicting the arbitrary pixel block and generating a
predicted arbitrary pixel block; a first subtractor for subtracting
the predicted arbitrary pixel block from the arbitrary pixel block
and generating an arbitrary pixel residual block; a first
transformer for transforming the arbitrary pixel residual block; a
first quantizer for performing quantization with respect to the
transformed arbitrary pixel residual block; a first inverse
quantizer for performing inverse quantization with respect to the
quantized arbitrary pixel residual block; a first inverse
transformer for performing inverse transform with respect to the
inverse quantized arbitrary pixel residual block; a first adder for
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; a second predictor for predicting the
remaining pixels in the respective sub-blocks by using adjacent
pixels to the respective sub-blocks and each of reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block and
generating respective predicted sub-blocks; a second subtractor for
subtracting the respective predicted sub-blocks from the respective
sub-blocks and generating respective residual sub-blocks; a
combiner for combining each of inverse transformed arbitrary
residual pixels in the inverse transformed arbitrary pixel residual
block to the corresponding positions of the respective residual
sub-blocks; a second transformer for transforming the respective
residual sub-blocks having the respective inverse transformed
arbitrary residual pixels combined; a second quantizer for
performing quantization with respect to the respective transformed
residual sub-blocks; and an encoder for encoding the respective
quantized residual sub-blocks and generating a bitstream, wherein
encoding and decoding with respect to a sub-block are carried out
after encoding and decoding with respect to another sub-block are
done.
11. The apparatus of claim 10, wherein the sequence of encoding and
decoding of the respective sub-blocks is the sequence of encoding
and decoding of intra.sub.--4.times.4 blocks in H.264/AVC or a
raster scanning direction.
12. The apparatus of claim 10, wherein the second predictor decides
a prediction direction of each of the sub-blocks and predicts the
remaining pixels in the respective sub-blocks in the decided
prediction directions.
13. The apparatus of claim 12, wherein the second predictor decides
as the prediction direction the direction of one of the adjacent
pixels to the respective sub-blocks toward the reconstructed
arbitrary pixel with the least difference in value between the
adjacent pixel.
14. The apparatus of claim 12, wherein the second predictor
generates information on a prediction mode according to the
prediction direction of each of the sub-blocks and transmits the
information to the encoder.
15. The apparatus of claim 1, wherein the first transformer
performs a transform and then an inverse transform with respect to
each of arbitrary pixel residual signals of the arbitrary pixel
residual block exclusively and independently.
16. The apparatus of claim 15, wherein the first transformer
performs the transform with respect to the respective arbitrary
pixel residual signals of the arbitrary pixel residual block
exclusively and independently by transforming the respective
arbitrary pixel residual signals with a modified basic vector.
17. The apparatus of claim 16, wherein the first transformer
further performs Hadamard Transform after performing the
transform.
18. The apparatus of claim 1, wherein the first inverse transformer
performs the inverse transform with respect to the respective
arbitrary pixel residual signals exclusively and independently by
inverse transforming frequency coefficients from the transform of
the respective arbitrary pixel residual signals by using a modified
basic vector.
19. The apparatus of claim 18, wherein the first inverse
transformer performs inverse Hadamard Transform with respect to the
frequency coefficients generated from the transform of the
respective arbitrary pixel residual signals and then performs the
inverse transform.
20. The apparatus of claim 10, wherein the second transformer uses
basic vectors determined according to the sequence of a first
one-dimensional transform direction decided according to a
prediction direction of the first predictor.
21. The apparatus of claim 20, wherein the second transformer, when
applying a horizontal one-dimensional transform first and then a
vertical one-dimensional transform, performs the horizontal
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third rows
and a modified basic vector in their fourth rows, and performs the
vertical one-dimensional transform with respect to the respective
4.times.4 blocks using the modified basic vector in their first
columns and the modified basic vector in their second to fourth
columns.
22. The apparatus of claim 20, wherein the second transformer, when
applying a vertical one-dimensional transform first and then a
horizontal one-dimensional transform, performs the vertical
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third
columns and a modified basic vector in their fourth columns, and
performs the horizontal one-dimensional transform with respect to
the respective 4.times.4 blocks using the modified basic vector in
their first rows and the modified basic vector in their second to
fourth rows.
23. The apparatus of claim 10, further comprising: a second inverse
quantizer for performing inverse quantization with respect to the
respective quantized residual sub-blocks; a second inverse
transformer for performing inverse transform with respect to the
respective inverse quantized residual sub-blocks; a second adder
for adding the respective inverse transformed residual sub-blocks
to the respective predicted sub-blocks and generating corresponding
reconstructed sub-blocks; and a combiner for combining the
respective reconstructed arbitrary pixels in the reconstructed
arbitrary pixel block to the corresponding positions of the
respective reconstructed sub-blocks, wherein the first predictor
and the second predictor store the respective sub-blocks having the
respective reconstructed arbitrary pixels combined and predict the
subsequent sub-blocks using the respective stored sub-blocks.
24. The apparatus of claim 23, wherein the second inverse
transformer uses basic vectors determined according to the sequence
of a first one-dimensional inverse transform direction.
25. The apparatus of claim 24, wherein the second inverse
transformer decides the sequence of the first one-dimensional
inverse transform direction in the reverse order of the sequence of
the first one-dimensional transform direction decided at the second
transformer.
26. The apparatus of claim 24, wherein the second inverse
transformer, when applying a horizontal one-dimensional inverse
transform first and then a vertical one-dimensional inverse
transform, performs the horizontal one-dimensional inverse
transform with respect to respective 4.times.4 blocks using a
modified basic vector in their first rows and an existing basic
vector in their second to fourth rows, and performs the vertical
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third columns and the modified basic vector in their fourth
columns.
27. The apparatus of claim 24, wherein the second inverse
transformer, when applying a vertical one-dimensional inverse
transform first and then a horizontal one-dimensional inverse
transform, performs the vertical one-dimensional inverse transform
with respect to respective 4.times.4 blocks using a modified basic
vector in their first columns and an existing basic vector in their
second to fourth columns, and performs the horizontal
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third rows and the modified basic vector in their fourth rows.
28. The apparatus of claim 23, wherein the second inverse
transformer performs inverse transform with respect to frequency
coefficients using the existing basic vector.
29. The apparatus of claim 23, wherein the second inverse
transformer uses some of the frequency coefficients and some of the
respective reconstructed arbitrary pixels to predict the remaining
frequency coefficients.
30. The apparatus of claim 1, wherein the first predictor performs
a low-pass filtering on the adjacent pixels to the current block
and predicts the arbitrary pixel block using down-sampled
pixels.
31. The apparatus of claim 1, wherein the first and second
predictors predict in the same direction as a prediction direction
according to the prediction mode of the current block.
32. A method for encoding a video comprising: extracting arbitrary
pixels from each sub-block of a current block of the video and
generating an arbitrary pixel block; firstly predicting the
arbitrary pixel block and generating a predicted arbitrary pixel
block; firstly subtracting the predicted arbitrary pixel block from
the arbitrary pixel block and generating an arbitrary pixel
residual block; firstly transforming the arbitrary pixel residual
block; firstly performing quantization with respect to the
transformed arbitrary pixel residual block; firstly performing
inverse quantization with respect to the quantized arbitrary pixel
residual block; firstly performing inverse transform with respect
to the inverse quantized arbitrary pixel residual block; firstly
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; secondly predicting the remaining pixels in
the respective sub-blocks by using adjacent pixels to the current
block and each of reconstructed arbitrary pixels in the
reconstructed arbitrary pixel block and generating predicted
blocks; secondly subtracting the predicted blocks from the current
block and generating residual blocks; combining each of inverse
transformed arbitrary residual pixels in the inverse transformed
arbitrary pixel residual block to the corresponding positions of
the residual block; secondly transforming the residual blocks
having the respective inverse transformed arbitrary residual pixels
combined; secondly performing quantization with respect to the
transformed residual blocks; and encoding the quantized residual
blocks and generating a bitstream.
33. The method of claim 32, further comprising: secondly performing
inverse quantization with respect to the quantized residual block;
secondly performing inverse transform with respect to the inverse
quantized residual block; secondly adding the inverse transformed
residual block to the predicted block and generating a
reconstructed current block; and combining the respective
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block to the corresponding positions of the reconstructed current
block.
34. A method for encoding a video comprising: extracting arbitrary
pixels from each sub-block of a current block of the video and
generating an arbitrary pixel block; firstly predicting the
arbitrary pixel block and generating a predicted arbitrary pixel
block; firstly subtracting the predicted arbitrary pixel block from
the arbitrary pixel block and generating an arbitrary pixel
residual block; firstly transforming the arbitrary pixel residual
block; firstly performing quantization with respect to the
transformed arbitrary pixel residual block; firstly performing
inverse quantization with respect to the quantized arbitrary pixel
residual block; firstly performing inverse transform with respect
to the inverse quantized arbitrary pixel residual block; firstly
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; secondly predicting the remaining pixels in
the respective sub-blocks by using adjacent pixels to the
respective sub-blocks and each of reconstructed arbitrary pixels in
the reconstructed arbitrary pixel block and generating respective
predicted sub-blocks; secondly subtracting the respective predicted
sub-blocks from the respective sub-blocks and generating respective
residual sub-blocks; combining each of inverse transformed
arbitrary residual pixels in the inverse transformed arbitrary
pixel residual block to the corresponding positions of the
respective residual sub-blocks; secondly transforming the
respective residual sub-blocks having the respective inverse
transformed arbitrary residual pixels combined; secondly performing
quantization with respect to the respective transformed residual
sub-blocks; and encoding the respective quantized residual
sub-blocks and generating a bitstream, wherein encoding and
decoding with respect to a sub-block are carried out after encoding
and decoding with respect to another sub-block are done.
35. The method of claim 34, further comprising: secondly performing
inverse quantization with respect to the respective quantized
residual sub-blocks; secondly performing inverse transform with
respect to the respective inverse quantized residual sub-blocks;
secondly adding the respective inverse transformed residual
sub-blocks to the respective predicted sub-blocks and generating
corresponding reconstructed sub-blocks; and combining the
respective reconstructed arbitrary pixels in the reconstructed
arbitrary pixel block to the corresponding positions of the
respective reconstructed sub-blocks, wherein the firstly predicting
and the secondly predicting store the respective sub-blocks having
the respective reconstructed arbitrary pixels combined and predict
the subsequent sub-blocks using the respective stored
sub-blocks.
36. An apparatus for decoding a video comprising: a decoder for
decoding a bitstream and extracting a residual block and a
prediction mode; an inverse quantizer for performing inverse
quantization with respect to the residual block; an extractor for
extracting frequency coefficients corresponding to positions of
arbitrary pixels in each of sub-blocks from the inverse quantized
residual block; a first inverse transformer for performing inverse
transform with respect to the extracted frequency coefficients and
generating an inverse transformed arbitrary pixel residual block; a
first predictor for predicting an arbitrary pixel block in a
prediction direction according to the prediction mode and
generating a predicted arbitrary pixel block; a first adder for
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; a second inverse transformer for performing
inverse transform with respect to the inverse quantized residual
block and generating an inverse transformed residual block; a
second predictor for predicting a current block of the video in a
prediction direction according to the prediction mode by using
adjacent pixels to the current block and each of reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block and
generating a predicted current block; a second adder for generating
a reconstructed current block by using the inverse transformed
residual block and the predicted current block; and a combiner for
combining the respective reconstructed arbitrary pixels to the
corresponding positions of the reconstructed current block.
37. The apparatus of claim 36, wherein the second inverse
transformer uses basic vectors determined according to the sequence
of a first one-dimensional inverse transform direction.
38. The apparatus of claim 37, wherein the second inverse
transformer decides the sequence of the first one-dimensional
inverse transform direction in the reverse order of the sequence of
the first one-dimensional transform direction decided at the second
transformer.
39. The apparatus of claim 37, wherein the second inverse
transformer, when applying a horizontal one-dimensional inverse
transform first and then a vertical one-dimensional inverse
transform, performs the horizontal one-dimensional inverse
transform with respect to respective 4.times.4 blocks using a
modified basic vector in their first rows and an existing basic
vector in their second to fourth rows, and performs the vertical
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third columns and the modified basic vector in their fourth
columns.
40. The apparatus of claim 37, wherein the second inverse
transformer, when applying a vertical one-dimensional inverse
transform first and then a horizontal one-dimensional inverse
transform, performs the vertical one-dimensional inverse transform
with respect to respective 4.times.4 blocks using a modified basic
vector in their first columns and an existing basic vector in their
second to fourth columns, and performs the horizontal
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third rows and the modified basic vector in their fourth rows.
41. The apparatus of claim 36, wherein the first predictor performs
a low-pass filtering on the adjacent pixels to the current block
and predicts the arbitrary pixel block using down-sampled
pixels.
42. The apparatus of claim 36, wherein the first and second
predictors predict in the same direction as a prediction direction
according to the prediction mode of the current block.
43. A method for decoding a video comprising: decoding a bitstream
to extract a residual block and a prediction mode; performing
inverse quantization with respect to the residual block; extracting
frequency coefficients corresponding to positions of arbitrary
pixels in each sub-block from the inverse quantized residual block;
firstly performing inverse transform with respect to the extracted
frequency coefficients and generating a reconstructed arbitrary
pixel residual block; firstly predicting an arbitrary pixel block
in a prediction direction according to the prediction mode and
generating a predicted arbitrary pixel block; firstly adding the
reconstructed arbitrary pixel residual block to the predicted
arbitrary pixel block and generating a reconstructed arbitrary
pixel block; secondly performing inverse transform with respect to
the inverse quantized residual block and generating an inverse
transformed residual block; secondly predicting a current block of
the video in a prediction direction according to the prediction
mode by using adjacent pixels to the current block and each of
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block and generating a predicted current block; secondly adding for
generating a reconstructed current block by using the inverse
transformed residual block and the predicted current block; and
combining the respective reconstructed arbitrary pixels to the
corresponding positions of the reconstructed current block.
44. The apparatus of claim 10, wherein the first transformer
performs a transform and then an inverse transform with respect to
each of arbitrary pixel residual signals of the arbitrary pixel
residual block exclusively and independently.
45. The apparatus of claim 10, wherein the first inverse
transformer performs the inverse transform with respect to the
respective arbitrary pixel residual signals exclusively and
independently by inverse transforming frequency coefficients from
the transform of the respective arbitrary pixel residual signals by
using a modified basic vector.
46. The apparatus of claim 10, wherein the first predictor performs
a low-pass filtering on the adjacent pixels to the current block
and predicts the arbitrary pixel block using down-sampled pixels.
Description
TECHNICAL FIELD
[0001] The present disclosure relates to a video encoding/decoding
method and apparatus. More particularly, the present disclosure
relates to a video encoding/decoding method and apparatus with more
accurate prediction of pixels of a block to currently encode or
decode in order to improve the encoding efficiency.
BACKGROUND ART
[0002] The statements in this section merely provide background
information related to the present disclosure and may not
constitute prior art.
[0003] Moving Picture Experts Group (MPEG) and Video Coding Experts
Group (VCEG) have developed an improved and excellent video
compression technology over existing MPEG-4 Part 2 and H.263
standards. The new standard is named H.264/AVC (Advanced Video
Coding) and was released simultaneously as MPEG-4 Part 10 AVC and
ITU-T Recommendation H.264. Such H.264/AVC (hereinafter referred to
as `H.264`) uses a spatial predictive coding method, which is
different from conventional video coding international standards
such as MPEG-1, MPEG-2, MPEG-4 Part 2 Visual and the like.
[0004] Conventional video coding methods use "intra prediction" for
coefficients transformed in Discrete Cosine Transform Domain (or
DCT Transform Domain) to seek higher encoding efficiency resulting
in degradation of the subjective video quality at low band
transmission bit rates. However, H.264 adopts a method of encoding
based on a spatial intra prediction in a spatial domain rather than
in a transform domain.
[0005] Encoders using an encoding method based on the conventional
spatial intra predictions predict a block to currently encode from
information of the previously encoded and reproduced previous
blocks, encode information on just the difference of the predicted
block from the actual block to encode and, transmit the encoded
information to a decoder. At the same time, the encoder may
transmit parameters needed for prediction of the block to the
decoder, or the encoder and decoder may be synchronized, so that
they may share the needed parameters for the decoder to predict the
block. At the decoder, prediction of the current block is carried
out by using neighboring blocks upon completion of their decoding
and then the difference information transmitted from the encoder is
added to the predicted current block to reconstruct the target
block information to presently decode. Then, again, if the
parameters needed for the prediction are transmitted from the
encoder, the decoder uses the parameters in predicting the current
block.
[0006] However, when a prediction is performed with respect to the
current block according to the typical video encoding/decoding
methods, the previously encoded adjacent pixels included in the
blocks neighboring (mostly at the left and upper sides of) the
block to presently encode, i.e. current block. In this occasion,
the current block pixels which are farther from the adjacent pixels
in the neighboring blocks are predicted by using the adjacent
pixels that have a greater distance and thus the accuracy of
prediction may be lowered. The less accurate prediction will
increase the difference between the actual pixel value and the
predicted pixel value, resulting in decreased compression
efficiency.
DISCLOSURE
Technical Problem
[0007] Therefore, the present disclosure has been made in view of
the above mentioned problems by improving the prediction accuracy
of intra predicting the block to presently encode/decode in order
to increase the encoding efficiency.
Technical Solution
[0008] One embodiment of the present disclosure provides an
apparatus for encoding a video comprising an extractor for
extracting arbitrary pixels from each sub-block of a current block
of the video and generating an arbitrary pixel block; a first
predictor for predicting the arbitrary pixel block and generating a
predicted arbitrary pixel block; a first subtractor for subtracting
the predicted arbitrary pixel block from the arbitrary pixel block
and generating an arbitrary pixel residual block; a first
transformer for transforming the arbitrary pixel residual block; a
first quantizer for performing quantization with respect to the
transformed arbitrary pixel residual block; a first inverse
quantizer for performing inverse quantization with respect to the
quantized arbitrary pixel residual block; a first inverse
transformer for performing an inverse transform with respect to the
inverse quantized arbitrary pixel residual block; a first adder for
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; a second predictor for predicting the
remaining pixels in the respective sub-blocks by using adjacent
pixels to the current block and each of reconstructed arbitrary
pixels in the reconstructed arbitrary pixel block and generating
predicted blocks; a second subtractor for subtracting the predicted
blocks from the current block and generating residual blocks; a
combiner for combining each of inverse transformed arbitrary
residual pixels in the inverse transformed arbitrary pixel residual
block to the corresponding positions of the residual block; a
second transformer for transforming the residual blocks having the
respective inverse transformed arbitrary residual pixels combined;
a second quantizer for performing quantization with respective to
the transformed residual blocks; and an encoder for encoding the
quantized residual blocks and generating a bitstream.
[0009] Another embodiment of the present disclosure provides an
apparatus for encoding a video comprising an extractor for
extracting arbitrary pixels from each sub-block of a current block
of the video and generating an arbitrary pixel block; a first
predictor for predicting the arbitrary pixel block and generating a
predicted arbitrary pixel block; a first subtractor for subtracting
the predicted arbitrary pixel block from the arbitrary pixel block
and generating an arbitrary pixel residual block; a first
transformer for transforming the arbitrary pixel residual block; a
first quantizer for performing quantization with respect to the
transformed arbitrary pixel residual block; a first inverse
quantizer for performing inverse quantization with respect to the
quantized arbitrary pixel residual block; a first inverse
transformer for performing an inverse transform with respect to the
inverse quantized arbitrary pixel residual block; a first adder for
adding the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; a second predictor for predicting the
remaining pixels in the respective sub-blocks by using adjacent
pixels to the respective sub-blocks and each of reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block and
generating respective predicted sub-blocks; a second subtractor for
subtracting the respective predicted sub-blocks from the respective
sub-blocks and generating respective residual sub-blocks; a
combiner for combining each of inverse transformed arbitrary
residual pixels in the inverse transformed arbitrary pixel residual
block to the corresponding positions of the respective residual
sub-blocks; a second transformer for transforming the respective
residual sub-blocks having the respective inverse transformed
arbitrary residual pixels combined; a second quantizer for
performing quantization with respective to the respective
transformed residual sub-blocks; and an encoder for encoding the
respective quantized residual sub-blocks and generating a
bitstream, wherein encoding and decoding with respect to a
sub-block are carried out after encoding and decoding with respect
to another sub-block are done.
[0010] Yet another embodiment of the present disclosure provides a
method for encoding a video comprising extracting arbitrary pixels
from each sub-block of a current block of the video and generating
an arbitrary pixel block; firstly predicting the arbitrary pixel
block and generating a predicted arbitrary pixel block; firstly
subtracting the predicted arbitrary pixel block from the arbitrary
pixel block and generating an arbitrary pixel residual block;
firstly transforming the arbitrary pixel residual block; firstly
performing quantization with respect to the transformed arbitrary
pixel residual block; firstly performing inverse quantization with
respect to the quantized arbitrary pixel residual block; firstly
performing an inverse transform with respect to the inverse
quantized arbitrary pixel residual block; firstly adding the
inverse transformed arbitrary pixel residual block to the predicted
arbitrary pixel block and generating a reconstructed arbitrary
pixel block; secondly predicting the remaining pixels in the
respective sub-blocks by using adjacent pixels to the current block
and each of reconstructed arbitrary pixels in the reconstructed
arbitrary pixel block and generating predicted blocks; secondly
subtracting the predicted blocks from the current block and
generating residual blocks; combining each of inverse transformed
arbitrary residual pixels in the inverse transformed arbitrary
pixel residual block to the corresponding positions of the residual
block; secondly transforming the residual blocks having the
respective inverse transformed arbitrary residual pixels combined;
secondly performing quantization with respective to the transformed
residual blocks; and encoding the quantized residual blocks and
generating a bitstream.
[0011] Yet another embodiment of the present disclosure provides a
method for encoding a video comprising extracting arbitrary pixels
from each sub-block of a current block of the video and generating
an arbitrary pixel block; firstly predicting the arbitrary pixel
block and generating a predicted arbitrary pixel block; firstly
subtracting the predicted arbitrary pixel block from the arbitrary
pixel block and generating an arbitrary pixel residual block;
firstly transforming the arbitrary pixel residual block; firstly
performing quantization with respect to the transformed arbitrary
pixel residual block; firstly performing inverse quantization with
respect to the quantized arbitrary pixel residual block; firstly
performing an inverse transform with respect to the inverse
quantized arbitrary pixel residual block; firstly adding the
inverse transformed arbitrary pixel residual block to the predicted
arbitrary pixel block and generating a reconstructed arbitrary
pixel block; secondly predicting the remaining pixels in the
respective sub-blocks by using adjacent pixels to the respective
sub-blocks and each of reconstructed arbitrary pixels in the
reconstructed arbitrary pixel block and generating respective
predicted sub-blocks; secondly subtracting the respective predicted
sub-blocks from the respective sub-blocks and generating respective
residual sub-blocks; combining each of inverse transformed
arbitrary residual pixels in the inverse transformed arbitrary
pixel residual block to the corresponding positions of the
respective residual sub-blocks; secondly transforming the
respective residual sub-blocks having the respective inverse
transformed arbitrary residual pixels combined; secondly performing
quantization with respect to the respective transformed residual
sub-blocks; and encoding the respective quantized residual
sub-blocks and generating a bitstream, wherein encoding and
decoding with respect to a sub-block are carried out after encoding
and decoding with respect to another sub-block are done.
[0012] Yet another embodiment of the present disclosure provides an
apparatus for decoding a video comprising a decoder for decoding a
bitstream to extract a residual block and a prediction mode; an
inverse quantizer for performing inverse quantization with respect
to the residual block; an extractor for extracting frequency
coefficients corresponding to positions of arbitrary pixels in
respective sub-blocks from the inverse quantized residual block; a
first inverse transformer for performing an inverse transform with
respect to the extracted frequency coefficients and generating an
inverse transformed arbitrary pixel residual block; a first
predictor for predicting an arbitrary pixel block in a prediction
direction according to the prediction mode and generating a
predicted arbitrary pixel block; a first adder for adding the
inverse transformed arbitrary pixel residual block to the predicted
arbitrary pixel block and generating a reconstructed arbitrary
pixel block; a second inverse transformer for performing the
inverse transform with respect to the inverse quantized residual
block and generating an inverse transformed residual block; a
second predictor for predicting a current block of the video in a
prediction direction according to the prediction mode by using
adjacent pixels to the current block and each of reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block and
generating a predicted current block; a second adder for generating
a reconstructed current block by using the inverse transformed
residual block and the predicted current block; and a combiner for
combining the respective reconstructed arbitrary pixels to the
corresponding positions of the reconstructed current block.
[0013] Yet another embodiment of the present disclosure provides a
method for decoding a video comprising decoding a bitstream to
extract a residual block and a prediction mode; performing inverse
quantization with respect to the residual block; extracting
frequency coefficients corresponding to positions of arbitrary
pixels in respective sub-blocks from the inverse quantized residual
block; firstly performing an inverse transform with respect to the
extracted frequency coefficients and generating a reconstructed
arbitrary pixel residual block; firstly predicting an arbitrary
pixel block in a prediction direction according to the prediction
mode and generating a predicted arbitrary pixel block; firstly
adding the reconstructed arbitrary pixel residual block to the
predicted arbitrary pixel block and generating a reconstructed
arbitrary pixel block; secondly performing the inverse transform
with respect to the inverse quantized residual block and generating
an inverse transformed residual block; secondly predicting a
current block of the video in a prediction direction according to
the prediction mode by using adjacent pixels to the current block
and each of reconstructed arbitrary pixels in the reconstructed
arbitrary pixel block and generating a predicted current block;
secondly adding for generating a reconstructed current block by
using the inverse transformed residual block and the predicted
current block; and combining the respective reconstructed arbitrary
pixels to the corresponding positions of the reconstructed current
block.
ADVANTAGEOUS EFFECTS
[0014] As described above, according to the disclosure for encoding
and decoding a video, an intra prediction of pixels in a block to
currently encode or decode is improved in accuracy of prediction to
increase the video coding efficiency.
[0015] Further areas of applicability will become apparent from the
description provided herein. It should be understood that the
description and specific examples are intended for purposes of
illustration only and are not intended to limit the scope of the
present disclosure.
DESCRIPTION OF DRAWINGS
[0016] The drawings described herein are for illustration purposes
only and are not intended to limit the scope of the present
disclosure in any way.
[0017] FIG. 1 is a block diagram of a video encoding apparatus;
[0018] FIGS. 2 and 3 are exemplary diagrams showing intra
prediction modes;
[0019] FIGS. 4 and 5 are exemplary diagrams showing basic vectors
for DCT transform and inverse transform, respectively;
[0020] FIG. 6 is an exemplary diagram showing arbitrary pixels in a
sub-block according to an embodiment;
[0021] FIG. 7 is a block diagram of a video encoding apparatus
according to an embodiment of the present disclosure;
[0022] FIG. 8 is an exemplary diagram showing an extraction of an
arbitrary pixel block from a current macroblock;
[0023] FIG. 9 is an exemplary diagram showing predictions of an
arbitrary pixel block using adjacent pixels to the current
macroblock;
[0024] FIGS. 10 and 11 are exemplary diagrams showing modified
basic vectors for forward and inverse DCT transforms,
respectively;
[0025] FIGS. 12 to 15 are exemplary diagrams showing predictions of
the remaining pixels in each of sub-blocks;
[0026] FIG. 16 is an exemplary diagram showing a one-dimensional
transform in preparation for a two-dimensional DCT transform;
[0027] FIGS. 17 and 18 are exemplary diagrams showing an inverse
transform procedure using the modified basic vectors according to
the sequential directions of the inverse transform;
[0028] FIG. 19 is a flow diagram for illustrating a video encoding
method according to an embodiment;
[0029] FIG. 20 is a flow diagram for illustrating the procedure of
reconstructing a current macroblock according to an embodiment;
[0030] FIG. 21 is a block diagram of a video decoding apparatus
according to an embodiment;
[0031] FIG. 22 is a flow diagram of a video decoding method
according to an embodiment of the present disclosure;
[0032] FIG. 23 is an exemplary diagram showing the sequence of
encoding the respective sub-blocks according to another
embodiment;
[0033] FIG. 24 is an exemplary diagram showing decisions of
prediction directions of the respective sub-blocks according to yet
another embodiment;
[0034] FIG. 25 is an exemplary diagram showing the sequences of
encoding and decoding of the respective sub-blocks according to the
direction of raster scanning;
[0035] FIG. 26 is an exemplary diagram showing predictions of the
respective sub-blocks according to newly decided optimal prediction
directions; and
[0036] FIGS. 27 and 28 are exemplary diagrams showing the
procedures of using partial frequency coefficients to predict the
remaining frequency coefficients according to an embodiment and the
alternative embodiment, respectively.
MODE FOR INVENTION
[0037] Hereinafter, an exemplary embodiment of the present
disclosure will be described with reference to the accompanying
drawings. In the following description, the same elements will be
designated by the same reference numerals although they are shown
in different drawings. Further, in the following description of the
present disclosure, a detailed description of known functions and
configurations incorporated herein will be omitted when it may make
the subject matter of the present disclosure rather unclear.
[0038] Also, in describing the components of the present
disclosure, there may be terms used like first, second, A, B, (a),
and (b). These are solely for the purpose of differentiating one
component from the other but not to imply or suggest the
substances, order or sequence of the components. If a component
were described as `connected`, `coupled`, or `linked` to another
component, they may mean the components are not only directly
`connected`, `coupled`, or `linked` but also are indirectly
`connected`, `coupled`, or `linked` via a third component.
[0039] FIG. 1 is a schematic electronic representation of a video
encoding apparatus 100 according to an embodiment of the present
disclosure.
[0040] Video encoding apparatus 100 comprises a predictor 110, a
subtractor 120, a transformer 130, a quantizer 140, an encoder 150,
an inverse quantizer 160, an inverse transformer 170, and an adder
180.
[0041] Predictor 110 predicts the current video block that is to be
encoded at the present time and generates a predicted block. In
other words, predictor 110 predicts the pixel value of each of
pixels in current block to encode out of a video according to a
predetermined optimal prediction mode to generate a predicted block
having a predicted pixel value. Predictor 110 also supplies encoder
150 with prediction mode information, which may then be
encoded.
[0042] Here, the optimal prediction mode refers to the prediction
mode determined as having the lowest encoding cost among various
intra prediction modes (i.e., respectively nine modes in the
intra.sub.--8.times.8 prediction and the intra.sub.--4.times.4
prediction and four modes in the intra.sub.--16.times.16
prediction).
[0043] FIGS. 2 and 3 are exemplary diagrams showing intra
prediction modes.
[0044] Referring to FIG. 2, the intra.sub.--4.times.4 prediction
has 9 kinds of prediction modes including a vertical mode, a
horizontal mode, a DC (Direct Current) mode, a diagonal down-left
mode, a diagonal down-right mode, a vertical-right mode, a
horizontal-down mode, a vertical-left mode, and a horizontal-up
mode. The intra.sub.--8.times.8 prediction has 9 kinds of
prediction modes similar to those of the intra.sub.--4.times.4
prediction. Referring also to FIG. 3, the intra.sub.--16.times.16
prediction has 4 kinds of prediction modes including a vertical
mode, a horizontal mode, a DC mode, and a plane mode.
[0045] Predictor 110 may calculate encoding costs according to
either the illustrated nine prediction modes or the four prediction
modes depending on the block modes or the block size of the current
block and then decide the prediction mode with the lowest cost to
encode as the optimal prediction mode.
[0046] Subtractor 120 generates a residual block by subtracting the
predicted block from the current block. Specifically, subtractor
120 generates a residual block that is a residual signal from the
calculated difference between the pixel values of the respective
pixels in the current block and the predicted pixel values of the
respective pixels in the predicted block at predictor 110.
[0047] Transformer 130 transforms the residual block to the
frequency domain to effect the transform of the respective pixel
values of the residual block into frequency coefficients. Here, the
transform of the residual signals into the frequency domain at
transformer 130 may be through various techniques of transforming
video signals on the time axis into the frequency axis such as
Hadamard transform, discrete cosine transform (DCT) based transform
and others where the residual signal transformed into the frequency
domain becomes the frequency coefficient.
[0048] FIGS. 4 and 5 are exemplary diagrams showing basic vectors
for DCT transform and inverse transform, respectively. For example,
transformer 130 uses the basic vectors created according to
Equation 1 as shown in FIG. 4 to perform the one-dimensional DCT in
the horizontal direction doing the same in the vertical direction
to complete a two-dimensional DCT, thereby transforming the pixel
value of the spatial domain into the frequency domain. At this
time, if the block mode is intra.sub.--16.times.16, the DC values
of 16 (4.times.4) frequency coefficients are exclusively and
respectively collected for the 16 sub-blocks in the 4.times.4 block
size and then subject to the Hadamard transform as in Equation
2.
T dct ( i , j ) = C i cos ( 2 j + 1 ) 2 N , C i = { 1 N , i = 0 2 N
, otherwise ##EQU00001## T dct = [ 1 2 1 2 1 2 1 2 1 2 cos ( .pi. 8
) 1 2 cos ( 3 .pi. 8 ) - 1 2 cos ( 3 .pi. 8 ) - 1 2 cos ( .pi. 8 )
1 2 - 1 2 - 1 2 1 2 1 2 cos ( 3 .pi. 8 ) - 1 2 cos ( .pi. 8 ) 1 2
cos ( .pi. 8 ) - 1 2 cos ( 3 .pi. 8 ) ] Equation 1 Forward Y D = 1
2 [ 1 1 1 1 1 - 1 1 - 1 1 1 - 1 - 1 1 - 1 - 1 1 ] [ W D ] = 1 2 [ 1
1 1 1 1 1 - 1 - 1 1 - 1 - 1 1 1 - 1 1 - 1 ] Equation 2
##EQU00001.2##
[0049] Quantizer 140 performs a quantization with respect to the
residual block containing the residual signal, which has been
transformed into the frequency domain by transformer 130. Here,
quantizer 140 may use various quantizing techniques such as a dead
zone uniform threshold quantization (DZUTQ), quantization weighted
matrix, their improvements, or others.
[0050] Encoder 150 encodes the transformed and quantized residual
block by using the entropy coding technique to generate a
bitstream. In the process, encoder 150 may employ various methods
for scanning the quantized frequency coefficients of the quantized
residual block to generate a quantized frequency coefficient
sequence, which may then be encoded through the technique such as
an entropy encoding for generating a bitstream output. At the same
time, encoder 150 encodes information on the prediction mode in
which the current block was predicted at predictor 110.
[0051] Inverse quantizer 160 performs inverse quantization with
respect to the quantized residual block from quantizer 140.
Specifically, inverse quantizer 160 performs inverse quantization
with respect to the quantized frequency coefficients of the
quantized residual block and generates the residual block having
the frequency coefficient.
[0052] Inverse transformer 170 performs an inverse transform with
respective to the inverse quantized residual block from inverse
quantizer 160. Specifically, inverse transformer 170 performs the
inverse transform with respect to the frequency coefficients of the
inverse quantized residual block. Here, inverse transformer 170 may
use the reverse of the transform process at transformer 130 in its
operation. For example, inverse transformer 170 gathers the DC
values of frequency coefficients to perform an inverse Hadamard
transform as in Equation 3, and use the basic vectors shown in FIG.
5 to carry out a one-dimensional inverse DCT on the current block
vertically and doing the same horizontally, thereby inverse
transforming the frequency coefficient of the frequency domain into
the spatial domain pixel value.
Inverse ##EQU00002## W QD = 1 2 [ 1 1 1 1 1 1 - 1 - 1 1 - 1 - 1 1 1
- 1 1 - 1 ] [ Z D ] = 1 2 [ 1 1 1 1 1 - 1 1 - 1 1 1 - 1 - 1 1 - 1 -
1 1 ] Equation 3 ##EQU00002.2##
[0053] Adder 180 adds the reconstructed residual block from inverse
transformer 170 to the predicted block at predictor 110 to
reconstruct the current block. The reconstructed current block is
stored in predictor 110 as a reference picture for use in encoding
the next block to the current block or in a future encoding of
another block.
[0054] Although not shown in FIG. 1, between predictor 110 and
adder 180, a deblocking filter (not shown) may be additionally
connected. The deblocking filter performs the deblocking filtering
on the reconstructed current block received from adder 180. Here,
the deblocking filtering refers to the operation for reducing
blocking artifact stemming from encoding the video block-by-block
and it can be implemented through applying a deblocking filter at
the block boundary and macroblock boundary, applying the deblocking
filter or saving the use thereof at only the macroblock
boundary.
[0055] FIG. 6 is an exemplary diagram showing arbitrary pixels in a
sub-block according to an embodiment.
[0056] When encoding a macroblock of an arbitrary size, the present
disclosure performs the encoding by unit of sub-blocks of arbitrary
sizes through first predicting, encoding, and decoding an arbitrary
pixel having an arbitrary location in the sub-block to have the
arbitrary pixel reconstructed for being then used to predict the
remaining pixels in the sub-block so that the entire pixels in the
macroblock are predicted starting from the closest pixels rather
than remote pixels to prevent a decrease of the prediction
accuracy.
[0057] Here, the arbitrarily sized macroblock 610 may be designated
in size by a user as N.times.M, and the arbitrary size of the
sub-block 620 in the arbitrarily sized macroblock 610 may be
designated by the user to be O.times.P (where O.ltoreq.N,
P.ltoreq.M).
[0058] In FIG. 6, the arbitrarily sized macroblock 610 to be
currently encoded is a 16.times.16 block having the sub-blocks 620
that are 4.times.4 blocks shown as compared to the arbitrary pixels
630. In the following embodiment, the disclosure will be described
through the exemplary illustrations. Additionally, for the sake of
description, the arbitrary pixels in the present disclosure are
determined to be located at the bottom right end 630 in the
sub-blocks and called X.sub.i. However, such a macroblock size,
sub-block size, and arbitrary pixel position are parts of exemplary
variables, which may be implemented into diverse block sizes and
pixels at different locations.
[0059] FIG. 7 is a block diagram of a video encoding apparatus 700
in an electronic configuration according to an embodiment of the
present disclosure.
[0060] In order to encode videos, video encoding apparatus 700 may
comprise an arbitrary pixel extractor 702, a first predictor 704, a
first subtractor 706, a first transformer 708, a first quantizer
710, a first inverse quantizer 712, a first inverse transformer
714, a first adder 716, a second predictor 718, a second subtractor
720, a combiner 722 for combining arbitrary residual pixels, a
second transformer 724, a second quantizer 726, an encoder 728, a
second inverse quantizer 730, a second inverse transformer 732, a
second adder 734, and a combiner 736 for combining reconstructed
arbitrary pixels.
[0061] Video encoding apparatus 700 may be a personal computer or
PC, notebook or laptop computer, personal digital assistant or PDA,
portable multimedia player or PMP, PlayStation Portable or PSP, or
mobile communication terminal or such devices, and may represent a
variety of apparatuses equipped with, for example, a communication
system such as a modem for carrying out communications between
various devices or wired/wireless communication networks, a memory
for storing various programs for encoding videos and related data,
and a microprocessor for executing the programs to effect
operations and controls.
[0062] Arbitrary pixel extractor 702 extracts arbitrary pixels from
each of sub-blocks of a current block of the video and generate an
arbitrary pixel block. Arbitrary pixel extractor 702 extracts the
arbitrary pixels 630 X.sub.0.about.X.sub.15 at arbitrary positions
among the pixels in the respective sub-blocks 620 of the current
macroblock 610 as illustrated in FIG. 6 and generate the arbitrary
pixel block shown in FIG. 8. FIG. 8 is an exemplary diagram showing
an operation of extracting an arbitrary pixel block from a current
macroblock.
[0063] First predictor 704 predicts the arbitrary pixel block and
generates a predicted arbitrary pixel block. First predictor 704
utilizes pixels V.sub.i (V.sub.0.about.V.sub.15) and pixels H.sub.i
(H.sub.0.about.H.sub.15), which are close to the current macroblock
among pixels at its left and upper sides and encoded prior to the
encoding of the current macroblock, as a means of adjacent pixels
for predicting the arbitrary pixel block, and generates the
predicted arbitrary pixel block having predicted arbitrary pixels
from predictions of each of the arbitrary pixels in the arbitrary
pixel block.
[0064] In addition, first predictor 704 performs a low-pass
filtering on the adjacent pixels to the current macroblock and
predicts the arbitrary pixel block using down-sampled pixels. FIG.
9 is an exemplary diagram for showing predictions of the arbitrary
pixel block using adjacent pixels to the current macroblock.
Referring to FIG. 9, first predictor 704 may perform the low-pass
filtering on the selected pixels V.sub.i (V.sub.0.about.V.sub.15)
adjacent to current macroblock and then the down-sampling on the
same to calculate pixels V.sub.a, V.sub.b, V.sub.c, and V.sub.d,
thereby obtaining the left-side adjacent pixels. Similarly, first
predictor 704 may perform the low-pass filtering on the selected
pixels H.sub.i (H.sub.0.about.H.sub.15) adjacent to current
macroblock and then the down-sampling on the same to calculate
pixels H.sub.a, H.sub.b, H.sub.c, and H.sub.d, thereby obtaining
the upper-side adjacent pixels.
[0065] At this time, first predictor 704 may predict the arbitrary
pixel block in the same direction as the prediction direction
according to the prediction mode of the current macroblock. Here,
the prediction direction of the current macroblock may be that of
the intra.sub.--16.times.16 prediction as shown in FIG. 3, and it
can be the optimal prediction mode determined as having the lowest
cost to encode among the respective prediction modes as information
on such a prediction mode may be supplied to encoder 728.
[0066] First subtractor 706 subtracts the predicted arbitrary pixel
block from the arbitrary pixel block and generates an arbitrary
pixel residual block. Specifically, first subtractor 706 generates
the arbitrary pixel residual block that is residual signals equal
to the differences between the respective predicted arbitrary pixel
block pixel values and the respective arbitrary pixel block pixel
values.
[0067] First transformer 708 transforms the arbitrary pixel
residual block. First transformer transforms each arbitrary
residual signal in the arbitrary pixel residual block into the
frequency domain to generate the frequency coefficients for the
arbitrary residual pixels. In addition, first transformer 708
transforms exclusively and independently the respective residual
signals in the arbitrary pixel residual block. For this purpose,
first transformer 708 may perform the DCT transform with respect to
the arbitrary residual signals in the arbitrary pixel block to
independently transform the respective arbitrary pixel residual
blocks only.
[0068] Furthermore, first transformer 708 may carry out the
Hadamard transform after the DCT transform performed. For example,
if the current macroblock is an intra.sub.--16.times.16 block,
first transformer 708 may do the Hadamard transform over the DCT
transformed frequency coefficients, but it may not add the Hadamard
transform if the current macroblock is an intra.sub.--4.times.4
block. This is because the current macroblock determined to be the
intra.sub.--4.times.4 block has little correlation between images
and the Hadamard transform would rather decrease the efficiency of
the process.
[0069] First quantizer 710 performs a quantization with respect to
the transformed arbitrary pixel residual block. First quantizer
performs quantization with respect to the frequency coefficients of
the arbitrary pixel residual block to have the arbitrary pixel
residual block generated with the quantized frequency
coefficients.
[0070] First inverse quantizer 710 performs an inverse quantization
with respect to the quantized arbitrary pixel residual block. First
inverse quantizer 710 performs the inverse quantization with
respect to the quantized frequency coefficients of the arbitrary
pixel residual block to have the arbitrary pixel residual block
generated with the inverse quantized frequency coefficients.
[0071] First inverse transformer 714 performs an inverse transform
with respect to the inverse quantized arbitrary pixel residual
block. First inverse transformer 714 transforms the frequency
coefficients of the arbitrary pixel residual block into the spatial
domain to have a reconstructed arbitrary pixel residual block
generated with reconstructed arbitrary residual signals. At this
time, first inverse transformer 714 performs the inverse transform
with respect to the respective arbitrary pixel residual signals
exclusively and independently by performing an inverse DCT
transform with respect to the frequency coefficients from the
transform of the respective arbitrary pixel residual signals by
using modified basic vectors for the inverse DCT transform. In
addition, first inverse transformer 714 may perform the inverse DCT
transform over the inverse Hadamard transform of the frequency
coefficients from the transform of the respective arbitrary pixel
residual signals.
[0072] Here, the modified basic vectors for the forward and inverse
DCT transform may mean such basic vectors modified from the forward
and inverse DCT transformed basic vectors shown in FIGS. 4 and 5 in
order to perform the forward and inverse transforms on the
arbitrary pixels independently of the remaining pixels in the
respective sub-blocks and they may be determined as shown in FIGS.
10 and 11. FIGS. 10 and 11 are exemplary diagrams showing the
forward and inverse DCT transforming modified basic vectors,
respectively. That is to say, first transformer 708 and first
inverse transformer 714 use the modified vectors in FIGS. 10 and 11
in order to allow the forward and inverse transforms on the
arbitrary residual signals Xd.sub.i exclusively and independently
when second transformer 724 and second inverse transformer 732
perform forward and inverse transforms on the remaining pixels in
the sub-blocks.
[0073] For example, when using the modified basic vectors of the
DCT transform shown in FIG. 10 to perform a one-dimensional
transform with respect to four pixels P.sub.0.about.3, with the
basic vector (0, 0, 0, 2) corresponding to the transformed first
frequency coefficient (Coeff.sub.0) and directed only to the
component of arbitrary residual signal Xd.sub.i, the first
frequency coefficient may be eventually calculated as in Equation
4.
Coeff.sub.0=0.times.P.sub.0+0.times.P.sub.1+0.times.P.sub.2+2.times.P.su-
b.3=2.times.P.sub.3 Equation 4
[0074] Therefore, in the transform, using the modified basic
vectors for
[0075] DCT transform shown in FIG. 10 will give the first frequency
coefficient comprised of the single component of pixel P.sub.3.
Thus, at varying values of the pixels P.sub.0.about.2, the first
frequency coefficient may remain unchanged and thus the arbitrary
pixels may enter into transforms independently of the remaining
pixels P.sub.0.about.2.
[0076] In order to transform the arbitrary pixel residual signal
Xd.sub.i, which is the residual signal of the arbitrary pixel
extracted from each sub-block in the current macroblock,
independently of the remaining pixels in the sub-block, first
transformer 708 performs the total of two-dimensional transform
with the basic vector (0, 0, 0, 2) corresponding to the transformed
first frequency coefficient (Coeff.sub.0). As the one-dimensional
transform equals to 2 times the value of the arbitrary pixel
residual signal Xd.sub.i, performing the two-dimensional transform
equals to 4 times the value of the arbitrary pixel residual signal
Xd.sub.i. This is like performing the two-dimensional transform of
the respective sub-blocks with the modified basic vectors for DCT
transform shown in FIG. 10 first and then gathering just the
frequency coefficients located at the top left sides of the
respective sub-blocks. Thus, first transformer 708 performs the
two-dimensional transform in both the vertical and horizontal
directions by multiplying the values of the 16 (4.times.4)
arbitrary pixel residual signal Xd.sub.i values extracted from the
respective sub-blocks in the current macroblock by 4, and then
perform the Hadamard transform using the equation 2 with respect to
the transformed frequency coefficients as with the frequency
coefficients of the DC components of the respective sub-blocks in
the intra.sub.--16.times.16 block in H.264.
[0077] Likewise, the frequency coefficients that correspond to the
arbitrary pixel residual signals are inverse transformed
independently of the other frequency coefficients
(Coeff.sub.1.about.3). The corresponding frequency coefficients at
the inverse transform may be calculated as in Equation 5.
P.sub.3'=Coeff.sub.0/2 Equation 5
[0078] First adder 716 adds the inverse transformed arbitrary pixel
residual block to the predicted arbitrary pixel block to generate a
reconstructed arbitrary pixel block.
[0079] Second predictor 718 predicts the remaining pixels in the
respective sub-blocks by using adjacent pixels to the current
macroblock and each of reconstructed arbitrary pixels in the
reconstructed arbitrary pixel block to generate predicted blocks.
Specifically, when predicting the respective sub-blocks in the
current macroblock, second predictor 718 predicts the remaining
pixels but the arbitrary pixels of the respective sub-blocks.
[0080] FIGS. 12 to 15 are exemplary diagrams showing extractions of
the remaining pixels in each of sub-blocks. As shown in FIGS. 12 to
15, second predictor 718 predicts the remaining pixels of the
respective sub-blocks by using the respectively reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block after
encoding and decoding thereof and the adjacent pixels
V.sub.i(V.sub.0.about.V.sub.15) and H.sub.i(H.sub.0.about.H.sub.15)
that are adjacent to the current macroblock and inside the
macroblocks neighboring the current macroblock. At this juncture,
the prediction direction of second predictor 718 is same as that of
first predictor 704.
[0081] Referring to FIG. 12, at the vertical prediction direction,
the remaining pixels located at coordinates (0, 0).about.(0, 15)
are predicted to be the pixel H.sub.0 value, the remaining pixels
located at coordinates (1, 0).about.(1, 15) to be the pixel H.sub.1
value, and the remaining pixels located at coordinates (2,
0).about.(2, 15) to be the pixel H.sub.2 value. However, among the
pixels at coordinates (3, 0).about.(3, 15) traversing the
reconstructed arbitrary pixels X.sub.0', X.sub.4', X.sub.8', and
X.sub.12', those pixels within coordinates (3, 0).about.(3, 2) are
to be predicted as the pixel H.sub.3 value while pixels within
coordinates (3, 4).about.(3, 6) are to be predicted as the
reconstructed arbitrary pixel X.sub.0', pixels within coordinates
(3, 8).about.(3, 6) are to be predicted as the reconstructed
arbitrary pixel X.sub.4', and pixels within coordinates (3,
12).about.(3, 14) are to be predicted as the reconstructed
arbitrary pixel) X.sub.8'. The remaining columns of the current
macroblock can be similarly predicted.
[0082] Referring to FIG. 13, at the horizontal prediction
direction, in a similar way to the vertical direction, the
remaining pixels located at coordinates (0, 0).about.(15, 0), (0,
1).about.(15, 1), and (0, 2).about.(15, 2) are predicted to be the
pixel V.sub.0, V.sub.1, and V.sub.2 values, but among the pixels at
coordinates (0, 3).about.(15, 3) traversing the reconstructed
arbitrary pixels X.sub.0', X.sub.1', X.sub.2', and X.sub.3', those
pixels within coordinates (0, 3).about.(2, 3) are to be predicted
as the pixel V.sub.3 value while pixels within coordinates (4,
3).about.(6, 3) are to be predicted as X.sub.0', pixels within
coordinates (8, 3).about.(10, 3) are to be predicted as X.sub.1',
and pixels within coordinates (12, 3).about.(14, 3) are to be
predicted as X.sub.2'.
[0083] Referring to FIG. 14, at a prediction direction of the DC
average prediction, the adjacent pixels
V.sub.i(V.sub.0.about.V.sub.15) inside the macroblock facing the
left side of the current macroblock, the adjacent pixels
H.sub.i(H.sub.0.about.H.sub.15) in its top macroblock, and the
respective reconstructed arbitrary pixels X.sub.i' of each
sub-block are weighted with an arbitrary value before performing
the predictions by calculating the average.
[0084] Referring to FIG. 15, at the plain prediction direction,
those pixels at coordinates (3, 0).about.(3, 2) are predicted by
the pixel value H.sub.3 and the value X.sub.0' weighted with an
arbitrary value according to its distance, and the pixels at
coordinates (0, 3).about.(2, 3) are predicted by the pixel value
V.sub.3 and the value X.sub.0' weighted with an arbitrary value
according to its distance, and then arbitrary weighting values are
applied to the predicted pixels at coordinates (3, 0).about.(3, 2)
and (0, 3).about.(2, 3), the reconstructed arbitrary pixel
X.sub.0', and the adjacent pixels V.sub.0.about.V.sub.3 and
H.sub.0.about.H.sub.3, thereby predicting the pixels at coordinates
(0, 0).about.(2, 2). Other sub-blocks are predicted in a similar
way.
[0085] Second subtractor 720 subtracts the predicted blocks from
the current macroblock of the actual video to generate residual
macroblocks. In other words, second subtractor 720 calculates the
residual signals of the remaining pixels other than the arbitrarily
positioned pixel X.sub.i of each sub-block in the current
macroblock, thereby generating a residual macroblock having the
residual signals.
[0086] Combiner 722 combines each of inverse transformed arbitrary
residual pixels in the inverse transformed arbitrary pixel residual
block to the corresponding positions of the residual macroblocks.
In other words, combiner 722 combines each of inverse transformed
and reconstructed arbitrary residual pixels Xd.sub.i' in the
inverse transformed arbitrary pixel residual block from inverse
transformer 714 to the corresponding pixel positions of each
corresponding sub-block in the residual macroblocks from second
subtractor 720. Therefore, the residual macroblocks generated at
second subtractor 720 carried only the residual signals of the
remaining pixels, but upon having the reconstructed arbitrary
residual pixels combined by combiner 722, it becomes a wholesome
residual macroblock. As such, by combining the previously
reconstructed arbitrary residual signals Xd.sub.i' to the positions
of the arbitrary residual pixels in each residual sub-block,
possible quantization errors associated with the transform and
quantization can be mitigated. In other words, because the
reconstructed arbitrary residual pixels already carry the
quantization errors reflected in them, both the video encoding and
decoding apparatuses will be able to operate with respect to the
reconstructed arbitrary residual signals Xd.sub.i' with no
discrepancies regardless of the quantization errors. Therefore, the
encoding and decoding processes take only the quantization errors
in the remaining pixels into account and this mitigates the
quantization errors better than using the arbitrary residual
signals Xd.sub.i' with no quantization errors reflected in
them.
[0087] Second transformer 724 transforms the residual macroblocks
having the inverse transformed arbitrary residual pixels combined.
Second transformer 724 transforms the respective residual signals
of the residual macroblocks having the inverse transformed
arbitrary residual pixels combined by combiner 722 into the
frequency domain, thereby generating residual macroblocks having
frequency coefficients.
[0088] In addition, second transformer 724 transforms the residual
macroblock by unit of 4.times.4 block using DCT transform while it
uses basic vectors determined according to the sequence of a first
one-dimensional transform direction. Here, the sequence of the
first one-dimensional transform direction may be decided according
to a prediction direction of the first predictor 704.
[0089] FIG. 16 is an exemplary diagram showing a one-dimensional
transform in preparation for a two-dimensional DCT transform.
Referring to FIG. 16, second transformer 724 may get the
two-dimensional DCT transform done by performing a horizontal
one-dimensional transform first and then a vertical one-dimensional
transform or vice versa where the sequence of first performing the
one-dimensional transform in the horizontal or vertical direction
may be decided by the direction of prediction on the arbitrary
pixel block in first predictor 704.
[0090] Depending on such decided sequence of the first
one-dimensional transform, second transformer 724 may apply the
different basic vectors for use in the one-dimensional transform.
FIGS. 17 and 18 show an inverse transform procedure using the
modified basic vectors according to the sequential directions of
the inverse transform. When applying a horizontal one-dimensional
transform first and then a vertical one-dimensional transform, as
shown in FIG. 17, second transformer 724 may perform the horizontal
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third rows
and a modified basic vector in their fourth rows, and perform the
vertical one-dimensional transform with respect to the respective
4.times.4 blocks using the modified basic vector in their first
columns and the modified basic vector in their second to fourth
columns.
[0091] Additionally, when applying a vertical one-dimensional
transform first and then a horizontal one-dimensional transform, as
shown in FIG. 18, second transformer 724 may perform the vertical
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third
columns and a modified basic vector in their fourth columns, and
perform the horizontal one-dimensional transform with respect to
the respective 4.times.4 blocks using the modified basic vector in
their first rows and the modified basic vector in their second to
fourth rows.
[0092] As above, determining the sequence of the first
one-dimensional transform direction according to the prediction
direction and accordingly taking either the existing basic vector
or the modified basic vector for the conventional basic vectors
used in transforms will not only reduce high-frequency components
in the pixel signals but can have the residual signals for the
remaining pixels in the respective sub-blocks transformed
independently of the respective arbitrary pixel residual signals.
Here, the existing basic vectors and the modified basic vectors are
same as those described referring to FIG. 10.
[0093] Second quantizer 726 performs a quantization with respect to
the transformed residual macroblocks. Second quantizer 726 performs
the quantization with respect to the frequency coefficients of the
transformed residual macroblocks from second transformer 724 to
generate the quantized residual macroblocks, which are the residual
macroblocks having the quantized frequency coefficients.
[0094] Encoder 728 encodes a quantized residual macroblocks to
generate a bitstream. Specifically, encoder 728 generates the
quantized frequency coefficient sequence by performing scanning
such as zig-zag scanning or others with respective to the quantized
frequency coefficients of the quantized residual macroblocks and
outputs the bitstream by encoding the generated quantized frequency
coefficient sequence by using various encoding techniques including
an entropy encoding. At this time, encoder 728 may encode not only
the quantized residual macroblocks but also information on the
prediction modes delivered from first predictor 704 or second
predictor 718.
[0095] Second inverse quantizer 730 performs an inverse
quantization with respect to the quantized residual macroblocks.
Second inverse quantizer 730 performs the inverse quantization with
respect to the quantized frequency coefficients of the quantized
residual macroblocks from second quantizer 726 to generate the
inverse quantized residual macroblocks having the resulting
frequency coefficients.
[0096] Second inverse transformer 732 performs an inverse transform
with respect to the inverse quantized residual macroblocks. Second
inverse transformer 732 performs the inverse transform with respect
to such frequency coefficients of the inverse quantized residual
block from second inverse quantizer 730 into the spatial domain and
then reconstructs the arbitrary pixel residual signals in order to
generate a reconstructed arbitrary pixel residual block.
[0097] Second inverse transformer 732 inverse-transforms the
inverse quantized residual macroblock by unit of 4'4 block using an
inverse DCT transform while it uses basic vectors determined
according to the sequence of a first one-dimensional inverse
transform direction in the inverse DCT transform. In addition,
second inverse transformer 732 may decide the sequence of the first
one-dimensional inverse transform direction as being the reverse of
the sequence of the first one-dimensional transform direction at
second transformer 724.
[0098] In other words, second inverse transformer 732, when
applying a horizontal one-dimensional inverse transform first and
then a vertical one-dimensional inverse transform, as shown in FIG.
17, may perform the horizontal one-dimensional inverse transform
with respect to respective 4.times.4 blocks using a modified basic
vector in their first rows and an existing basic vector in their
second to fourth rows, and performs the vertical one-dimensional
inverse transform with respect to the respective 4.times.4 blocks
using the existing basic vector in their first to third columns and
the modified basic vector in their fourth columns.
[0099] Additionally, second inverse transformer 732, when applying
a vertical one-dimensional inverse transform first and then a
horizontal one-dimensional inverse transform, as shown in FIG. 18,
may perform the vertical one-dimensional inverse transform with
respect to respective 4.times.4 blocks using a modified basic
vector in their first columns and an existing basic vector in their
second to fourth columns, and performs the horizontal
one-dimensional inverse transform with respect to the respective
4.times.4 blocks using the existing basic vector in their first to
third rows and the modified basic vector in their fourth rows.
[0100] Second adder 734 is designed to add the inverse transformed
residual block to the predicted macroblock to generate a
reconstructed current block. Specifically, second adder 734 adds
the respective residual signals of the inverse transformed residual
macroblocks from second transformer 734 to the respective pixels of
the predicted block from second predictor 718 to generate a
reconstructed current macroblocks, which are the macroblocks having
the reconstructed pixels.
[0101] Combiner 736 is designed to combine the respective
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block to the corresponding positions of the reconstructed current
macroblock. Specifically, combiner 736 combines the respective
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block from first adder 716 to the corresponding positions of the
reconstructed current macroblock from second adder 734 to generate
the finally reconstructed current macroblocks. Such finally
reconstructed current macroblocks are stored in a memory (not
shown) for being used by first predictor 704 and second predictor
718 in encoding the next macroblock.
[0102] FIG. 19 is a flow diagram for illustrating a video encoding
method according to an embodiment.
[0103] Upon receiving a video to encode, video encoding apparatus
700 divides the video by unit of macroblock and elects from various
block modes the optimal block mode, in which the apparatus 700
predicts the macroblock to currently encode and then completes the
encoding.
[0104] If the intra mode is the encoding mode of choice to perform
the intra prediction, video encoding apparatus 700 extracts
arbitrary pixels from each sub-block of a current macroblock and
generates an arbitrary pixel block at step S1910, predicts the
arbitrary pixel block and generates a predicted arbitrary pixel
block, and subtracts the predicted arbitrary pixel block from the
arbitrary pixel block to generate an arbitrary pixel residual block
at S1920.
[0105] Video encoding apparatus 700 is configured to perform a
transform on the arbitrary pixel residual block, a quantization
with respect to the transformed arbitrary pixel residual block, an
inverse quantization with respect to the quantized arbitrary pixel
residual block, and an inverse transform with respect to the
inverse quantized arbitrary pixel residual block at S1930, and adds
the inverse transformed arbitrary pixel residual block to the
predicted arbitrary pixel block to generate a reconstructed
arbitrary pixel block at S1940. Video encoding apparatus 700
predicts the remaining pixels in the respective sub-blocks by using
adjacent pixels to the current macroblock and each of reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block to
generate predicted macroblocks, and subtracts the predicted
macroblocks from the current macroblock to generate residual
macroblocks at S1950.
[0106] In addition, video encoding apparatus 700 is designed to
combine each of inverse transformed arbitrary residual pixels in
the inverse transformed arbitrary pixel residual block to the
corresponding positions of the residual macroblock at S1960,
transforms the residual macroblocks having the respective inverse
transformed arbitrary residual pixels combined, perform
quantization with respect to the transformed residual macroblocks,
and encodes the quantized residual macroblocks to generate a
bitstream at S1970.
[0107] Thereafter, as shown in FIG. 20, video encoding apparatus
700 may reconstruct the current macroblock. In particular, video
encoding apparatus 700 performs an inverse quantization with
respect to the quantized residual macroblock at S1960 and perform
an inverse transform with respect to the inverse quantized residual
block at S2010, adds the inverse transformed residual macroblock to
the predicted macroblock to generate a reconstructed current
macroblock at S2020, and combines the respective reconstructed
arbitrary pixels in the reconstructed arbitrary pixel block to the
corresponding positions of the reconstructed current macroblocks to
generate the finally reconstructed current macroblocks at
S2030.
[0108] As described above, the bitstream of video encoded by video
encoding apparatus 700 may then be transmitted in real time or
non-real-time to video decoding apparatuses described below for
decoding the same before its reconstruction and reproduction into
the video via a wired/wireless communication network including the
Internet, a short range wireless communication network, a wireless
LAN network, WiBro (Wireless Broadband) also known as WiMax
network, and mobile communication network or a communication
interface such as cable or USB (universal serial bus).
[0109] FIG. 21 is a block diagram of a video decoding apparatus
2100 according to an embodiment.
[0110] Video decoding apparatus 2100 for decoding a video may
comprise a decoder 2102, an inverse quantizer 2104, an extractor
2106 for extracting frequency coefficients, a first inverse
transformer 2108, a first predictor 2110, a first adder 2112, a
second inverse transformer 2114, a second predictor 2116, a second
adder 2118, and a pixel combiner 2120.
[0111] Decoder 2102 is designed to decode a bitstream and extracts
residual macroblocks and prediction modes. Specifically, decoder
2101 decodes the bitstream to extract the quantized frequency
coefficient sequence and the prediction modes, and performs various
inverse scanning including an inverse zig-zag scanning with
respective to the quantized frequency coefficient sequence to
generate the residual macroblocks.
[0112] Inverse quantizer 2104 performs an inverse quantization with
respect to the residual macroblocks, and extractor 2106 extracts
the frequency coefficients corresponding to positions of arbitrary
pixels of each sub-block from the inverse quantized residual
macroblocks. Here, the frequency coefficients corresponding to
positions of arbitrary pixels may be the DC component frequency
coefficients.
[0113] First inverse transformer 2108 performs an inverse transform
with respect to the extracted frequency coefficients from extractor
2106 to generate an inverse transformed arbitrary pixel residual
block. First inverse transformer 2108 operates on the extracted
frequency coefficients in the similar or same way the first inverse
transformer 714 performs the inverse transform as described with
reference to FIG. 7 and the detailed descriptions of the same is
omitted.
[0114] First predictor 2110 predicts an arbitrary pixel block in a
prediction direction according to the extracted prediction mode
from decoder 2102 to generate a predicted arbitrary pixel block.
First predictor 2110 operates on the arbitrary pixel block in the
prediction direction in the similar or same way the first predictor
704 performs the prediction as described with reference to FIG. 7
and the detailed descriptions of the same is omitted. First adder
2112 adds the inverse transformed arbitrary pixel residual block to
the predicted arbitrary pixel block to generate a reconstructed
arbitrary pixel block.
[0115] Second inverse transformer 2114 performs an inverse
transform with respect to the inverse quantized residual macroblock
to generate an inverse transformed residual macroblock. Second
inverse transformer 2114 operates on the frequency coefficients of
the inverse quantized residual macroblocks in the similar or same
way the second transformer 724 performs the inverse transform as
described with reference to FIG. 7 and the detailed descriptions of
the same is omitted.
[0116] Second predictor 2116 predicts a current macroblock of the
video in a prediction direction according to the prediction mode by
using adjacent pixels to the current macroblock and each of the
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block to generate a predicted current macroblock. This operation of
second predictor 2116 predicting the current macroblock of the
video in a prediction direction according to the prediction mode by
using adjacent pixels to the current macroblock and each of the
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block is similar to or same as the second predictor 718 performs
the prediction as described with reference to FIG. 7 and the
detailed descriptions of the same is omitted.
[0117] Second adder 2118 generates a reconstructed current
macroblock by using the inverse transformed residual macroblock and
the predicted current macroblock. Combiner 2120 combines the
respective reconstructed arbitrary pixels to the corresponding
positions of the reconstructed current macroblock. As such, the
combined current macroblock becomes the finally reconstructed
current macroblock, and such finally reconstructed current
macroblocks grouped by units of picture are outputted as a
reconstructed video. In addition, the finally reconstructed
macroblock is employed by first predictor 2110 and second predictor
2116 to encode the next macroblock.
[0118] FIG. 22 is a flow diagram of a video decoding method
according to an embodiment of the present disclosure.
[0119] Video decoding apparatus 2100, that has received and stored
the bitstream for a video via a wired/wireless communication
network or cable, decodes and reconstructs the video in order to
reproduce it according to a user selected algorithm or another
algorithm of a program in execution.
[0120] In order to achieve this, video decoding apparatus 2100
decodes the bitstream and extract a residual macroblock and a
prediction mode at step S2210, and perform inverse quantization
with respect to the residual macroblock at S2220.
[0121] Video decoding apparatus 2100 extracts frequency
coefficients corresponding to positions of arbitrary pixels in each
sub-block from the inverse quantized residual macroblock at S2230,
performs inverse transform with respect to the extracted frequency
coefficients to generate a reconstructed arbitrary pixel residual
block at S2240, and predicts an arbitrary pixel block in a
prediction direction according to the prediction mode to generate a
predicted arbitrary pixel block at S2250. Furthermore, video
decoding apparatus 2100 adds the inverse transformed reconstructed
arbitrary pixel residual block from S2240 to the predicted
arbitrary pixel block from S2250 to generate a reconstructed
arbitrary pixel block at S2250.
[0122] Video decoding apparatus 2100 performs an inverse transform
with respect to the inverse quantized residual macroblock to
generate an inverse transformed residual macroblock at S2260, and
predicts a current macroblock of the video in a prediction
direction according to the prediction mode by using adjacent pixels
to the current macroblock and each of reconstructed arbitrary
pixels in the reconstructed arbitrary pixel block to generate a
predicted current macroblock at S2270. Video decoding apparatus
2100 generates a reconstructed current macroblock by using the
inverse transformed residual macroblock and the predicted current
macroblock at S2280, and combines the respective reconstructed
arbitrary pixels to the corresponding positions of the
reconstructed current macroblock at S2290.
[0123] In the following, a video encoding apparatus according to
another embodiment will be described. The present embodiment of the
video encoding apparatus is same as the first embodiment of the
video encoding apparatus by their components, although the
component functions are distinctive.
[0124] Specifically, in encoding and decoding each sub-block of the
current macroblock to be currently encoded, the first embodiment of
the video encoding apparatus 700 encodes the entire sub-blocks and
then decodes them to perform the transforms and quantizations by
unit of current macroblock. Whereas, this embodiment of the video
encoding apparatus, in encoding and decoding the entire sub-blocks,
processes one sub-block of both encoding and decoding after
processing another sub-block of encoding and decoding to go through
the entire sub-blocks. Therefore, in this embodiment, the
transforms and quantizations are performed by unit of
sub-block.
[0125] Referring to FIG. 23, the first embodiment of the video
encoding apparatus 700 completes the encoding of the respective
sub-blocks of the current macroblock in the numbered sequence as
shown and thereafter decodes the respective sub-blocks. In
particular, the first embodiment of the video encoding apparatus
700 sequentially predicts the sub-blocks numbered 1 to 16 with
succeeding subtractions by using adjacent pixels to the current
macroblock and the reconstructed arbitrary pixels in the respective
sub-blocks for being subjected to transforms and quantizations by
unit of macroblock until the entire sub-blocks are thus encoded,
and then continues to perform an inverse quantization and inverse
transform on the quantized macroblock and then subtract the
respective predicted sub-blocks from the transformed residual
macroblock to eventually decode the sub-blocks at 1 to 16.
[0126] In contrast, this embodiment of the video encoding apparatus
may comprise second predictor 178 for predicting the sub-block 1 by
using its adjacent pixels and the reconstructed arbitrary pixels,
second subtractor 720 for generating the residual sub-block for the
sub-block 1, second transformer 724 for transforming the residual
sub-block, second quantizer 726 for performing quantization with
respect to the transformed residual sub-block, second inverse
quantizer 730 for performing an inverse quantization with respect
to the respective quantized residual sub-blocks, and second inverse
transformer 732 for performing inverse transform with respect to
the inverse quantized residual sub-blocks and adding with the
predicted sub-block to generate a reconstructed sub-block 1.
Thereafter, second predictor 178 predicts the sub-block 2 by using
not just the adjacent pixels to the current macroblock but the
pixels located in the reconstructed sub-block 1 close to the
sub-block 2 as the left-side adjacent pixels and the adjacent
pixels to the current macroblock as the upper-side adjacent pixels,
and in particular these adjacent pixels and the reconstructed
arbitrary pixels within the sub-block 2 are used to predict the
sub-block 2. Subtracting the predicted sub-block 2 from the
sub-block 2 generates a residual sub-block, which is then subjected
to a transform, quantization, inverse transform and inverse
quantization to reconstruct the sub-block 2. These steps are
repeatedly carried out in sequence to encode the respective
sub-blocks while using the respective sub-blocks reconstructed.
[0127] This embodiment of the video encoding apparatus extracts
arbitrary pixels from each sub-block of the current macroblock of
the video to generate an arbitrary pixel block, predicts the
arbitrary pixel block to generate a predicted arbitrary pixel
block, subtracts the predicted arbitrary pixel block from the
arbitrary pixel block to generate an arbitrary pixel residual
block, transforms the arbitrary pixel residual block and perform
quantization with respect to the transformed arbitrary pixel
residual block, performs an inverse quantization with respect to
the quantized arbitrary pixel residual block and performs an
inverse transform with respect to the inverse quantized arbitrary
pixel residual block, and then adds the inverse transformed
arbitrary pixel residual block to the predicted arbitrary pixel
block to generate a reconstructed arbitrary pixel block. The
apparatus continues to predict the remaining pixels in the
respective sub-blocks by using adjacent pixels to the respective
sub-blocks and the respective reconstructed arbitrary pixels in the
reconstructed arbitrary pixel block to generate the respective
predicted blocks, subtract the respective predicted sub-blocks from
the sub-blocks to generate the respective residual sub-blocks,
combine each of inverse transformed arbitrary residual pixels in
the inverse transformed arbitrary pixel residual block to the
corresponding positions of the respective residual sub-blocks,
transform the respective residual sub-blocks having the respective
inverse transformed arbitrary residual pixels combined and perform
quantization with respect to the respective transformed residual
sub-blocks, and then encode the respective quantized residual
sub-blocks to generate a bitstream. Throughout the respective
sub-blocks, one sub-block is encoded and subsequently decoded after
the encoding and decoding on another sub-block.
[0128] This embodiment of the video encoding apparatus also
performs an inverse quantization with respect to the respective
quantized residual sub-blocks, performs an inverse transform with
respect to the respective inverse quantized residual sub-blocks and
adds the respective inverse transformed residual sub-blocks to the
respective predicted sub-blocks to generate the respective
reconstructed sub-blocks, and then combines the respective
reconstructed arbitrary pixels in the reconstructed arbitrary pixel
block to the corresponding positions of the respective
reconstructed sub-blocks. In predicting the remaining pixels of the
respective sub-blocks, the respective sub-blocks having the
respective reconstructed arbitrary pixels combined are stored for
use in predicting the subsequent sub-blocks.
[0129] In addition, this embodiment of the video encoding apparatus
may not use the prediction direction of the current macroblock or
the prediction direction of first predictor 704 but rather decide a
prediction direction for the respective sub-blocks and predict the
remaining pixels of the respective sub-blocks in the decided
prediction directions. To this end, this embodiment of the video
encoding apparatus may include second predictor 718 for use in
deciding as the prediction directions the directions of the
adjacent pixels to the respective sub-blocks facing toward the
respective reconstructed arbitrary pixels with the least difference
in value therebetween. Optionally, this second predictor 718 may
generate information on the prediction mode according to the
prediction directions of the respective sub-blocks for supplying to
encoder 728. Upon receiving the prediction mode information of the
respective sub-blocks delivered, encoder 728 may encode the
respective sub-block prediction mode information to inset the same
in the bitstream so that the video decoding apparatus in predicting
the respective sub-blocks may use such information to synchronize
the prediction directions. If there is no sub-block prediction mode
information encoded, the video decoding apparatus operates similar
to the encoding apparatus in deciding the prediction directions of
the respective sub-blocks and making predictions in the decided
directions.
[0130] Referring to FIG. 24, second predictor 718 in this
embodiment does not rely on the prediction directions of first
predictor 704 but decides the respective new optimal prediction
directions by comparing the previously reconstructed pixels at the
left V.sub.i and the top H.sub.i of the sub-blocks against the
reconstructed arbitrary pixels X.sub.i', and predicts the
respective sub-blocks in the newly decided optimal prediction
directions. Newly capturing the optimal predetermined directions is
through predicting edges by using the arbitrary pixels X.sub.i and
adjacent pixels V.sub.0.about.V.sub.3, H.sub.0.about.H.sub.7
adjacent to the current macroblock.
[0131] For example, if X.sub.i and H.sub.3 have the strongest
resemblance, the optimal prediction direction of the corresponding
sub-block is set to be vertical, and if X.sub.i and V.sub.3 have
the strongest resemblance, the optimal prediction direction of the
corresponding sub-block is set to be horizontal. DC prediction mode
is decided when the values V.sub.0.about.V.sub.3,
H.sub.0.about.H.sub.7 resemble to X.sub.i. Diagonal-left is decided
when there are resemblances between V.sub.0 and H.sub.0, V.sub.1
and H.sub.1, V.sub.2 and H.sub.2, V.sub.3 and H.sub.3, and/or
H.sub.7 and X.sub.i. Diagonal-right is decided when there are
resemblances between N and X.sub.i. Vertical-right is decided when
there are resemblances between H.sub.1 and X.sub.i. Horizontal-down
is decided when there are resemblances between V.sub.2 and X.sub.i.
Vertical-left is decided when there are resemblances between
H.sub.5 and X.sub.i and when H.sub.0 and V.sub.1, H.sub.1 and
V.sub.2, and/or H.sub.2 and V.sub.3 have resemblances.
Horizontal-up is decided when there are resemblances between
H.sub.1 and V.sub.0, H.sub.2 and V.sub.1, H.sub.3 and V.sub.2,
and/or H.sub.4 and V.sub.3. In this way, second predictor 718 in
this embodiment may decide the optimal directions for the
respective sub-blocks before predictions in order to increase the
prediction accuracy. In other words, for the respective sub-blocks,
predictions are performed on the remaining pixels by using the
reconstructed arbitrary pixels and therefore in accordance with the
relations between the reconstructed arbitrary pixels and their
adjacent pixels the respective sub-blocks may have the respectively
decided optimal prediction directions therefor, whereby such
changes of prediction directions will improve the efficiency of
compressing videos through the encoding and decoding.
[0132] Here, the sequence of encoding and decoding of the
respective sub-blocks by this embodiment of the video encoding
apparatus may be either the sequence of encoding and decoding of
intra.sub.--4.times.4 blocks in H.264/AVC or a raster scanning
direction. The sequence in FIG. 23 depicts the encoding and
decoding sequence on intra.sub.--4.times.4 blocks in H.264/AVC
standard, whereas FIG. 25 depicts the encoding and decoding
sequence on the respective sub-blocks according to the raster
scanning direction. As shown, this embodiment of the video encoding
apparatus is subject a sub-block to the prediction, transform,
quantization, inverse quantization, and inverse transform in an
arbitrary sequence to reconstruct it, and then uses the previously
reconstructed sub-blocks to process the subsequent sub-blocks
through the prediction, transform, quantization, inverse
quantization, and inverse transform as a way to encode and decode
the respective sub-blocks.
[0133] FIG. 26 illustrates the process of predicting the remaining
pixels according to newly decided optimal prediction directions by
using the reconstructed arbitrary pixels X and adjacent pixels H,
V, N. In this embodiment, predictions are made with respective to
the remaining pixels of the respective sub-blocks in the
illustrated method.
[0134] In addition, this embodiment of the video encoding apparatus
contains the rest of the features of video encoding apparatus 700.
This means that first transformer 708 of the video encoding
apparatus in this embodiment may exclusively and independently
transform and then inverse transform each of arbitrary pixel
residual signals of the arbitrary pixel residual block, perform a
DCT transform on each of arbitrary pixel residual signals of the
arbitrary pixel residual block by using the modified basic vectors
to exclusively and independently transform the respective arbitrary
pixel residual signals and additionally perform the Hadamard
transform.
[0135] First inverse transformer 714 of the video encoding
apparatus in this embodiment may also perform an inverse DCT
transform on the frequency coefficients resulting from transforms
of the respective arbitrary pixel residual signals by using the
modified basic vectors to exclusively and independently inverse
transform the respective arbitrary pixel residual signals, and then
process the frequency coefficients from the transforms of the
respective arbitrary pixel residual signals through an inverse
Hadamard transform followed by an inverse DCT transform.
[0136] Second transformer 724 of the video encoding apparatus in
this embodiment may also process the residual macroblock by unit of
4.times.4 block through a DCT transform while using basic vectors
determined according to the sequence of a first one-dimensional
transform direction of the DCT transform. At this time, the
sequence of the first one-dimensional transform direction may be
determined according to the prediction direction of the first
predictor. In addition, second transformer 724, when applying a
horizontal one-dimensional transform first and then a vertical
one-dimensional transform, may perform the horizontal
one-dimensional transform with respect to respective 4.times.4
blocks using an existing basic vector in their first to third rows
and a modified basic vector in their fourth rows, and performs the
vertical one-dimensional transform with respect to the respective
4.times.4 blocks using the modified basic vector in their first
columns and the modified basic vector in their second to fourth
columns, and in a reversed sequence of applying a vertical
one-dimensional transform first and then a horizontal
one-dimensional transform, it performs the vertical one-dimensional
transform with respect to respective 4.times.4 blocks using an
existing basic vector in their first to third columns and a
modified basic vector in their fourth columns, and performs the
horizontal one-dimensional transform with respect to the respective
4.times.4 blocks using the modified basic vector in their first
rows and the modified basic vector in their second to fourth
rows.
[0137] Second inverse transformer 732 of the video encoding
apparatus in this embodiment may also process the frequency
coefficients by unit of 4.times.4 block through an inverse DCT
transform while using basic vectors determined according to the
sequence of a first one-dimensional inverse transform direction of
the inverse DCT transform and the sequence of the first
one-dimensional inverse transform direction may be determined to be
the reverse of the sequence of the first one-dimensional transform
direction. Additionally, second inverse transformer 732, when
applying a horizontal one-dimensional inverse transform first and
then a vertical one-dimensional inverse transform, may perform the
horizontal one-dimensional inverse transform with respect to
respective 4.times.4 blocks using a modified basic vector in their
first rows and an existing basic vector in their second to fourth
rows, and perform the vertical one-dimensional inverse transform
with respect to the respective 4.times.4 blocks using the existing
basic vector in their first to third columns and the modified basic
vector in their fourth columns, and in a reversed sequence of
applying a vertical one-dimensional inverse transform first and
then a horizontal one-dimensional inverse transform, it may perform
the vertical one-dimensional inverse transform with respect to
respective 4.times.4 blocks using a modified basic vector in their
first columns and an existing basic vector in their second to
fourth columns, and perform the horizontal one-dimensional inverse
transform with respect to the respective 4.times.4 blocks using the
existing basic vector in their first to third rows and the modified
basic vector in their fourth rows. Second inverse transformer 732
may also process the frequency coefficients by unit of 4.times.4
block through the inverse DCT transform while using existing basic
vectors and use some of the frequency coefficients and some of the
respective reconstructed arbitrary pixels to predict the remaining
frequency coefficients.
[0138] First predictor 704 in this embodiment may perform a
low-pass filtering on the adjacent pixels to the current block and
predict the arbitrary pixel block using down-sampled pixels, and it
may do the prediction in the same direction as a prediction
direction according to the prediction mode of the current
macroblock.
[0139] On the other hand, in both of the first embodiment of the
video encoding apparatus and this embodiment of the video encoding
apparatus, second inverse transformer 732 may use some of the
frequency coefficients and some of the respective reconstructed
arbitrary pixels to predict the remaining frequency coefficients,
whereby they may not need the modified basic vectors but use the
existing basic vectors in performing the inverse DCT transform
while having arbitrary pixels inverse transformed independently of
the remaining pixels.
[0140] In such a two-dimensional inverse transform, the DCT inverse
transform basic vectors shown in FIG. 5, i.e. the existing basic
vectors are used rather than the modified basic vectors from second
inverse transformer 732 in operation. Upon completing the inverse
transform, the quantized frequency coefficients substitute the DC
frequency coefficient positions in order to be transmitted.
[0141] FIGS. 27 and 28 are exemplary diagrams showing the
procedures of using partial frequency coefficients to predict the
remaining frequency coefficients according to an embodiment and the
alternative embodiment, respectively.
[0142] When second transformer 724 performs a horizontal
one-dimensional transform first and then a vertical one-dimensional
transform, second inverse transformer 732 may calculate DC.sub.P
through Equation 6. Equation 6 is to perform the one-dimensional
inverse DCT transform by using the existing basic vector for the
inverse transform as shown in FIG. 5.
[0143] FIG. 27 shows DC.sub.00 at (1) which is the frequency
coefficient transformed in first transformer 708 and the remaining
frequency coefficients AC.sub.01.about.AC.sub.33 that are the
frequency coefficients transformed in second transformer 724, and
if the reconstructed arbitrary pixel X.sub.i' in the corresponding
sub-block is same as (2), then vertical one-dimensional inverse DCT
transforms are performed in their columns 1 to 3 only to obtain
AC{circle around (1)}, AC{circle around (2)}, AC{circle around (3)}
as in (3), and calculate with AC{circle around (1)}, AC{circle
around (2)}, AC{circle around (3)}, and the reconstructed arbitrary
pixels X.sub.i' to give DC.sub.P.
0.5.times.Coef{circle around (1)}-0.6533.times.Coef{circle around
(2)}+0.5.times.Coef{circle around (3)}-0.2706.times.Coef{circle
around (4)}=X.sub.i'' Equation 6
[0144] Equation 6 rearranged for Coef{circle around (1)} will
derive Equation 7.
2.times.(-0.6533.times.Coef{circle around
(2)}+0.5.times.Coef{circle around (3)}-0.2706.times.Coef{circle
around (4)}-X.sub.i'')=Coef{circle around (1)} Equation 7 [0145]
where Coef{circle around (2)}, Coef{circle around (3)}, and
Coef{circle around (4)} respectively correspond to AC{circle around
(1)}, AC{circle around (2)}, and AC{circle around (3)} in FIGS. 27
and 28, and Coef{circle around (1)} is DC.sub.P.
[0146] Thereafter, AC.sub.10, AC.sub.20, AC.sub.30, and DC.sub.P
are used to calculate DC.sub.00p as at (5). The method of
calculating DC.sub.00p is same as calculating the above described
DC.sub.P. By using the calculated DC.sub.00p and AC.sub.10,
AC.sub.20, AC.sub.30, a vertical one-dimensional inverse transform
is performed on the remaining column 1 as at (6). As to the
remaining horizontal one-dimensional inverse transform, the DCT
inverse transform basic vectors shown in FIG. 5, i.e. the existing
basic vectors are used to perform the inverse transform as at (7)
to output the reconstructed pixels of the corresponding sub-block
as at (8). In this way, reductions in AC component frequency
coefficients will be greater as compared to using the modified
basic vectors as in FIG. 11. In addition, by using the existing
basic vectors in possession without having to prepare the modified
basic vectors, it is possible to inverse transform the arbitrary
pixels independently of the remaining pixels of the sub-block or
the frequency coefficients of the remaining pixels.
[0147] In the description above, although all of the components of
the embodiments of the present disclosure may have been explained
as assembled or operatively connected as a unit, the present
disclosure is not intended to limit itself to such embodiments.
Rather, within the objective scope of the present disclosure, the
respective components may be selectively and operatively combined
in any numbers. Every one of the components may be also implemented
by itself in hardware while the respective ones can be combined in
part or as a whole selectively and implemented in a computer
program having program modules for executing functions of the
hardware equivalents. Codes or code segments to constitute such a
program may be easily deduced by a person skilled in the art. The
computer program may be stored in computer readable media, which in
operation can realize the embodiments of the present disclosure. As
the computer readable media, the candidates include magnetic
recording media, optical recording media, and carrier wave
media.
[0148] Also, terms like `include`, `comprise`, and `have` should be
interpreted in default as inclusive or open rather than exclusive
or closed unless expressly defined to the contrary. All the terms
that are technical, scientific or otherwise agree with the meanings
as understood by a person skilled in the art unless defined to the
contrary. Common terms as found in dictionaries should be
interpreted in the context of the related technical writings not
too ideally or impractically unless the present disclosure
expressly defines them so.
[0149] Although exemplary embodiments of the present disclosure
have been described for illustrative purposes, those skilled in the
art will appreciate that various modifications, additions and
substitutions are possible, without departing from essential
characteristics of the disclosure. Therefore, exemplary embodiments
of the present disclosure have not been described for limiting
purposes. Accordingly, the scope of the disclosure is not to be
limited by the above embodiments but by the claims and the
equivalents thereof.
INDUSTRIAL APPLICABILITY
[0150] As described above, the present disclosure is highly useful
for application in a video encoding or decoding apparatus to intra
predict the pixels of the block to currently encode or decode with
an improved prediction accuracy to increase the video coding
efficiency.
* * * * *