U.S. patent application number 13/333529 was filed with the patent office on 2013-04-04 for two-dimensional motion compensation filter operation and processing.
This patent application is currently assigned to BROADCOM CORPORATION. The applicant listed for this patent is Ba-Zhong Shen. Invention is credited to Ba-Zhong Shen.
Application Number | 20130083852 13/333529 |
Document ID | / |
Family ID | 47992566 |
Filed Date | 2013-04-04 |
United States Patent
Application |
20130083852 |
Kind Code |
A1 |
Shen; Ba-Zhong |
April 4, 2013 |
Two-dimensional motion compensation filter operation and
processing
Abstract
Two-dimensional motion compensation filter operation and
processing. A video bitstream or signal corresponding thereto
undergoes motion compensation operations simultaneously or in
parallel with respect to at least two respective dimensions (e.g.,
at least horizontal and vertical) in accordance with generating
coefficient values employed for generating a decoded and/or output
video signal. The simultaneous and in parallel operations made with
respect to more than one dimension associated with the video
bitstream or signal may employ a two-dimensional discrete cosine
transform (2-D DCT) implemented to operate on more than one
dimension simultaneously. Same or different respective
fractional-pel distances may be employed with respect to multiple
respective dimensions (e.g., common/same fractional-pel distance
for all of the multiple respective dimensions, or different
respective fractional-pel distances with respect to each of the
multiple respective dimensions [such as a first fractional-pel
distance for a first dimension, a second fractional-pel distance
for a second dimension, etc.]).
Inventors: |
Shen; Ba-Zhong; (Irvine,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Shen; Ba-Zhong |
Irvine |
CA |
US |
|
|
Assignee: |
BROADCOM CORPORATION
IRVINE
CA
|
Family ID: |
47992566 |
Appl. No.: |
13/333529 |
Filed: |
December 21, 2011 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61541938 |
Sep 30, 2011 |
|
|
|
Current U.S.
Class: |
375/240.16 ;
375/E7.226 |
Current CPC
Class: |
H04N 19/40 20141101;
H04N 19/164 20141101; H04N 19/86 20141101; H04N 19/895 20141101;
H04N 19/30 20141101; H04N 19/103 20141101 |
Class at
Publication: |
375/240.16 ;
375/E07.226 |
International
Class: |
H04N 7/32 20060101
H04N007/32 |
Claims
1. An apparatus, comprising: a motion compensation module for
performing fractional pixel position interpolation, by employing
two-dimensional discrete cosine transform (2-D DCT) processing, of
a first plurality of pixels of a first frame with respect to a
first dimension and a second dimension simultaneously or in
parallel thereby generating a plurality of coefficient values for
use in generating a second plurality of pixels of a second frame;
and wherein: each of the plurality of coefficient values being
based on a respective at least one fractional-pel value in the
first dimension and a respective at least one fractional-pel value
in the second dimension; the at least one fractional-pel value in
the first dimension based on a first fractional-pel distance; the
at least one fractional-pel value in the second dimension based on
a second fractional-pel distance; and the second frame being
situated prior to or after the first frame in a sequence of
frames.
2. The apparatus of claim 1, wherein: the first fractional-pel
distance being the second fractional-pel distance.
3. The apparatus of claim 1, wherein: the apparatus being a video
decoder for generating the second plurality of pixels of the second
frame using at least one of the plurality of coefficient values, at
least one motion vector relating at least one of the second
plurality of pixels of the second frame and at least one of the
plurality of coefficient values, and at least one residual value
corresponding to at least one of the second plurality of pixels of
the second frame.
4. The apparatus of claim 1, wherein: the apparatus being a
destination device including an input for receiving an input
bitstream corresponding to the first frame and the second frame or
a signal corresponding to the input bitstream from a source device
or a middling device via at least one communication network.
5. The apparatus of claim 1, wherein: the apparatus being a
communication device operative within at least one of a satellite
communication system, a wireless communication system, a wired
communication system, a fiber-optic communication system, and a
mobile communication system.
6. An apparatus, comprising: a motion compensation module for
performing fractional pixel position interpolation of a first
plurality of pixels of a first frame with respect to a first
dimension and a second dimension simultaneously or in parallel
thereby generating a plurality of coefficient values for use in
generating a second plurality of pixels of a second frame.
7. The apparatus of claim 6, wherein: the apparatus being a video
decoder for generating the second plurality of pixels of the second
frame using at least one of the plurality of coefficient values, at
least one motion vector relating at least one of the second
plurality of pixels of the second frame and at least one of the
plurality of coefficient values, and at least one residual value
corresponding to at least one of the second plurality of pixels of
the second frame.
8. The apparatus of claim 6, wherein: the first frame being a
reference frame in a sequence of frames; and the second frame being
a current frame in the sequence of frames.
9. The apparatus of claim 6, wherein: the second frame being
situated prior to or after the first frame in a sequence of
frames.
10. The apparatus of claim 6, wherein: a motion compensation module
for employing two-dimensional discrete cosine transform (2-D DCT)
processing of the first plurality of pixels of the first frame with
respect to the first dimension and the second dimension
simultaneously or in parallel thereby generating the plurality of
coefficient values.
11. The apparatus of claim 6, wherein: each of the plurality of
coefficient values being based on a respective at least one
fractional-pel value in the first dimension and a respective at
least one fractional-pel value in the second dimension; and each of
the at least one fractional-pel value in the first dimension and
the at least one fractional-pel value in the second dimension based
on a common fractional-pel distance.
12. The apparatus of claim 6, wherein: each of the plurality of
coefficient values being based on a respective at least one
fractional-pel value in the first dimension and a respective at
least one fractional-pel value in the second dimension; the at
least one fractional-pel value in the first dimension based on a
first fractional-pel distance; and the at least one fractional-pel
value in the second dimension based on a second fractional-pel
distance.
13. The apparatus of claim 6, wherein: the apparatus being a
destination device including an input for receiving an input
bitstream corresponding to the first frame and the second frame or
a signal corresponding to the input bitstream from a source device
or a middling device via at least one communication network.
14. The apparatus of claim 6, wherein: the apparatus being a
middling device including an input for receiving an input bitstream
corresponding to the first frame and the second frame or a signal
corresponding to the input bitstream from a source device or at
least one additional middling device via at least one communication
network; and the apparatus including an output for outputting an
output bitstream, a signal corresponding to an output bitstream, or
an output video signal to at least one destination device via the
at least one communication network or at least one additional
communication network.
15. The apparatus of claim 6, wherein: the apparatus being a
communication device operative within at least one of a satellite
communication system, a wireless communication system, a wired
communication system, a fiber-optic communication system, and a
mobile communication system.
16. A method for operating a video processing device, the method
comprising: via an input, receiving an input bitstream; operating a
motion compensation module for performing fractional pixel position
interpolation of a first plurality of pixels of a first frame with
respect to a first dimension and a second dimension simultaneously
or in parallel thereby generating a plurality of coefficient values
for use in generating a second plurality of pixels of a second
frame; and employing the first frame and the second frame for
generating an output video signal.
17. The method of claim 16, wherein: the video processing device
being a video decoder for generating the second plurality of pixels
of the second frame using at least one of the plurality of
coefficient values, at least one motion vector relating at least
one of the second plurality of pixels of the second frame and at
least one of the plurality of coefficient values, and at least one
residual value corresponding to at least one of the second
plurality of pixels of the second frame.
18. The method of claim 16, wherein: the first frame being a
reference frame in a sequence of frames; and the second frame being
a current frame in the sequence of frames.
19. The method of claim 16, wherein: the second frame being
situated prior to or after the first frame in a sequence of
frames.
20. The method of claim 16, further comprising: operating the
motion compensation module for employing two-dimensional discrete
cosine transform (2-D DCT) processing of the first plurality of
pixels of the first frame with respect to the first dimension and
the second dimension simultaneously or in parallel thereby
generating the plurality of coefficient values.
21. The method of claim 16, wherein: each of the plurality of
coefficient values being based on a respective at least one
fractional-pel value in the first dimension and a respective at
least one fractional-pel value in the second dimension; and each of
the at least one fractional-pel value in the first dimension and
the at least one fractional-pel value in the second dimension based
on a common fractional-pel distance.
22. The method of claim 16, wherein: each of the plurality of
coefficient values being based on a respective at least one
fractional-pel value in the first dimension and a respective at
least one fractional-pel value in the second dimension; the at
least one fractional-pel value in the first dimension based on a
first fractional-pel distance; and the at least one fractional-pel
value in the second dimension based on a second fractional-pel
distance.
23. The method of claim 16, wherein: the video processing device
being a destination device including the input for receiving an
input bitstream corresponding to the first frame and the second
frame or a signal corresponding to the input bitstream from a
source device or a middling device via at least one communication
network.
24. The method of claim 16, wherein: the video processing device
being a middling device including the input for receiving an input
bitstream corresponding to the first frame and the second frame or
a signal corresponding to the input bitstream from a source device
or at least one additional middling device via at least one
communication network; and the video processing device including an
output for outputting the video signal to at least one destination
device via the at least one communication network or at least one
additional communication network.
25. The method of claim 16, wherein: the video processing device
operative within at least one of a satellite communication system,
a wireless communication system, a wired communication system, a
fiber-optic communication system, and a mobile communication
system.
Description
CROSS REFERENCE TO RELATED PATENTS/PATENT APPLICATIONS
Provisional Priority Claims
[0001] The present U.S. Utility patent application claims priority
pursuant to 35 U.S.C. .sctn.119(e) to the following U.S.
Provisional patent application which is hereby incorporated herein
by reference in its entirety and made part of the present U.S.
Utility patent application for all purposes: [0002] 1. U.S.
Provisional Patent Application Ser. No. 61/541,938, entitled
"Coding, communications, and signaling of video content within
communication systems," (Attorney Docket No. BP23215), filed Sep.
30, 2011, pending.
Incorporation by Reference
[0003] The following standards/draft standards are hereby
incorporated herein by reference in their entirety and are made
part of the present U.S. Utility patent application for all
purposes: [0004] 1. "WD4: Working Draft 4 of High-Efficiency Video
Coding, Joint Collaborative Team on Video Coding (JCT-VC)," Joint
Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and
ISO/IEC JTC1/SC29/WG11 6th Meeting: Torino, IT, 14-22 Jul. 2011,
Document: JCTVC-F803 d4, 230 pages. [0005] 2. International
Telecommunication Union, ITU-T, TELECOMMUNICATION STANDARDIZATION
SECTOR OF ITU, H.264 (March 2010), SERIES H: AUDIOVISUAL AND
MULTIMEDIA SYSTEMS, Infrastructure of audiovisual services--Coding
of moving video, Advanced video coding for generic audiovisual
services, Recommendation ITU-T H.264, also alternatively referred
to as International Telecomm ISO/IEC 14496-10--MPEG-4 Part 10, AVC
(Advanced Video Coding), H.264/MPEG-4 Part 10 or AVC (Advanced
Video Coding), ITU H.264/MPEG4-AVC, or equivalent.
BACKGROUND OF THE INVENTION
[0006] 1. Technical Field of the Invention
[0007] The invention relates generally to digital video processing;
and, more particularly, it relates to performing motion
compensation filter processing in accordance with video processing
in accordance with such digital video processing.
[0008] 2. Description of Related Art
[0009] Communication systems that operate to communicate digital
media (e.g., images, video, data, etc.) have been under continual
development for many years. With respect to such communication
systems employing some form of video data, a number of digital
images are output or displayed at some frame rate (e.g., frames per
second) to effectuate a video signal suitable for output and
consumption. Within many such communication systems operating using
video data, there can be a trade-off between throughput (e.g.,
number of image frames that may be transmitted from a first
location to a second location) and video and/or image quality of
the signal eventually to be output or displayed. The present art
does not adequately or acceptably provide a means by which video
data may be transmitted from a first location to a second location
in accordance with providing an adequate or acceptable video and/or
image quality, ensuring a relatively low amount of overhead
associated with the communications, relatively low complexity of
the communication devices at respective ends of communication
links, etc.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0010] FIG. 1 illustrates an embodiment of communication
system.
[0011] FIG. 2A illustrates an embodiment of a computer.
[0012] FIG. 2B illustrates an embodiment of a laptop computer.
[0013] FIG. 2C illustrates an embodiment of a high definition (HD)
television.
[0014] FIG. 2D illustrates an embodiment of a standard definition
(SD) television.
[0015] FIG. 2E illustrates an embodiment of a handheld media
unit.
[0016] FIG. 2F illustrates an embodiment of a set top box
(STB).
[0017] FIG. 2G illustrates an embodiment of a digital video disc
(DVD) player.
[0018] FIG. 2H illustrates an embodiment of a generic digital image
and/or video processing device.
[0019] FIG. 3 is a diagram illustrating an embodiment of video
encoding architecture.
[0020] FIG. 4 is a diagram illustrating an embodiment of
intra-prediction processing.
[0021] FIG. 5 is a diagram illustrating an embodiment of
inter-prediction processing.
[0022] FIG. 6 is a diagram illustrating an embodiment of a video
decoding architecture.
[0023] FIG. 7, FIG. 8, and FIG. 9 illustrate various embodiments,
respectively, of discrete cosine transform (DCT) processing with
respect to pixels of a frame or picture.
[0024] FIG. 10 is a diagram illustrating an embodiment of integer
samples (blocks with upper-case letters) and fractional sample
positions (blocks with lower-case letters).
[0025] FIG. 11 illustrates an embodiment of two-dimensional
processing with respect to pixels of a frame or picture.
[0026] FIG. 12 illustrates an embodiment showing simultaneous or in
parallel generation of coefficients in multiple dimensions in
accordance with video processing operations.
[0027] FIG. 13A and FIG. 13B illustrate various embodiments of
methods for performing various video processing operations.
DETAILED DESCRIPTION OF THE INVENTION
[0028] Within many devices that use digital media such as digital
video, respective images thereof, being digital in nature, are
represented using pixels. Within certain communication systems,
digital media can be transmitted from a first location to a second
location at which such media can be output or displayed. The goal
of digital communications systems, including those that operate to
communicate digital video, is to transmit digital data from one
location, or subsystem, to another either error free or with an
acceptably low error rate. As shown in FIG. 1, data may be
transmitted over a variety of communications channels in a wide
variety of communication systems: magnetic media, wired, wireless,
fiber, copper, and/or other types of media as well.
[0029] FIG. 1 illustrates an embodiment of communication system
100.
[0030] Referring to FIG. 1, this embodiment of a communication
system 100 is a communication channel 199 that communicatively
couples a communication device 110 (including a transmitter 112
having an encoder 114 and including a receiver 116 having a decoder
118) situated at one end of the communication channel 199 to
another communication device 120 (including a transmitter 126
having an encoder 128 and including a receiver 122 having a decoder
124) at the other end of the communication channel 199. In some
embodiments, either of the communication devices 110 and 120 may
only include a transmitter or a receiver. There are several
different types of media by which the communication channel 199 may
be implemented (e.g., a satellite communication channel 130 using
satellite dishes 132 and 134, a wireless communication channel 140
using towers 142 and 144 and/or local antennae 152 and 154, a wired
communication channel 150, and/or a fiber-optic communication
channel 160 using electrical to optical (E/O) interface 162 and
optical to electrical (O/E) interface 164)). In addition, more than
one type of media may be implemented and interfaced together
thereby forming the communication channel 199.
[0031] It is noted that such communication devices 110 and/or 120
may be stationary or mobile without departing from the scope and
spirit of the invention. For example, either one or both of the
communication devices 110 and 120 may be implemented in a fixed
location or may be a mobile communication device with capability to
associate with and/or communicate with more than one network access
point (e.g., different respective access points (APs) in the
context of a mobile communication system including one or more
wireless local area networks (WLANs), different respective
satellites in the context of a mobile communication system
including one or more satellite, or generally, different respective
network access points in the context of a mobile communication
system including one or more network access points by which
communications may be effectuated with communication devices 110
and/or 120.
[0032] To reduce transmission errors that may undesirably be
incurred within a communication system, error correction and
channel coding schemes are often employed. Generally, these error
correction and channel coding schemes involve the use of an encoder
at the transmitter end of the communication channel 199 and a
decoder at the receiver end of the communication channel 199.
[0033] Any of various types of ECC codes described can be employed
within any such desired communication system (e.g., including those
variations described with respect to FIG. 1), any information
storage device (e.g., hard disk drives (HDDs), network information
storage devices and/or servers, etc.) or any application in which
information encoding and/or decoding is desired.
[0034] Generally speaking, when considering a communication system
in which video data is communicated from one location, or
subsystem, to another, video data encoding may generally be viewed
as being performed at a transmitting end of the communication
channel 199, and video data decoding may generally be viewed as
being performed at a receiving end of the communication channel
199.
[0035] Also, while the embodiment of this diagram shows
bi-directional communication being capable between the
communication devices 110 and 120, it is of course noted that, in
some embodiments, the communication device 110 may include only
video data encoding capability, and the communication device 120
may include only video data decoding capability, or vice versa
(e.g., in a uni-directional communication embodiment such as in
accordance with a video broadcast embodiment).
[0036] Digital image and/or video processing of digital images
and/or media (including the respective images within a digital
video signal) may be performed by any of the various devices
depicted below in FIG. 2A-2H to allow a user to view such digital
images and/or video. These various devices do not include an
exhaustive list of devices in which the image and/or video
processing described herein may be effectuated, and it is noted
that any generic digital image and/or video processing device may
be implemented to perform the processing described herein without
departing from the scope and spirit of the invention.
[0037] FIG. 2A illustrates an embodiment of a computer 201. The
computer 201 can be a desktop computer, or an enterprise storage
devices such a server, of a host computer that is attached to a
storage array such as a redundant array of independent disks (RAID)
array, storage router, edge router, storage switch and/or storage
director. A user is able to view still digital images and/or video
(e.g., a sequence of digital images) using the computer 201.
Oftentimes, various image and/or video viewing programs and/or
media player programs are included on a computer 201 to allow a
user to view such images (including video).
[0038] FIG. 2B illustrates an embodiment of a laptop computer 202.
Such a laptop computer 202 may be found and used in any of a wide
variety of contexts. In recent years, with the ever-increasing
processing capability and functionality found within laptop
computers, they are being employed in many instances where
previously higher-end and more capable desktop computers would be
used. As with the computer 201, the laptop computer 202 may include
various image viewing programs and/or media player programs to
allow a user to view such images (including video).
[0039] FIG. 2C illustrates an embodiment of a high definition (HD)
television 203. Many HD televisions 203 include an integrated tuner
to allow the receipt, processing, and decoding of media content
(e.g., television broadcast signals) thereon. Alternatively,
sometimes an HD television 203 receives media content from another
source such as a digital video disc (DVD) player, set top box (STB)
that receives, processes, and decodes a cable and/or satellite
television broadcast signal. Regardless of the particular
implementation, the HD television 203 may be implemented to perform
image and/or video processing as described herein. Generally
speaking, an HD television 203 has capability to display HD media
content and oftentimes is implemented having a 16:9 widescreen
aspect ratio.
[0040] FIG. 2D illustrates an embodiment of a standard definition
(SD) television 204. Of course, an SD television 204 is somewhat
analogous to an HD television 203, with at least one difference
being that the SD television 204 does not include capability to
display HD media content, and an SD television 204 oftentimes is
implemented having a 4:3 full screen aspect ratio. Nonetheless,
even an SD television 204 may be implemented to perform image
and/or video processing as described herein.
[0041] FIG. 2E illustrates an embodiment of a handheld media unit
205. A handheld media unit 205 may operate to provide general
storage or storage of image/video content information such as joint
photographic experts group (JPEG) files, tagged image file format
(TIFF), bitmap, motion picture experts group (MPEG) files, Windows
Media (WMA/WMV) files, other types of video content such as MPEG4
files, etc. for playback to a user, and/or any other type of
information that may be stored in a digital format. Historically,
such handheld media units were primarily employed for storage and
playback of audio media; however, such a handheld media unit 205
may be employed for storage and playback of virtual any media
(e.g., audio media, video media, photographic media, etc.).
Moreover, such a handheld media unit 205 may also include other
functionality such as integrated communication circuitry for wired
and wireless communications. Such a handheld media unit 205 may be
implemented to perform image and/or video processing as described
herein.
[0042] FIG. 2F illustrates an embodiment of a set top box (STB)
206. As mentioned above, sometimes a STB 206 may be implemented to
receive, process, and decode a cable and/or satellite television
broadcast signal to be provided to any appropriate display capable
device such as SD television 204 and/or HD television 203. Such an
STB 206 may operate independently or cooperatively with such a
display capable device to perform image and/or video processing as
described herein.
[0043] FIG. 2G illustrates an embodiment of a digital video disc
(DVD) player 207. Such a DVD player may be a Blu-Ray DVD player, an
HD capable DVD player, an SD capable DVD player, an up-sampling
capable DVD player (e.g., from SD to HD, etc.) without departing
from the scope and spirit of the invention. The DVD player may
provide a signal to any appropriate display capable device such as
SD television 204 and/or HD television 203. The DVD player 205 may
be implemented to perform image and/or video processing as
described herein.
[0044] FIG. 2H illustrates an embodiment of a generic digital image
and/or video processing device 208. Again, as mentioned above,
these various devices described above do not include an exhaustive
list of devices in which the image and/or video processing
described herein may be effectuated, and it is noted that any
generic digital image and/or video processing device 208 may be
implemented to perform the image and/or video processing described
herein without departing from the scope and spirit of the
invention.
[0045] FIG. 3 is a diagram illustrating an embodiment 300 of video
encoding architecture.
[0046] Referring to embodiment 300 of FIG. 3, with respect to this
diagram depicting an alternative embodiment of a video encoder,
such a video encoder carries out prediction, transform, and
encoding processes to produce a compressed output bit stream. Such
a video encoder may operate in accordance with and be compliant
with one or more video encoding protocols, standards, and/or
recommended practices such as ISO/IEC 14496-10--MPEG-4 Part 10, AVC
(Advanced Video Coding), alternatively referred to as H.264/MPEG-4
Part 10 or AVC (Advanced Video Coding), ITU H.264/MPEG4-AVC.
[0047] It is noted that a corresponding video decoder, such as
located within a device at another end of a communication channel,
is operative to perform the complementary processes of decoding,
inverse transform, and reconstruction to produce a respective
decoded video sequence that is (ideally) representative of the
input video signal.
[0048] As may be seen with respect to this diagram and as may be
understood with respect to various embodiments, alternative
arrangements and architectures may be employed for effectuating
video encoding. Generally speaking, an encoder processes an input
video signal (e.g., typically composed in units of macro-blocks,
often times being square in shape and including N.times.N pixels
therein). The video encoding determines a prediction of the current
macro-block based on previously coded data. That previously coded
data may come from the current frame (or picture) itself (e.g.,
such as in accordance with intra-prediction) or from one or more
other frames (or pictures) that have already been coded (e.g., such
as in accordance with inter-prediction). The video encoder
subtracts the prediction of the current macro-block to form a
residual.
[0049] Generally speaking, intra-prediction is operative to employ
block sizes of one or more particular sizes (e.g., 16.times.16,
8.times.8, or 4.times.4) to predict a current macro-block from
surrounding, previously coded pixels within the same frame (or
picture). Generally speaking, inter-prediction is operative to
employ a range of block sizes (e.g., 16.times.16 down to 4.times.4)
to predict pixels in the current frame (or picture) from regions
that are selected from within one or more previously coded frames
(or pictures).
[0050] With respect to the transform and quantization operations, a
block of residual samples may undergo transformation using a
particular transform (e.g., 4.times.4 or 8.times.8). One possible
embodiment of such a transform operates in accordance with discrete
cosine transform (DCT). The transform operation outputs a group of
coefficients such that each respective coefficient corresponds to a
respective weighting value of one or more basis functions
associated with a transform. After undergoing transformation, a
block of transform coefficients is quantized (e.g., each respective
coefficient may be divided by an integer value and any associated
remainder may be discarded, or they may be multiplied by an integer
value). The quantization process is generally inherently lossy, and
it can reduce the precision of the transform coefficients according
to a quantization parameter (QP). Typically, many of the
coefficients associated with a given macro-block are zero, and only
some nonzero coefficients remain. Generally, a relatively high QP
setting is operative to result in a greater proportion of
zero-valued coefficients and smaller magnitudes of non-zero
coefficients, resulting in relatively high compression (e.g.,
relatively lower coded bit rate) at the expense of relatively
poorly decoded image quality; a relatively low QP setting is
operative to allow more nonzero coefficients to remain after
quantization and larger magnitudes of non-zero coefficients,
resulting in relatively lower compression (e.g., relatively higher
coded bit rate) with relatively better decoded image quality.
[0051] The video encoding process produces a number of values that
are encoded to form the compressed bit stream. Examples of such
values include the quantized transform coefficients, information to
be employed by a decoder to re-create the appropriate prediction,
information regarding the structure of the compressed data and
compression tools employed during encoding, information regarding a
complete video sequence, etc. Such values and/or parameters (e.g.,
syntax elements) may undergo encoding within an entropy encoder
operating in accordance with CABAC, CAVLC, or some other entropy
coding scheme, to produce an output bit stream that may be stored,
transmitted (e.g., after undergoing appropriate processing to
generate a continuous time signal that comports with a
communication channel), etc.
[0052] In an embodiment operating using a feedback path, the output
of the transform and quantization undergoes inverse quantization
and inverse transform. One or both of intra-prediction and
inter-prediction may be performed in accordance with video
encoding. Also, motion compensation and/or motion estimation may be
performed in accordance with such video encoding.
[0053] The signal path output from the inverse quantization and
inverse transform (e.g., IDCT) block, which is provided to the
intra-prediction block, is also provided to a de-blocking filter.
In certain optional embodiments, the output from the de-blocking
filter is provided to one or more other in-loop filters (e.g.,
implemented in accordance with adaptive loop filter (ALF), sample
adaptive offset (SAO) filter, and/or any other filter type)
implemented to process the output from the inverse transform block.
For example, such an ALF is applied to the decoded picture before
it is stored in a picture buffer (again, sometimes alternatively
referred to as a DPB, digital picture buffer). Such an ALF is
implemented to reduce coding noise of the decoded picture, and the
filtering thereof may be selectively applied on a slice by slice
basis, respectively, for luminance and chrominance whether or not
such an ALF is applied either at slice level or at block level.
Two-dimensional 2-D finite impulse response (FIR) filtering may be
used in application of such an ALF. The coefficients of the filters
may be designed slice by slice at the encoder, and such information
is then signaled to the decoder (e.g., signaled from a transmitter
communication device including a video encoder [alternatively
referred to as encoder] to a receiver communication device
including a video decoder [alternatively referred to as
decoder]).
[0054] One embodiment is operative to generate the coefficients in
accordance with Wiener filtering design. In addition, it may be
applied on a block by block based at the encoder whether the
filtering is performed and such a decision is then signaled to the
decoder (e.g., signaled from a transmitter communication device
including a video encoder [alternatively referred to as encoder] to
a receiver communication device including a video decoder
[alternatively referred to as decoder]) based on quadtree
structure, where the block size is decided according to the
rate-distortion optimization. It is noted that the implementation
of using such 2-D filtering may introduce a degree of complexity in
accordance with both encoding and decoding. For example, by using
2-D filtering in accordance and implementation of an ALF, there may
be some increasing complexity within encoder implemented within the
transmitter communication device as well as within a decoder
implemented within a receiver communication device.
[0055] As mentioned with respect to other embodiments, the use of
an ALF can provide any of a number of improvements in accordance
with such video processing, including an improvement on the
objective quality measure by the peak to signal noise ratio (PSNR)
that comes from performing random quantization noise removal. In
addition, the subjective quality of a subsequently encoded video
signal may be achieved from illumination compensation, which may be
introduced in accordance with performing offset processing and
scaling processing (e.g., in accordance with applying a gain) in
accordance with ALF processing.
[0056] With respect to any video encoder architecture implemented
to generate an output bitstream, it is noted that such
architectures may be implemented within any of a variety of
communication devices. The output bitstream may undergo additional
processing including error correction code (ECC), forward error
correction (FEC), etc. thereby generating a modified output
bitstream having additional redundancy deal therein. Also, as may
be understood with respect to such a digital signal, it may undergo
any appropriate processing in accordance with generating a
continuous time signal suitable for or appropriate for transmission
via a communication channel That is to say, such a video encoder
architecture may be of limited within a communication device
operative to perform transmission of one or more signals via one or
more communication channels. Additional processing may be made on
an output bitstream generated by such a video encoder architecture
thereby generating a continuous time signal that may be launched
into a communication channel.
[0057] FIG. 4 is a diagram illustrating an embodiment 400 of
intra-prediction processing. As can be seen with respect to this
diagram, a current block of video data (e.g., often times being
square in shape and including generally N.times.N pixels) undergoes
processing to estimate the respective pixels therein. Previously
coded pixels located above and to the left of the current block are
employed in accordance with such intra-prediction. From certain
perspectives, an intra-prediction direction may be viewed as
corresponding to a vector extending from a current pixel to a
reference pixel located above or to the left of the current pixel.
Details of intra-prediction as applied to coding in accordance with
H.264/AVC are specified within the corresponding standard (e.g.,
International Telecommunication Union, ITU-T, TELECOMMUNICATION
STANDARDIZATION SECTOR OF ITU, H.264 (March 2010), SERIES H:
AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Infrastructure of audiovisual
services--Coding of moving video, Advanced video coding for generic
audiovisual services, Recommendation ITU-T H.264, also
alternatively referred to as International Telecomm ISO/IEC
14496-10--MPEG-4 Part 10, AVC (Advanced Video Coding), H.264/MPEG-4
Part 10 or AVC (Advanced Video Coding), ITU H.264/MPEG4-AVC, or
equivalent) that is incorporated by reference above.
[0058] The residual, which is the difference between the current
pixel and the reference or prediction pixel, is that which gets
encoded. As can be seen with respect to this diagram,
intra-prediction operates using pixels within a common frame (or
picture). It is of course noted that a given pixel may have
different respective components associated therewith, and there may
be different respective sets of samples for each respective
component.
[0059] FIG. 5 is a diagram illustrating an embodiment 500 of
inter-prediction processing. In contradistinction to
intra-prediction, inter-prediction is operative to identify a
motion vector (e.g., an inter-prediction direction) based on a
current set of pixels within a current frame (or picture) and one
or more sets of reference or prediction pixels located within one
or more other frames (or pictures) within a frame (or picture)
sequence. As can be seen, the motion vector extends from the
current frame (or picture) to another frame (or picture) within the
frame (or picture) sequence. Inter-prediction may utilize sub-pixel
interpolation, such that a prediction pixel value corresponds to a
function of a plurality of pixels in a reference frame or
picture.
[0060] A residual may be calculated in accordance with
inter-prediction processing, though such a residual is different
from the residual calculated in accordance with intra-prediction
processing. In accordance with inter-prediction processing, the
residual at each pixel again corresponds to the difference between
a current pixel and a predicted pixel value. However, in accordance
with inter-prediction processing, the current pixel and the
reference or prediction pixel are not located within the same frame
(or picture). While this diagram shows inter-prediction as being
employed with respect to one or more previous frames or pictures,
it is also noted that alternative embodiments may operate using
references corresponding to frames before and/or after a current
frame. For example, in accordance with appropriate buffering and/or
memory management, a number of frames may be stored. When operating
on a given frame, references may be generated from other frames
that precede and/or follow that given frame.
[0061] Coupled with the CU, a basic unit may be employed for the
prediction partition mode, namely, the prediction unit, or PU. It
is also noted that the PU is defined only for the last depth CU,
and its respective size is limited to that of the CU.
[0062] FIG. 6 is a diagram illustrating an embodiment 600 of a
video decoding architecture.
[0063] Generally speaking, such video decoding architectures
operate on an input bitstream. It is of course noted that such an
input bitstream may be generated from a signal that is received by
a communication device from a communication channel. Various
operations may be performed on a continuous time signal received
from the communication channel, including digital sampling,
demodulation, scaling, filtering, etc. such as may be appropriate
in accordance with generating the input bitstream. Moreover,
certain embodiments, in which one or more types of error correction
code (ECC), forward error correction (FEC), etc. may be
implemented, may perform appropriate decoding in accordance with
such ECC, FEC, etc. thereby generating the input bitstream. That is
to say, in certain embodiments in which additional redundancy may
have been made in accordance with generating a corresponding output
bitstream (e.g., such as may be launched from a transmitter
communication device or from the transmitter portion of a
transceiver communication device), appropriate processing may be
performed in accordance with generating the input bitstream.
Overall, such a video decoding architectures and lamented to
process the input bitstream thereby generating an output video
signal corresponding to the original input video signal, as closely
as possible and perfectly in an ideal case, for use in being output
to one or more video display capable devices.
[0064] Referring to the embodiment 600 of FIG. 6, generally
speaking, a decoder such as an entropy decoder (e.g., which may be
implemented in accordance with CABAC, CAVLC, etc.) processes the
input bitstream in accordance with performing the complementary
process of encoding as performed within a video encoder
architecture. The input bitstream may be viewed as being, as
closely as possible and perfectly in an ideal case, the compressed
output bitstream generated by a video encoder architecture. Of
course, in a real-life application, it is possible that some errors
may have been incurred in a signal transmitted via one or more
communication links. The entropy decoder processes the input
bitstream and extracts the appropriate coefficients, such as the
DCT coefficients (e.g., such as representing chroma, luma, etc.
information) and provides such coefficients to an inverse
quantization and inverse transform block. In the event that a DCT
transform is employed, the inverse quantization and inverse
transform block may be implemented to perform an inverse DCT (IDCT)
operation. Subsequently, A/D blocking filter is implemented to
generate the respective frames and/or pictures corresponding to an
output video signal. These frames and/or pictures may be provided
into a picture buffer, or a digital picture buffer (DPB) for use in
performing other operations including motion compensation.
Generally speaking, such motion compensation operations may be
viewed as corresponding to inter-prediction associated with video
encoding. Also, intra-prediction may also be performed on the
signal output from the inverse quantization and inverse transform
block. Analogously as with respect to video encoding, such a video
decoder architecture may be implemented to perform mode selection
between performing it neither intra-prediction nor
inter-prediction, inter-prediction, or intra-prediction in
accordance with decoding an input bitstream thereby generating an
output video signal.
[0065] In certain optional embodiments, one or more in-loop filters
(e.g., implemented in accordance with adaptive loop filter (ALF),
sample adaptive offset (SAO) filter, and/or any other filter type)
such as may be implemented in accordance with video encoding as
employed to generate an output bitstream, a corresponding one or
more in-loop filters may be implemented within a video decoder
architecture. In one embodiment, an appropriate implementation of
one or more such in-loop filters is after the de-blocking
filter.
[0066] FIG. 7, FIG. 8, and FIG. 9 illustrate various embodiments
1100, 1200, and 1300, respectively, of discrete cosine transform
(DCT) processing with respect to pixels of a frame or picture.
[0067] Referring to the embodiment 700 of FIG. 7, 1/4-pel 8-tap DCT
as proposed in accordance with the HEVC Working Draft is
pictorially illustrated. Such an implementation employs 3 filters
at position 1/4, 1/2 and 3/4 and as modified as follows:
H ( 1 / 4 ) = ( - 1 , 4 , - 10 , 57 , 19 , - 7 , 3 , - 1 ) / 2 B =
( h 1 , h 2 , h 3 , h 4 , h 5 , h 6 , h 7 , h 8 ) ##EQU00001## H (
1 / 2 ) = ( - 1 , 4 , - 11 , 40 , 40 , - 11 , 4 , - 1 ) / 2 B *
symmetric * H ( 3 / 4 ) = ( h 8 h 7 h 6 , h 5 , h 4 , h 3 , h 2 h 1
) ##EQU00001.2##
[0068] The pixels, a, b, and c are determined as follows:
[0069] a: Use H(1/4) with horizontal 8 full pixel samples (4 left
and 4 right)
[0070] b: Use H(1/2) with horizontal 8 full pixel samples (4 left
and 4 right)
[0071] c: Use H(3/4) with horizontal 8 full pixel samples (4 left
and 4 right)
[0072] Referring to the embodiment 800 of FIG. 8, the pixels, a, b,
and c are determined as follows:
a=(A.sub.-3,0A.sub.-2,0A.sub.-1,0A.sub.0,0A.sub.1,0A.sub.2,0A.sub.3,0A.s-
ub.4,0)*H(1/4).sup.T
b=(A.sub.-3,0A.sub.-2,0A.sub.-1,0A.sub.0,0A.sub.1,0A.sub.2,0A.sub.3,0A.s-
ub.4,0)*H(1/2).sup.T
c=(A.sub.-3,0A.sub.-2,0A.sub.-1,0A.sub.0,0A.sub.1,0A.sub.2,0A.sub.3,0A.s-
ub.4,0)*H(3/4).sup.T
[0073] a: Use H(1/4) with horizontal 12 full pixel samples (6 left
and 6 right)
[0074] b: Use H(1/2) with horizontal 12 full pixel samples (6 left
and 6 right)
[0075] c: Use H(3/4) with horizontal 12 full pixel samples (6 left
and 6 right)
[0076] The pixels, d, h, and n are determined as follows:
d=(A.sub.0,-3A.sub.0,-2A.sub.0,-1A.sub.0,0A.sub.0,1A.sub.0,2A.sub.0,3A.s-
ub.0,4)*H(1/4).sup.T
h=(A.sub.0,-3A.sub.0,-2A.sub.0,-1A.sub.0,0A.sub.0,1A.sub.0,2A.sub.0,3A.s-
ub.0,4)*H(1/2).sup.T
n=(A.sub.0,-3A.sub.0,-2A.sub.0,-1A.sub.0,0A.sub.0,1A.sub.0,2A.sub.0,3A.s-
ub.0,4)*H(3/4).sup.T
[0077] d: Use H(1/4) with vertical 8 full pixel samples (4 above
and 4 down)
[0078] h: Use H(1/2) with vertical 8 full pixel samples (4 above
and 4 down)
[0079] n: Use H(3/4) with vertical 8 full pixel samples (4 above
and 4 down)
[0080] Referring to the embodiment 900 of FIG. 9, the intermediate
values are determined as follows:
d.sub.i,0=(A.sub.i,-3A.sub.i,-2A.sub.i,-1A.sub.i,0A.sub.i,1A.sub.i,2A.su-
b.i,3A.sub.i,4)*H(1/4).sup.T
h.sub.i,0=(A.sub.i,-3A.sub.i,-2A.sub.i,-1A.sub.i,0A.sub.i,1A.sub.i,2A.su-
b.i,3A.sub.i,4)*H(1/2).sup.T
n.sub.i,0=(A.sub.i,-3A.sub.i,-2A.sub.i,-1A.sub.i,0A.sub.i,1A.sub.i,2A.su-
b.i,3A.sub.i,4)*H(3/4).sup.T
[0081] i=-3, -2, -1, 1, 2, 3, 4
[0082] The processing operates to compute intermediate values in
left 3 columns of full pixel samples and right 4 columns of full
pixel samples. The same 3 filters are used with the total being 7
times filter computations. This is operative to obtain values (di,
hi, ni), i=-3, -2, -1, 1, 2, 3, 4 as follows:
e=(d.sub.-3,0d.sub.-2,0d.sub.-1,0d.sub.0,0d.sub.1,0d.sub.2,0d.sub.3,0d.s-
ub.4,0)H(1/4).sup.T
f=(d.sub.-3,0d.sub.--2,0d.sub.-1,0d.sub.0,0d.sub.1,0d.sub.2,0d.sub.3,0d.-
sub.4,0)H(1/2).sup.T
g=(d.sub.-3,0d.sub.-2,0d.sub.-1,0d.sub.0,0d.sub.1,0d.sub.2,0d.sub.3,0d.s-
ub.4,0)H(3/4).sup.T
i=(h.sub.-3,0h.sub.-2,0h.sub.-1,0h.sub.0,0h.sub.1,0h.sub.2,0h.sub.3,0h.s-
ub.4,0)H(1/4).sup.T
j=(h.sub.-3,0h.sub.-2,0h.sub.-1,0h.sub.0,0h.sub.1,0h.sub.2,0h.sub.3,0h.s-
ub.4,0)H(1/2).sup.T
k=(h.sub.-3,0h.sub.-2,0h.sub.-1,0h.sub.0,0h.sub.1,0h.sub.2,0h.sub.3,0h.s-
ub.4,0)H(3/4).sup.T
p=(n.sub.-3,0n.sub.-2,0n.sub.-1,0n.sub.0,0n.sub.1,0n.sub.2,0n.sub.3,0n.s-
ub.4,0)H(1/4).sup.T
q=(n.sub.-3,0n.sub.-2,0n.sub.-1,0n.sub.0,0n.sub.1,0n.sub.2,0n.sub.3,0n.s-
ub.4,0)H(1/2).sup.T
r=(n.sub.-3,0n.sub.-2,0n.sub.-1,0n.sub.0,0n.sub.1,0n.sub.2,0n.sub.3,0n.s-
ub.4,0)H(3/4).sup.T
[0083] Use filter H(1/4), H(1/2) and H(3/4) with 8 intermediate
value di, i=-3, . . . , 4 to get e, f and g
[0084] Use filter H(1/4), H(1/2) and H(3/4) with 8 intermediate
value hi, i=-3, . . . , 4 to get i, j and k
[0085] Use filter H(1/4), H(1/2) and H(3/4) with 8 intermediate
value ni, i=-3, . . . , 4 to get p, q and r
[0086] At least one drawback may be understood, in that, the
computation associated with fractional-pixels e, f, g, i, j, k, p,
q, r cannot be carried out at the same time as that of sub-pixels
a, b, c, d, h. n. As may be understood, the fact that these
operations can be performed simultaneously, in parallel, etc.
prohibits the respective implementation in accordance with
parallelism. Also, the fractional-pixels are always up-sampled
(estimated) in one direction, either horizontal or vertical.
However, as may be understood with respect to an actual picture or
frame, the picture or frame is actually a 2 dimensional object.
That is to say, by performing such sampling only with respect to a
single dimension, the operations associated therewith are
inherently limited and imperfect.
[0087] FIG. 10 is a diagram illustrating an embodiment of integer
samples (blocks with upper-case letters) and fractional sample
positions (blocks with lower-case letters).
[0088] With respect to this diagram, the uppercase or capital
letters represent actual pixel values. The lowercase letters
represent fractional sample positions interveningly situated
between actual pixel values. With respect to calculating the actual
pixel values for a current picture or frame, motion compensation
operations, such as may be performed in accordance with a video
decoding architecture, operate by using a motion vector which
relates a current pixel within a current frame or picture to a
particular location within another frame or picture (e.g., which
may be viewed as a reference frame or picture that is situated
either before or after the current frame). The motion vector
identifies a particular location within the reference frame or
picture, and this particular location may sometimes not correspond
exactly with a given pixel location within that reference frame or
picture. In such situations, interpolation is then performed with
respect to the fractional sample positions interveningly situated
between actual pixel values within the reference frame or picture.
For example, this diagram particularly shows three fractional
sample positions intervening between respective actual pixel values
in both the vertical and horizontal directions. When interpolation
needs to be performed (e.g., such as when a motion vector points to
a location within a reference frame or picture that is not exactly
an actual pixel value within the reference frame or picture), then
calculation of the respective fractional sample positions between
the actual pixel values is performed.
[0089] In accordance with such video processing, such interpolation
may be employed for calculating the actual pixel values of a
current frame or picture. For example, based upon such
interpolation as may be performed to identify the appropriate value
associated with an actual location within a reference frame or
picture (e.g., again, which may not correspond exactly to an actual
pixel value within the reference frame or picture and for which
interpolation may be performed), these calculated filter
coefficients may be employed in conjunction with a residual (e.g.,
as described above with respect to various video processing
operations) to generate the actual pixel values of a current frame
or picture.
[0090] In accordance with in accordance with various aspects, and
their equivalents, of the invention, such calculation of these
respective fractional sample positions intervening between actual
pixel values may be performed simultaneously within more than one
dimension. For example, such calculations may be performed
simultaneously in both the vertical and horizontal directions. For
example, considering 1/4 sub-pixel sampling with respect to the
four respective pixels A.sub.0,0, A.sub.1,0, A.sub.0,1, and
A.sub.1,1, then the corresponding to one the one fractional sample
positions of a.sub.0,0, b.sub.0,0, c.sub.0,0, d.sub.0,0, e.sub.0,0,
f.sub.0,0, g.sub.0,0, h.sub.0,0, i.sub.0,0, j.sub.0,0, k.sub.0,0,
n.sub.0,0, p.sub.0,0, q.sub.0,0, and r.sub.0,0, and also a.sub.0,1,
b.sub.0,1, c.sub.0,1, and also d.sub.1,0, h.sub.1,0, n.sub.1,0, may
all be calculated simultaneously and/or in parallel with one
another. As may be understood, such calculations that are performed
simultaneously and/or in parallel with one another are also
performed in both the vertical and horizontal directions. Again,
the calculation of these multiple respective filter coefficients in
multiple respective directions are performed simultaneously and/or
in parallel with one another.
[0091] FIG. 11 illustrates an embodiment 1100 of two-dimensional
(2-D) processing with respect to pixels of a frame or picture.
Generally speaking, instead of performing DCT-IF operations in only
the horizontal and the vertical direction at a given time (e.g.,
successively performing operations in one direction firstly and
then followed by performing operations in the other direction), a
diagonal approach may be performed which provides a significant
savings in terms of the number steps to be performed.
[0092] As can be seen, two-dimensional DCT processing is performed
with respect to this diagram. The use of 2-D motion compensation
filters may be employed for fractional-pel accuracy.
[0093] 2-D fractional-pel interpolation may be performed as
follows:
[0094] Considering the signal, x(t,s) as being a continuous
two-dimensional signal with respective sampled sequences as
follows:
[0095] x(t, s): continuous 2-D signal with sampled sequences
x.sub.d(n,m)=x(nT.sub.s+d.sub.s,mT.sub.t+d.sub.t), n=0, . . . ,N-1,
m=0, . . . ,M-1
[0096] where,
-T.sub.s<d.sub.s<T.sub.s,-T.sub.t<d.sub.t<T.sub.t
[0097] Full pixel: x.sub.0(n,m)=x(nT.sub.x,mT.sub.y), n=0, . . .
,N-1, m=0, . . . ,M-1
[0098] As such, there are 21 possible d:
[0099] (0, T.sub.t/4), (0, T.sub.t/2), (0, 3T.sub.t/4)
[0100] (T.sub.s/4, 0), (T.sub.s/4, T.sub.t/4), (T.sub.s/4,
T.sub.t/2), (T.sub.s/4, 3T.sub.t/4), (T.sub.s/4, T.sub.t)
[0101] (T.sub.s/2, 0), (T.sub.s/2, T.sub.t/4), (T.sub.s/2,
T.sub.t/2), (T.sub.s/2, 3T.sub.t/4), (T.sub.s/2, T.sub.t)
[0102] (3T.sub.s/4, 0), (3T.sub.s/4, T.sub.t/4), (3T.sub.s/4,
T.sub.t/2), (3T.sub.s/4, 3T.sub.t/4), (3T.sub.s/4, T.sub.t)
[0103] (T.sub.s, T.sub.t/4), (T.sub.s, T.sub.t/2), (T.sub.s,
3T.sub.t/4)
[0104] Basically we can have 21 filters:
[0105] f.sub.d(u,v) such that
x d ( n , m ) = u = A B v = C D f d ( u , v ) x 0 ( n + u , m + v )
##EQU00002##
where A<B, C<D are integers
[0106] Possible embodiment,
[0107] Use 2-D DCT to get the filter value
2 D - DCT : X d ( u , v ) = 2 .alpha. ( u ) .alpha. ( v ) N n = 0 N
- 1 m = 0 N - 1 x d ( n , m ) cos ( 2 n + 1 ) u .pi. 2 N cos ( 2 m
+ 1 ) v .pi. 2 N ##EQU00003## .alpha. ( j ) = 1 / 2 if j = 0 and N
, 1 otherwise ##EQU00003.2## 2 D - IDCT : x d ( n , m ) = 2 N u = 0
N - 1 v = 0 N - 1 .alpha. ( u ) .alpha. ( v ) X d ( u , v ) cos ( 2
n + 1 ) u .pi. 2 N cos ( 2 m + 1 ) v .pi. 2 N ##EQU00003.3##
[0108] Proposition 1
[0109] If a continuous signal x(t, s) is bandlimited
[0110] (its Fourie Transform X(.OMEGA.) is bandlimited in the
baseband (-(.pi./T), (.pi./T))
[0111] (i.e. X(.OMEGA.)=0 when |.OMEGA.|.gtoreq..pi./T), Then
X d ( u , v ) = 2 .alpha. ( u ) .alpha. ( v ) N n = 0 N - 1 m = 0 M
- 1 x 0 ( n , m ) cos ( 2 ( n + d s / T s ) + 1 ) u .pi. 2 N cos (
2 ( m + d t / T t ) + 1 ) v .pi. 2 N ##EQU00004## 2 D - IDCT : x d
( n , m ) = 2 N u = 0 N - 1 v = 0 M - 1 .alpha. ( u ) .alpha. ( v )
X d ( u , v ) cos ( 2 n + 1 ) u .pi. 2 N cos ( 2 m + 1 ) v .pi. 2 N
##EQU00004.2## X d ( u , v ) = 2 .alpha. ( u ) .alpha. ( v ) N n =
0 N - 1 m = 0 M - 1 x 0 ( n , m ) cos ( 2 ( n + d s / T s ) + 1 ) u
.pi. 2 N cos ( 2 ( m + d t / T t ) + 1 ) v .pi. 2 N ##EQU00004.3##
x d ( n , m ) = 2 N u = 0 N - 1 v = 0 M - 1 .alpha. ( u ) .alpha. (
v ) { 2 .alpha. ( u ) .alpha. ( v ) N p = 0 N - 1 q = 0 M - 1 x 0 (
p , q ) cos ( 2 ( p + d s / T s ) + 1 ) u .pi. 2 N cos ( 2 ( q + d
t / T t ) + 1 ) v .pi. 2 N } cos ( 2 n + 1 ) u .pi. 2 N cos ( 2 m +
1 ) v .pi. 2 N ##EQU00004.4##
[0112] The filter coefficients.
f d ( p , q ) = 4 N 2 u = 0 N - 1 v = 0 M - 1 [ .alpha. ( u )
.alpha. ( v ) ] 2 cos ( 2 ( p + d s / T s ) + 1 ) u .pi. 2 N cos (
2 ( q + d t / T t ) + 1 ) v .pi. 2 N cos ( 2 n + 1 ) u .pi. 2 N cos
( 2 m + 1 ) v .pi. 2 N ##EQU00005##
[0113] As can be seen with respect to the FIG. 11, as well as with
respect to the one possible embodiment of generating the
coefficients (e.g., filter coefficients) associated with an input
bitstream, such processing in operation is effectuated such that
more than one respective dimension is employed simultaneously
and/or in parallel. That is to say, with respect to an input
bitstream, at least with respect to a two-dimensional image,
processing of the input bitstream is made with respect to a first
dimension (e.g., vertical) and a second dimension (e.g.,
horizontal) simultaneously or in parallel thereby generating the
coefficient values. Moreover, as may also be seen, at least one
fractional-pel value in the first dimension and at least one
fractional-pel value in the second dimension are employed. Looking
at the particular example in which the fractional-pel distance
corresponds to one fourth shift in the vertical and horizontal
directions (e.g., d=(T.sub.s/4, T.sub.t/4)), it may be seen that at
least one fractional-pel value [corresponding to that
fractional-pel distance] in the vertical dimension is employed and
at least one fractional-pel value corresponding to that same
fractional-pel distance] in the horizontal dimension is employed.
It is noted that the use of such a fractional-pel distance
corresponding to a one fourth shift is exemplary, and generally
speaking, any desired fractional-pel distance may be employed
without departing from the scope and spirit of the invention (e.g.,
fractional-pel distance generally of being 1/N, where N is an
integer, such that d=(T.sub.s/N, T.sub.t/N)).
[0114] Moreover, it is noted that while such exemplary embodiments
are described with respect to a given fractional-pel distance
employed commonly within more than one dimension, it is noted that
different respective fractional-pel distances may be employed with
respect to the different dimensions. For example, a first
fractional-pel distance may be employed in a first dimension (e.g.,
vertical), a second fractional-pel distance may be employed in the
second dimension (e.g., horizontal), and so on (e.g.,
d=(T.sub.s/N.sub.1, T.sub.t/N.sub.2), where N.sub.1 and N.sub.2 are
different respective integers).
[0115] Also, while certain of the embodiments and/or diagrams
included herein are directed towards image or video signals
corresponding to two dimensions, it is noted that such operations
and/or processing may be generally extended to image or video
signals corresponding to more than two dimensions (e.g.,
three-dimensional image or video signals). For example, such
operations and/or processing may be generally extended to more than
two dimensions in accordance with generating such a function,
f.sub.d(p, q, r), for a three-dimensional signal, or such a
function, f.sub.d(p, q, r, s), for a four-dimensional signal, and
so on. For example, in an embodiment in which the same
fractional-pel distance would be employed in each respective
dimension of a three-dimensional signal, such a fractional-pel
distance may be shown as d=(T.sub.s/U, T.sub.t/U, T.sub.u/U)),
where U is an integer. Analogously, in an embodiment in which the
same fractional-pel distance would be employed in each respective
dimension of a four-dimensional signal, such a fractional-pel
distance may be shown as d=(T.sub.s/U, T.sub.t/U, T.sub.u/U,
T.sub.v/U)), where U is an integer, and so on.
[0116] Alternatively, if different respective fractional-pel
justices would be employed in each respective dimension of a
three-dimensional signal, such a fractional-pel distance may be
shown as d=(T.sub.s/U.sub.1, T.sub.t/U.sub.2, T.sub.u/U.sub.3)),
where each of U.sub.1, U.sub.2, and U.sub.3 are integers. Of
course, it is noted that any to respective dimensions may employ
the very same fractional-pel distance, such that either one of
U.sub.2 and U.sub.3 may be the same as U.sub.1. Analogously, if
different respective fractional-pel justices would be employed in
each respective dimension of a three-dimensional signal, such a
fractional-pel distance may be shown as d=(T.sub.s/U.sub.1,
T.sub.t/U.sub.2, T.sub.u/U.sub.3, T.sub.v/U.sub.4)), where each of
U.sub.1, U.sub.2, U.sub.3, and U.sub.4 are integers, and so on.
[0117] As may be understood, with respect to the generation of
coefficient values in accordance with more than one dimension
effectuated simultaneously and/or in parallel, such operations may
be extended towards signals having any of a number of multiple
dimensions. For example, with respect to generation of signals in
accordance with three dimensions, as described generally above,
such simultaneous and/or in parallel processing may be made with
respect to each of the respective three dimensions. Alternatively,
with respect to three-dimensional image or video signaling
corresponding to at least two respective cameras operating
simultaneously from different respective viewpoints, such motion
compensation processing may be applied towards the respective image
or video signals generated from those at least two respective
cameras. That is to say, while each respective camera may be
operative for generating a two-dimensional image or video signal,
such motion compensation processing as described herein with
respect to two-dimensional signals may be applied respectively to
the respective image or video signals generated from those of these
two respective cameras. In any event, with respect to any given
image or video signal, information related to more than one
respective dimension may be employed simultaneously and/or in
parallel in accordance with generating coefficient values.
[0118] Also, respect to the FIG. 11, as well as with respect other
embodiments and/or diagrams herein, it is noted that the manner by
which the function, f.sub.d(p, q) (or a higher order dimension
function), has been derived with respect to this exemplary
embodiment is but one manner by which such a function may be
generated for use in deriving filter coefficients for use in
accordance with motion compensation processing. Other such
processes, such as those not being effectuated in accordance with
discrete cosine transform (DCT), and it's respective inverse, may
alternatively be employed in other embodiments without departing
from the scope and spirit of the invention.
[0119] At a minimum, with respect to this exemplary embodiment
performed with respect to more than one dimension, it can be seen
that such filter coefficients may be generated with respect to each
of at least a first dimension and a second dimension simultaneously
and/or in parallel (e.g., simultaneously and/or in parallel with
respect to each of a vertical dimension and a horizontal dimension
as with respect to a two-dimensional image or video signal).
[0120] FIG. 12 illustrates an embodiment 1200 showing simultaneous
or in parallel generation of coefficients in multiple dimensions in
accordance with video processing operations. This diagram
pictorially illustrates an input bitstream undergoing simultaneous
or in parallel video processing operations in at least two
respective dimensions. While various embodiments are directed to
performing video processing in accordance with video decoding
operations (e.g., such as in accordance with motion compensation
operations and/or processing in a video decoding architecture), it
is noted that certain operations may also be employed in accordance
with performing video encoding operations.
[0121] As may be understood in accordance with in accordance with
various aspects, and their equivalents, of the invention,
coefficient values associated with a video signal are generated
with respect to each of two or more dimensions simultaneously
and/or in parallel. The generation of these respective coefficient
values is based on at least one respective fractional-pel value in
each of the respective dimensions (e.g., a first at least one
fractional-pel value in a first dimension, a second at least one
fractional-pel value in a second dimension, etc. such that
different respective fractional-pel values are employed in
different respective dimensions). Moreover, the same fractional-pel
distance may be employed within each of the respective dimensions
in certain embodiments. Alternatively, different respective
fractional-pel distance may be employed within each of the
respective dimensions in other embodiments (e.g., a first
fractional-pel distance may be employed in a first dimension (e.g.,
vertical), a second fractional-pel distance may be employed in the
second dimension (e.g., horizontal), and so on).
[0122] The coefficient values generated in accordance with
performing such video processing (e.g., including motion
compensation processing) may be used in accordance with any of a
number of additional video encoding processes, video decoding
processes, and/or operational steps by any of a number of
additional circuitries, modules, functional blocks, etc.
[0123] FIG. 13A and FIG. 13B illustrate various embodiments of
methods 1300 and 1301, respectively, for performing various video
processing operations.
[0124] Referring to the method 1300 of FIG. 13A, the method 1300
begins by receiving an input bitstream via an input, as shown in a
block 1310. The method 1300 continues by performing motion
compensation (e.g., such as may be performed using emotion
compensation module, such as in accordance with a video decoding
architecture) including performing fractional pixel position
interpolation of a first plurality of pixels of a first frame with
respect to a first dimension and a second dimension simultaneously
or in parallel thereby generating a plurality of coefficient values
for use in generating a second plurality of pixels of the second
frame, as shown in a block 1320. The method 1300 then operates by
employing the first frame and the second frame for generating
output video signal, as shown in a block 1330.
[0125] As may be understood in accordance with such motion
compensation operations associated in accordance with video
processing, a novel approaches presented herein by which such
respective coefficient values may be calculated simultaneously
and/or in parallel with one another in accordance with more than
one respective dimension. That is to say, in certain embodiments in
from certain perspectives, the calculation of such coefficient
values may be viewed as being made in both the vertical and
horizontal dimensions simultaneously.
[0126] Referring to method 1301 of FIG. 13B, the method 1301
operates by processing an input plurality of pixels, with respect
to a first dimension and a second dimension simultaneously and/or
in parallel thereby generating a plurality of coefficient values,
as shown in a block 1311. The operations associated with the block
1311 may be performed such that each of the plurality of
coefficient values is based on a respective at least one
fractional-pel value in the first dimension and a respective at
least one fractional-pel value in the second dimension. For
example, the same fractional-pel value may be used with respect to
both dimensions or different respective fractional-pel values may
be employed respectively within different respective
dimensions.
[0127] It is noted that such video processing operations may be
performed within any of a variety of different types of devices
including destination devices, middling devices, and/or generally
any communication device that is operative within one or more of a
satellite communication system, a wireless communication system, a
wired communication system, and a fiber-optic communication system.
With respect to the destination device, such a device may be viewed
generally as including an input for receiving an input bitstream or
a signal corresponding to the input bitstream from a source device
or a middling device via one or more communication networks. With
respect to the middling device, such a device may be viewed
generally as including an input for receiving an input bitstream or
a signal corresponding to the input bitstream from a source device
or another middling device via one or more communication networks,
as well as including an output for outputting an output bitstream
or a signal corresponding to an output bitstream or an output video
signal to at least one destination device via the one or more
communication networks or one or more additional communication
networks.
[0128] It is also noted that the various operations and functions
as described with respect to various methods herein may be
performed within a communication device, such as using a baseband
processing module and/or a processing module implemented therein
and/or other component(s) therein.
[0129] As may be used herein, the terms "substantially" and
"approximately" provides an industry-accepted tolerance for its
corresponding term and/or relativity between items. Such an
industry-accepted tolerance ranges from less than one percent to
fifty percent and corresponds to, but is not limited to, component
values, integrated circuit process variations, temperature
variations, rise and fall times, and/or thermal noise. Such
relativity between items ranges from a difference of a few percent
to magnitude differences. As may also be used herein, the term(s)
"operably coupled to", "coupled to", and/or "coupling" includes
direct coupling between items and/or indirect coupling between
items via an intervening item (e.g., an item includes, but is not
limited to, a component, an element, a circuit, and/or a module)
where, for indirect coupling, the intervening item does not modify
the information of a signal but may adjust its current level,
voltage level, and/or power level. As may further be used herein,
inferred coupling (i.e., where one element is coupled to another
element by inference) includes direct and indirect coupling between
two items in the same manner as "coupled to". As may even further
be used herein, the term "operable to" or "operably coupled to"
indicates that an item includes one or more of power connections,
input(s), output(s), etc., to perform, when activated, one or more
its corresponding functions and may further include inferred
coupling to one or more other items. As may still further be used
herein, the term "associated with", includes direct and/or indirect
coupling of separate items and/or one item being embedded within
another item. As may be used herein, the term "compares favorably",
indicates that a comparison between two or more items, signals,
etc., provides a desired relationship. For example, when the
desired relationship is that signal 1 has a greater magnitude than
signal 2, a favorable comparison may be achieved when the magnitude
of signal 1 is greater than that of signal 2 or when the magnitude
of signal 2 is less than that of signal 1.
[0130] As may also be used herein, the terms "processing module",
"processing circuit", and/or "processing unit" may be a single
processing device or a plurality of processing devices. Such a
processing device may be a microprocessor, micro-controller,
digital signal processor, microcomputer, central processing unit,
field programmable gate array, programmable logic device, state
machine, logic circuitry, analog circuitry, digital circuitry,
and/or any device that manipulates signals (analog and/or digital)
based on hard coding of the circuitry and/or operational
instructions. The processing module, module, processing circuit,
and/or processing unit may be, or further include, memory and/or an
integrated memory element, which may be a single memory device, a
plurality of memory devices, and/or embedded circuitry of another
processing module, module, processing circuit, and/or processing
unit. Such a memory device may be a read-only memory, random access
memory, volatile memory, non-volatile memory, static memory,
dynamic memory, flash memory, cache memory, and/or any device that
stores digital information. Note that if the processing module,
module, processing circuit, and/or processing unit includes more
than one processing device, the processing devices may be centrally
located (e.g., directly coupled together via a wired and/or
wireless bus structure) or may be distributedly located (e.g.,
cloud computing via indirect coupling via a local area network
and/or a wide area network). Further note that if the processing
module, module, processing circuit, and/or processing unit
implements one or more of its functions via a state machine, analog
circuitry, digital circuitry, and/or logic circuitry, the memory
and/or memory element storing the corresponding operational
instructions may be embedded within, or external to, the circuitry
comprising the state machine, analog circuitry, digital circuitry,
and/or logic circuitry. Still further note that, the memory element
may store, and the processing module, module, processing circuit,
and/or processing unit executes, hard coded and/or operational
instructions corresponding to at least some of the steps and/or
functions illustrated in one or more of the Figures. Such a memory
device or memory element can be included in an article of
manufacture.
[0131] The present invention has been described above with the aid
of method steps illustrating the performance of specified functions
and relationships thereof. The boundaries and sequence of these
functional building blocks and method steps have been arbitrarily
defined herein for convenience of description. Alternate boundaries
and sequences can be defined so long as the specified functions and
relationships are appropriately performed. Any such alternate
boundaries or sequences are thus within the scope and spirit of the
claimed invention. Further, the boundaries of these functional
building blocks have been arbitrarily defined for convenience of
description. Alternate boundaries could be defined as long as the
certain significant functions are appropriately performed.
Similarly, flow diagram blocks may also have been arbitrarily
defined herein to illustrate certain significant functionality. To
the extent used, the flow diagram block boundaries and sequence
could have been defined otherwise and still perform the certain
significant functionality. Such alternate definitions of both
functional building blocks and flow diagram blocks and sequences
are thus within the scope and spirit of the claimed invention. One
of average skill in the art will also recognize that the functional
building blocks, and other illustrative blocks, modules and
components herein, can be implemented as illustrated or by discrete
components, application specific integrated circuits, processors
executing appropriate software and the like or any combination
thereof.
[0132] The present invention may have also been described, at least
in part, in terms of one or more embodiments. An embodiment of the
present invention is used herein to illustrate the present
invention, an aspect thereof, a feature thereof, a concept thereof,
and/or an example thereof A physical embodiment of an apparatus, an
article of manufacture, a machine, and/or of a process that
embodies the present invention may include one or more of the
aspects, features, concepts, examples, etc. described with
reference to one or more of the embodiments discussed herein.
Further, from figure to figure, the embodiments may incorporate the
same or similarly named functions, steps, modules, etc. that may
use the same or different reference numbers and, as such, the
functions, steps, modules, etc. may be the same or similar
functions, steps, modules, etc. or different ones.
[0133] While such circuitries in the above described figure(s) may
including transistors, such as field effect transistors (FETs), as
one of ordinary skill in the art will appreciate, such transistors
may be implemented using any type of transistor structure
including, but not limited to, bipolar, metal oxide semiconductor
field effect transistors (MOSFET), N-well transistors, P-well
transistors, enhancement mode, depletion mode, and zero voltage
threshold (VT) transistors.
[0134] Unless specifically stated to the contra, signals to, from,
and/or between elements in a figure of any of the figures presented
herein may be analog or digital, continuous time or discrete time,
and single-ended or differential. For instance, if a signal path is
shown as a single-ended path, it also represents a differential
signal path. Similarly, if a signal path is shown as a differential
path, it also represents a single-ended signal path. While one or
more particular architectures are described herein, other
architectures can likewise be implemented that use one or more data
buses not expressly shown, direct connectivity between elements,
and/or indirect coupling between other elements as recognized by
one of average skill in the art.
[0135] The term "module" is used in the description of the various
embodiments of the present invention. A module includes a
processing module, a functional block, hardware, and/or software
stored on memory for performing one or more functions as may be
described herein. Note that, if the module is implemented via
hardware, the hardware may operate independently and/or in
conjunction software and/or firmware. As used herein, a module may
contain one or more sub-modules, each of which may be one or more
modules.
[0136] While particular combinations of various functions and
features of the present invention have been expressly described
herein, other combinations of these features and functions are
likewise possible. The present invention is not limited by the
particular examples disclosed herein and expressly incorporates
these other combinations.
* * * * *