U.S. patent application number 16/010833 was filed with the patent office on 2018-10-18 for video coding with degradation of residuals.
The applicant listed for this patent is GOOGLE LLC. Invention is credited to Shunyao Li, Peyman Milanfar, Debargha Mukherjee.
Application Number | 20180302643 16/010833 |
Document ID | / |
Family ID | 62599060 |
Filed Date | 2018-10-18 |
United States Patent
Application |
20180302643 |
Kind Code |
A1 |
Mukherjee; Debargha ; et
al. |
October 18, 2018 |
VIDEO CODING WITH DEGRADATION OF RESIDUALS
Abstract
A method for encoding a video signal using a computing device,
the video signal having a plurality of frames, each frame having a
plurality of blocks, and each block having a plurality of pixels.
The method includes generating a residual block from an original
block of a current frame and a prediction block, degrading the
residual block to decrease a bit-cost for encoding the residual
block, and encoding the residual block into an encoded residual
block.
Inventors: |
Mukherjee; Debargha;
(Cupertino, CA) ; Li; Shunyao; (Sunnyvale, CA)
; Milanfar; Peyman; (Menlo Park, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
GOOGLE LLC |
Mountain View |
CA |
US |
|
|
Family ID: |
62599060 |
Appl. No.: |
16/010833 |
Filed: |
June 18, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14969675 |
Dec 15, 2015 |
10009622 |
|
|
16010833 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 19/86 20141101;
H04N 19/132 20141101; H04N 19/59 20141101; H04N 19/52 20141101;
H04N 19/124 20141101; H04N 19/176 20141101; H04N 19/82 20141101;
H04N 19/146 20141101; H04N 19/61 20141101; H04N 19/117 20141101;
H04N 19/182 20141101; H04N 19/91 20141101; H04N 19/44 20141101;
H04N 19/107 20141101 |
International
Class: |
H04N 19/52 20060101
H04N019/52; H04N 19/182 20060101 H04N019/182; H04N 19/176 20060101
H04N019/176; H04N 19/107 20060101 H04N019/107; H04N 19/86 20060101
H04N019/86; H04N 19/91 20060101 H04N019/91; H04N 19/59 20060101
H04N019/59; H04N 19/44 20060101 H04N019/44; H04N 19/82 20060101
H04N019/82; H04N 19/124 20060101 H04N019/124; H04N 19/61 20060101
H04N019/61 |
Claims
1. A method for encoding a block of a video frame, the method
comprising: generating a prediction residual block for the block;
removing high-frequency information from the prediction residual
block to produce a degraded residual block; and encoding the
degraded residual block to produce an encoded block, wherein a
header of the encoded block includes data indicative of a
block-based post filter available for recovering at least some of
the high frequency information.
2. The method of claim 1, wherein the high-frequency information is
removed from the prediction residual block by steps of an encoder
that are different than steps performed by a transform stage of the
encoder and by a quantization stage of the encoder.
3. The method of claim 1, wherein removing the high-frequency
information from the prediction residual block to produce the
degraded residual block comprises: downscaling the prediction
residual block to produce a downscaled prediction residual block of
pixel values.
4. The method of claim 1, further comprising: recovering the
high-frequency information by applying the block-based post filter
to a reconstructed block produced using the encoded block.
5. The method of claim 4, further comprising: subsequent to
recovering the high-frequency information, producing a filtered
block using the high-frequency information; and using the filtered
block to generate a prediction residual for another block of the
video frame or for a block of another video frame.
6. The method of claim 4, further comprising: decoding the encoded
block to produce a decoded block; upscaling the decoded block to
produce an upscaled block; and generating the reconstructed block
using the upscaled block.
7. The method of claim 1, wherein the data indicative of the
block-based post filter includes an identifier corresponding to a
predefined filter.
8. The method of claim 1, wherein the data indicative of the
block-based post filter includes one or more filter parameters.
9. A method for encoding a block of a video frame, the method
comprising: selecting a block-based post filter for recovering high
frequency information removed from a prediction residual block; and
encoding data indicative of the block-based post filter to a header
of an encoded block produced using the prediction residual
block.
10. The method of claim 9, further comprising: removing the
high-frequency information from the prediction residual block to
produce a degraded residual block; and encoding the degraded
residual block to produce the encoded block.
11. The method of claim 10, wherein removing the high-frequency
information decreases a bit-cost for encoding the prediction
residual block.
12. The method of claim 9, further comprising: decoding the encoded
block to produce a decoded block; upscaling the decoded block to
produce an upscaled block; and generating a reconstructed block
using the upscaled block.
13. The method of claim 12, further comprising: recovering the
high-frequency information by applying the block-based post filter
to a reconstructed block.
14. The method of claim 13, further comprising: using the
high-frequency information recovered by applying the block-based
post filter for predicting another block of the video frame or for
a block of another video frame.
15. The method of claim 9, wherein the high-frequency information
is removed from the prediction residual block by steps of an
encoder that are different than steps performed by a transform
stage of the encoder and by a quantization stage of the
encoder.
16. A method for decoding an encoded block of an encoded video
frame, the method comprising: applying a block-based post filter to
a reconstructed block to recover high-frequency information removed
from a prediction residual, wherein the prediction residual is used
to produce the encoded block and the reconstructed block is
produced using the encoded block.
17. The method of claim 16, further comprising: storing a filtered
block produced using the high-frequency information recovered by
applying the block-based post filter.
18. The method of claim 16, further comprising: decoding the
encoded block to produce a decoded block; upscaling the decoded
block to produce an upscaled block; and generating the
reconstructed block using the upscaled block.
19. The method of claim 16, further comprising: decoding data
indicative of the block-based post filter from a header of the
encoded block.
20. The method of claim 19, wherein the data indicative of the
block-based post filter includes at least one of an identifier
corresponding to a predefined filter or one or more filter
parameters.
Description
CROSS-REFERENCE TO RELATED APPLICATION(S)
[0001] This disclosure is a continuation of U.S. patent application
Ser. No. 14/969,675, filed Dec. 15, 2015, the disclosure of which
is incorporated by reference herein in its entirety.
TECHNICAL FIELD
[0002] This disclosure relates to encoding and decoding visual
data, such as video stream data, for transmission or storage and
subsequent display.
BACKGROUND
[0003] Digital video streams typically represent video using a
sequence of frames or still images. Each frame can include a number
of blocks, which in turn may contain information describing the
value of color, brightness or other attributes for pixels. The
amount of data in a typical video stream is large, and transmission
and storage of video can use significant computing or
communications resources. Various approaches have been proposed to
reduce the amount of data in video streams, including compression
and other encoding techniques. In addition, video data can be
transmitted or stored at varying spatial resolutions in order to
save transmission or storage bandwidth.
[0004] Modern video codecs use block-based coding. Each frame is
divided into different sizes of blocks. Prediction methods exploit
spatial and temporal similarities between blocks to achieve high
compression ratios. These prediction methods include
inter-prediction methods that exploit temporal redundancies in the
data by utilizing information from other frames to generate a
prediction, and intra-prediction methods that exploit spatial
redundancies to generate a prediction using information only from
the current frame. To encode a block in a video frame, a prediction
for a current block is created by identifying a best matching block
from a reference frame and calculating a prediction residual by
subtracting the reference block from the current block. The
residual is then encoded and written to the bitstream. The decoder
decodes the bitstream and gets the residual, adds it to the
prediction to generate a reconstruction of the block. The
reconstruction is used for further blocks in current or future
frames as reference, which forms a closed-loop scheme.
[0005] For example, in certain video compression schemes, a video
frame is first divided into basic encoding units called super
blocks. The super blocks are further divided into rectangular or
square partitions. For 64.times.64 super blocks, the partition
sizes can range from 4.times.4 to 64.times.64. Prediction is then
performed at the partition level. For each partition, the
prediction residuals are generated, are transformed, and then the
transform coefficients are quantized and coded before they are
written to the bitstream.
SUMMARY
[0006] One aspect of the disclosed implementations is a method for
encoding a block of a video frame. The method comprises generating
a prediction residual block for the block. The method further
comprises removing high-frequency information from the prediction
residual block to produce a degraded residual block. The method
further comprises encoding the degraded residual block to produce
an encoded block. A header of the encoded block includes data
indicative of a block-based post filter available for recovering at
least some of the high frequency information.
[0007] Another aspect of the disclosed implementations is a method
for encoding a block of a video frame. The method comprises
selecting a block-based post filter for recovering high frequency
information removed from a prediction residual block. The method
further comprises encoding data indicative of the block-based post
filter to a header of an encoded block produced using the
prediction residual block.
[0008] Another aspect of the disclosed implementations is a method
for decoding an encoded block of an encoded video frame. The method
comprises applying a block-based post filter to a reconstructed
block to recover high-frequency information removed from a
prediction residual. The prediction residual is used to produce the
encoded block and the reconstructed block is produced using the
encoded block.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The description herein makes reference to the accompanying
drawings wherein like reference numerals refer to like parts
throughout the several views, and wherein:
[0010] FIG. 1 is a schematic of a video encoding and decoding
system;
[0011] FIG. 2 is a block diagram of an exemplary computing device
that can implement a transmitting station or a receiving
station;
[0012] FIG. 3 is a diagram of a typical video stream to be encoded
and subsequently decoded;
[0013] FIG. 4 is a block diagram of a video compression system in
accordance with an aspect of this disclosure;
[0014] FIG. 5 is a block diagram of a video decompression system in
accordance with another aspect of this disclosure;
[0015] FIG. 6 is a block diagram showing encoding applied to an
original block of a video signal according to a first example;
[0016] FIG. 7 is a block diagram showing decoding applied to an
encoded block of a video bitstream according to the first
example;
[0017] FIG. 8 is a flowchart showing an encoding process according
to the first example;
[0018] FIG. 9 is a flowchart showing a decoding process according
to the first example;
[0019] FIG. 10 is a block diagram showing encoding applied to an
original block of a video signal according to a second example;
[0020] FIG. 11 is a block diagram showing decoding applied to an
encoded block of a video bitstream according to the second
example;
[0021] FIG. 12 is a flowchart showing an encoding process according
to the second example; and
[0022] FIG. 13 is a flowchart showing a decoding process according
to the second example.
DETAILED DESCRIPTION
[0023] This disclosure is directed to video compression and
decompression techniques in which the residual blocks are degraded
during encoding, and block-based post filtering is utilized during
decoding to recover high frequency information that is lost as a
result of the degradation. For example, a linear filter or a
non-linear filter can be applied during final reconstruction. The
newly filtered block would then be used as the final reconstruction
for future reference in the coding loop.
[0024] In some implementations, the post filter is applied directly
to the reconstruction. In other implementations, the post filter is
not utilized. Thus, information can be included in the bitstream
that indicates whether or not the filter is to be used. In some
implementations, the encoder may choose to use or not use the post
filter based on rate distortion cost. In addition, the encoder may
encode information into the bitstream regarding the filter to be
used. In some implementations, this information identifies a
specific filter index from a family of post filters. In other
implementations, this information includes parameters for a
parameterized family of filters. In some implementations, this
information includes the actual filter taps at an appropriate
precision.
[0025] FIG. 1 is a schematic of a video encoding and decoding
system 100 in which the systems and methods described herein can be
implemented. An exemplary transmitting station 112 can be, for
example, a computer having an internal configuration of hardware
such as that described in FIG. 2. However, other suitable
implementations of the transmitting station 112 are possible. For
example, the processing of transmitting station 112 can be
distributed among multiple devices.
[0026] A network 128 can connect the transmitting station 112 and a
receiving station 130 for encoding and decoding of a video stream.
Specifically, the video stream can be encoded in transmitting
station 112 and the encoded video stream can be decoded in
receiving station 130. Network 128 can be, for example, the
Internet. Network 128 can also be a local area network (LAN), wide
area network (WAN), virtual private network (VPN), cellular
telephone network or any other means of transferring the video
stream from transmitting station 112 to, in this example, receiving
station 130.
[0027] Receiving station 130, in one example, can be a computer
having an internal configuration of hardware such as that described
in FIG. 2. However, other suitable implementations of receiving
station 130 are possible. For example, the processing of receiving
station 130 can be distributed among multiple devices.
[0028] Other implementations of video encoding and decoding system
100 are possible. For example, an implementation can omit network
128. In another implementation, a video stream can be encoded and
then stored for transmission at a later time to receiving station
130 or any other device having memory. In one implementation, the
receiving station 130 receives (e.g., via network 128, a computer
bus, and/or some communication pathway) the encoded video stream
and stores the video stream for later decoding. In an exemplary
implementation, a real-time transport protocol (RTP) is used for
transmission of the encoded video over network 128. In another
implementation, a transport protocol other than RTP may be used,
e.g., an HTTP-based video streaming protocol.
[0029] As will be explained further herein, the transmitting
station 112 and the receiving station 130 are examples of devices
that can be included in the video encoding and decoding system 100.
Additional devices can be included, such as a server that relays
transmissions from the transmitting station 112 to the receiving
station 130.
[0030] FIG. 2 is a block diagram of an exemplary computing device
200 that can implement a transmitting station or a receiving
station. For example, computing device 200 can implement one or
both of transmitting station 112 and receiving station 130 of FIG.
1. Computing device 200 can be in the form of a computing system
including multiple computing devices, or in the form of a single
computing device, for example, a mobile phone, a tablet computer, a
laptop computer, a notebook computer, a desktop computer, and the
like.
[0031] A CPU 224 in computing device 200 can be a conventional
central processing unit. Alternatively, CPU 224 can be any other
type of device, or multiple devices, capable of manipulating or
processing information now-existing or hereafter developed.
Although the disclosed implementations can be practiced with a
single processor as shown, e.g., CPU 224, advantages in speed and
efficiency can be achieved using more than one processor.
[0032] A memory 226 in computing device 200 can be a read only
memory (ROM) device or a random access memory (RAM) device in an
implementation. Any other suitable type of storage device can be
used as the memory 226. Memory 226 can include code and data 227
that is accessed by CPU 224 using a bus 230. Memory 226 can further
include an operating system 232 and application programs 234, the
application programs 234 including at least one program that
permits CPU 224 to perform the methods described here. As shown,
for example, application programs 234 can include applications 1
through N, which further include an application that performs one
or more of the methods described herein. Computing device 200 can
also include a secondary storage 236 that can be, for example, a
memory card used with a mobile computing device 200. Because the
video communication sessions may contain a significant amount of
information, they can be stored in whole or in part in secondary
storage 236 and loaded into memory 226 as needed for
processing.
[0033] Computing device 200 can also include one or more output
devices, such as a display 228. Display 228 may be, in one example,
a touch sensitive display that combines a display with a touch
sensitive element that is operable to sense touch inputs. Display
228 can be coupled to CPU 224 via bus 230. Other output devices
that permit a user to program or otherwise use computing device 200
can be provided in addition to or as an alternative to display 228.
When the output device is or includes a display, the display can be
implemented in various ways, including by a liquid crystal display
(LCD), a cathode-ray tube (CRT) or light emitting diode (LED)
display, such as an OLED display.
[0034] Computing device 200 can also include or be in communication
with an image-sensing device 238, for example a camera, or any
other image-sensing device 238 now existing or hereafter developed
that can sense an image such as the image of a user operating
computing device 200. Image-sensing device 238 can be positioned
such that it is directed toward the user operating computing device
200. In an example, the position and optical axis of image-sensing
device 238 can be configured such that the field of vision includes
an area that is directly adjacent to display 228 and from which
display 228 is visible.
[0035] Computing device 200 can also include or be in communication
with a sound-sensing device 240, for example a microphone, or any
other sound-sensing device now existing or hereafter developed that
can sense sounds near computing device 200. Sound-sensing device
240 can be positioned such that it is directed toward the user
operating computing device 200 and can be configured to receive
sounds, for example, speech or other utterances, made by the user
while the user operates computing device 200.
[0036] Although FIG. 2 depicts CPU 224 and memory 226 of computing
device 200 as being integrated into a single unit, other
configurations can be utilized. The operations of CPU 224 can be
distributed across multiple machines (each machine having one or
more of processors) that can be coupled directly or across a local
area or other network. Memory 226 can be distributed across
multiple machines such as a network-based memory or memory in
multiple machines performing the operations of computing device
200. Although depicted here as a single bus, bus 230 of computing
device 200 can be composed of multiple buses. Further, secondary
storage 236 can be directly coupled to the other components of
computing device 200 or can be accessed via a network and can
comprise a single integrated unit such as a memory card or multiple
units such as multiple memory cards. Computing device 200 can thus
be implemented in a wide variety of configurations.
[0037] FIG. 3 is a diagram of an example of a video 350 to be
encoded and subsequently decoded. Video 350 includes a video
sequence 352. At the next level, video sequence 352 includes a
number of adjacent frames 354. While three frames are depicted as
adjacent frames 354, video sequence 352 can include any number of
adjacent frames 354. Adjacent frames 354 can then be further
subdivided into individual frames, e.g., a single frame 356. At the
next level, single frame 356 can be divided into a series of blocks
358, which can contain data corresponding to, for example,
16.times.16 pixels in frame 356. The blocks can also be arranged in
planes of data. For example, a corresponding block in each plane
can respectively contain luminance and chrominance data for the
pixels of the block. Blocks 58 can also be of any other suitable
size such as 16.times.8 pixel groups or 8.times.16 pixel groups and
can be further subdivided into smaller blocks depending on the
application. Unless otherwise noted, the terms block and macroblock
are used interchangeably herein.
[0038] FIG. 4 is a block diagram of an encoder 470 in accordance
with an aspect of this disclosure. Encoder 470 can be implemented,
as described above, in transmitting station 112 such as by
providing a computer software program stored in memory, for
example, memory 226. The computer software program can include
machine instructions that, when executed by a processor such as CPU
224, cause transmitting station 112 to encode video data in the
manner described in FIG. 4. Encoder 470 can also be implemented as
specialized hardware included, for example, in transmitting station
112. Encoder 470 has the following stages to perform the various
functions in a forward path (shown by the solid connection lines)
to produce an encoded or compressed bitstream 488 using input video
350: an intra/inter prediction stage 472, a transform stage 474, a
quantization stage 476, and an entropy encoding stage 478. Encoder
470 may also include a reconstruction path (shown by the dotted
connection lines) to reconstruct a frame for encoding of future
blocks. In FIG. 4, encoder 470 has the following stages to perform
the various functions in a reconstruction path: a dequantization
stage 480, an inverse transform stage 482, a reconstruction stage
484, and a loop filtering stage 486. Other structural variations of
encoder 470 can be used to encode video 350.
[0039] When video 350 is presented for encoding, each frame 356
within the video 350 can be processed in units of blocks 358. At
the intra/inter prediction stage 472, each block can be encoded
using intra-frame prediction (prediction using blocks within a
single frame) or inter-frame prediction (prediction using blocks
from a different frame). In any case, a prediction block can be
formed. In the case of intra-prediction, a prediction block can be
formed from samples in the current frame that have been previously
encoded and reconstructed. In the case of inter-prediction, a
prediction block can be formed from samples in one or more
previously constructed reference frames.
[0040] Next, still referring to FIG. 4, the prediction block can be
subtracted from the current block at intra/inter prediction stage
472 to produce a residual block (also called a residual). Transform
stage 474 transforms the residual into transform coefficients in,
for example, the frequency domain. Examples of block-based
transforms include the Karhunen-Loeve Transform (KLT), the Discrete
Cosine Transform (DCT), and the Singular Value Decomposition
Transform (SVD). In one example, the DCT transforms the block into
the frequency domain. In the case of DCT, the transform coefficient
values are based on spatial frequency, with the lowest frequency
(DC) coefficient at the top-left of the matrix and the highest
frequency coefficient at the bottom-right of the matrix.
[0041] Quantization stage 476 converts the transform coefficients
into discrete quantum values, which are referred to as quantized
transform coefficients, using a quantizer value or a quantization
level. The quantized transform coefficients are then entropy
encoded by entropy encoding stage 478. The entropy-encoded
coefficients, together with other information used to decode the
block, which may include for example the type of prediction used,
motion vectors and quantizer value, are then output to compressed
bitstream 488. Compressed bitstream 488 can be formatted using
various techniques, such as variable length coding (VLC) or
arithmetic coding. Compressed bitstream 488 can also be referred to
as an encoded video stream and the terms are used interchangeably
herein.
[0042] The reconstruction path in FIG. 4 (shown by the dotted
connection lines) can be used to ensure that both encoder 470 and a
decoder 500 (described below) use the same reference frames to
decode compressed bitstream 488. The reconstruction path performs
functions that are similar to functions that take place during the
decoding process that are discussed in more detail below, including
dequantizing the quantized transform coefficients at dequantization
stage 480 and inverse transforming the dequantized transform
coefficients at inverse transform stage 482 to produce a derivative
residual block (also called a derivative residual). At
reconstruction stage 484, the prediction block that was predicted
at the intra/inter prediction stage 472 can be added to the
derivative residual to create a reconstructed block. Loop filtering
stage 486 can be applied to the reconstructed block to reduce
distortion such as blocking artifacts.
[0043] Other variations of encoder 470 can be used to encode
compressed bitstream 488. For example, a non-transform based
encoder 470 can quantize the residual signal directly without
transform stage 474. In another implementation, an encoder 470 can
have quantization stage 476 and dequantization stage 480 combined
into a single stage.
[0044] FIG. 5 is a block diagram of a decoder 500 in accordance
with an implementation. Decoder 500 can be implemented in receiving
station 130, for example, by providing a computer software program
stored in memory 226. The computer software program can include
machine instructions that, when executed by a processor such as CPU
224, cause receiving station 130 to decode video data in the manner
described in FIG. 5. Decoder 500 can also be implemented in
hardware included, for example, in transmitting station 112 or
receiving station 130.
[0045] Decoder 500, similar to the reconstruction path of encoder
470 discussed above, includes in one example the following stages
to perform various functions to produce an output video stream 516
from compressed bitstream 488: an entropy decoding stage 502, a
dequantization stage 504, an inverse transform stage 506, an
intra/inter prediction stage 508, a reconstruction stage 510, a
filtering stage 512, which can include loop filtering and/or
deblocking and a frame buffering stage 514. Other structural
variations of decoder 500 can be used to decode compressed
bitstream 488.
[0046] When compressed bitstream 488 is presented for decoding, the
data elements within compressed bitstream 488 can be decoded by
entropy decoding stage 502 (using, for example, arithmetic coding)
to produce a set of quantized transform coefficients.
Dequantization stage 504 dequantizes the quantized transform
coefficients, and inverse transform stage 506 inverse transforms
the dequantized transform coefficients to produce a derivative
residual that can be identical to that created by inverse transform
stage 482 in encoder 470. Using header information decoded from
compressed bitstream 488 such as modes and motion vectors, decoder
500 can use intra/inter prediction stage 508 to create the same
prediction block as was created in encoder 470, e.g., at
intra/inter prediction stage 472. At reconstruction stage 510, the
prediction block can be added to the derivative residual to create
a reconstructed block. Filtering stage 512 can be applied to the
reconstructed block to reduce blocking artifacts. Information can
then be held in a frame buffer at frame buffering stage 514 for
subsequent use in decoding or output. A post-processing stage can
be applied to the reconstructed block to further refine the image.
The result of the process performed by the decoder 500 is output as
output video stream 516. Output video stream 516 can also be
referred to as a decoded video stream and the terms are used
interchangeably herein.
[0047] Other variations of decoder 500 can be used to decode
compressed bitstream 488. For example, decoder 500 can produce
output video stream 516 without post-processing.
[0048] FIG. 6 is a block diagram showing video encoding 600 that
incorporates degradation of residual blocks and block-based post
filtering according to a first example, applied to an original
block 610 from a video signal having a plurality of frames, each
frame having a plurality of blocks, and each block having a
plurality of pixels. Video encoding 600 can be implemented, for
example, in the encoder 470, and the description of the encoder 470
is applicable except as described herein.
[0049] A prediction block 620 is identified for use in generating a
residual block 630. The prediction block 620 can be a block from a
different frame than the current frame if an inter-prediction mode
is used, or can be a block from the current frame if an
intra-prediction mode is used. The prediction block 620 is selected
in a conventional manner according to the prediction mode used,
such that the prediction block 620 has a high degree of
correspondence to the original block 610. In order to generate the
residual block 630, the prediction block 620 is subtracted from the
original block 610, and the remaining information is utilized as
the residual block 630.
[0050] In order to reduce the bit cost for encoding the residual
block 630, the residual block 630 is intentionally degraded to
generate a degraded residual block 640. Degrading the residual
block 630 can be performed in any manner that reduces the quality
and the bit cost required to transmit the degraded residual block
640. One example of a degradation that can be applied to the
residual block 630 is modification of the step size used for
quantization of the residual block 630. For example, quantization
can be performed during encoding of the residual in the manner
described with regard to the quantization stage 476 of the encoder
470, but using a larger quantization step size than otherwise would
be used. This results in a decrease in the bit cost to encode the
residual. Thus, in an implementation where a nominal quantization
step size is utilized for the current frame, degrading the residual
can include selecting a larger quantization step size for the
current block, as compared to the normal quantization step size. As
another example, the larger quantization step size can be selected
by setting the quantization step size can be larger than that used
for other coding modes that are applied to the current frame.
[0051] The degraded residual block 640 is encoded to define an
encoded residual block 650. Encoding can be performed as described
with respect to the video encoder 470, including transform stage
474, quantization stage 476, and entropy encoding in stage 478. The
encoded residual block 650 can be stored or transmitted. In some
implementations, filter information is stored or transmitted along
with the encoded residual block 160. The filter information can
indicate whether a filter is to be applied during decoding, can
include information such as an identifier that corresponds to a
predefined filter to be used for filtering, and/or can include a
plurality of filter parameters that can be utilized during
decoding.
[0052] Subsequent to generation of the encoded residual block 650,
the information from the encoded residual block 650 is recovered
using a process analogous to that which will be applied at the
decoder side in order to generate a prediction block that can be
used for encoding of subsequent blocks in the current frame or a
different frame. Thus, the encoded residual block 650 is decoded to
generate a decoded residual block 660, such as in the manner
described with respect to empty decoding stage 502, de-quantization
stage 504, and inverse transform stage 506 of the decoder 500. The
decoded residual block 660 is then added to the prediction block
620 that was previously utilized to generate the residual block 630
from the original block 610. The result of adding the decoded
residual block 660 to the prediction block 620 is a reconstructed
block 670.
[0053] In order to recover some of the high frequency information
that was lost during degradation of the residual block 630 to
define the degraded residual block 640, a block-based post-filter
is applied to the reconstructed block 670. In some implementations,
the filter is a block-based loop filter. The block-based loop
filter that is utilized for filtering the reconstructed block 670
can be designed in various ways. In one implementation, the
block-based loop filter is a Weiner filter, which minimizes the
mean square area between the reconstructed block 670 and the
original block 610. For a given filter window size, the filter can
be designed as a two-dimensional Weiner filter using linear
estimation theory. The two-dimensional Weiner filter is then
decomposed into two one-dimensional filters. The two-dimensional
filter can be decomposed into two one-dimensional filters using
decomposition techniques such as singular value decomposition and
iterative optimization.
[0054] The block-based loop filters can be generated in advance of
encoding or can be generated during the encoding process.
Predefined filters can be defined in advance of encoding by
training a plurality of filters using a variety of sample video
input filters. The trained filters are then clustered, and the
filter at the center of each cluster can be utilized to form a
filter family. The filter family is then made available at both the
end coder and the decoder. During encoding, a particular family can
be selected. As one example, the filter can be selected by
comparing the rate distortion performance for a plurality of the
filters from the filter family as applied to at least a portion of
the video signals being encoded. The selected filter can then be
applied to other portions of the video signal. Selection of the
filter can be made at any level of granularity. Thus, the same
filter could be selected for a series of frames, a single frame, a
macroblock, or the filter could be selected on a block-by-block
basis. In another implementation, the filter to be used for
filtering the reconstructed block 670 is designed during encoding,
with the result of filtered selection being a plurality of filter
parameters that can be transmitted to the decoded for use in
decoding the video bit stream.
[0055] As previously indicated, information identifying the filter
selected for filtering the reconstructed block 670 can be stored in
association with the video bit stream. In one implementation,
information identifying the filter to be utilized can be encoded
within the block header for each block. In implementations where a
predefined filter is used, information identifying the predefined
filter is transmitted or stored with the video bit stream. In
implementations where a filter is designed during encoding, the
parameters describing the filter can be transmitted or stored with
the video bit stream.
[0056] The result of filtering the reconstructed block 670
utilizing the block-based post filter is the filtered reconstructed
block 680. The filtered reconstructed block can be stored at the
encoder for use in subsequent prediction operations.
[0057] FIG. 7 is a block diagram showing video decoding 700 that
incorporates recovery of information lost as a result of
intentional degradation of residual blocks by using block-based
post filtering. Video decoding 700 is applied to an encoded
residual block 710 from a video bit stream having a plurality of
frames, each frame having a plurality of blocks, and each block
having a plurality of pixels. The encoded residual block 710 can be
formed in the manner described with respect to video encoding
600.
[0058] The encoded residual block 710 is decoded into a decoded
residual block 720. Decoding of the encoded residual block 720.
Decoding of the encoded residual block 710 can be performed in the
manner described with respect to the entropy decoding stage 502,
the dequantization stage 504, and the inverse transform stage 506
of the decoder 500. Decoding the encoded residual block 710 can
include decoding filter information from the video bitstream
regarding the block-based post filter for the decoded residual
block 720.
[0059] The decoded residual block 720 is combined with a prediction
block 730 by adding the decoded residual block 720 to the
prediction block 730. The result is a reconstructed block 740. The
reconstructed block 740 is of reduced quality relative to the
original block 610 owing to degradation of a residual block 630
when generating the degraded residual block 640 during video
encoding 600. In order to restore some of the high frequency
information that was lost during degradation, a block-based filter
is applied to the reconstructed block 740, which results in a
filtered reconstructed block 750. The filter applied to generate
the filtered reconstructed block 750 can be identified from
information in the video bit stream, such as filter information
regarding the block-based post filter to be utilized in filtering
the reconstructed block 740. As previously discussed, the filter
information can be received from the video bit stream, such as by
decoding the filter information from the header of the encoded
residual block 710. The filter information can include, for
example, an identifier that corresponds to a predefined filter or a
plurality of filter parameters that define the filter.
[0060] The filtered reconstructed block 750 can be output as a
video signal. For example, the filtered reconstructed block 750
could be output for display. The filtered reconstructed block is
also stored for use in decoding of other blocks by storing the
filtered reconstructed block 750 as the prediction block 730.
[0061] FIG. 8 shows an example of an encoding process 800. The
encoding process 800 can be implemented, for example, as a software
program that is executed by computing devices such as the
transmitting station 112 or the receiving station 130. The software
program can include machine-readable instructions that are stored
in a memory such as memory 226 that, when executed by a processor
such as CPU 224, cause the computing device to perform the encoding
process 800. The encoding process 800 can also be implemented using
hardware. As explained above, some computing devices may have
multiple memories and multiple processors, and the steps of the
encoding process 800 may in such cases be distributed using
different processors and memories. Use of the terms "processor" and
"memory" in the singular encompasses computing devices that have
only one processor or one memory as well as devices having multiple
processors or memories that may each be used in the performance of
some but not necessarily all of the recited steps.
[0062] Operation 810 includes generating the residual block 630
from the original block 610 and the prediction block 620. This can
be performed in a conventional manner as previously described. This
operation can further include obtaining the original block 610. The
original block 610 can be obtained in any manner such as by
receiving it in a video signal or accessing it from a storage
device. This operation can also include identifying the prediction
block 620 that will be used with the original block 610 to define
the residual block 630. This can be performed using any of a number
of well-known algorithms that search for matching reference blocks
in the current frame or in other frames of the video signal. Thus,
the prediction block 620 can be identified by comparing the
contents of the original block 610 with the contents of a plurality
of blocks from the video signal and choosing the best matching
block as the prediction block 620.
[0063] Operation 820 includes degrading the residual block 630,
which can be performed in the manner described with respect to the
degraded residual block 640. Subsequent to degrading the degraded
residual block 640, it is then encoded at operation 830.
[0064] Operation 840 includes selecting a block-based post filter
that will be utilized to recover some of the high frequency
information that was removed from the residual block 630 in order
to reduce the bit cost for encoding it to define the degraded
residual block 640. This information can be encoded with the
encoded residual block 650, such as by placing it in the header of
the encoded residual block 650.
[0065] Subsequent to encoding the encoded residual block 650 and
the filter information, the remainder of the process 800 is
directed to decoding and reconstruction of the encoded residual
block such that it can be used as a prediction block in further
encoding operations. In operation 850, the encoded residual block
650 is decoded. The result of operation 850 is the decoded residual
block 660. At operation 860, the decoded residual block 660 is
combined with the prediction block 620 that was previously utilized
at operation 810 to generate the residual block 630. The result of
this operation is the reconstructed block 670. In operation 870,
the block-based post filter is applied to the reconstructed block
670 to recover some of the high frequency information that was
previously discarded in the process of generating the degraded
residual block 640 at operation 820. Subsequent to filtering, the
filtered reconstructed block 680 is stored at operation 880 for
subsequent use as the prediction block 620 in a subsequent encoding
operation for a different block.
[0066] FIG. 9 shows an example of a decoding process 900. The
decoding process 900 can be implemented, for example, as a software
program that is executed by computing devices such as the
transmitting station 112 or the receiving station 130. The software
program can include machine-readable instructions that are stored
in a memory such as memory 226 that, when executed by a processor
such as CPU 224, cause the computing device to perform the decoding
process 900. The decoding process 900 can also be implemented using
hardware. As explained above, some computing devices may have
multiple memories and multiple processors, and the steps of the
decoding process 900 may in such cases be distributed using
different processors and memories. Use of the terms "processor" and
"memory" in the singular encompasses computing devices that have
only one processor or one memory as well as devices having multiple
processors or memories that may each be used in the performance of
some but not necessarily all of the recited steps.
[0067] Operation 910 includes decoding the encoded residual block
710 into the decoded residual block 720 as previously described. In
operation 920, filter information is decoded from the video bit
stream, such as from the header of the encoded residual block 710.
In operation 930, the reconstructed block 740 is generated by
adding the decoded residual block 720 to the prediction block 730.
In operation 940, a block-based post filter is applied to the
reconstructed block 740 to recover information lost during
degradation. The filtered reconstructed block 750 can then be
output for display and can also be stored at operation 950, as the
prediction block 730 for use in a subsequent decoding
operation.
[0068] FIG. 10 is a block diagram showing video encoding 1000 that
incorporates downscaling of residual blocks and block-based post
filtering according to a first example, applied to an original
block 1010 from a video signal having a plurality of frames, each
frame having a plurality of blocks, and each block having a
plurality of pixels. Video encoding 1000 can be implemented, for
example, in the encoder 470, and the description of the encoder 470
is applicable except as described herein. The original block 1010
has a first resolution, which can be, but is not necessarily, a
maximum resolution for the video signal.
[0069] A prediction block 1020 is identified for use in generating
a residual block 1030. The resolutions of the prediction block 1020
can be the same as the resolution of the original block 1010. The
prediction block 1020 can be a block from a different frame than
the current frame if an inter-prediction mode is used, or can be a
block from the current frame if an intra-prediction mode is used.
The prediction block 1020 is selected in a conventional manner
according to the prediction mode used, such that the prediction
block 1020 has a high degree of correspondence to the original
block 1010. In order to generate the residual block 1030, the
prediction block 1020 is subtracted from the original block 1010,
and the remaining information is utilized as the residual block
1030.
[0070] In order to reduce the bit cost for encoding the residual
block 1030, the residual block 1030 is downscaled to generate a
downscaled residual block 1040. Downscaling the residual block 1030
can be performed in any manner that reduces the resolution of the
residual block 1030. As an example, one common method of
downscaling takes the average of multiple adjacent pixel values and
sets a corresponding value in the downscaled image to the average
value. Suitable downscaling operations may reduce the resolution of
the encoded residual block 1030 to a second resolution that is, as
examples, half or one-quarter the first resolution at which the
residual block 1030 was calculated. Any other suitable magnitude of
downscaling can be utilized. In some implementations, the
downscaling is performed in the context of a coding mode that
applies a predetermined magnitude of downscaling to the residual
block 1030 in order to generate the downscaled residual block 1040.
In other implementations, the magnitude of the downscaling is
selected based on the content of the video signal, the plurality of
frames, the current frame, or the block. This selection can be
made, for example, by comparing the rate-distortion performance of
multiple alternatives.
[0071] The downscaled residual block 1040 is encoded to define an
encoded residual block 1050. Encoding can be performed as described
with respect to the encoder 470, including transform stage 474,
quantization stage 476, and entropy encoding in stage 478. The
encoded residual block 1050 can be stored or transmitted. In some
implementations, filter information is stored or transmitted along
with the encoded residual block 1050. The filter information can
indicate whether a filter is to be applied to the during decoding,
can include information such as an identifier that corresponds to a
predefined filter to be used for filtering during decoding, and/or
can include a plurality of filter parameters that can be utilized
during decoding.
[0072] Subsequent to generation of the encoded residual block 1050,
the information from the encoded residual block 1050 is recovered
using a process analogous to that which will be applied at the
decoder side in order to generate a prediction block that can be
used for encoding of subsequent blocks in the current frame or a
different frame. Thus, the encoded residual block 1050 is decoded
to generate a decoded residual block 1060, such as in the manner
described with respect to entropy decoding stage 502,
de-quantization stage 504, and inverse transform stage 506 of the
decoder 500. The decoded residual block 1060 is at the second
resolution, which is the same resolution as the downscaled residual
block 1050 prior to encoding. The decoded residual block 1060 is
then upscaled to define an upscaled decoded residual block 1065.
Upscaling can be performed by any one of numerous well known
methods.
[0073] The upscaled decoded residual block 1065 is then added to
the prediction block 1020 that was previously utilized to generate
the residual block 1030 from the original block 1010. The result of
adding the upscaled decoded residual block 1065 to the prediction
block 1020 is a reconstructed block 1070.
[0074] In order to recover some of the high frequency information
that was lost as a result of downscaling and subsequently upscaling
the residual block 1030, a block-based post-filter is applied to
the reconstructed block 1070. In some implementations, the filter
is a block-based loop filter. The block-based loop filter that is
utilized for filtering the reconstructed block 1070 can be designed
in various ways. In one implementation, the block-based loop filter
is a Weiner filter, which minimizes the mean square area between
the reconstructed block 1070 and the original block 1010. For a
given filter window size, the filter can be designed as a
two-dimensional Weiner filter using linear estimation theory. The
two-dimensional Weiner filter is then decomposed into two
one-dimensional filters. The two-dimensional filter can be
decomposed into two one-dimensional filters using decomposition
techniques such as singular value decomposition and iterative
optimization.
[0075] The block-based loop filters can be generated in advance of
encoding or can be generated during the encoding process.
Predefined filters can be defined in advance of encoding by
training a plurality of filters using a variety of sample video
input filters. The trained filters are then clustered, and the
filter at the center of each cluster can be utilized to form a
filter family. The filter family is then made available at both the
end coder and the decoder. During encoding, a particular family can
be selected. As one example, the filter can be selected by
comparing the rate distortion performance for a plurality of the
filters from the filter family as applied to at least a portion of
the video signals being encoded. The selected filter can then be
applied to other portions of the video signal. Selection of the
filter can be made at any level of granularity. Thus, the same
filter could be selected for a series of frames, a single frame, a
macroblock, or the filter could be selected on a block-by-block
basis. In another implementation, the filter to be used for
filtering the reconstructed block 1070 is designed during encoding,
with the result of filtered selection being a plurality of filter
parameters that can be transmitted to the decoded for use in
decoding the video bit stream.
[0076] As previously indicated, information identifying the filter
selected for filtering the reconstructed block 1070 can be stored
in association with the video bit stream. In one implementation,
information identifying the filter to be utilized can be encoded
within the block header for each block. In implementations where a
predefined filter is used, information identifying the predefined
filter is transmitted or stored with the video bit stream. In
implementations where a filter is designed during encoding, the
parameters describing the filter can be transmitted or stored with
the video bit stream.
[0077] The result of filtering the reconstructed block 1070
utilizing the block-based post filter is the filtered reconstructed
block 1080. The filtered reconstructed block can be stored at the
encoder for use in subsequent prediction operations.
[0078] FIG. 11 is a block diagram showing video decoding 1100 that
incorporates upscaling of residual blocks and block-based post
filtering. Video decoding 1100 is applied to an encoded residual
block 1110 from a video bit stream having a plurality of frames,
each frame having a plurality of blocks, and each block having a
plurality of pixels. The encoded residual block 1110 can be
generated in the manner described with respect to video encoding
1000.
[0079] The encoded residual block 1110 is decoded into a decoded
residual block 1120. Decoding of the encoded residual block 1120.
Decoding of the encoded residual block 1110 can be performed in the
manner described with respect to the entropy decoding stage 502,
the dequantization stage 504, and the inverse transform stage 506
of the decoder 500. The encoded residual block 1110 is upscaled in
order to generate an upscaled decoded residual block 1125. The
decoded residual block 1110 has a first resolution and the upscaled
decoded residual block 1125 has a second resolution that is greater
than the first resolution. As examples, the second resolution can
be double or quadruple the first resolution.
[0080] The upscaled decoded residual block 1125 is combined with a
prediction block 1130 by adding the upscaled decoded residual block
1125 to the prediction block 1130. The result is a reconstructed
block 1140. The reconstructed block 1140 is of reduced quality
relative to the original block 1010 owing to loss of information in
from the residual block 1030 as a result of downscaling during
video encoding 1000 and upscaling the residual generating during
video decoding 1100. In order to restore some of the high frequency
information that was lost during downscaling, a block-based filter
is applied to the reconstructed block 1140, which results in a
filtered reconstructed block 1150. The filter applied to generate
the filtered reconstructed block 1150 can be identified from
information in the video bit stream, such as filter information
regarding the block-based post filter to be utilized in filtering
the reconstructed block 1140. As previously discussed, the filter
information can be received from the video bit stream, such as by
decoding the filter information from the header of the encoded
residual block 1110. The filter information can include, for
example, an identifier that corresponds to a predefined filter or a
plurality of filter parameters that define the filter.
[0081] The filtered reconstructed block 1150 can be output as a
video signal. For example, the filtered reconstructed block 1150
could be output for display. The filtered reconstructed block is
also stored for use in decoding of other blocks by storing the
filtered reconstructed block 1150 as the prediction block 1130.
[0082] FIG. 12 shows an example of an encoding process 1200. The
encoding process 1200 can be implemented, for example, as a
software program that is executed by computing devices such as the
transmitting station 112 or the receiving station 130. The software
program can include machine-readable instructions that are stored
in a memory such as memory 226 that, when executed by a processor
such as CPU 224, cause the computing device to perform the encoding
process 1200. The encoding process 1200 can also be implemented
using hardware. As explained above, some computing devices may have
multiple memories and multiple processors, and the steps of the
encoding process 1200 may in such cases be distributed using
different processors and memories. Use of the terms "processor" and
"memory" in the singular encompasses computing devices that have
only one processor or one memory as well as devices having multiple
processors or memories that may each be used in the performance of
some but not necessarily all of the recited steps.
[0083] Operation 1210 includes generating the residual block 1030
from the original block 1010 and the prediction block 1020. This
can be performed in a conventional manner as previously described.
This operation can further include obtaining the original block
1010. The original block 1010 can be obtained in any manner such as
by receiving it in a video signal or accessing it from a storage
device. This operation can also include identifying the prediction
block 1020 that will be used with the original block 1010 to define
the residual block 1030. This can be performed using any of a
number of well-known algorithms that search for matching reference
blocks in the current frame or in other frames of the video signal.
Thus, the prediction block 1020 can be identified by comparing the
contents of the original block 1010 with the contents of a
plurality of blocks from the video signal and choosing the best
matching block as the prediction block 1020.
[0084] Operation 1220 includes downscaling the residual block 1030,
which can be performed in the manner described with respect to the
downscaled residual block 1040. The downscaled residual block 1040
is then encoded at operation 1230.
[0085] Operation 1240 includes selecting a block-based post filter
that will be utilized to recover some of the high frequency
information that was removed from the residual block 1030 in order
to reduce the bit cost for encoding it to define the downscaled
residual block 1040. This information can be encoded with the
encoded residual block 1050, such as by placing it in the header of
the encoded residual block 1050.
[0086] Subsequent to encoding the encoded residual block 1050 and
the filter information, the remainder of the process 1200 is
directed to decoding and reconstruction of the encoded residual
block such that it can be used as a prediction block in further
encoding operations. In operation 1250, the encoded residual block
1050 is decoded. The result of operation 1250 is the decoded
residual block 1060. At operation 1255, the decoded residual block
1060 is upscaled, which results in generation of the upscaled
decoded residual block 1065 At operation 1260, the upscaled decoded
residual block 1065 is combined with the prediction block 1020 that
was previously utilized at operation 1210 to generate the residual
block 1030. The result of this operation is the reconstructed block
1070. In operation 1270, the block-based post filter is applied to
the reconstructed block 1070 to recover some of the high frequency
information that was previously discarded in the process of
generating the downscaled residual block 1040 at operation 1220.
Subsequent to filtering, the filtered reconstructed block 1080 is
stored at operation 1280 for subsequent use as the prediction block
1020 in a subsequent encoding operation for a different block.
[0087] FIG. 13 shows an example of a decoding process 1300. The
decoding process 1300 can be implemented, for example, as a
software program that is executed by computing devices such as the
transmitting station 112 or the receiving station 130. The software
program can include machine-readable instructions that are stored
in a memory such as memory 226 that, when executed by a processor
such as CPU 224, cause the computing device to perform the decoding
process 1300. The decoding process 1300 can also be implemented
using hardware. As explained above, some computing devices may have
multiple memories and multiple processors, and the steps of the
decoding process 1300 may in such cases be distributed using
different processors and memories. Use of the terms "processor" and
"memory" in the singular encompasses computing devices that have
only one processor or one memory as well as devices having multiple
processors or memories that may each be used in the performance of
some but not necessarily all of the recited steps.
[0088] Operation 1310 includes decoding the encoded residual block
1110 into the decoded residual block 1120 as previously described.
In operation 1320, filter information is decoded from the video bit
stream, such as from the header of the encoded residual block 1110.
In operation 1325, the decoded residual block in upscaled, which
results in the upscaled decoded residual block 1125.
[0089] In operation 1330, the reconstructed block 1140 is generated
by adding the upscaled decoded residual block 1125 to the
prediction block 1130. In operation 1340, a block-based post filter
is applied to the reconstructed block 1140 to recover information
lost as a result of downscaling and subsequent upscaling. The
filtered reconstructed block 1150 can then be output for display
and can also be stored at operation 1350, as the prediction block
1130 for use in a subsequent decoding operation.
[0090] The aspects of encoding and decoding described above
illustrate some exemplary encoding and decoding techniques.
However, it is to be understood that encoding and decoding, as
those terms are used in the claims, could mean compression,
decompression, transformation, or any other processing or change of
data.
[0091] The words "example" or "exemplary" are used herein to mean
serving as an example, instance, or illustration. Any aspect or
design described herein as "example" or "exemplary" is not
necessarily to be construed as preferred or advantageous over other
aspects or designs. Rather, use of the words "example" or
"exemplary" is intended to present concepts in a concrete fashion.
As used in this application, the term "or" is intended to mean an
inclusive "or" rather than an exclusive "or". That is, unless
specified otherwise, or clear from context, "X includes A or B" is
intended to mean any of the natural inclusive permutations. That
is, if X includes A; X includes B; or X includes both A and B, then
"X includes A or B" is satisfied under any of the foregoing
instances. In addition, the articles "a" and "an" as used in this
application and the appended claims should generally be construed
to mean "one or more" unless specified otherwise or clear from
context to be directed to a singular form. Moreover, use of the
term "an implementation" or "one implementation" throughout is not
intended to mean the same embodiment or implementation unless
described as such.
[0092] Implementations of transmitting station 112 and/or receiving
station 130 (and the algorithms, methods, instructions, etc.,
stored thereon and/or executed thereby, including by encoder 470
and decoder 500) can be realized in hardware, software, or any
combination thereof. The hardware can include, for example,
computers, intellectual property (IP) cores, application-specific
integrated circuits (ASICs), programmable logic arrays, optical
processors, programmable logic controllers, microcode,
microcontrollers, servers, microprocessors, digital signal
processors or any other suitable circuit. In the claims, the term
"processor" should be understood as encompassing any of the
foregoing hardware, either singly or in combination. The terms
"signal" and "data" are used interchangeably. Further, portions of
transmitting station 112 and receiving station 130 do not
necessarily have to be implemented in the same manner.
[0093] Further, in one aspect, for example, transmitting station
112 or receiving station 130 can be implemented using a general
purpose computer or general purpose processor with a computer
program that, when executed, carries out any of the respective
methods, algorithms and/or instructions described herein. In
addition or alternatively, for example, a special purpose
computer/processor can be utilized which can contain other hardware
for carrying out any of the methods, algorithms, or instructions
described herein.
[0094] Transmitting station 112 and receiving station 130 can, for
example, be implemented on computing devices of any type. For
instance, the transmitting station 112 can be a personal computer
that includes a video capture device for obtain raw video to be
encoded and the receiving station 130 can be a personal computer
that includes a video display device for displaying decoded video.
Alternatively, transmitting station 112 can be implemented on a
server and receiving station 130 can be implemented on a device
separate from the server, such as a hand-held communications
device. In this instance, transmitting station 112 can encode
content using an encoder 470 into an encoded video signal and
transmit the encoded video signal to the communications device. In
turn, the communications device can then decode the encoded video
signal using a decoder 500. Alternatively, the communications
device can decode content stored locally on the communications
device, for example, content that was not transmitted by
transmitting station 112. Other suitable transmitting station 112
and receiving station 130 implementation schemes are available. As
one example, receiving station 130 can be a generally stationary
personal computer rather than a portable communications device. As
another example, a device that includes the encoder 470 may also
include the decoder 500.
[0095] Further, all or a portion of implementations of the present
invention can take the form of a computer program product
accessible from, for example, a tangible computer-usable or
computer-readable medium. A computer-usable or computer-readable
medium can be any device that can, for example, tangibly contain,
store, communicate, or transport the program for use by or in
connection with any processor. The medium can be, for example, an
electronic, magnetic, optical, electromagnetic, or a semiconductor
device. Other suitable mediums are also available.
[0096] The above-described embodiments, implementations and aspects
have been described in order to allow easy understanding of the
present invention and do not limit the present invention. On the
contrary, the invention is intended to cover various modifications
and equivalent arrangements included within the scope of the
appended claims, which scope is to be accorded the broadest
interpretation so as to encompass all such modifications and
equivalent structure as is permitted under the law.
* * * * *