U.S. patent application number 17/051531 was filed with the patent office on 2021-11-25 for joint source channel coding with h.265 video compression.
The applicant listed for this patent is Amimon Ltd.. Invention is credited to Zvi Reznic.
Application Number | 20210368177 17/051531 |
Document ID | / |
Family ID | 1000005781807 |
Filed Date | 2021-11-25 |
United States Patent
Application |
20210368177 |
Kind Code |
A1 |
Reznic; Zvi |
November 25, 2021 |
Joint Source Channel Coding with H.265 Video Compression
Abstract
A wireless video transmission system includes: a coarse
compression module to compress a video frame and to generate coarse
data of the video frame; a coarse decompression module to generate
coarse frame information from the coarse data; a distortions
extractor to generate coarse distortions from the coarse frame and
from the video frame; a refinement data encoder to generate
refinement data from coarse distortions based on the coarse frame
information; and a data combining and modulation module to combine
and transmit the coarse data and the refinement data.
Inventors: |
Reznic; Zvi; (Tel Aviv,
IL) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Amimon Ltd. |
Ra'anana |
|
IL |
|
|
Family ID: |
1000005781807 |
Appl. No.: |
17/051531 |
Filed: |
May 14, 2019 |
PCT Filed: |
May 14, 2019 |
PCT NO: |
PCT/IL2019/050541 |
371 Date: |
October 29, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62800504 |
Feb 3, 2019 |
|
|
|
62777159 |
Dec 9, 2018 |
|
|
|
62675778 |
May 24, 2018 |
|
|
|
62672889 |
May 17, 2018 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 19/91 20141101;
H04N 19/18 20141101; H04N 19/176 20141101; H04N 19/124 20141101;
H04N 19/122 20141101 |
International
Class: |
H04N 19/124 20060101
H04N019/124; H04N 19/122 20060101 H04N019/122; H04N 19/18 20060101
H04N019/18; H04N 19/91 20060101 H04N019/91; H04N 19/176 20060101
H04N019/176 |
Claims
1. A wireless video transmission system comprising: a coarse
compression module to compress a video frame and to generate coarse
data of said video frame; a coarse decompression module to generate
a coarse frame information from said coarse data; a distortions
extractor to generate coarse distortions from said coarse frame
information and from said video frame; a refinement data encoder to
generate refinement data from said coarse distortions based on said
coarse frame information; and a data combining and modulation
module to combine and transmit said coarse data and said refinement
data.
2. The system according to claim 1, wherein said refinement data
encoder is adapted to generate quantized transform coefficients
from said coarse data.
3. The system according to claim 2, wherein said quantized
transform coefficients comprise quantized DCT coefficients.
4. The system according to claim 3, wherein said refinement data
encoder is further adapted to detect block energy in said quantized
transform coefficients.
5. The system according to claim 4, wherein said refinement data
encoder is further adapted to estimate a probability density of the
quantization error in said quantized transform coefficients.
6. The system according to claim 5, wherein said refinement data
encoder is adapted to generate and reorder transform coefficients
from said coarse distortions.
7. The system according to claim 6, wherein said refinement data
encoder is further adapted to apply a gain to said reordered
transform coefficients, wherein said gain is derived from said
coarse data.
8. The system according to claim 7, wherein said reordered
transform coefficients comprise quantized DCT coefficients.
9. The system according to claim 8, wherein said refinement data
encoder is further adapted to entropy encode said reordered
transfer coefficients.
10. The system according to claim 9, wherein an entropy coding
codebook used for said entropy encoding is based on said
probability density of the quantization error.
11. The system according to claim 10, wherein said refinement data
encoder is further adapted to compand said reordered transfer
coefficients.
12. The system according to claim 11, wherein said refinement data
encoder is further adapted to perform a linear transform on said
reordered transfer coefficients.
13. The system according to claim 12, wherein said coarse
compression module divides said video frame into a plurality of
blocks.
14. A method of wireless video transmission, the method comprising:
compressing a video frame to generate coarse data of said video
frame; decompressing said coarse data to generate coarse frame
information from said coarse data; generating coarse distortions
from said coarse frame information and from said video frame;
generating refinement data from said coarse distortions based on
information derived from said coarse data; and combining and
transmitting said coarse data and said refinement data.
15. The method according to claim 14, further comprising:
generating quantized transform coefficients from said coarse
data.
16. The method according to claim 15, wherein said quantized
transform coefficients comprise quantized DCT coefficients.
17. The method according to claim 16, further comprising: detecting
block energy in said quantized transform coefficients.
18. The method according to claim 17, further comprising:
estimating a probability density in said quantized transform
coefficients.
19. The method according to claim 18, further comprising:
generating and reordering transform coefficients from said coarse
distortions.
20. The method according to claim 19, further comprising: applying
a gain to said reordered transfer coefficients.
21. The method according to claim 20, wherein said reordered
transform coefficients comprise quantized DCT coefficients.
22. The method according to claim 21, further comprising: entropy
encoding said reordered transfer coefficients.
23. The method according to claim 22, further comprising:
companding said reordered transfer coefficients.
24. The method according to claim 23, further comprising:
performing a linear transform on said reordered transfer
coefficients.
25. The method according to claim 24, further comprising: dividing
said video frame into a plurality of blocks.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This patent application claims priority and benefit: (I)
from U.S. 62/672,889, filed on May 17, 2018; and (II) from U.S.
62/675,778, filed on May 24, 2018; and (III) from U.S. 62/777,159,
filed on Dec. 9, 2018; and (IV) from U.S. 62/800,504, filed on Feb.
3, 2019; all of which are hereby incorporated by reference in their
entirety.
FIELD
[0002] The present invention relates to video communications
generally and more particularly to a system for hierarchal video
transmission and reception.
BACKGROUND
[0003] In some applications such as, for example, home AV
(Audio-Video) systems, two-piece TV, VR (Virtual Reality), AR
(Augmented Reality) and MR (Mixed Reality), there may be a need to
wirelessly transmit raw video. In many cases, the video may be
generated in an uncompressed manner (e.g. by a GPU overlaying
graphics over movies) such that the input to the wireless system at
one end may consist of pixels and the output of the system at the
other end may be a reproduction of the input pixels.
[0004] These wireless systems should generally provide for
relatively high quality where the output pixels are as similar as
possible to the input pixels. The systems should additionally
provide for relatively low latency where the total delay of the
transmission is low and for relatively high robustness to temporal
degradations in the capacity of the wireless channel. They are
generally configured to include a video encoder and a wireless
transmitter. The video encoder may be based on a known compression
standard such as, for example, H.264 or H.265. An advantage of
using compression schemes based on known compression standards is
that there is a relatively large offering of commercially available
designs which may be purchased as hardware and/or software and may
be integrated into new or existing systems. This may facilitate the
development of "standard-compression" based systems. The wireless
transmitter may additionally include a modem which may be based on
a known communication standard such as, for example, Wi-Fi.
[0005] In the wireless systems, tradeoffs are generally required
between quality, latency, and robustness. To improve the tradeoff
between quality, latency and robustness, methods of
Joint-Source-Channel-Coding (JSCC) where introduced. Consequently,
the systems may utilize the wireless spectrum efficiently.
SUMMARY
[0006] There is provided, in accordance with an embodiment of the
present invention, a wireless video transmission system which may
include a coarse compression module which may compress a video
frame and to generate coarse data of the video frame, a coarse
decompression module which may generate coarse frame information
from the coarse data, distortions extractor which may generate
coarse distortions from the coarse frame information and from the
video frame, a refinement data encoder which may generate
refinement data from the coarse frame distortions based on
information derived from the coarse frame information, and a data
combining and modulation module which may combine and transmit the
coarse data and the refinement data.
[0007] In some embodiments, the refinement data encoder may
generate quantized transform coefficients from the coarse data.
Optionally, the quantized transform coefficients may include
quantized DCT coefficients. The refinement data encoder may
additionally detect block energy in the quantized transform
coefficients and may additionally estimate a probability density of
the quantization error in the quantized transform coefficients.
[0008] In some embodiments, the refinement data encoder may
generate and reorder transform coefficients from the coarse
distortions. The refinement data encoder may additionally apply a
gain to the reordered transform coefficients, wherein the gain is
derived from the coarse data. Optionally, the reordered transform
coefficients may include quantized DCT coefficients.
[0009] In some embodiments, the refinement data encoder may entropy
encode the reordered transfer coefficients. Optionally, the entropy
coding codebook used for the entropy encoding may be based on the
probability density of the quantization error. The refinement data
encoder may additionally compand the reordered transfer
coefficients. The refinement data encoder may additionally perform
a linear transform on the reordered transfer coefficients.
[0010] In some embodiments, the coarse compression module may
divide the video frame into a plurality of blocks.
[0011] There is provided, in accordance with an embodiment of the
present invention, a method of wireless video transmission which
may include compressing a video frame to generate coarse data of
the video frame, decompressing the coarse data to generate coarse
frame information from the coarse data, generating coarse
distortions from the coarse frame information and from the video
frame, generating refinement data from the coarse distortions based
on information derived from the coarse frame information, and
combining and transmitting the coarse data and the refinement
data.
[0012] In some embodiments, the method may include generating
quantized transform coefficients from the coarse data. Optionally,
the quantized transform coefficients may include quantized DCT
coefficients. The method may additionally include detecting block
energy in the quantized transform coefficients. The method may
additionally include estimating a probability density in the
quantized transform coefficients.
[0013] In some embodiments, the method may include generating and
reordering transform coefficients from the coarse distortions. The
method may additionally include applying a gain to the reordered
transfer coefficients. Optionally, the reordered transform
coefficients may include quantized DCT coefficients. The method may
additionally include entropy encoding the reordered transfer
coefficients. The method may additionally include companding the
reordered transfer coefficients. The method may additionally
include performing a linear transform on the reordered transfer
coefficients.
[0014] In some embodiments, the method may include dividing the
video frame into a plurality of blocks.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The subject matter regarded as the invention is particularly
pointed out and distinctly claimed in the concluding portion of the
specification. The invention, however, both as to organization and
method of operation, together with objects, features, and
advantages thereof, may best be understood by reference to the
following detailed description when read with the accompanying
drawings in which:
[0016] FIG. 1 schematically illustrates an exemplary two-layer
video compression and transmission system for transmitting video
over a communication channel, according to an embodiment of the
present invention;
[0017] FIG. 2 schematically illustrates an exemplary system for
reception and reconstruction of video from a signal received over a
communication channel, according to an embodiment of the present
invention;
[0018] FIG. 3 schematically illustrates an exemplary refinement
coding module, which may be used in the video compression and
transmission system of FIG. 1, according to an embodiment of the
present invention;
[0019] FIG. 4 schematically illustrates an exemplary refinement
decoding module, which may be used in the system for reception and
reconstruction of video of FIG. 2, according to an embodiment of
the present invention;
[0020] FIG. 5A schematically illustrates an exemplary refinement
coding module, which may be used in the video compression and
transmission system of FIG. 1, according to an embodiment of the
present invention;
[0021] FIG. 5B schematically illustrates three exemplary graphs
denoted (a), (b) and (c), representing probability density
functions of three DCT taps, respectively, according to an
embodiment of the present invention;
[0022] FIG. 6 schematically illustrates an exemplary refinement
decoding module, which may be used in the system for reception and
reconstruction of video of FIG. 2, according to an embodiment of
the present invention;
[0023] FIG. 7 schematically illustrates an exemplary DCT taps
processor, which may be used with the refinement coding module of
FIG. 5, according to an embodiment of the present invention;
and
[0024] FIG. 8 schematically illustrates several different types of
compander functions, which may be used with the DCT taps processor
of FIG. 7, according to an embodiment of the present invention.
[0025] It will be appreciated that for simplicity and clarity of
illustration, elements shown in the figures have not necessarily
been drawn to scale. For example, the dimensions of some of the
elements may be exaggerated relative to other elements for clarity.
Further, where considered appropriate, reference numerals may be
repeated among the figures to indicate corresponding or analogous
elements.
DETAILED DESCRIPTION OF THE PRESENT INVENTION
[0026] In the following detailed description, numerous specific
details are set forth in order to provide a thorough understanding
of the invention. However, it will be understood by those skilled
in the art that the present invention may be practiced without
these specific details. In other instances, well-known methods,
procedures, and components have not been described in detail so as
not to obscure the present invention.
[0027] Applicants have realized that wireless video transmission
systems which combine use of JSCC with known compression block
techniques including compression standards such as, for example,
H.265, may suffer from bandwidth overhead. For example, consider a
system which may combine H.265 data with "analog" refinement data.
The system may employ "coarse" bins and "fine" bins where the
coarse bins are used for sending H.265 data (including interframe
prediction) and the fine bins are used for sending the data for
memoryless refinement of the received image. The analog refinement
may have video-content-dependent power, and may suffer from
additive noise and therefore, it may be desired to apply a
video-content-dependent gain to bring the transmitted power to the
highest level available by the hardware capabilities and the radio
regulation. A drawback using content-dependent gain is that the
gain should be conveyed to the receiver, and the message which
describes that gain, creates a bandwidth overhead.
Video-content-dependent gain may be applied to single blocks (that
is, each block will have a different gain), or to group of blocks
(that is, each group will have different gain, but all the blocks
in the group will have the same gain).
[0028] Applicants have realized that bandwidth overhead in video
transmission systems combining use of known compression block
techniques (e.g. H.264 or H.265) with JSCC may be reduced,
optionally eliminated, by utilizing information associated with the
coarse data to generate control data for the refinement encoder,
where the refinement encoder generates refinement data which may be
transmitted to a video receiver for memoryless refinement of a
video image. The transmission side may include a refinement encoder
which may be adapted to perform a two-dimensional DCT transform on
coarse distortions which may be in the form of error frames (in the
pixel domain) or in the form of quantization errors of DCT taps
generated from the coarse data to generate DCT taps for each block.
The refinement encoder may additionally calculate the block energy
to apply a gain to the coarse distortions DCT taps based on the
power of the coarse DCT taps. Alternatively, the refinement encoder
may estimate a probability density for the coarse DCT taps which
may then be used to entropy encode the coarse distortions DCT taps.
A compander may be used to apply non-linear gain to the DCT taps of
the refinement encoder.
[0029] The receiving side may include a refinement decoder which
may decode the received refinement data based on the received
coarse DCT taps. Similarly to the refinement encoder in the
transmission side, the refinement decoder may calculate the block
energy to apply a gain to the received refinement DCT taps based on
the power of the coarse DCT taps. Alternatively, the refinement
decoder may estimate a probability density for the coarse DCT taps
which may then be used to entropy decode the received refinement
DCT taps. As the received coarse data is substantially the same as
the transmitted coarse data except for a small bit error rate
associated with the packet error rate, the same gain applied by the
refinement encoder may be applied by the refinement decoder,
thereby eliminating the need to send the gain information to the
receiver and reducing bandwidth overhead. Alternatively, the same
probability density function may be used for the entropy decoding,
thereby eliminating the need to send the probability density
function information to the receiver and reducing bandwidth
overhead.
[0030] Reference is made to FIG. 1, which schematically illustrates
an exemplary two-layer video compression and transmission system
100 for transmitting video over a communication channel 150,
according to an embodiment of the present invention. System 100 may
include a coarse compression module 102, a coarse decompression
module 104, a coarse distortions extractor module 106, a refinement
coding module 108, a data combining and modulation module 110.
[0031] Coarse compression module 102 may receive pixels of an
original video frame 10 and may generate a compressed description
of the video which may be denoted as coarse data 12. Coarse
compression module 102 may use inter-frame prediction in performing
the data compression to generate coarse data 12. Inter frame
prediction may include use of a forward inter-frame predictor to
generate P-frames and optionally also a bi-directional inter-frame
predictor to generate B-frames. In order to reduce latency, coarse
compression module 102 may wholly or partially avoid generating
B-frames (bidirectionally predicted frames). Coarse decompression
module 104 may parse and decode the coarse data 12 to generate a
coarse frame 14.
[0032] Coarse distortions extractor module 106 may receive original
frame 10 and may apply coarse frame information 14 to generate
coarse distortions 16. To generate the coarse distortions, coarse
distortions extractor 106 may perform pixel subtraction optionally
followed by addition of a fixed offset and optional limiting to
generate an error frame. For example, a first pixel in a first line
in original frame 10 may have RGB values of 98, 61 and 80, and in
coarse frame 14 the same pixel may have the values of 100, 60 and
80. The coarse distortions extractor module 106 may add an offset
of 128 to the difference, and may additionally limit the result
from the bottom by 0 and from the top by 255. As a result, the
first pixel in the first line in error frame 16 may be min (255,
max (0, 128+98-100,128+61-60,128+80-80))=126,129,128.
[0033] Alternatively, the coarse distractions extractor may operate
in the DCT domain, and subtract the DCT taps of the original frame
from the DCT taps of the Coarse frame to generate quantization
errors of the DCT taps.
[0034] Refinement coding block module 108 may compress coarse
distortions 16 and may generate a refinement data description
(refinement data 18). Refinement coding block module 108 may adapt
its coding parameters according to the content of the coarse data.
For example, it may perform two dimensional DCT and may send only a
subset of the DCT taps, for example, the first N taps. It may
additionally select the value of N based on the coarse description
of that block according to a pre-defined algorithm for selecting N.
Since the coarse data is transmitted as well, the receiver may
select the same value of N for decoding the refinement data,
according to a pre-defined algorithm for selecting N, which is
equivalent to the pre-defined algorithm in the encoder. This may
eliminate the need to specifically encode and send the value of
N.
[0035] Data combining and modulation module 110 may combine coarse
data 12 and refinement data 18, and transmit them over a
communication channel 150. Data combining and modulation module 110
may modulate the signal using OFDM, with a portion of the bins
("digital bins") allocated for the coarse data and a portion of the
bins ("fine bins") allocated to the refinement data. Alternatively,
a different modulation and coding scheme may be used for the
digital bins and the fine bins, and the fine bins may include
properties similar to analog transmission. Alternatively, the same
modulation and modulation scheme may be used for the coarse data
and the fine data.
[0036] Reference is made to FIG. 2, which schematically illustrates
an exemplary system 200 for reception and reconstruction of video
from a signal received over a communication channel 250, according
to an embodiment of the present invention. System 200 may include a
demodulation and data splitting module 202, a coarse decompression
module 204, a refinement decoding module 206, and a frame combiner
module 208. System 200 may be used to receive a video signal
transmitted by transmission system 100 over channel 150, in which
case channel 150 and channel 250 is a same channel.
[0037] Demodulation and data splitter module 202 may demodulate the
received signal and may split the signal into coarse data 20 and
received refinement data 22. Coarse data 20 may be substantially
identical to coarse data 12 sent by transmission system 100, and
may include a small bit error rate compared to coarse data 12
associated with, for example, packet errors.
[0038] Coarse decompression module 204 may decode the video and may
generate coarse frame information 24. Coarse decompression module
204 may optionally be a standard video decoder such as, for
example, a MPEG decoder, a H.264 decoder, or a H.265 decoder, among
other standard video decoders.
[0039] Refinement decoding module 206 may receive refinement data
22 and may generate reconstructed coarse distortions 26. Refinement
decoding module 206 may adapt its decoding parameters according to
the content of coarse data 20. For example, it may decode from
coarse data 20 the value N of the number of transmitted DCT taps
and may then reconstruct the DCT values by zeroing all the taps
starting from tap N+1.
[0040] Frame combiner module 208 may combine the coarse frame
information 24 with reconstructed coarse data 26, for example by
summing the two frames together pixel by pixel, optionally reducing
a fixed offset (e.g. 128 for 8 bits pixels) and optionally limit
the result between two values, for example, 0 and 255. Frame
combiner module 28 may then generate a reconstructed frame 28.
Alternatively, frame combiner module may combine the coarse frame
information 24 with the reconstructed coarse data which may include
the quantization errors of the DCT taps to generate the
reconstructed frame 28.
[0041] Reference is made to FIG. 3 which schematically illustrates
an exemplary implementation of refinement coding module 108, which
may be used in video compression and transmission system 100,
according to an embodiment of the present invention. Refinement
coding module 108 may include a quantized DCT taps extractor 302, a
block energy detector module 304, an optional refinement DCT taps
generator module 306, and a DCT taps processor 308.
[0042] In an exemplary mode of operation, original frame 10 may be
divided into blocks, for example, block of 8.times.8 pixel or
4.times.4 pixels or 16.times.16 pixel blocks. Coarse compression
module 102 may perform a certain transform, for example a
two-dimensional DCT, on each block. Coarse compression module 102
may additionally reorder the DCT taps according to a certain
predefined order, for example in a. zig-zag order, and may
additionally quantize the DCT taps.
[0043] The value of quantized DCT tap number j of color c of block
number i may be denoted DCTQi,c,j.
[0044] DCT tap extractor module 302 may extract from coarse data 12
the quantized DCT taps and may transfer this data as quantized DCT
taps 13 to block energy detector module 304. Block energy detector
module 304 may determine, for each block, a certain gain,
represented by refinement block gain 15, which may be applied by
DCT taps processor 308 to reordered DCT taps 17 to generate
refinement data 18.
[0045] Refinement DCT taps Generator module 306 may perform a
certain transform, for example a two-dimensional DCT, on each block
of error frame in coarse distortions 16 if in the pixel domain DCT
& reordering module 306 may additionally reorder the DCT taps
according to a certain predefined order, for example in a. zig-zag
order, and may additionally quantize the DCT taps to generate
reordered refinement DC taps 17. Alternatively, if coarse
distortions include quantization errors of the DCT taps, DCT taps
processor 308 may generate refinement data 18 which may be a linear
transform of the reordered DC taps 17 following the processor
applying a gain to the DCT taps or to the linear transform of the
DCT taps based on the refinement block gain determined by block
energy detector module 304.
[0046] In some embodiments, an exemplary mode of operation of block
energy detector module 304 may be as follows:
[0047] a. Block energy detector 304 may first calculate for each
block i the block energy Ei by, for example:
E.sub.i=.SIGMA..sub.j=0.sup.M-1.SIGMA..sub.c=1.sup.3DCTQ.sub.i,c,j.sup.2
[0048] wherein M is the number of DCT taps per block, and it is
assumed that the 2D DCT is performed separately over each of the 3
colors.
[0049] b. Block energy detector module 304 may then use a certain
function, for example,
G.sub.i=f(E.sub.i)
to generate refinement block gain Gi. An example for such a
function may be given by:
G i = min .function. ( G max , max .function. ( G min , E r .times.
e .times. f E i ) ) ##EQU00001##
where the following are constants: Gmin, Gmax, Eref. Another
example of the function may be given by:
G i = { G H if .times. .times. E i < T G L if .times. .times. E
i .gtoreq. T ##EQU00002##
for some threshold T.
[0050] c. Alternatively or additionally, the frame may be divided
into superblocks, with N blocks per superblock (e.g., N=240), and
an additional refinement block gain G'i may be applied, where the
additional refinement block gain is substantially identical for all
the blocks in the superblock.
[0051] In some embodiments, another exemplary mode of operation of
block energy detector module 304 may be as follows:
[0052] a. Block energy detector module 304 may first calculate for
each superblock, the superblock energy, denoted Esuperblock,
by:
E.sub.superblock=.SIGMA..sub.i=1.sup.N.SIGMA..sub.j=0.sup.M-1.SIGMA..sub-
.c=1.sup.3DCTQ.sub.i,c,j.sup.2
where M is the number of DCT taps per block, and it is assumed that
the DCT is performed over 3 colors.
[0053] b. Block energy detector module 304 may then use a function,
such as:
G.sub.i'=f(E.sub.superblock)
to generate the refinement block gain G'i. An example of such a
function may be given by:
G i ' = min ( G max ' , max .function. ( G min ' .times. E ref ' E
superblock ) ##EQU00003##
where the following are constants: G'min, G'max, E'ref.
[0054] Reference is made to FIG. 4, which schematically illustrates
an exemplary implementation of refinement decoding module 206,
which may be used in system 200 for reception and reconstruction of
video, according to an embodiment of the present invention.
Refinement decoding module 206 may include a quantized DCT taps
extractor module 402, a block energy detector module 404, a DCT
taps extractor 406, and an optional reordering & IDCT module
408.
[0055] In some embodiments, as previously discussed with reference
to FIG. 2, coarse data 20 may be substantially identical to coarse
data 12 sent by transmission system 100, and may include a small
bit error rate compared to coarse data 12 associated with the
packet error rate. As such, quantized DCT taps extractor module 402
and block energy detector module 404 in refinement decoding module
206 may be substantially identical to quantized DCT taps extractor
module 302 and block energy detector module 304 in refinement
coding module 300 in FIG. 3. Additionally, refinement block gain 25
calculated by block energy detector module 404 may be substantially
identical to refinement block gain 15 calculated by block energy
detector module 304 of FIG. 3. It may be appreciated by the skilled
person that since the quantized DCT taps extractor module and the
energy block detector module are substantially identical to that in
transmission system 100, and since the values of DCTQ,i,c,j will be
made available to refinement decoding module 206 in receiver system
200, there is no need to send the value of Gi and of G'i over
channel 150, thereby allowing for a reduction (saving) in
bandwidth. That is, the values Gi and G'i, which were calculated in
refinement coding module 300 in FIG. 3, may be regenerated in
refinement decoding module 206, and there is no need to send
them.
[0056] DCT taps extractor 406 may extract the DCT taps from the
received refinement data 22 and may generate received reordered
refinement DCT taps 27 from received refinement data 22 based on
refinement block gain 25 computed by block energy detector module
404. Optional reordering & IDCT module 408 may reorder received
reordered DCT taps 27 and may perform an inverse transform on the
DCT taps to generate reconstructed coarse distortions 26 in the
form of error frames if received reordered refinement DCT taps 27
is in the pixel domain. If received reordered refinement DCT taps
27 includes quantization errors of the DCT taps, reordering &
IDCT module 408 may be obviated.
[0057] Reference is now made to FIG. 5A, which schematically
illustrates an exemplary refinement coding module 500 which may be
used in video compression and transmission system 100, according to
an embodiment of the present invention. Refinement coding module
500 may include a quantized DCT taps extractor 502, a refinement
probability function estimator module 504, an optional refinement
DCT taps generator module 506, and a DCT taps processor 508.
[0058] In an exemplary mode of operation, original frame 10 may be
divided into blocks; for example, block of 8.times.8 pixels, or
4.times.4 pixel or 16.times.16 pixel blocks. Coarse compression
module 102 may perform a certain transform, for example a
two-dimensional DCT, on each block. Coarse compression module 102
may additionally reorder the DCT taps according to a certain
predefined order, for example in a. zig-zag order, and may
additionally quantize the DCT taps. The DCT taps may be denoted as
a random variable x and its quantized value may be denoted as
Q(x).
[0059] Quantized DCT taps extractor module 502 and DCT &
reordering module 506 may be substantially similar to quantized DCT
taps extractor module 302 and refinement DCT taps module 306,
respectively, in FIG. 3. Refinement probability density function
estimator module 504 may estimate, for each Quantized DCT tap 13,
the probability density function of the corresponding DCT tap of
the error frame. DCT taps processor 508 may use refinement
probability density function per DCT tap 30 to process reordered
refinement DCT taps 17 and generate refinement data 18.
[0060] In some embodiments, DCT taps processor 508 may further
quantize reordered DCT taps 17 using a quantizer which may include
quantization cell size smaller than that in the quantizer used by
coarse compression module 102. Following quantization, DCT taps
processor 508 may use entropy encoding to compress the index of the
quantization cell and generate a string of bits which may indicate
the index of the fine quantization cell and where the string of
bits may be optimized to reduce the average number of bits in the
stream.
[0061] Entropy encoders are generally optimized for a certain
probability density of the source. To yield a good code book, the
probability density of the source is estimated. However, as shown
in FIG. 5B, the conditional probability density function of a
certain DCT tap may vary and may depend on its quantized value.
Refinement Probability density function estimator module 504 may
receive the quantized value at its input, and responsively, may
estimate the conditional probability density function to generate
refinement probability density function per DCT tap 30. DCT taps
processor 508 may then use refinement probability density function
per DCT tap 30 to select the proper entropy code-book which is best
fitted to the probability density function.
[0062] Reference is now made to FIG. 5B, which schematically
illustrates three exemplary graphs (a), (b) and (c) representing
probability density functions 550, 552, and 554 of three DCT taps,
respectively, according to an embodiment of the present invention.
Graph (a) describes the probability density function 550 of a
certain DCT tap. Also shown in the graph are the quantization
points and quantization cell boundaries of a uniform quantizer with
quantization cell size of .DELTA. and quantization points at 0,
.+-..DELTA., .+-.2.DELTA., .+-.3.DELTA., . . . . Graph (b)
probability density function 552 of a certain DCT tap whose
quantization point is 0. Graph (c) describes the probability
density function 554 of a certain DCT whose quantization point is
+2.DELTA.. It may be appreciated that
f(x-Q(x)|Q(x)=0)
is very different from
f(x-Q(x)|Q(x)=2.DELTA.)
and consequently, it may be preferable to use different codebooks
to compress X in each of the cases.
[0063] Reference is made to FIG. 6, which schematically illustrates
an exemplary refinement decoding module 600, which may be used in
system 200 for reception and reconstruction of video, according to
an embodiment of the present invention. Refinement decoding module
600 may include a quantized DCT taps extractor module 602, a
refinement probability density function estimator module 604, a DCT
taps extractor 606, and an optional reordering & IDCT module
608.
[0064] In some embodiments, as previously discussed with reference
to FIG. 2, coarse data 20 may be substantially identical to coarse
data 12 sent by transmission system 100, and may include a small
bit error rate compared to coarse data 12 associated with the
packet error rate. As such, quantized DCT taps extractor module 602
and refinement probability density function estimator module 604 in
refinement decoding module 600 may be substantially identical to
quantized DCT taps extractor module 502 and refinement probability
density function estimator module 604 in refinement coding module
500 in FIG. 5A. Additionally, refinement probability density
function per DCT tap 32 determined by refinement probability
density function estimator module 604 may be substantially
identical to refinement probability density function per DCT tap 30
determined by refinement probability density function estimator
module 504 of FIG. 5A. It may be appreciated by the skilled person
that, since refinement probability density function per DCT tap 37
estimated at refinement decoding block 600 is substantially
identical to refinement probability density function per DCT tap 30
estimated at the refinement encoding block 500 of FIG. 5A, there is
no need for the transmitter to send the refinement probability
density function per DCT tap to the receiver, which may allow for a
reduction in bandwidth.
[0065] DCT taps extractor 606 may extract the DCT taps from the
received refinement data and may generate received reordered
refinement DCT taps 27 from received refinement data 22 based on
refinement probability density function per DCT tap 37 determined
by refinement probability density function estimator module 604.
Optional reordering & IDCT module 608 may reorder received
reordered DCT taps 27 and may perform an inverse transform on the
DCT taps to generate reconstructed coarse distortions 26 in the
form of error frames if received reordered refinement DCT taps 27
is in the pixel domain. If received reordered refinement DCT taps
27 includes quantization errors of the DCT taps, reordering &
IDCT module 408 may be obviated.
[0066] Reference is made to FIG. 7, which schematically illustrates
an exemplary DCT taps processor 700, which may be used with
refinement coding module 500 in video transmission system 100,
according to an embodiment of the present invention. DCT taps
processor 700 may include a compander selector per DCT tap module
702, a compander module 704, and a linear transform module 706. In
some embodiments, DCT taps processor 700 may be the same as DCT
taps processor 508 in FIG. 5.
[0067] Compander module 704 may generate companded DCT taps 34 and
may include use of a non-liner function to apply a gain to
reordered refinement DCT taps 17. In some embodiments, a non-linear
function which may be used in Compander 704 may be, for example, as
in the following Equation (1):
C(x)=sign(x)C.sub.1 {square root over (|x|)}
where C1 is a constant, X is the input, and C(x) is the output.
[0068] Another non-linear function which may be used, may be as in
the following Equation (2):
C .function. ( x ) = sign .function. ( x ) { G 1 .times. x if
.times. .times. .times. x < T G 1 .times. T + G 2 ( x - T ) if
.times. .times. .times. x .gtoreq. T ##EQU00004##
[0069] for some constants G1 and G2 and T, and where X is the
input, and C(x) is the output.
[0070] In wireless systems the transmission power is usually
limited both by RF (radio frequency) regulations and by the
capabilities of the power amplifier. To optimize transmission range
and receiver SNR (signal-to-noise ratio), it is desired to optimize
the values of G1 and G2 and T in the above Equation (2), such that
the signal may be transmitted with the highest gain, but without
breaching RF regulations, or compressing and/or overloading the
power amplifier. The optimal choice of the constants G1 and G2 and
T generally depends on the probability density function of the
input X. For example, if X is known to be zero-mean with low
variance, then large values of G1 may be used (which may improve
the receiver SNR). But if X is known to be zero-mean with high
variance, then small values of G1 and G2 are generally used, to
avoid breaching the RF regulations or compressing and/or
overloading the power amplifier.
[0071] Compander selector per DCT taps module 702 receives, for
each DCT tap, the refinement probability density as indicated by
refinement probability density function per DCT tap 30, and uses it
to select the type of compander 704 as indicated by compander type
33. That is, compander selector per DCT taps 702 may chose the
values of G1 and G2 and T according to the refinement probability
density. Linear transform module 706 may apply a linear transform
to companded DCT taps 18 to generate refinement data 18.
[0072] Reference is made to FIG. 8, which schematically illustrates
several different types of compander functions 800, which may be
used with DCT taps processor 700, according to an embodiment of the
present invention. Type 1 compander function 802 may be suitable
for high power DCT taps; Type 2 compander 804 may be suitable for
medium power DCT taps; and Type 3 compander 806 may be suitable for
low power DCT taps.
[0073] Although portions of the discussion herein relate, for
demonstrative purposes, to wired links and/or wired communications,
some embodiments of the present invention are not limited in this
regard, and may include one or more wired or wireless links, may
utilize one or more components of wireless communication, may
utilize one or more methods or protocols of wireless communication,
or the like. Some embodiments may utilize wired communication
and/or wireless communication.
[0074] The present invention may be implemented by using hardware
units, software units, processors, CPUs, DSPs, integrated circuits,
memory units, storage units, wireless communication modems or
transmitters or receivers or transceivers, cellular transceivers, a
power source, input units, output units, Operating System (OS),
drivers, applications, and/or other suitable components.
[0075] The present invention may be implemented by using code or
program code or machine-readable instructions or machine-readable
code, which is stored on a non-transitory storage medium or
non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a
physical memory unit, a physical storage unit), such that the
program or code or instructions, when executed by a processor or a
machine or a computer, cause such device to perform a method in
accordance with the present invention.
[0076] Embodiments of the present invention may be utilized with a
variety of devices or systems having a touch-screen or a
touch-sensitive surface; for example, a smartphone, a cellular
phone, a mobile phone, a smart-watch, a tablet, a handheld device,
a portable electronic device, a portable gaming device, a portable
audio/video player, an Augmented Reality (AR) device or headset or
gear, a Virtual Reality (VR) device or headset or gear, a "kiosk"
type device, a vending machine, an Automatic Teller Machine (ATM),
a laptop computer, a desktop computer, a vehicular computer, a
vehicular dashboard, a vehicular touch-screen, or the like.
[0077] The system(s) and/or device(s) of the present invention may
optionally comprise, or may be implemented by utilizing suitable
hardware components and/or software components; for example,
processors, processor cores, Central Processing Units (CPUs),
Digital Signal Processors (DSPs), circuits, Integrated Circuits
(ICs), controllers, memory units, registers, accumulators, storage
units, input units (e.g., touch-screen, keyboard, keypad, stylus,
mouse, touchpad, joystick, trackball, microphones), output units
(e.g., screen, touch-screen, monitor, display unit, audio
speakers), microphone(s) and/or sensor(s), wired or wireless modems
or transceivers or transmitters or receivers, GPS receiver or GPS
element or other location-based or location-determining unit or
system, network elements (e.g., routers, switches, hubs, antennas),
and/or other suitable components and/or modules.
[0078] The system(s) and/or devices of the present invention may
optionally be implemented by utilizing co-located components,
remote components or modules, "cloud computing" servers or devices
or storage, client/server architecture, peer-to-peer architecture,
distributed architecture, and/or other suitable architectures or
system topologies or network topologies.
[0079] In accordance with embodiments of the present invention,
calculations, operations and/or determinations may be performed
locally within a single device, or may be performed by or across
multiple devices, or may be performed partially locally and
partially remotely (e.g., at a remote server) by optionally
utilizing a communication channel to exchange raw data and/or
processed data and/or processing results.
[0080] Some embodiments may be implemented by using a
special-purpose machine or a specific-purpose device that is not a
generic computer, or by using a non-generic computer or a
non-general computer or machine. Such system or device may utilize
or may comprise one or more components or units or modules that are
not part of a "generic computer" and that are not part of a
"general purpose computer", for example, cellular transceivers,
cellular transmitter, cellular receiver, GPS unit,
location-determining unit, accelerometer(s), gyroscope(s),
device-orientation detectors or sensors, device-positioning
detectors or sensors, or the like.
[0081] Some embodiments may be implemented as, or by utilizing, an
automated method or automated process, or a machine-implemented
method or process, or as a semi-automated or partially-automated
method or process, or as a set of steps or operations which may be
executed or performed by a computer or machine or system or other
device.
[0082] Some embodiments may be implemented by using code or program
code or machine-readable instructions or machine-readable code,
which may be stored on a non-transitory storage medium or
non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a
physical memory unit, a physical storage unit), such that the
program or code or instructions, when executed by a processor or a
machine or a computer, cause such processor or machine or computer
to perform a method or process as described herein. Such code or
instructions may be or may comprise, for example, one or more of:
software, a software module, an application, a program, a
subroutine, instructions, an instruction set, computing code,
words, values, symbols, strings, variables, source code, compiled
code, interpreted code, executable code, static code, dynamic code;
including (but not limited to) code or instructions in high-level
programming language, low-level programming language,
object-oriented programming language, visual programming language,
compiled programming language, interpreted programming language, C,
C++, C#, Java, JavaScript, SQL, Ruby on Rails, Go, Cobol, Fortran,
ActionScript, AJAX, XML, JSON, Lisp, Eiffel, Verilog, Hardware
Description Language (HDL, BASIC, Visual BASIC, Matlab, Pascal,
HTML, HTML5, CSS, Perl, Python, PHP, machine language, machine
code, assembly language, or the like.
[0083] Discussions herein utilizing terms such as, for example,
"processing", "computing", "calculating", "determining",
"establishing", "analyzing", "checking", "detecting", "measuring",
or the like, may refer to operation(s) and/or process(es) of a
processor, a computer, a computing platform, a computing system, or
other electronic device or computing device, that may automatically
and/or autonomously manipulate and/or transform data represented as
physical (e.g., electronic) quantities within registers and/or
accumulators and/or memory units and/or storage units into other
data or that may perform other suitable operations.
[0084] Some embodiments of the present invention may perform steps
or operations such as, for example, "determining", "identifying",
"comparing", "checking", "querying", "searching", "matching",
and/or "analyzing", by utilizing, for example: a pre-defined
threshold value to which one or more parameter values may be
compared; a comparison between (i) sensed or measured or calculated
value(s), and (ii) pre-defined or dynamically-generated threshold
value(s) and/or range values and/or upper limit value and/or lower
limit value and/or maximum value and/or minimum value; a comparison
or matching between sensed or measured or calculated data, and one
or more values as stored in a look-up table or a legend table or a
list of reference value(s) or a database of reference values or
ranges; a comparison or matching or searching process which
searches for matches and/or identical results and/or similar
results and/or sufficiently-close results, among multiple values or
limits that are stored in a database or look-up table; utilization
of one or more equations, formula, weighted formula, and/or other
calculation in order to determine similarity or a match between or
among parameters or values; utilization of comparator units, lookup
tables, threshold values, conditions, conditioning logic, Boolean
operator(s) and/or other suitable components and/or operations.
[0085] The terms "plurality" and "a plurality", as used herein,
include, for example, "multiple" or "two or more". For example, "a
plurality of items" includes two or more items.
[0086] References to "one embodiment", "an embodiment",
"demonstrative embodiment", "various embodiments", "some
embodiments", and/or similar terms, may indicate that the
embodiment(s) so described may optionally include a particular
feature, structure, or characteristic, but not every embodiment
necessarily includes the particular feature, structure, or
characteristic. Repeated use of the phrase "in one embodiment" does
not necessarily refer to the same embodiment, although it may.
Repeated use of the phrase "in some embodiments" does not
necessarily refer to the same set or group of embodiments, although
it may.
[0087] As used herein, and unless otherwise specified, the
utilization of ordinal adjectives such as "first", "second",
"third", "fourth", and so forth, to describe an item or an object,
merely indicates that different instances of such like items or
objects are being referred to; and does not intend to imply as if
the items or objects so described must be in a particular given
sequence, either temporally, spatially, in ranking, or in any other
ordering manner.
[0088] Some embodiments may comprise, or may be implemented by
using, an "app" or application which may be downloaded or obtained
from an "app store" or "applications store", for free or for a fee,
or which may be pre-installed on a computing device or electronic
device, or which may be transported to and/or installed on such
computing device or electronic device.
[0089] Functions, operations, components and/or features described
herein with reference to one or more embodiments of the present
invention, may be combined with, or may be utilized in combination
with, one or more other functions, operations, components and/or
features described herein with reference to one or more other
embodiments of the present invention. The present invention may
comprise any possible combinations, re-arrangements, assembly,
re-assembly, or other utilization of some or all of the modules or
functions or components that are described herein, even if they are
discussed in different locations or different chapters of the above
discussion, or even if they are shown across different drawings or
multiple drawings.
[0090] While certain features of the present invention have been
illustrated and described herein, many modifications,
substitutions, changes, and equivalents may occur to those skilled
in the art. Accordingly, the claims are intended to cover all such
modifications, substitutions, changes, and equivalents.
* * * * *