U.S. patent application number 11/454475 was filed with the patent office on 2007-04-26 for reverse link lower layer assisted video error control.
Invention is credited to Vikram Gupta, Yen-Chi Lee, Christopher Gerard Lott, Peerapol Tinnakornsrisuphap.
Application Number | 20070091816 11/454475 |
Document ID | / |
Family ID | 37814397 |
Filed Date | 2007-04-26 |
United States Patent
Application |
20070091816 |
Kind Code |
A1 |
Lee; Yen-Chi ; et
al. |
April 26, 2007 |
Reverse link lower layer assisted video error control
Abstract
The disclosure relates to reverse link lower layer assisted
video error control. A method may encode video data, form a packet
with the encoded video data, and transmit the packet across a
wireless channel to an access network. A medium access control
(MAC) layer may receive a negative acknowledgement (NAK) from the
access network. The method may determine whether the received NAK
is associated with a packet that contains video data. If the
received NAK is associated with a packet that contains video data,
the method may perform error control.
Inventors: |
Lee; Yen-Chi; (San Diego,
CA) ; Lott; Christopher Gerard; (San Diego, CA)
; Tinnakornsrisuphap; Peerapol; (San Diego, CA) ;
Gupta; Vikram; (San Diego, CA) |
Correspondence
Address: |
QUALCOMM INCORPORATED
5775 MOREHOUSE DR.
SAN DIEGO
CA
92121
US
|
Family ID: |
37814397 |
Appl. No.: |
11/454475 |
Filed: |
June 15, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
11315399 |
Dec 21, 2005 |
|
|
|
11454475 |
Jun 15, 2006 |
|
|
|
60729017 |
Oct 21, 2005 |
|
|
|
Current U.S.
Class: |
370/252 ;
370/394; 375/E7.013; 375/E7.025; 375/E7.28; 375/E7.281 |
Current CPC
Class: |
H04N 21/6583 20130101;
H04N 21/6181 20130101; H04N 21/6473 20130101; H04N 19/895 20141101;
H04N 21/2662 20130101; H04N 21/6131 20130101; H04N 21/64776
20130101 |
Class at
Publication: |
370/252 ;
370/394 |
International
Class: |
H04J 1/16 20060101
H04J001/16 |
Claims
1. A method comprising: encoding video data; forming an Internet
protocol (IP) packet with the encoded video data; transmitting the
packet across a wireless channel to an access network; at a medium
access control (MAC) layer, receiving a negative acknowledgement
(NAK) from the access network; determining whether the received NAK
is associated with a packet that contains video data; determining
which IP packet contains data that was lost in transmission; and if
the received NAK is associated with a packet that contains video
data, performing error control.
2. The method of claim 1, wherein the packet comprises macroblocks
(MBs) of video data, the method further comprising passing
information of the macroblocks from a video encoder to a real-time
transport protocol (RTP) layer.
3. The method of claim 1, further comprising: after encoding a
frame of video data and before encoding a new frame of video data,
querying the MAC layer to determine whether the MAC layer received
a NAK; and if the MAC layer received a NAK, performing error
control.
4. The method of claim 1, further comprising informing a packet
consolidation protocol (PCP) module of the received NAK.
5. The method of claim 1, further comprising, if the received NAK
is associated with a packet that contains video data, informing a
radio link protocol (RLP) module of the received NAK.
6. The method of claim 5, further comprising determining which RLP
queue contains a packet that experienced an error in
transmission.
7. The method of claim 1, further comprising informing an
application layer of the packet associated with the received
NAK.
8. The method of claim 1, further comprising: maintaining a map
between Internet protocol (IP) packets and macroblocks of video
data; after determining which IP packet contains data that was lost
in transmission using the map to convert the IP packet into
macroblocks of a video frame to determine which macroblocks were
lost in transmission; and performing error control for the
macroblocks that were lost in transmission.
9. The method of claim 8, wherein error control comprises at least
one of (a) restricting a motion estimation search range so that a
damaged part will not be used as a prediction, (b) intra-code
co-located macroblocks and neighboring macroblocks, and (c) using
different reference frames for co-located macroblocks.
10. The method of claim 1, wherein the wireless channel comprises a
code division multiple access 2000 1.times. Evolution Data
Optimized Revision A (CDMA2000 1.times. EV-DO Rev A) channel.
11. The method of claim 1, wherein the wireless channel comprises a
wideband code division multiple access (WCDMA) High Speed Uplink
Packet Access (HSUPA) channel.
12. A method comprising: encoding a first frame of video data;
forming a packet with the encoded first frame of video data;
transmitting the packet across a wireless channel to an access
network; before encoding a second frame of video data, determining
whether a medium access control (MAC) layer received a negative
acknowledgement (NAK) from the access network; determining whether
the received NAK is associated with a packet that contains video
data; and if the received NAK is associated with a packet that
contains video data, then performing error control.
13. The method of claim 12, wherein determining whether a medium
access control (MAC) layer received a negative acknowledgement
(NAK) from the access network comprises checking whether a flag is
set in a radio link protocol (RLP) module.
14. The method of claim 12, further comprising informing an
application layer of the received NAK.
15. The method of claim 12, further comprising informing a video
encoder of the received NAK.
16. The method of claim 12, further comprising setting a one-bit
flag in a radio link protocol (RLP) module to indicate the MAC
layer received a NAK.
17. The method of claim 12, wherein error control comprises at
least one of (a) encoding the second frame as an I-frame, (b)
increasing percentage of intra-coded macroblocks, and (c) using a
frame before the first frame as a reference for motion
estimation.
18. An apparatus comprising a machine-readable memory storing a set
of instructions configured to: encode video data; form an Internet
protocol (IP) packet with the encoded video data; transmit the
packet across a wireless channel to an access network; at a medium
access control (MAC) layer, receive a negative acknowledgement
(NAK) from the access network; determine whether the received NAK
is associated with a packet that contains video data; determine
which IP packet contains data that was lost in transmission; and if
the received NAK is associated with a packet that contains video
data, perform error control.
19. The apparatus of claim 18, wherein the packet comprises
macroblocks (MBs) of video data, the instructions further
comprising passing information of the macroblocks from a video
encoder to a real-time transport protocol (RTP) layer.
20. The apparatus of claim 18, wherein the instructions are further
configured to: after encoding a frame of video data and before
encoding a new frame of video data, query the MAC layer to
determine whether the MAC layer received a NAK; and if the MAC
layer received a NAK, perform error control.
21. The apparatus of claim 18, wherein the instructions are farther
configured to inform a packet consolidation protocol (PCP) module
of the received NAK.
22. The apparatus of claim 18, wherein the instructions are further
configured to, if the received NAK is associated with a packet that
contains video data, inform a radio link protocol (RLP) module of
the received NAK.
23. The apparatus of claim 22, wherein the instructions are further
configured to determine which RLP queue contains a packet that
experienced an error in transmission.
24. The apparatus of claim 18, wherein the instructions are further
configured to inform an application layer of the packet associated
with the received NAK.
25. The apparatus of claim 18, wherein the instructions are further
configured to: maintain a map between Internet protocol (IP)
packets and macroblocks of video data; after determining which IP
packet contains data that was lost in transmission use the map to
convert the IP packet into macroblocks of a video frame to
determine which macroblocks were lost in transmission; and perform
error control for the macroblocks that were lost in
transmission.
26. The apparatus of claim 25, wherein error control comprises at
least one of (a) restricting a motion estimation search range so
that a damaged part will not be used as a prediction, (b)
intra-code co-located macroblocks and neighboring macroblocks, and
(c) using different reference frames for co-located
macroblocks.
27. The apparatus of claim 18, wherein the wireless channel
comprises a code division multiple access 2000 1.times. Evolution
Data Optimized Revision A (CDMA2000 1.times. EV-DO Rev A)
channel.
28. The apparatus of claim 18, wherein the wireless channel
comprises a wideband code division multiple access (WCDMA) High
Speed Uplink Packet Access (HSUPA) channel.
29. An apparatus comprising a machine-readable memory storing a set
of instructions configured to: encode a first frame of video data;
form a packet with the encoded first frame of video data; transmit
the packet across a wireless channel to an access network; before
encoding a second frame of video data, determine whether a medium
access control (MAC) layer received a negative acknowledgement
(NAK) from the access network; determine whether the received NAK
is associated with a packet that contains video data; and if the
received NAK is associated with a packet that contains video data,
then perform error control.
30. The apparatus of claim 29, wherein determining whether a medium
access control (MAC) layer received a negative acknowledgement
(NAK) from the access network comprises checking whether a flag is
set in a radio link protocol (RLP) module.
31. The apparatus of claim 29, the instructions being further
configured to inform an application layer of the received NAK.
32. The apparatus of claim 29, the instructions being further
configured to inform a video encoder of the received NAK.
33. The apparatus of claim 29, the instructions being further
configured to set a one-bit flag in a radio link protocol (RLP)
module to indicate the MAC layer received a NAK.
34. The apparatus of claim 29, wherein error control comprises at
least one of (a) encoding the second frame as an I-frame, (b)
increasing percentage of intra-coded macroblocks, and (c) using a
frame before the first frame as a reference for motion
estimation.
35. An apparatus comprising: a video encoder configured to encode
video data; a module configured to form an Internet protocol (IP)
packet with the encoded video data; a transceiver configured to (a)
transmit the packet across a wireless channel to an access network,
and (b) receive a medium access control (MAC) layer negative
acknowledgement (NAK) from the access network; wherein the module
is configured to determine whether the received NAK is associated
with a packet that contains video data and determine which IP
packet contains data that was lost in transmission; and if the
received NAK is associated with a packet that contains video data,
the video encoder is configured to perform error control.
36. An apparatus comprising: a video encoder configured to encode a
first frame of video data; a first module configured to form a
packet with the encoded first frame of video data; a transceiver
configured to (a) transmit the packet across a wireless channel to
an access network and (b) receive a medium access control (MAC)
layer negative acknowledgement (NAK) from the access network;
before encoding a second frame of video data, a second module
configured to determine whether the MAC layer received a NAK from
the access network and determine whether the received NAK is
associated with a packet that contains video data; and an error
control module configured to perform error control if the received
NAK is associated with a packet that contains video data.
Description
CLAIM OF PRIORITY
[0001] This application is a continuation-in-part application and
claims priority to co-assigned U.S. patent application Ser. No.
11/315,399, filed on Dec. 21, 2005, entitled "METHODS AND SYSTEMS
FOR ADAPTIVE ENCODING OF REAL-TIME INFORMATION IN PACKET-SWITCHED
WIRELESS COMMUNICATION SYSTEMS" (attorney docket no. 051126), which
claims priority to U.S. Provisional Application No. 60/729,017,
filed on Oct. 21, 2005, which are hereby incorporated by reference
in their entirety.
TECHNICAL FIELD
[0002] The disclosure relates to video encoding and, more
particularly, to reverse link lower layer assisted video error
control.
BACKGROUND
[0003] A cellular phone may include an audio capture device, such
as a microphone or speech synthesizer, and an audio encoder to
generate audio packets (or frames). The phone may use communication
protocol layers and modules, such as a radio link protocol (RLP)
module, a medium access control (MAC) layer, and a physical (PHY)
layer. The phone may place audio packets in a RLP queue. A MAC
layer module may generate MAC layer packets from contents of the
RLP queue. The MAC layer packets may be converted to PHY layer
packets for transmission across a communication channel to another
communication device.
SUMMARY
[0004] One aspect relates to a method comprising: encoding video
data; forming an Internet protocol (IP) packet with the encoded
video data; transmitting the packet across a wireless channel to an
access network; at a medium access control (MAC) layer, receiving a
negative acknowledgement (NAK) from the access network; determining
whether the received NAK is associated with a packet that contains
video data; determining which IP packet contains data that was lost
in transmission; and if the received NAK is associated with a
packet that contains video data, performing error control.
[0005] Another aspect relates to a method comprising: encoding a
first frame of video data; forming a packet with the encoded first
frame of video data; transmitting the packet across a wireless
channel to an access network; before encoding a second frame of
video data, determining whether a medium access control (MAC) layer
received a negative acknowledgement (NAK) from the access network;
determining whether the received NAK is associated with a packet
that contains video data; and if the received NAK is associated
with a packet that contains video data, then performing error
control.
[0006] Another aspect relates to an apparatus comprising a
machine-readable memory storing a set of instructions configured
to: encode video data; form an Internet protocol (IP) packet with
the encoded video data; transmit the packet across a wireless
channel to an access network; at a medium access control (MAC)
layer, receive a negative acknowledgement (NAK) from the access
network; determine whether the received NAK is associated with a
packet that contains video data; determine which IP packet contains
data that was lost in transmission; and if the received NAK is
associated with a packet that contains video data, perform error
control.
[0007] Another aspect relates to an apparatus comprising a
machine-readable memory storing a set of instructions configured
to: encode a first frame of video data; form a packet with the
encoded first frame of video data; transmit the packet across a
wireless channel to an access network; before encoding a second
frame of video data, determine whether a medium access control
(MAC) layer received a negative acknowledgement (NAK) from the
access network; determine whether the received NAK is associated
with a packet that contains video data; and if the received NAK is
associated with a packet that contains video data, then perform
error control.
[0008] Another aspect relates to an apparatus comprising: a video
encoder configured to encode video data; a module configured to
form an Internet protocol (IP) packet with the encoded video data;
a transceiver configured to (a) transmit the packet across a
wireless channel to an access network, and (b) receive a medium
access control (MAC) layer negative acknowledgement (NAK) from the
access network; wherein the module is configured to determine
whether the received NAK is associated with a packet that contains
video data and determine which IP packet contains data that was
lost in transmission; and if the received NAK is associated with a
packet that contains video data, the video encoder is configured to
perform error control.
[0009] An apparatus comprising: a video encoder configured to
encode a first frame of video data; a first module configured to
form a packet with the encoded first frame of video data; a
transceiver configured to (a) transmit the packet across a wireless
channel to an access network and (b) receive a medium access
control (MAC) layer negative acknowledgement (NAK) from the access
network; before encoding a second frame of video data, a second
module configured to determine whether the MAC layer received a NAK
from the access network and determine whether the received NAK is
associated with a packet that contains video data; and an error
control module configured to perform error control if the received
NAK is associated with a packet that contains video data.
[0010] The details of one or more embodiments are set forth in the
accompanying drawings and the description below.
BRIEF DESCRIPTION OF DRAWINGS
[0011] FIG. 1 illustrates a communication system with a video and
audio encoder device sending data across a transmission channel to
a decoder device.
[0012] FIG. 2 illustrates a method and structure for a first
approach to detect and control errors, which may be used in the
system of FIG. 1.
[0013] FIG. 3 illustrates a method and structure for a second
approach to detect and control errors, which may be used in the
system of FIG. 1.
[0014] FIG. 4 further illustrates the first and second approaches
of FIGS. 2 and 3.
[0015] FIG. 5 illustrates an example of detecting a lost packet and
forcing an intra refresh, which minimizes error propagation caused
by a lost packet.
DETAILED DESCRIPTION
[0016] Video and Audio Encoding and Transmission
[0017] FIG. 1 illustrates a communication system 10 with a video
and audio encoder device 12 sending data across a transmission
channel 16 (including an access network component 15) to a decoder
device 14. The encoder device 12 and decoder device 14 may be
dispersed throughout the system 10. The encoder device 12 and
decoder device 14 may represent various types of devices, such as a
wireless phone, a cellular phone, a laptop computer, a wireless
multimedia device, a wireless communication personal computer (PC)
card, a personal digital assistant (PDA), an external or internal
modem, or any device that communicates through a wireless
channel.
[0018] The encoder device 12 and decoder device 14 may each have
various names, such as access terminal (AT), access unit,
subscriber unit, mobile station, mobile device, mobile unit, mobile
phone, mobile, remote station, remote terminal, remote unit, user
device, user equipment, handheld device, etc.
[0019] Each device 12, 14 may communicate with an access network
(AN) component 15 on a forward link (FL) and/or a reverse link
(RL). The FL (or downlink) refers to transmission from the AN 15 to
a device 12, 14. The reverse link (or uplink) refers to
transmission from a device 12, 14 to the AN 15. The AN 15 may refer
to the network portion of a communication system, and may include
(but is not limited to) or implement the function of a base station
(BS), a base-station transceiver system (BTS), an access point
(AP), a modem pool transceiver (MPT), a Node B (e.g., in a WCDMA
type system), etc.
[0020] The encoder device 12 may be in a first video communication
device and may include an audio source 17, video source 18, video
encoder 20, audio encoder 22, real-time transport protocol
(RTP)/user datagram protocol (UDP)/Internet protocol (IP)
conversion module 26, radio link protocol (RLP) queue 28, MAC layer
module 30 and physical (PHY) layer module 32. Other embodiments of
the encoder device 12 may include other elements instead of or in
addition to the elements shown in FIG. 1. Other embodiments of the
encoder device 12 may include fewer elements than those shown in
FIG. 1.
[0021] The decoder device 14 may be in another video communication
device and may include a PHY layer module 34, MAC layer module 36,
RLP queue 38, RTP/UDP/IP conversion module 40, video decoder 42,
audio decoder 44, audio output unit 46 and video output unit 48.
Other embodiments of the decoder device 14 may include other
elements instead of or in addition to the elements shown in FIG. 1.
Other embodiments of the decoder device 14 may include fewer
elements than those shown in FIG. 1.
[0022] The system 10 may provide bi-directional video and audio
transmission, such as video telephony (VT), via the channel 16.
Reciprocal encoding, decoding, and conversion modules may be
provided on opposite ends of the channel 16. In some embodiments,
the encoder device 12 and decoder device 14 may be embodied within
video communication devices such as wireless mobile terminals
equipped for video streaming, VT, or both. The mobile terminals may
support VT according to packet-switched standards such as RTP, UDP
or IP.
[0023] The video source 18 may be a video capture device, such as a
video camera, one or more video archives, or a combination of a
video camera and video archives. The video encoder 20 generates
encoded video data packets according to a video compression method,
such as MPEG-4. Other video compression methods may be used, such
as the International Telecommunication Union (ITU) H.263, ITU
H.264, or MPEG-2 methods. The video encoder 20 may provide a video
source rate control scheme that is generally CODEC-dependent. For
example, the video encoder 20 may be adapted for video encoding
according to MPEG4, ITU H.263 or ITU H.264. Video encoder 20 may be
implemented by a DSP or embedded logic core.
[0024] The audio source 17 may be an audio capture device, such as
a microphone, or a speech synthesizer device. The audio encoder 22
may encode audio data and generate audio packets to accompany the
video data. The audio data may be encoded according to an audio
compression method, such as adaptive multi-rate narrow band
(AMR-NB), or other techniques. For VT applications, the video will
permit viewing of a party to a VT conference, and the audio will
permit the speaking voice of that party to be heard.
[0025] The RTP/UDP/IP conversion module 26 obtains video and audio
data packets from video encoder 20 and audio encoder 22. The
RTP/UDP/IP conversion module 26 may add appropriate RTP/UDP/IP
header information to the audio and video data packets received
from the audio encoder 22 and video encoder 20, and place the data
packets in the RLP queue 28. RTP may run on top of UDP. UDP may run
on top of IP. IP may run on top of a point-to-point protocol (PPP)
layer in one configuration.
[0026] The MAC layer module 30 may retrieve RLP packets from RLP
queue 28 and generate MAC layer packets. Each MAC layer packet may
carry RTP/UDP/IP header information and audio or video packet data
that is contained within RLP queue 28.
[0027] Audio packets may be inserted into RLP queue 28
independently of video packets. In some cases, a MAC layer packet
generated from the contents of RLP queue 28 will carry only header
information and video packet data. In other cases, the MAC layer
packet will carry only header information and audio packet data. In
other cases, the MAC layer packet will carry header information,
audio packet data and video packet data, depending on the contents
of RLP queue 28. The MAC layer packets may be configured according
to a radio link protocol (RLP), and may be referred to as MAC RLP
packets.
[0028] The PHY layer module 32 may convert the MAC RLP packets into
PHY layer packets for transmission over the channel 16. The channel
16 carries the PHY layer packets to the decoder device 14.
[0029] In the decoding device 14, the PHY layer module 34 and MAC
layer module 36 may operate in a reciprocal manner. The PHY layer
module 34 may identify the MAC layer packets from the PHY layer
packets and convert/reassemble PHY layer packets received from
channel 16 to MAC RLP packets.
[0030] The MAC layer module 36 may reassemble the contents of the
MAC RLP packets to provide video and audio packets for insertion of
the MAC RLP packets into the RLP queue 38. The RTP/UDP/IP
conversion module 40 may remove/strip the accompanying header
information from the data in the RLP queue 38, and reassemble the
video and audio data for delivery to the video decoder 42 and audio
decoder 44, respectively.
[0031] Video decoder 42 decodes the video data frames to produce a
stream of video data for use in driving a display device (video
output) 48. Audio decoder 44 decodes the audio data to produce
audio information for presentation to a user, e.g., via an audio
speaker (audio output) 46.
[0032] Video telephony (VT) refers to real-time communication of
audio and video packets between at least two devices, such as
systems 12 and 14. In mobile VT applications, a VT device (wireless
terminal) receives PHY layer packets via a wireless forward link
(FL) (i.e., "downlink") from a base station. A VT device transmits
PHY layer packets via a wireless reverse link (RL) (i.e., "uplink")
to the base station.
[0033] The system 10 and channel 16 may be designed to support one
or more wireless communication technologies such as code division
multiple access (CDMA), frequency division multiple access (FDMA),
time division multiple access (TDMA), or orthogonal frequency
divisional multiplexing (OFDM), or another suitable wireless
technique. The above wireless communication technologies may be
delivered according to any of a variety of radio access
technologies. For example, CDMA may be delivered according to
cdma2000 or wideband CDMA (WCDMA) standards. TDMA may be delivered
according to the Global System for Mobile Communications (GSM)
standard. The Universal Mobile Telecommunication System (UMTS)
standard permits GSM or WCDMA operation. For VT applications, the
system 10 may be designed to support high data rate (HDR)
technologies such as cdma2000 1.times. EV-DO, Release 0, Revision
A, or subsequent EV-DO releases.
[0034] Channel conditions may be a concern for wireless channels,
but may be especially problematic for mobile VT applications, in
which channel conditions may suffer from fading or network
congestion.
[0035] Video Error Control
[0036] Lower layer assisted video error control methods may address
errors that occur to video data during transmission on the reverse
link. The methods may enable the video encoder 20 to use channel
error information from the MAC layer 30 and immediately apply error
control, without waiting for feedback from the receiving device 14.
The methods may use mechanisms to retrieve/receive error
information that can be understood by the video encoder 20.
Depending on desired implementation complexity, the error
information from the MAC layer 30 can be transformed into two
different formats.
[0037] After receiving error messages from lower layers, the video
encoder 20 may (a) use a different reference frame for a new frame,
(b) increase macroblock (MB)-intra refresh rate, or (c) insert an
intra-frame (I-frame) to stop error propagation at the receiver.
Inserting an I-frame is basically the same as forcing an
intra-refresh. In an I-frame, every MB is intra-coded, i.e., it
does not depend on the previous frame, which can stop the error
propagation. For MB-intra refresh, only some MBs are intra-coded.
Option (c) may avoid using periodic I-frames (or decrease
intra-frame refresh rate) to improve coding efficiency and video
delay, as well as provide better video quality when errors occur.
The video quality can be better because the video encoder 20 may
wipe out the errors at once, instead of gradually refreshing each
MB in the subsequent frames.
[0038] FIG. 5 illustrates an example of detecting a packet loss and
forcing an intra refresh, which minimizes error propagation caused
by a lost packet.
[0039] A CDMA2000 1.times. EV-DO Rev A reverse link is used below
as one example of receiving/retrieving error information from the
MAC layer 30. The methods may also be applied to other
communication systems, such as WCDMA High Speed Uplink Packet
Access (HSUPA) system, which use H-ARQ, NAK or other similar
approach to detect transmission errors and/or lost information.
[0040] The methods described below may provide a cross-layer
optimization for error performance in multimedia over wireless
networks, including video telephony and video sharing, where
transmission errors may be inevitable.
[0041] For forward link (downlink) transmission errors, some
methods may use end-to-end feedback to provide error information.
However, if these methods solely rely on end-to-end feedback, they
may not respond to RL transmission errors quickly enough to reduce
artifacts caused by lost packets. The RL methods described herein
may compliment the end-to-end feedback approach.
[0042] MAC-Assisted Video Error Control using NAK Information
[0043] This section describes two MAC-assisted video error control
methods that use negative acknowledgment (NAK) information from the
access network 15. In both methods, the video encoder 20 or error
control module 206 may query the MAC layer 30 on a frame-by-frame
basis, i.e., before encoding a new frame, the video encoder 20 may
receive error information from one or more lower layers and perform
error control if needed. Based on an error information message
type, the video encoder 20 may perform different error control
schemes to mitigate error propagation.
[0044] First Approach: Error Control using Video Flow IP Packet
NAK
[0045] FIG. 2 illustrates a method and structure for a first
approach where the video encoder 20 or error control module 206 is
informed of which macroblocks (MBs) in a video frame were lost in
transmission. Each video frame may comprise many macroblocks.
[0046] When a transmission error occurs, the access network 15
sends a NAK to the reverse traffic channel MAC (RTCMAC) layer 30.
The MAC layer 30 informs a packet consolidation protocol (PCP)
module 210 of the MAC packet associated with the NAK.
[0047] The PCP module 210 determines if the lost MAC packet
contains any video flow data. Audio and video data can be placed in
the same MAC packet, even though quality of service (QoS) is
supported in the EV-DO Rev A reverse link. If there is video data
in the lost MAC packet, the PCP module 210 will inform the RLP
module 208 which RLP queue (there may be several RLP queues)
contains a packet that experienced an error in transmission with a
RLP packet NAK.
[0048] The RLP module 208 determines which IP packet contains the
lost data. The RLP module 208 has knowledge of an IP packet. For
EV-DO Rev A, a video flow is configured such that a Link Flow data
unit is an IP packet. Then the RLP module 208 informs a RTP layer
202 in an application layer which IP packet has been lost with a
video flow IP packet NAK.
[0049] The RTP layer 202 maintains a map or table that will convert
the lost IP packet into lost MBs in a frame. Then the information
of the lost MBs will be passed to the error control module 206
and/or video encoder 20 for further error control.
[0050] This first approach may involve modifications at three
different layers: [0051] The video encoder 20 passes information of
MBs in each packet to the RTP layer 202. [0052] The RTP layer 202
maintains a conversion map or table between IP packets (indexed by
sequence numbers) and macroblocks. If an IP packet is lost, the RTP
layer 202 can convert the IP packet to macroblocks and inform the
video encoder 20 of the lost MBs. The sequence number associated
with each IP packet may also be passed to the RLP module 208.
[0053] The RLP module 208 keeps track of how the IP packets are
fragmented into RLP packets and determines which IP packet is lost
by checking the sequence number.
[0054] For error control, since the video encoder 20 knows which
MBs are lost, the video encoder 20 or error control module 206 can
perform one or more of the following error control methods: [0055]
Restrict motion estimation search range so that the damaged (lost)
part will not be used as the prediction. [0056] Intra-code the
co-located MBs and their neighboring MBs. [0057] Use different
reference frames for the co-located MBs.
[0058] Second Approach: Error Control using Video Flow NAK
[0059] The first approach described above may provide good error
control capability and performance, but the implementation
complexity may be relatively high. The first approach may need to
modify the RTP and RLP layers 202, 208. A second approach may
provide a simplified yet effective approach for error control.
[0060] FIG. 3 illustrates a method and structure for the second
approach. Compared to FIG. 2, FIG. 3 may not require modification
of the RTP layer 202. In fact, the second approach may not need the
RTP layer 202 to perform any operation. Instead, the video encoder
or error control module 206 may query the RLP module 208 before
encoding each new frame of video data, and error information may be
passed directly to the video encoder 20 or error control module 206
for error control. This error message may simply be a flag (e.g., a
binary bit) set in the RLP module 208 that informs the video
encoder 20 if there has been any data lost on a video flow since
the last time the MAC layer 30 was queried. In one embodiment, the
MAC layer 30 may automatically send a NAK to the PCP module 210,
which sends a NAK to the RLP module 208, which sets a flag without
any queries to the PCP or MAC layers 210, 30.
[0061] Between encoding two video frames, the video encoder 20 may
just need to know whether any data was lost in the previous frame
so the video encoder 20 or error control module 206 can perform
error control on the current frame. Since the encoder 20 or error
control module 206 queries the RLP module 208 on a frame-by-frame
basis, i.e., before encoding a new frame, knowing if there is any
transmission error may be just as good as knowing which IP packet
was lost.
[0062] If the MAC layer automatic repeat request (MARQ) is used for
the video flow, this flag should be set only when the MARQ packet
containing video data is not positively acknowledged (ACK) at the
physical layer.
[0063] This second approach greatly may simplify the implementation
complexity and provide useful information for the video encoder 20
to perform proper error control. However, since the flag may only
indicate if there has been an error or not, the video encoder 20
may not know which part of the frame is lost. Therefore, the error
control should assume the error may happen in any part of any frame
since the last query.
[0064] Once this flag is set, the video encoder 20 may perform one
or more of the following error control methods, which may be
different from the first approach: [0065] Encode the current frame
as an I-frame. [0066] Increase percentage of intra-coded MBs.
[0067] Use the frame before the previous frame as a reference for
motion estimation.
[0068] FIG. 4 further illustrates the first and second approaches.
EV-DO Rev A reserve link channel is used as an example, but other
types of channels, standards and communication protocols may be
used. Near the bottom of FIG. 4, the MAC layer 30 sends a MAC
packet containing audio and video data to the physical layer 32 as
a first subpacket in time slot n. For example, there may be 50
packet transmissions per second at the MAC layer 30, assuming
12-slot termination and all these MAC packets carry video data. An
EV-DO physical layer may enable an access terminal to detect a lost
packet over the reverse link almost instantaneously.
[0069] The MAC layer 30 receives a NAK from a base station through
hybrid ARQ (H-ARQ) channel. The MAC layer 30 tries sending a second
subpacket during time slot n+3. The MAC layer 30 receives a second
NAK from the base station again through H-ARQ channel. The MAC
layer 30 tries sending a third subpacket during time slot n+6. The
MAC layer 30 receives a third NAK from the base station again
through H-ARQ channel. The MAC layer 30 tries sending a fourth
subpacket during time slot n+9. The MAC layer 30 receives a NAK
from a base station through last ARQ (L-ARQ) channel and also
receives a NAK from a base station through packet ARQ (P-ARQ)
channel. This causes the MAC layer 30 to inform the PCP module
210.
[0070] The second approach may allow the video encoder 20 to
quickly recover after the errors, and the user may not see degraded
quality video. The second approach may insert an I-frame right
after the errors to immediately stop error propagation. When there
is no error, the second approach may provide better quality and
less frame skipping due to I-frames than other methods.
[0071] Video encoder device 12 may have a dedicated memory for
storing instructions and data, as well as dedicated hardware,
software, firmware, or combinations thereof. If implemented in
software, the techniques may be embodied as instructions on a
computer-readable medium such as random access memory (RAM),
read-only memory (ROM), non-volatile random access memory (NVRAM),
electrically erasable programmable read-only memory (EEPROM), FLASH
memory, or the like. The instructions cause one or more processors
to perform certain aspects of the functionality described in this
disclosure.
[0072] The techniques described in this disclosure may be
implemented within a general purpose microprocessor, digital signal
processor (DSP), application specific integrated circuit (ASIC),
field programmable gate array (FPGA), or other equivalent logic
devices. For example, video encoder device 12, video decoder device
14, and associated components and modules, may be implemented as
parts of an encoding process, or coding/decoding (CODEC) process,
running on a digital signal processor (DSP) or other processing
device. Accordingly, components described as modules may form
programmable features of such a process, or a separate process.
[0073] Video encoder device 12 may have a dedicated memory for
storing instructions and data, as well as dedicated hardware,
software, firmware, or combinations thereof. If implemented in
software, the techniques may be embodied as instructions executable
by one or more processors. The instructions may be stored on a
computer-readable medium such as random access memory (RAM),
read-only memory (ROM), non-volatile random access memory (NVRAM),
electrically erasable programmable read-only memory (EEPROM), FLASH
memory, magnetic or optical data storage device, or the like. The
instructions cause one or more processors to perform certain
aspects of the functionality described in this disclosure.
[0074] Various embodiments have been described. These and other
embodiments are within the scope of the following claims.
* * * * *