U.S. patent application number 14/219773 was filed with the patent office on 2014-07-24 for system and method for correcting for lost data in a digital audio signal.
This patent application is currently assigned to HUAWEI TECHNOLOGIES CO., LTD.. The applicant listed for this patent is Huawei Technologies Co., Ltd.. Invention is credited to Yang Gao, Lei Miao, Herve Marcel Taddei.
Application Number | 20140207445 14/219773 |
Document ID | / |
Family ID | 43049981 |
Filed Date | 2014-07-24 |
United States Patent
Application |
20140207445 |
Kind Code |
A1 |
Gao; Yang ; et al. |
July 24, 2014 |
System and Method for Correcting for Lost Data in a Digital Audio
Signal
Abstract
In an embodiment, a method of receiving a digital audio signal,
using a processor, includes generating a high band time domain
signal; generating low band time domain signal; estimating an
energy ratio between the high band and the low band from a last
good frame; keeping the energy ratio for following frame-erased
frames by applying an energy correction scaling gain to a high band
signal segment by segment in the time domain; and combining the low
band signal and the high band signal into a final output.
Inventors: |
Gao; Yang; (Mission Viejo,
CA) ; Taddei; Herve Marcel; (Voorburg, NL) ;
Miao; Lei; (Beijing, CN) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Huawei Technologies Co., Ltd. |
Shenzhen |
|
CN |
|
|
Assignee: |
HUAWEI TECHNOLOGIES CO.,
LTD.
Shenzhen
CN
|
Family ID: |
43049981 |
Appl. No.: |
14/219773 |
Filed: |
March 19, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12773668 |
May 4, 2010 |
8718804 |
|
|
14219773 |
|
|
|
|
61175463 |
May 5, 2009 |
|
|
|
Current U.S.
Class: |
704/220 |
Current CPC
Class: |
G10L 19/0017 20130101;
G10L 19/005 20130101 |
Class at
Publication: |
704/220 |
International
Class: |
G10L 19/00 20060101
G10L019/00 |
Claims
1. A method of receiving a digital audio signal using a processor,
generating a high band time domain signal; generating low band time
domain signal; estimating an energy ratio between the high band and
the low band from a last good frame; keeping the energy ratio for
following frame-erased frames by applying an energy correction
scaling gain to a high band signal segment by segment in the time
domain; and combining the low band signal and the high band signal
into a final output.
2. The method of claim 1, wherein the scaling gain is smoothed
sample by sample from one segment to a next segment of the high
band signal.
3. The method of claim 1, wherein the energy ratio is estimated as:
Ratio = E HB E LB = i T env ( i ) s ^ LB ( n ) 2 , ##EQU00019##
where T.sub.env(i) is a temporal energy envelope of a last good
high band signal, E HB = i T env ( i ) ##EQU00020## is a high band
energy, E.sub.LB.parallel.s.sub.LB(n).parallel..sup.2 is a low band
energy.
4. The method of claim 1, further comprising calculating an energy
correction gain factor g.sub.i for an i-th sub-segment of following
erased frames according to: g i = Ratio s ^ LB i ( j ) 2 s ^ HB i (
j ) 2 ##EQU00021## if g.sub.i>1, g.sub.i=1 where
.parallel.s.sub.LB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the low band signal
s.sub.LB.sup.i(j)=s.sub.LB(20i+j) and
.parallel.s.sub.HB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the high band signal
s.sub.HB.sup.i(j)=s.sub.HB(20i+j), and j represents a sample
index.
5. The method of claim 1, further comprising calculating an energy
correction gain factor g.sub.i for an i-th sub-segment of following
erased frames according to: g i = Ratio s ^ LB i ( j ) 2 s ^ HB i (
j ) 2 ##EQU00022## if g.sub.i>1, g.sub.i=1 where
.parallel.s.sub.LB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the low band signal
s.sub.LB.sup.i(j)=s.sub.LB(Li+j) and
.parallel.s.sub.HB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the high band signal
s.sub.HB.sup.i(j)=s.sub.HB(Li+j), L is an integer and j represents
a sample index.
6. The method of claim 1, further comprising calculating a smoothed
correction gain factor g.sub.i(j) and applying the smoothed
correction gain factor g.sub.i(j) to an i-th sub-segment high band
signal s.sub.HB.sup.i(j)=s.sub.HB(20i+j) according to:
g.sub.i(j)0.95 g.sub.i(j-1)+0.05g.sub.i
s.sub.HB.sup.i(j)s.sub.HB.sup.i(j) g.sub.i(j), wherein g.sub.i
comprises an energy correction gain factor for the an i-th
sub-segment, and wherein g.sub.i(j) is smoothed from one segment to
a next segment, sample by sample, and j represents a sample
index.
7. The method of claim 1, further comprising calculating a smoothed
correction gain factor g.sub.i(j) and applying the smoothed
correction gain factor g.sub.i(j) to an i-th sub-segment high band
signal s.sub.HB.sup.i(j)=s.sub.HB(Li+j) according to:
g.sub.i(j).lamda. g.sub.i(j-1)+(1-.lamda.)g.sub.i
s.sub.HB.sup.i(j)s.sub.HB.sup.i(j) g.sub.i(j), wherein g.sub.i
comprises an energy correction gain factor for the an i-th
sub-segment, and wherein g.sub.i(j) is smoothed from one segment to
a next segment, sample by sample, j represents a sample index, L is
an integer and 0.ltoreq..lamda..ltoreq.1.
8. A method of correcting for missing audio data, the method
comprising: copying frequency domain coefficients of a digital
audio signal from a previous frame; using a processor, adaptively
adding random noise coefficients to the copied frequency domain
coefficients; scaling the random noise coefficients and the copied
frequency domain coefficients to form recovered frequency domain
coefficients, wherein scaling is controlled with a parameter
representing a periodicity or harmonicity of a received digital
audio signal; generating a high band time domain signal by
inverse-transforming high band frequency domain coefficients of the
recovered frequency domain coefficients; generating low band time
domain signal; estimating an energy ratio between the high band and
the low band from a last good frame; keeping the energy ratio for
following frame-erased frames by applying an energy correction
scaling gain to a high band signal, segment by segment in the time
domain; and combining the low band signal and the high band signal
to form a final output.
9. The method of claim 8, wherein the frequency domain is a MDCT
domain, DFT domain or FFT domain.
10. The method of claim 8, wherein the parameter representing the
periodicity or harmonicity comprises a voicing factor, a pitch
gain, or a spectral sharpness.
11. The method of claim 8, wherein the processor comprises an audio
decoder in a voice over internet protocol (VOIP) system.
12. A system for receiving a digital audio signal, the system
comprising: a processor; and a computer readable storage medium
storing programming for execution by the processor, the programming
including instructions to generate a high band time domain signal,
generate low band time domain signal, estimate an energy ratio
between the high band and the low band from a last good frame, keep
the energy ratio for following frame-erased frames by applying an
energy correction scaling gain to a high band signal segment by
segment in the time domain, and combine the low band signal and the
high band signal into a final output.
13. The system of claim 12, wherein the scaling gain is smoothed
sample by sample from one segment to a next segment of the high
band signal.
14. The system of claim 12, wherein the energy ratio is estimated
as: Ratio = E HB E LB = i T env ( i ) s ^ LB ( n ) 2 , ##EQU00023##
where T.sub.env(i) is a temporal energy envelope of a last good
high band signal, E HB = i T env ( i ) ##EQU00024## is a high band
energy, E.sub.LB=.parallel.s.sub.LB(n).parallel..sup.2 is a low
band energy.
15. The system of claim 12, wherein the programming further
includes instructions to calculate an energy correction gain factor
g.sub.i for an i-th sub-segment of following erased frames by
determining: g i = Ratio s ^ LB i ( j ) 2 s ^ HB i ( j ) 2
##EQU00025## if g.sub.i>1, g.sub.i=1 where
.parallel.s.sub.LB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the low band signal
s.sub.LB.sup.i(j)=s.sub.LB(20i+j) and
.parallel.s.sub.HB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the high band signal
s.sub.HB.sup.i(j)=s.sub.HB(20i+j), and j represents a sample
index.
16. The system of claim 12, wherein the programming further
includes instructions to calculate an energy correction gain factor
g.sub.i for an i-th sub-segment of following erased frames by
determining: g i = Ratio s ^ LB i ( j ) 2 s ^ HB i ( j ) 2
##EQU00026## if g.sub.i>1, g.sub.i=1 where
.parallel.s.sub.LB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the low band signal
s.sub.LB.sup.i(j)=s.sub.LB(Li+j) and
.parallel.s.sub.HB.sup.i(j).parallel..sup.2 represents energies of
i-th sub-segments of the high band signal
s.sub.HB.sup.i(j)=s.sub.HB(Li+j), L is an integer and j represents
a sample index.
17. The system of claim 12, wherein the programming further
includes instructions to calculate a smoothed correction gain
factor g.sub.i(j) and apply the smoothed correction gain factor
g.sub.i(j) to an i-th sub-segment high band signal
s.sub.HB.sup.i(j)=s.sub.HB(20i+j): g.sub.i(j)0.95
g.sub.i(j-1)+0.05g.sub.i s.sub.HB.sup.i(j)s.sub.HB.sup.i(j)
g.sub.i(j), wherein g.sub.i comprises an energy correction gain
factor for the an i-th sub-segment, and wherein g.sub.i(j) is
smoothed from one segment to a next segment, sample by sample, and
j represents a sample index.
18. The system of claim 12, wherein the programming further
includes instructions to calculate a smoothed correction gain
factor g.sub.i(j) and apply the smoothed correction gain factor
g.sub.i(j) to an i-th sub-segment high band signal
s.sub.HB.sup.i(j)=s.sub.HB(Li+j): g.sub.i(j).lamda.
g.sub.i(j-1)+(1-.lamda.).about.g.sub.i
s.sub.HB.sup.i(j)s.sub.HB.sup.i(j) g.sub.i(j), wherein g.sub.i
comprises an energy correction gain factor for the an i-th
sub-segment, and wherein g.sub.i(j) is smoothed from one segment to
a next segment, sample by sample, j represents a sample index, L is
an integer and 0.ltoreq..lamda..ltoreq.1.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This patent application is a continuation of U.S. patent
application Ser. No. 12/773,668, filed on May 4, 2010, which claims
priority to U.S. Provisional Application No. 61/175,463 filed on
May 5, 2009, entitled "Low Complexity FEC Algorithm for MDCT Based
Codec," which applications are incorporated by reference herein in
their entirety.
TECHNICAL FIELD
[0002] The present invention relates generally to audio signal
coding or compression, and more particularly to a system and method
for correcting for lost data in a digital audio signal.
BACKGROUND
[0003] In modern audio/speech digital signal communication systems,
a digital signal is compressed at an encoder and the compressed
information is packetized and sent to a decoder through a
communication channel, frame by frame, in real time. A system made
of an encoder and decoder together is called a CODEC.
[0004] Most communication channels can not guarantee that all
information packets sent by encoder reaches decoder side in real
time without any loss of data, or without the data being delayed to
the point where it becomes unusable. Generally, the packet loss
rate varies according to the channel quality. In order to
compensate for loss of sound quality due to the packet loss, some
audio decoders implement a Frame Erasure Concealment (FEC)
algorithm, also known as a Packet Loss Concealment (PLC) algorithm.
Different types of decoders usually employ different FEC
algorithms.
[0005] G.729.1 is a scalable codec having multiple layers working
at different bit rates. The lowest core layers of 8 kbps and 12
kbps implement a Code-Excited Linear Prediction (CELP) algorithm.
These two core layers encode and decode a narrowband signal from 0
to 4 kHz. At the bit rate of 14 kbps, a Band-Width Extension (BWE)
algorithm called a Time Domain Band-Width Extension (TDBWE)
encodes/decodes a high band from 4 kHz to 7 kHz by using an extra 2
kbps added to the 12 kbps bit rate to enhance audio quality. BWE
usually includes frequency and time envelope coding and fine
spectral structure generation. Since both frequency and time
envelope coding may take most of the bit budget, fine spectral
structure is often generated by spending very little or no bit
budget. The corresponding signal in time domain of the fine
spectral structure is called excitation. The frequency domain can
be defined in a Modified Discrete Cosine Transform (MDCT), a
Fast-Fourier Transform (FFT) domain, or other domain. The TDBWE
algorithm in G.729.1 is a BWE that generates an excitation signal
in the time domain and applies temporal shaping on the excitation
signal. The time domain excitation signal is then transformed into
the frequency domain with an FFT transformation, and the spectral
envelope is applied in FFT domain.
[0006] In the ITU G.729.1 standard, which is incorporated herein by
reference, at a 16 kbps layer or greater layers, the high frequency
band from 4 kHz to 7 kHz is encoded/decoded with an MDCT algorithm
when no information (bitstream packets) is lost in the channel.
When packet loss occurs, however, the FEC algorithm is based on a
TDBWE algorithm.
[0007] ITU-T Rec. G.729.1 is also called G.729EV, which is an 8-32
kbit/s scalable wideband (50-7000 Hz) extension of ITU-T Rec.
G.729. By default, the encoder input and decoder output are sampled
at 16 kHz. The bitstream produced by the encoder is scalable and
has 12 embedded layers, which will be referred to as Layers 1 to
12. Layer 1 is the core layer corresponding to a bit rate of 8
kbit/s. This layer is compliant with a G.729 bitstream, which makes
G.729EV interoperable with G.729. Layer 2 is a narrowband
enhancement layer adding 4 kbit/s, while Layers 3 to 12 are
wideband enhancement layers adding 20 kbit/s with steps of 2
kbit/s.
[0008] A G.729EV coder operates with a digital signal sampled at 16
kHz in a 16-bit linear pulse code modulated (PCM) format as an
encoder input. However, an 8 kHz input sampling frequency is also
supported. Similarly, the format of the decoder output is 16-bit
linear PCM with a sampling frequency of 8 or 16 kHz. Other
input/output characteristics are converted to 16-bit linear PCM
with 8 or 16 kHz sampling before encoding, or from 16-bit linear
PCM to the appropriate format after decoding.
[0009] The G.729EV coder is built upon a three-stage structure
using embedded CELP coding, TDBWE, and predictive transform coding
that will be referred to as Time-Domain Aliasing Cancellation
(TDAC). A TDAC algorithm can be viewed as specific type of MDCT
algorithm. The embedded CELP stage generates Layers 1 and 2 that
yield a narrowband synthesis (50-4000 Hz) at 8 kbit/s and 12
kbit/s. The TDBWE stage generates Layer 3 and allows the production
of a wideband output (50-7000 Hz) at 14 kbit/s. The TDAC stage
operates in the MDCT domain and generates Layers 4 to 12 to improve
quality from 16 to 32 kbit/s. The TDAC module jointly encodes the
weighted CELP coding error signal in the 50-4000 Hz band and the
input signal in the 4000-7000 Hz band for Layers 4 to 12. The FEC
algorithm for Layers 4 to 12, however, is still based on the TDBWE
algorithm.
[0010] The G.729EV coder operates using 20 ms frames. However, the
embedded CELP coding stage operates on 10 ms frames, like G.729. As
a result two 10 ms CELP frames are processed per 20 ms frame. To be
consistent with the text of ITU-T Rec. G.729, which is incorporated
herein by reference, the 20 ms frames used by G.729EV will be
referred to as superframes, whereas the 10 ms frames and the 5 ms
subframes involved in the CELP processing will be respectively
called frames and subframes.
[0011] As illustrated in FIG. 1, the TDBWE (Layer 3) encoder
extracts a fairly coarse parametric description from the
pre-processed and downsampled higher-band signal 101, s.sub.HB(n).
This parametric description includes time envelope 102 and
frequency envelope 103 parameters. The 20 ms input speech
superframe 101, s.sub.HB(n) is subdivided into 16 segments of
length 1.25 ms each, i.e., where each segment has 10 samples. The
16 time envelope parameters 102, T.sub.env(i), i=0, . . . , 15, are
computed as logarithmic subframe energies:
T env ( i ) = 1 2 log 2 ( 1 / 10 n = 0 9 S HB 2 ( n + i 10 ) ) , i
= 0 , , 15. ( 1 ) ##EQU00001##
[0012] TDBWE parameters T.sub.env (Z), i=0, . . . , 15, are
quantized by mean-removed split vector quantization. First, mean
time envelope 104 is calculated:
M T = 1 16 i = 0 15 T env ( i ) . ( 2 ) ##EQU00002##
[0013] The mean value 104, M.sub.T, is then scalar quantized with 5
bits using uniform 3 dB steps in log domain. This quantization
produces the quantized value 105, {circumflex over (M)}.sub.T. The
quantized mean is then subtracted:
T.sub.env.sup.M(i)=T.sub.env(i)-{circumflex over (M)}.sub.T, i=0, .
. . ,15. (3)
The mean-removed time envelope parameter set is then split into two
vectors of dimension 8:
T.sub.env,1=(T.sub.env.sup.M(0),T.sub.env.sup.M(1).sub.1, . . .
,T.sub.env.sup.M(7)) and
T.sub.env,2=(T.sub.env.sup.M(8),T.sub.env.sup.M(9), . . .
,T.sub.env.sup.M(15)). (4)
[0014] Finally, vector quantization using pre-trained quantization
tables is applied. Note that the vectors T.sub.env,1 and
T.sub.env,2 share the same vector quantization codebooks to reduce
storage requirements. The codebooks (or quantization tables) for
T.sub.env,1/T.sub.env,2 are generated by modifying generalized
Lloyd-Max centroids such that a minimal distance between two
centroids is verified. The codebook modification procedure includes
rounding Lloyd-Max centroids on a rectangular grid with a step size
of 6 dB in log domain.
[0015] For the computation of the 12 frequency envelope parameters
103, F.sub.env(i) j=0, . . . , 11, the signal 101, s.sub.HB(n), is
windowed by a slightly asymmetric analysis window w.sub.F(n). The
maximum of the window w.sub.F(n) is centered on the second 10 ms
frame of the current superframe. The window W.sub.F(n) is
constructed such that the frequency envelope computation has a
lookahead of 16 samples (2 ms) and a lookback of 32 samples (4 ms).
The windowed signal s.sub.HB.sup.w(n) is transformed by FFT.
Finally, the frequency envelope parameter set is calculated as
logarithmic weighted sub-band energies for 12 evenly spaced and
equally wide overlapping sub-bands in the FFT domain. The j-th
sub-band starts at the FFT bin of index 2j and spans a bandwidth of
3 FFT bins.
[0016] FIG. 2 illustrates the concept of the TDBWE decoder module.
The TDBWE received parameters are used to shape artificially
generated excitation signal 202, s.sub.HB.sup.exc), according to
desired time and frequency envelopes 209, {circumflex over
(T)}.sub.env(i), and 209, {circumflex over (F)}.sub.env(j). This
shaping is followed by a time-domain post-processing procedure.
[0017] The quantized parameter set includes the value {circumflex
over (M)}.sub.T and the following vectors: {circumflex over
(T)}.sub.env,1, {circumflex over (T)}.sub.env,2, {circumflex over
(F)}.sub.env,1, {circumflex over (F)}.sub.env,2 and {circumflex
over (F)}.sub.env,3. The split vectors are defined by Equations
(4). The quantized mean time envelope {circumflex over (M)}.sub.T
is used to reconstruct the time envelope and the frequency envelope
parameters from the individual vector components, i.e.:
{circumflex over (T)}.sub.env(i)={circumflex over
(T)}.sub.env.sup.M(i)+{circumflex over (M)}.sub.T, i=0, . . . ,15
(5)
and
{circumflex over (F)}.sub.env(i)={circumflex over
(F)}.sub.env.sup.M(i)+{circumflex over (M)}.sub.T, j=0, . . . ,11.
(6)
[0018] TDBWE excitation signal 201, exc(n), is generated by a 5 ms
subframe based on parameters that are transmitted in Layers 1 and 2
of the bitstream. Specifically, the following parameters are used:
the integer pitch lag T.sub.0=int(T.sub.1) or int(T.sub.2)
depending on the subframe, the fractional pitch lag frac, the
energy of the fixed codebook contributions
E c = n = 0 39 ( g ^ c c ( n ) + g ^ enh c ' ( n ) ) 2 ,
##EQU00003##
and the energy of the adaptive codebook contribution
E p = n = 0 39 ( g ^ p v ( n ) ) 2 . ##EQU00004##
The parameters of the excitation generation are computed for every
5 ms subframe. The excitation signal generation includes the
following steps: [0019] estimation of two gains g.sub.v and
g.sub.uv for the voiced and unvoiced contributions to the final
excitation signal 201, exc(n); [0020] pitch lag post-processing;
[0021] generation of the voiced contribution; [0022] generation of
the unvoiced contribution; and [0023] low-pass filtering.
[0024] The shaping of the time envelope of the excitation signal
202, s.sub.HB.sup.exc(n), utilizes decoded time envelope parameters
208, {circumflex over (T)}.sub.env(i), with i=0, . . . , 15 to
obtain a signal 203, s.sub.HB.sup.T(n), with a time envelope that
is nearly identical to the time envelope of the encoder side
higher-band signal 101, S.sub.HB(n). This is achieved by scalar
multiplication:
s.sub.HB.sup.T(n)=g.sub.T(n)s.sub.HB.sup.exc(n), n=0, . . . ,159.
(7)
[0025] In order to determine the gain function g.sub.T(n), the
excitation signal 202, S.sub.HB.sup.exc(n), is segmented and
analyzed in the same manner as the parameter extraction in the
encoder. The obtained analysis results are, again, time envelope
parameters {tilde over (T)}.sub.env(i) with i=0, . . . , 15. They
describe the observed time envelope of S.sub.HB.sup.exc(n). Then a
preliminary gain factor is calculated:
g'.sub.T(i)=2.sup.{circumflex over (T)}.sup.env.sup.(i)-{tilde over
(T)}.sup.env.sup.(i), i=0, . . . ,15 (8)
[0026] For each signal segment with index i=0, . . . , 15, these
gain factors are interpolated using a "flat-top" Hanning window
w.sub.t( ). This interpolation procedure finally yields the gain
function:
g T ( n + i 10 ) = { w t ( n ) g T ' ( i ) + w t ( n + 10 ) g T ' (
i - 1 ) n = 0 , , 4 w t ( n ) g T ' ( i ) n = 5 , , 9 , ( 9 )
##EQU00005##
where g'.sub.T(-1) is defined as the memorized gain factor
g'.sub.T(15) from the last 1.25 ms segment of the preceding
superframe.
[0027] Signal 204, s.sub.HB.sup.F(n), is obtained by shaping the
excitation signal s.sub.HB.sup.exc(n) (generated from parameters
estimated in lower-band by the CELP decoder) according to the
desired time and frequency envelopes. Generally, there is no
coupling between this excitation and the related envelope shapes
{circumflex over (T)}.sub.env(i) and {circumflex over
(F)}.sub.env(j). As a result, some clicks may be present in the
signal s.sub.HB.sup.F(n). To attenuate these artifacts, an adaptive
amplitude compression is applied to s.sub.HB.sup.F(n). Each sample
of s.sub.HB.sup.F(n) of the i-th 1.25 ms segment is compared to the
decoded time envelope {circumflex over (T)}.sub.env(i) and the
amplitude of s.sub.HB.sup.F(n) are compressed in order to attenuate
large deviations from this envelope. The TDBWE synthesis 205,
s.sub.HB.sup.bwe(n), is transformed to s.sub.HB.sup.bwe(k) by MDCT.
This spectrum is used by the TDAC decoder to extrapolate missing
sub-bands.
[0028] In case of packet loss, the G.729.1 decoder employs the
TDBWE algorithm to compensate for the HB part by estimating the
current spectral envelope and the temporal envelope using
information from the previous frame. The excitation signal is still
constructed by extracting information from the low band
(Narrowband) CELP parameters. As can be seen from the above
description, such an FEC process is quite complicated.
[0029] As mentioned above, G.729.1 employs a TDAC/MDCT based codec
algorithm to encode and decode the high band part for bit-rate
higher than 14 kbps. The TDAC encoder illustrated in FIG. 3 jointly
represents jointly two split MDCT spectra 301, D.sub.LB.sup.w(k),
and 302, S.sub.HB(k), by gain-shape vector quantization. Joint
spectrum 303, Y(k), is divided into sub-bands, where each sub-band
defines the spectral envelope. The sub-bands are represented in the
log domain by 304, log_rms(j). After quantization, the spectral
envelope is represented by the index 305, rms_index (j). The
spectral envelope information is also used to allocate a proper
number of bits 306, nbit(j), for each subband to code the MDCT
coefficients. The shape of each sub-band coefficients is encoded by
embedded spherical vector quantization using trained permutation
codes.
[0030] Lower-band CELP weighted error signal d.sub.LB.sup.w(n) and
higher-band signal s.sub.HB(n) are transformed into frequency
domain by MDCT with a superframe length of 20 ms and a window
length of 40 ms. D.sub.LB.sup.w(k) represents the MDCT coefficients
of the windowed signal d.sub.LB.sup.w(n) with 40 ms sinusoidal
windowing. MDCT coefficients, Y (k), in the 0-7000 Hz band are
split into 18 sub-bands. The j-th sub-band comprises nb_coef(j)
coefficients Y(k) with sb_bound(j).ltoreq.k<sb_bound (j+1). Each
subband of the first 17 sub-bands includes 16 coefficients (400 Hz
bandwidth), and the last sub-band includes 8 coefficients (200 Hz
bandwidth). The spectral envelope is defined as the root mean
square (rms) in log domain of the 18 sub-bands, which is then
quantized in encoder.
[0031] The perceptual importance 307, ip(j), j=0 . . . 17, of each
sub-band is defined as:
ip ( j ) = 1 2 log 2 ( rms_q ( j ) 2 .times. nb_coef ( j ) ) +
offset , ( 10 ) ##EQU00006##
where rms_q(j)=2.sup.1/2 rms.sup.--.sup.index(j) is the quantized
rms and rms_q(j).sup.2.times.nb_coef(j) corresponds to the
quantized sub-band energy. Consequently the perceptual importance
is equivalent to the sub-band log-energy. This information is
related to the quantized spectral envelope as follows:
ip ( j ) = 1 2 [ rms_index ( j ) + log 2 ( nb_coef ( j ) ) ] +
offset . ( 11 ) ##EQU00007##
[0032] The offset value is introduced to simplify further the
expression of ip(j). The sub-bands are then sorted by decreasing
perceptual importance. This perceptual importance ordering is used
for bit allocation and multiplexing of vector quantization
indices.
[0033] Each sub-band j=0, . . . , 17 of dimension nb_coef(j) is
encoded with nbit(j) bits by spherical vector quantization. This
operation is divided into two steps: search for a best code vector
and indexing of the selected code vector.
[0034] The bits associated with the HB spectral envelope coding are
multiplexed before the bits associated with the lower-band spectral
envelope coding. Furthermore, sub-band quantization indices are
multiplexed by order of decreasing perceptual importance. The
sub-bands that are perceptually more important (i.e., with the
largest perceptual importance ip(j)) are written first in the
bitstream. As a result, if just part of the coded spectral envelope
is received at the decoder, the higher-band envelope can be decoded
before that of the lower band. This property is used at the TDAC
decoder to perform a partial level-adjustment of the higher-band
MDCT spectrum.
[0035] The TDAC decoder pertaining to layers 4 to 12 is depicted in
FIG. 4. Received normalization factor (called norm_MDCT)
transmitted by the encoder with 4 bits is used in the TDAC decoder
to normalize MDCT coefficients 401, .sup.norm(k). The factor is
used to scale the signal reconstructed by two inverse MDCTs. The
higher-band spectral envelope 407, rms_q(j), is decoded first, then
index rms_index(j), j=11, . . . , 17, is reconstructed. If the
number of bits is insufficient to decode the higher-band spectral
envelope completely, decoded indices rms_index(j) are kept to allow
partial level-adjustment of the decoded HB spectrum. The bits
related to the lower band, i.e. rms_index(j), j=0, . . . , 9, are
decoded in a similar way as in the higher band. The decoded indices
are combined into a single vector [rms_index(0) rms_index(1) . . .
rms_index(17)], which represents the reconstructed spectral
envelope in log domain. The vector quantization indices are read
from the TDAC bitstream according to their perceptual importance
ip(j).
[0036] In sub-band j of dimension nb_coef(j) and non-zero bit
allocation nbit(j), the vector quantization index identifies a code
vector which constructs the sub-band j of .sup.norm(k). The missing
subbands are filled by the generated coefficients 408 from the
transform of the TDBWE signal. After filling the missing subbands,
the complete set of MDCT coefficients are named as 402,
.sup.ext(k), which will be subject to level adjustment by using the
spectral envelope information. Level-adjusted coefficients 403,
(k), are the input to the post-processing module. The
post-processing of MDCT coefficients is only applied to the higher
band, because the lower band is post-processed with a traditional
time-domain approach. For the high-band, there are no Linear
Prediction Coding (LPC) coefficients transmitted to the decoder.
The TDAC post-processing is performed on the available MDCT
coefficients at the decoder side. Reconstructed spectrum 404,
.sup.post (k), is split into a lower-band spectrum 406, {circumflex
over (D)}.sub.LB.sup.w(k), and a higher-band spectrum 405,
S.sub.HB(k). Both bands are transformed to the time domain using
inverse MDCT transforms.
[0037] Narrowband (NB) signal encoding is mainly contributed by the
CELP algorithm, and its concealment strategy is disclosed the ITU
G7.29.1 standard. Here, the concealment strategy includes replacing
the parameters of the erased frame based on the parameters from
past frames and the transmitted extra FEC parameters. Erased frames
are synthesized while controlling the energy. This concealment
strategy depends on the class of the erased superframe, and makes
use of other transmitted parameters that include phase information
and gain information.
SUMMARY OF THE INVENTION
[0038] In an embodiment, a method of receiving a digital audio
signal, using a processor, includes correcting the digital audio
signal from lost data. Correcting includes copying frequency domain
coefficients of the digital audio signal from a previous frame,
adaptively adding random noise coefficients to the copied frequency
domain coefficients, and scaling the random noise coefficients and
the copied frequency domain coefficients to form recovered
frequency domain coefficients. Scaling is controlled with a
parameter representing a periodicity or harmonicity of the digital
audio signal.
[0039] In another embodiment, a method of receiving a digital audio
signal using a processor, includes generating a high band time
domain signal, generating low band time domain signal, estimating
an energy ratio between the high band and the low band from a last
good frame, keeping the energy ratio for following frame-erased
frames by applying an energy correction scaling gain to a high band
signal segment by segment in the time domain, combining the low
band signal and the high band signal into a final output.
[0040] In a further embodiment, a method of correcting for missing
audio data includes copying frequency domain coefficients of the
digital audio signal from a previous frame, adaptively adding
random noise coefficients to the copied frequency domain
coefficients, scaling the random noise coefficients and the copied
frequency domain coefficients to form recovered frequency domain
coefficients. Scaling is controlled with a parameter representing a
periodicity or harmonicity of the digital audio signal. The method
also includes generating a high band time domain signal by
inverse-transforming high band frequency domain coefficients of the
recovered frequency domain coefficients, generating low band time
domain signal and estimating an energy ratio between the high band
and the low band from a last good frame. The method further
includes keeping the energy ratio for following frame-erased frames
by applying an energy correction scaling gain to a high band
signal, segment by segment in the time domain and combining the low
band signal and the high band signal to form a final output.
[0041] In a further embodiment, a system for receiving a digital
audio signal includes an audio decoder configured to copy frequency
domain coefficients of the digital audio signal from a previous
frame, adaptively add random noise coefficients to the copied
coefficients, and scale the random noise coefficients and the
copied frequency domain coefficients to form recovered frequency
domain coefficients. In an embodiment, scaling is controlled with a
parameter representing a periodicity or harmonicity of the digital
audio signal. The audio decoder is also configured to produce a
corrected audio signal from the recovered frequency domain
coefficients.
[0042] The foregoing has outlined, rather broadly, features of the
present invention. Additional features of the invention will be
described, hereinafter, which form the subject of the claims of the
invention. It should be appreciated by those skilled in the art
that the conception and specific embodiment disclosed may be
readily utilized as a basis for modifying or designing other
structures or processes for carrying out the same purposes of the
present invention. It should also be realized by those skilled in
the art that such equivalent constructions do not depart from the
spirit and scope of the invention as set forth in the appended
claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0043] For a more complete understanding of the present invention,
and the advantages thereof, reference is now made to the following
descriptions taken in conjunction with the accompanying drawing, in
which:
[0044] FIG. 1 illustrates a high-level block diagram of a G.729.1
TDBWE encoder;
[0045] FIG. 2 illustrates high-level block diagram of a G.729.1
TDBWE decoder;
[0046] FIG. 3 illustrates a high-level block diagram of a G.729.1
TDAC encoder;
[0047] FIG. 4 illustrates high-level block diagram of a G.729.1
TDAC decoder;
[0048] FIG. 5 illustrates an embodiment FEC algorithm in the
frequency domain;
[0049] FIG. 6 illustrates a block diagram an embodiment time domain
energy correction for FEC; and
[0050] FIG. 7 illustrates an embodiment communication system.
[0051] Corresponding numerals and symbols in different figures
generally refer to corresponding parts unless otherwise indicated.
The figures are drawn to clearly illustrate the relevant aspects of
embodiments of the present invention and are not necessarily drawn
to scale. To more clearly illustrate certain embodiments, a letter
indicating variations of the same structure, material, or process
step may follow a figure number.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
[0052] The making and using of the presently preferred embodiments
are discussed in detail below. It should be appreciated, however,
that the present invention provides many applicable inventive
concepts that can be embodied in a wide variety of specific
contexts. The specific embodiments discussed are merely
illustrative of specific ways to make and use the invention, and do
not limit the scope of the invention.
[0053] The present invention will be described with respect to
embodiments in a specific context, namely a system and method for
performing audio decoding for telecommunication systems.
Embodiments of this invention may also be applied to systems and
methods that utilize speech and audio transform coding.
[0054] In an embodiment, a FEC algorithm generates current MDCT
coefficients by combining old MDCT coefficients from previous frame
with adaptively added random noise. The copied MDCT component from
a previous frame and the added noise component are adaptively
scaled by using scaling factors which are controlled with a
parameter representing periodicity or harmonicity of signal. In the
time domain, the high band signal is obtained by an inverse MDCT
transformation of the generated MDCT coefficients, and is
adaptively scaled segment by segment while maintaining the energy
ratio between the high band and low band signals.
[0055] In the G.729.1 standard, even though the output signal may
be sampled at a 16 kHz sampling rate, the bandwidth is limited to 7
kHz, and the energy from 7 kHz to 8 kHz is set to zero. Recently,
the ITU-T has standardized a scalable extension of G.729.1 (having
G.729.1 as core), called here G.729.1 super-wideband extension. The
extended standard encodes/decodes a superwideband signal between 50
Hz and 14 kHz with a sampling rate of 32 kHz for the input/output
signal. In this case, the superwideband spectrum is divided into 3
bands. The first band from 0 to 4 kHz is called the Narrow Band (NB
or low band, the second band from 4 kHz to 7 kHz is called the Wide
Band (WB) or high band (HB), and the spectrum above 7 kHz is called
the superwideband (SWB) or super high band. The definitions of
these names may vary from application to application. Typically,
FEC algorithms for each band are different. Without losing the
generality, the example embodiments are directed toward the second
band (WB) high band area. Alternatively, embodiment algorithms can
be directed toward the first band, third band, or toward other
systems.
[0056] This section describes an embodiment modification of FEC in
the 4 kHz-7 kHz band for G.729.1 when the output sampling rate is
at 32 kHz. As mentioned hereinabove, one of the functions of TDBWE
algorithm in G.729.1 is to perform frame erasure concealment (FEC)
of the high band (4 kHz-7 kHz) not only for the 14 kbps layer, but
also for higher layers, although the layers higher than 14 kbps are
coded with a MDCT based codec algorithm in a no-FEC condition. Some
embodiment algorithms exploit the characteristics of MDCT based
codec algorithm to achieve a simpler FEC algorithm for those layers
higher than 14 kbps. Some embodiment FEC algorithms re-generates
non received MDCT coefficients of a given frame by using the MDCT
coefficients of the previous frame to which some random
coefficients are added in an adaptive fashion. In time domain, the
signal obtained by applying an inverse MDCT transform of the
generated MDCT coefficients is adaptively scaled, segment by
segment, while maintaining the energy ratio between the high band
and low band signals.
[0057] Some embodiment FEC algorithms generate MDCT domain
coefficients and correct temporal energy shape of the signal in
time domain in case of packet loss. In other embodiments, the
generation of MDCT coefficients and the correction of the signal
time domain shape can work separately. For example, in one
embodiment, the correction of signal time domain shape is applied
to a signal that is not generated using embodiment algorithms.
Further more, in other embodiments, the generation of MDCT
coefficients works independently on any frequency band without
considering the relationship with other frequency bands.
[0058] The TDBWE in G.729.1 has three functions: (1) producing the
layer of 14 kbps; (2) filling 0 bit subbands; and (3) performing
FEC for rates>=16 kbps. Some embodiments of the current
invention are adapted to replace the third function of the TDBWE in
the G.729.1 standard for super-wideband extension for rates greater
than or equal to 32 kbps at a sampling rate of 32 kHz. In some
embodiments, under the of rates greater than or equal to 32 kbps at
a sampling rate of 32 kHz, the layer of 14 kbps is not used, and
the second function of TDBWE is replaced with a simpler embodiment
algorithm, and the third function of TDBWE is also replaced with an
embodiment algorithm. The FEC algorithm of the high band of 4 kHz
to 7 kHz for rates greater than or equal to 32 kbps at the sampling
rate of 32 kHz exploits the characteristics of the MDCT based codec
algorithm.
[0059] In an embodiment, a FEC algorithm has two main functions:
generating MDCT domain coefficients and correcting the temporal
energy shape of the high band signal in the time domain, in case of
packet loss. The details of the two main functions are described as
follows:
[0060] With respect to the estimation of MDCT domain coefficients
in the case of packet loss, a simple solution is to copy the MDCT
domain coefficients from previous frame to current frame. However,
such a simple repetition of previous MDCT coefficients may cause
unnatural sound or too much periodicity (too high harmonicity) in
some situations. In an embodiment, in order to control the signal
periodicity and the sound naturalness, random noise components are
adaptively added to the copied MDCT coefficients (see FIG. 5):
S.sub.HB(k)=g.sub.1S.sub.HB.sup.old(k)+g.sub.2N(k), (12)
where S.sub.HB.sup.old(k) are copied MDCT coefficients 501 of the
high band [4-7 kHz] from previous frame, and all the MDCT
coefficients in the 7 kHz to 8 kHz band are set to zero in terms of
the codec definition; N(k) are random noise coefficients 502, the
energy of which is initially normalized to S.sub.HB.sup.old(k) in
each subband. In an embodiment, every 20 MDCT coefficients are
defined as one subband, resulting in 8 subbands from 4 kHz to 8
kHz. The last 2 subbands of the 7 kHz to 8 kHz band are set to
zero. In alternative embodiments, more than or less than 20 MDCT
coefficients can be defines as a subband. In Equation (12), g1 and
g.sub.2 are two gains estimated to control the energy ratio between
S.sub.HB.sup.old(k) and N(k) while maintaining an appropriate total
energy reduction compared to the previous frame during the FEC. If
G.sub.p, 0.ltoreq. G.sub.p.ltoreq.1 is a parameter defined to
measure the signal periodicity, G.sub.p=0 means no periodicity and
G.sub.p=1 represents full periodicity; g.sub.1 and g.sub.2 are
defined as follows:
g.sub.1=g.sub.r G.sub.p; and (13)
g.sub.2=g.sub.r(1- G.sub.p). (14)
Here, g.sub.r=0.9 is a gain reduction factor in MDCT domain to
maintain the energy of current frame lower than the one of previous
frame. In alternative embodiments g.sub.r can take on other values.
In some embodiments, aggressive energy control is not applied at
this stage and the temporal energy shape is corrected later in the
time domain. G.sub.p is the last smoothed voicing factor which is
expressed as G.sub.p0.75 G.sub.p+0.25 G.sub.p from one received
subframe to next received subframe. In some embodiments, G.sub.p is
expressed generally as G.sub.p.beta. G.sub.p+(1-.beta.)G.sub.p,
where .beta. is between 0 and 1. G.sub.p is based on the received
subframe and expressed as:
G p = E p E p + E c ( 15 ) ##EQU00008##
During FEC frames, G.sub.p is reduced by a factor 0.75 from current
to next frame: G.sub.p0.75 G.sub.p so that the periodicity keeps
decreasing when more consecutive FEC frames occur in embodiments.
In alternative embodiments, G.sub.p is reduced by a factor other
than 0.75. In equation (15), E.sub.p is the energy of the adaptive
codebook excitation component and E.sub.c is the energy of the
fixed codebook excitation component.
[0061] In an embodiment, another way of estimating the periodicity
is to define a pitch gain or a normalized pitch gain:
g p = n s ^ ( n ) s ^ ( n + T ) [ n s ^ ( n ) s ^ ( n ) ] [ n s ^ (
n + T ) s ^ ( n + T ) ] , ( 16 ) ##EQU00009##
where T is a pitch lag from last received frame for CELP algorithm,
s(n) is time domain signal which sometimes could be defined in
weighted signal domain or LPC residual domain, and g.sub.p is used
to replace G.sub.p.
[0062] In the case of music signals that have no available CELP
parameters, a frequency domain harmonic measure or a spectral
sharpness measure is used as a parameter to replace G.sub.p in
equations (13) and (14) in some embodiments. For example, the
spectral sharpness for one subband can be defined as the average
magnitude divided by the maximum magnitude:
Sharp = 1 N k S ^ HB ( k ) Max { S ^ HB ( k ) , k = 0 , 1 , , N } .
( 17 ) ##EQU00010##
Based on the definition in equations (17), a smaller value of Sharp
means a sharper spectrum or more harmonics in the spectral domain.
In most cases, however, a higher harmonic spectrum also means a
higher periodic signal. In an embodiment, the parameter of equation
(17) is mapped to another parameter varying from 0 to 1 before
replacing G.sub.p.
[0063] In an embodiment, after the generated MDCT coefficients 503,
S.sub.HB(k), are determined, they are inverse-transformed into the
time domain. During the inverse transformation, the contribution
under current MDCT window is interpolated with the one from a
previous MDCT window to get the estimated high band signal 504,
s.sub.HB(n).
[0064] With respect to time domain control of FEC based on the
energy ratio between the high band and the low band, FIG. 6
summarizes an embodiment time domain energy correction in case of
FEC. The low band and high band time domain synthesis signals are
noted as s.sub.LB(n) and s.sub.HB(n) respectively, and are sampled
at an 8 kHz sampling rate. In the case of an error free condition,
s.sub.LB(n) is a combination of CELP output and MDCT enhancement
layer output: s.sub.LB(n)=S.sub.LB.sup.celp(n)+{circumflex over
(d)}.sub.LB.sup.echo (n), and the MDCT enhancement layer time
domain output is the inverse MDCT transformation of {circumflex
over (D)}.sub.LB.sup.w(k). In some embodiments, the contribution of
the CELP output s.sub.LB.sup.celp(n) is normally dominant, and
s.sub.HB(n) is obtained by performing an inverse MDCT
transformation of S.sub.HB(k). The final output signal sampled at
16 kHz, S.sub.WB(n), is computed by upsampling both s.sub.LB(n) and
s.sub.HB(n), and by filtering the up-sampled signals with a
quadrature mirror filter (QMF) synthesis filter bank.
[0065] Because the time domain signal s.sub.HB(n) is obtained by
performing the inverse MDCT transformation of S.sub.HB(k),
s.sub.HB(n) has just one frame delay compared to the latest
received CELP frame or TDBWE frame in time domain, the correct
temporal envelope shape for the first FEC frame of s.sub.HB(n) can
be still obtained from the latest received TDBWE parameters. In an
embodiment, to evaluate the temporal energy envelope, one 20 ms
frame is divided into 8 small sub-segments of 2.5 ms, and the
temporal energy envelope noted as T.sub.env(i), i=0, 1, . . . 7,
represents the energy of each sub-segment. For the first FEC frame
of s.sub.HB(n), T.sub.env (i) is obtained by decoding the latest
received TDBWE parameters, and the corresponding low band CELP
output s.sub.LB.sup.celp(n) is still correct by decoding the latest
received CELP parameters. However, the contribution a {circumflex
over (d)}.sub.LB.sup.echo(n) from the MDCT enhancement layer is
only partially correct and is diminished to zero from the first FEC
frame to the second FEC frame. Here, CELP encodes/decodes frame by
frame, however, MDCT over-lap-adds a moving window of two frames,
so that the result of the current frame is the combination of the
previous frame and the current frame.
[0066] For the second FEC frame of s.sub.HB(n) and the following
FEC frames, the G.729.1 decoder already provides an FEC algorithm
to recover the corresponding low band output 605, s.sub.LB(n). High
band signal s.sub.HB(n) is first estimated by performing an inverse
MDCT transform of S.sub.HB(k) which is expressed in Equation (12).
Due to the fact that s.sub.LB(n) and s.sub.HB(n) are respectively
estimated in different paths with different methods, their relative
energy relationship may not be perceptually the best. While this
relative energy relationship is important from perceptual point of
view, the energy of s.sub.HB(n) could be too low or too high in the
time domain, compared to the energy of s.sub.LB(n). In an
embodiment, one way to address this issue is first to get the
energy ratio between 608, s.sub.LB(n), and 607, s.sub.HB(n), from
the last received frame or the first FEC frame of s.sub.HB(n), and
then keep this energy ratio for the following FEC frames.
[0067] In an embodiment, as the inverse MDCT transformation causes
one frame delay, an estimation of the energy ratio between the low
band signal and the high band signal is calculated during the first
FEC frame of s.sub.HB(n). The low band energy is from the low band
signal obtained from the G.729.1 decoder, and the high band energy
is the sum of the s.sub.LB(n) temporal energy envelope T.sub.env(i)
parameters evaluated from the latest received TDBWE parameters.
Energy ratio 601 is defined as
Ratio = E HB E LB = i T env ( i ) s ^ LB ( n ) 2 . ( 16 )
##EQU00011##
Equation (16) represents the average energy ratio for the whole
time domain frame.
[0068] In an embodiment, for the first FEC frame of s.sub.HB(n),
the temporal energy envelope T.sub.env(i) is directly applied by
multiplying each high band sub-segment 602,
s.sub.HB.sup.i(j)=s.sub.HB(20i+j), with a gain factor
g.sub.f(i):
g f ( i ) = 0.9 T env ( i ) j = 0 20 s ^ HB ( i 20 + j ) 2 , i = 0
, 1 , 7. ( 17 ) ##EQU00012##
the above gain factor is further smoothed sample by sample during
the gain factor multiplication:
g.sub.f(j)0.95 g.sub.f(j-1)+0.05g.sub.f(i); and (18)
s.sub.HB(i20+j).sub.HB(i20+j) g.sub.f(j) (19)
In equations (17), (18), and (19), i is sub-segment index and j is
sample index. It should be noted that in alternative embodiments,
the multiplying constant of 0.9 take on other values, more than or
less then 20 samples can be used in equation (17). In further
embodiments, g.sub.f(j) can be expressed generally as
g.sub.f(j).lamda. g.sub.f(j-1)+(1-.lamda.)g.sub.f(i),
0.ltoreq..lamda..ltoreq.1 and s.sub.HB(iL+j)s.sub.HB(iL+j)
g.sub.f(j), where L is an integer.
[0069] In an embodiment, for the second FEC frame of s.sub.HB(n),
and for the following FEC frames, each frame is also divided into 8
small sub-segments. The energy ratio correction is performed on
each small sub-segment. The energy correction gain factor g.sub.i
for i-th sub-segment is calculated in the following way:
g i = Ratio s ^ LB i ( j ) 2 s ^ HB i ( j ) 2 if g i > 1 , g i =
1. ( 20 ) ##EQU00013##
In Equation (20), .parallel.s.sub.LB.sup.i(j).parallel..sup.2 and
.parallel.s.sub.HB(j).parallel..sup.2 represent respectively the
energies of the i-th sub-segments of the low band signal 603,
s.sub.LB.sup.i(j)=s.sub.LB(20i+j), and the high band signal 602,
s.sub.HB.sup.i(j)=s.sub.HB(20i+j). The correction gain defined in
equation (20) is finally applied to the i-th sub-segment
s.sub.HB.sup.i(j) while smoothing the gain from one segment to next
segment, sample by sample:
g.sub.i(j)0.95 g.sub.i(j-1)+0.05g.sub.i; and (21)
s.sub.HB.sup.i(j)s.sub.HB.sup.i(j) g.sub.i(j) (22)
[0070] In a final step, the energy corrected high band signal 604,
s.sub.HB(n), and the low band signal 605, s.sub.LB(n), are
upsampled and filtered with a QMF filter bank to form the final
wideband output signal 606, s.sub.WB(n). It should be noted that in
alternative embodiments, g.sub.i(j) can be expressed generally
as
g.sub.i(j).lamda..sub.2 g.sub.i(j-1)+(1-.lamda..sub.2)g.sub.i,
0.ltoreq..lamda..sub.2.ltoreq.1, and
s.sub.HB(iL.sub.2+j)s.sub.HB(iL.sub.2+j) g.sub.i(j).
where L.sub.2 is an integer; normally, .lamda..sub.2=.lamda. and
L.sub.2=L, however, in some embodiments,
.lamda..sub.2.noteq..lamda. and/or L.sub.2.noteq.L.
[0071] FIG. 7 illustrates communication system 10 according to an
embodiment of the present invention. Communication system 10 has
audio access devices 6 and 8 coupled to network 36 via
communication links 38 and 40. In one embodiment, audio access
device 6 and 8 are voice over internet protocol (VoIP) devices and
network 36 is a wide area network (WAN), public switched telephone
network (PSTN) and/or the internet. Communication links 38 and 40
are wireline and/or wireless broadband connections. In an
alternative embodiment, audio access devices 6 and 8 are cellular
or mobile telephones, links 38 and 40 are wireless mobile telephone
channels and network 36 represents a mobile telephone network.
[0072] Audio access device 6 uses microphone 12 to convert sound,
such as music or a person's voice into analog audio input signal
28. Microphone interface 16 converts analog audio input signal 28
into digital audio signal 32 for input into encoder 22 of CODEC 20.
Encoder 22 produces encoded audio signal TX for transmission to
network 26 via network interface 26 according to embodiments of the
present invention. Decoder 24 within CODEC 20 receives encoded
audio signal RX from network 36 via network interface 26, and
converts encoded audio signal RX into digital audio signal 34.
Speaker interface 18 converts digital audio signal 34 into audio
signal 30 suitable for driving loudspeaker 14.
[0073] In embodiments of the present invention, where audio access
device 6 is a VoIP device, some or all of the components within
audio access device 6 are implemented within a handset. In some
embodiments, however, Microphone 12 and loudspeaker 14 are separate
units, and microphone interface 16, speaker interface 18, CODEC 20
and network interface 26 are implemented within a personal
computer. CODEC 20 can be implemented in either software running on
a computer or a dedicated processor, or by dedicated hardware, for
example, on an application specific integrated circuit (ASIC).
Microphone interface 16 is implemented by an analog-to-digital
(A/D) converter, as well as other interface circuitry located
within the handset and/or within the computer. Likewise, speaker
interface 18 is implemented by a digital-to-analog converter and
other interface circuitry located within the handset and/or within
the computer. In further embodiments, audio access device 6 can be
implemented and partitioned in other ways known in the art.
[0074] In embodiments of the present invention where audio access
device 6 is a cellular or mobile telephone, the elements within
audio access device 6 are implemented within a cellular handset.
CODEC 20 is implemented by software running on a processor within
the handset or by dedicated hardware. In further embodiments of the
present invention, audio access device may be implemented in other
devices such as peer-to-peer wireline and wireless digital
communication systems, such as intercoms, and radio handsets. In
applications such as consumer audio devices, audio access device
may contain a CODEC with only encoder 22 or decoder 24, for
example, in a digital microphone system or music playback device.
In other embodiments of the present invention, CODEC 20 can be used
without microphone 12 and speaker 14, for example, in cellular base
stations that access the PTSN.
[0075] In some embodiments of the present invention, embodiment
algorithms are implemented by CODEC 20. In further embodiments,
however, embodiment algorithms can be implemented using general
purpose processors, application specific integrated circuits,
general purpose integrated circuits, or a computer running
software.
[0076] In an embodiment, a method of receiving an audio signal
using a low complexity and high quality FEC or PLC includes copying
frequency domain coefficients from previous frame, adaptively
adding random noise to the copied coefficients, scaling the random
noise component and the copied component, wherein the scaling is
controlled with a parameter representing the periodicity or
harmonicity of the audio. In an embodiment, the frequency domain
can be represented, for example in the MDCT, DFT, or FFT domain. In
further embodiments, discrete frequency domains can be used. In an
embodiment, the parameter representing the periodicity or
harmonicity can be a voicing factor, pitch gain, or spectral
sharpness variable.
[0077] In an embodiment the recovered frequency domain (MDCT
domain) coefficients are expressed as,
s.sub.HB(k)=g.sub.1s.sub.HB.sup.old(k)+g.sub.2N(k),
where S.sub.HB.sup.old(k) are copied MDCT coefficients from
previous frame; N(k) are random noise coefficients, the energy of
which is initially normalized to S.sub.HB.sup.old(k) in each
subband, and g1 and g2 are adaptive controlling gains.
[0078] In a further embodiment, g.sub.1 and g.sub.2 are defined
as:
g.sub.1=g.sub.r G.sub.p, and
g.sub.2=g.sub.r(1- G.sub.p),
where g.sub.r=0.9 is a gain reduction factor in MDCT domain to
maintain the energy of current frame lower than the one of previous
frame, G.sub.p is the last smoothed voicing factor which represents
the periodicity or harmonicity, and G.sub.p is smoothed as
G.sub.p0.75 G.sub.p+0.25 G.sub.p from one received subframe to next
received subframe. During FEC frames, G.sub.p is reduced by a
factor 0.75 from current to next frame: G.sub.p0.75 G.sub.p so that
the periodicity keeps decreasing when more consecutive FEC frames
occur.
[0079] In an embodiment, G.sub.p has the definition from received
subframe:
G p = E p E p + E c , ##EQU00014##
where E.sub.p is the energy of the CELP adaptive codebook
excitation component and E.sub.c is the energy of the CELP fixed
codebook excitation component.
[0080] In some embodiments, wherein G.sub.p can be replaced by a
pitch gain or a normalized pitch gain:
g p = n s ^ ( n ) s ^ ( n + T ) [ n s ^ ( n ) s ^ ( n ) ] [ n s ^ (
n + T ) s ^ ( n + T ) ] , ##EQU00015##
where T is a pitch lag from last received frame for CELP algorithm,
s(n) is time domain signal which sometimes can be defined in
weighted signal domain or LPC residual domain.
[0081] In other embodiments, wherein G.sub.p can be replaced by the
spectral sharpness defined as the average frequency magnitude
divided by the maximum frequency magnitude:
Sharp = 1 N S ^ HB ( k ) Max { S ^ HB ( k ) , k = 0 , 1 , , N } .
##EQU00016##
[0082] In an embodiment, a method of low complexity and high
quality FEC or PLC includes generating high band time domain
signal, generating low band time domain signal, estimating the
energy ratio between the high band and the low band from last good
frame, keeping the energy ratio for the following frame-erased
frames by applying an energy correction scaling gain to the high
band signal segment by segment in time domain, and combining the
low band signal and the high band signal into the final output. In
some embodiments, the scaling gain is smoothed sample by sample
from one segment to next of the high band signal.
[0083] In an embodiment, the energy ratio from last good frame is
calculated as
Ratio = E HB E LB = i T env ( i ) s ^ LB ( n ) 2 , ##EQU00017##
where T.sub.env(i) is the temporal energy envelope of the last good
high band signal.
[0084] In an embodiment, wherein the energy correction gain factor
g.sub.i for i-th sub-segment of the following erased frames is
calculated in the following way:
g i = Ratio s ^ LB i ( j ) 2 s ^ HB i ( j ) 2 ##EQU00018## if g i
> 1 , g i = 1 ##EQU00018.2##
where .parallel.s.sub.LB.sup.i(j).parallel..sup.2 and
.parallel.s.sub.HB.sup.i(j).parallel..sup.2 represent respectively
the energies of the i-th sub-segments of the low band signal
s.sub.LB.sup.i(j)=s.sub.LB(20i+j) and the high band signal
s.sub.HB.sup.i(j)=s.sub.HB(20i+j).
[0085] In an embodiment, the correction gain factor g.sub.i is
finally applied to the i-th sub-segment high band signal
s.sub.HB(j)=s.sub.HB(20i+j) while smoothing the gain from one
segment to next segment, sample by sample:
g.sub.i(j)0.95 g.sub.i(j-1)+0.05g.sub.i
s.sub.HB.sup.i(j)s.sub.HB.sup.i(j) g.sub.i(j).
[0086] In an embodiment, a method of low complexity and high
quality FEC or PLC includes copying high band frequency domain
coefficients from previous frame, adaptively adding random noise to
the copied coefficients, scaling the random noise component and the
copied component, controlled with a parameter representing said
periodicity or harmonicity of said signal, generating high band
time domain signal by inverse-transforming the generated high band
frequency domain coefficients, generating low band time domain
signal, estimating the energy ratio between the high band and the
low band from last good frame, keeping the energy ratio for the
following frame-erased frames by applying an energy correction
scaling gain to the high band signal segment by segment in time
domain, and combining the low band signal and the high band signal
into the final output. In some embodiments, the frequency domain
can be MDCT domain, DFT (FFT) domain, or any other discrete
frequency domain. In some embodiments, the parameter representing
the periodicity or harmonicity can be voicing factor, pitch gain,
or spectral sharpness.
[0087] In some embodiments, the method is applied to operate for
systems configured to operate over a voice over internet protocol
(VoIP) system, or for systems that operate over a cellular
telephone network. In some embodiments, the method is applied to
operate within a receiver having an audio decoder configured to
receive the audio parameters and produce an output audio signal
based on the received audio parameters, wherein the output audio
signal comprises an improved FEC signals.
[0088] In embodiment, a MDCT based FEC algorithm replaces the TDBWE
based FEC algorithm for Layers 4 to 12 in a G.729EV based
system.
[0089] In a further embodiment, a method of correcting for missing
data of a digital audio signal includes copying frequency domain
coefficients of the digital audio signal from a previous frame,
adaptively adding random noise coefficients to the copied frequency
domain coefficients, scaling the random noise coefficients and the
copied frequency domain coefficients to form recovered frequency
domain coefficients. Scaling is controlled with a parameter
representing a periodicity or harmonicity of the digital audio
signal. The method also includes generating a high band time domain
signal by inverse-transforming high band frequency domain
coefficients of the recovered frequency domain coefficients,
generating low band time domain signal by a corresponding to low
band coding method and estimating an energy ratio between the high
band and the low band from a last good frame. The method further
includes keeping the energy ratio for following frame-erased frames
by applying an energy correction scaling gain to a high band
signal, segment by segment in the time domain and combining the low
band signal and the high band signal to form a final output.
[0090] In a further embodiment, a system for receiving a digital
audio signal includes an audio decoder configured to copy frequency
domain coefficients of the digital audio signal from a previous
frame, adaptively add random noise coefficients to the copied
coefficients, and scale the random noise coefficients and the
copied frequency domain coefficients to form recovered frequency
domain coefficients. In an embodiment, scaling is controlled with a
parameter representing a periodicity or harmonicity of the digital
audio signal. The audio decoder is further configured to produce a
corrected audio signal from the recovered frequency domain
coefficients.
[0091] In an embodiment, wherein the audio decoder is further
configured to receive audio parameters from the digital audio
signal. In an embodiment, the audio decoder is implemented within a
voice over internet protocol (VOID) system. In one embodiment, the
system further includes a loudspeaker coupled to the corrected
audio signal.
[0092] It should be appreciated that in alternate embodiments,
different sample rates and numbers of channels that are different
from the specific examples disclosed hereinabove can be used.
Furthermore, embodiment algorithms can be used to correct for lost
data in a variety of systems and contexts.
[0093] Advantages of embodiment algorithms include an ability to
achieve a simpler FEC algorithm for those layers higher than 14
kbps in G.729.1 SWB by exploiting characteristics of MDCT based
codec algorithms.
[0094] The above description contains specific information
pertaining to low complexity FEC algorithm for MDCT Based Codec.
However, one skilled in the art will recognize that embodiments of
the present invention may be practiced in conjunction with various
encoding/decoding algorithms different from those specifically
discussed in the present application. Moreover, some of the
specific details, which are within the knowledge of a person of
ordinary skill in the art, are not discussed to avoid obscuring the
present invention.
[0095] The drawings in the present application and their
accompanying detailed description are directed to merely example
embodiments of the invention. To maintain brevity, other
embodiments of the invention that use the principles of the present
invention are not specifically described in the present application
and are not specifically illustrated by the present drawings.
[0096] It will also be readily understood by those skilled in the
art that materials and methods may be varied while remaining within
the scope of the present invention. It is also appreciated that the
present invention provides many applicable inventive concepts other
than the specific contexts used to illustrate embodiments.
Accordingly, the appended claims are intended to include within
their scope such processes, machines, manufacture, compositions of
matter, means, methods, or steps.
* * * * *