U.S. patent application number 11/386192 was filed with the patent office on 2007-04-05 for method and system for information processing.
This patent application is currently assigned to Fondazione Torino Wireless. Invention is credited to Fereydoun Daneshgaran Bajastani, Massimiliano Laddomada, Marina Mondin.
Application Number | 20070079223 11/386192 |
Document ID | / |
Family ID | 34934438 |
Filed Date | 2007-04-05 |
United States Patent
Application |
20070079223 |
Kind Code |
A1 |
Mondin; Marina ; et
al. |
April 5, 2007 |
Method and system for information processing
Abstract
A novel technique for information processing is provided, in
which the decoding of channel encoded data is enhanced by using an
inherent correlation of the data. It is demonstrated that the
correlation is highly robust with respect to bit errors introduced
by the transmission channel. Thus, the correlation represents
additional information in decoding the data, thereby relaxing
constraints in view of power supply and computational resources at
the transmitter side and/or the transmission channel for a given
desired quality of service.
Inventors: |
Mondin; Marina; (Torino,
IT) ; Laddomada; Massimiliano; (Martina Fionca,
IT) ; Bajastani; Fereydoun Daneshgaran; (Torino,
IT) |
Correspondence
Address: |
MCDONNELL BOEHNEN HULBERT & BERGHOFF LLP
300 S. WACKER DRIVE
32ND FLOOR
CHICAGO
IL
60606
US
|
Assignee: |
Fondazione Torino Wireless
Corso Trento 21 10129
Torino
IT
Euroconcepts S.R.L.
Corso Tazzoli 215/13 10137
Torino
IT
|
Family ID: |
34934438 |
Appl. No.: |
11/386192 |
Filed: |
March 22, 2006 |
Current U.S.
Class: |
714/780 |
Current CPC
Class: |
H03M 13/3746
20130101 |
Class at
Publication: |
714/780 |
International
Class: |
H03M 13/00 20060101
H03M013/00 |
Foreign Application Data
Date |
Code |
Application Number |
Mar 22, 2005 |
EP |
05006313.0 |
Claims
1. A method of information processing comprising: generating a
first piece of information and a second piece of information in a
timely related manner; transmitting at least said first piece of
information from a first source to a second source over a first
transmission channel; and decoding at least said first piece of
information at said second source by using an estimated correlation
of said transmitted first piece of information and said second
piece of information available at said second source at the time of
decoding at least said first piece of information.
2. The method of claim 1, wherein decoding at least said first
piece of information comprises iteratively decoding said first
piece of information using a soft decision algorithm.
3. The method of claim 2, wherein iteratively decoding at least
said first piece of information comprises partially decoding said
first piece of information with a first iteration step, estimating
a first correlation value relating said partially decoded first
piece of information to said second piece of information and using
said first correlation value in decoding said first piece of
information in a second iterative step.
4. The method of claim 3, wherein said first correlation value is
used to readjust at least one decision criterion of said soft
decision algorithm.
5. The method of claim 3, wherein iteratively decoding said first
piece of information comprises partially decoding said first piece
of information as obtained after said second iterative step,
estimating a second correlation value relating said first piece of
information partially decoded twice to said second piece of
information and using said second correlation value in decoding
said first piece of information in a third iterative step and so on
until a desired fixed total number of iterations have been
achieved.
6. The method of claim 1, wherein said second piece of information
is transmitted to said second source via a second transmission
channel.
7. The method of claim 1, wherein said second piece of information
is transmitted via said first transmission channel.
8. The method of claim 1, wherein said first piece of information
is generated at said first source and said second piece of
information is generated at said second source.
9. The method of claim 6, wherein said first piece of information
is generated at said first source and said second piece of
information is generated at a third source.
10. The method of claim 7, wherein said first and second pieces of
information are generated at said first source.
11. The method of claim 1, wherein said first piece of information
is one of a plurality of first pieces of information that are
transmitted via a plurality of first transmission channels
including said first transmission channel to a plurality of second
sources including said second source, each of the plurality of
first sources at least transmitting at least one of the plurality
of first pieces of information, each of the plurality of second
sources receiving at least one of said plurality of first pieces of
information, each of the plurality of second sources having access
to at least one of a plurality of second pieces of information
including said second piece of information, the method further
comprising decoding said plurality of first pieces of information
at said plurality of second sources using respective estimated
correlations of said plurality of first pieces of information with
said plurality of second pieces of information.
12. The method of claim 1, further comprising transmitting said
first piece of information with or without data compression prior
to any channel encoding of the first piece of information.
13. The method of claim 1, further comprising determining said
estimated correlation by comparing first data bits representing
said first piece of information with second data bits representing
said second piece of information by a logic operation.
14. The method claim 13, further comprising obtaining said
estimated correlation by determining a comparison result on the
basis of a number of agreements of the comparison and normalizing
said comparison result.
15. The method of claim 1, wherein said first and second pieces of
information are iteratively decoded.
16. The method of claim 3, further comprising determining said
first correlation value after a first iterative step for the first
and second pieces of information and using said first correlation
value in a second step of decoding the first and second pieces of
information.
17. The method of claim 1, further comprising channel encoding at
least said first piece of information.
18. The method of claim 17, wherein channel encoding at least said
first piece of information comprises low density parity check
encoding or any other block coding said first piece of
information.
19. The method of claim 17, wherein channel encoding at least said
first piece of information comprises using a serially concatenated
convolutional code or any other convolutional encoding scheme with
or without concatenation of more than one code, be it concatenated
block, convolutional or mixed block and convolutional codes.
20. The method of claim 17, wherein said second piece of
information is channel encoded by the same encoding method as the
first piece of information.
21. A method of channel decoding at least first data representing a
first piece of information generated by a first source and second
data representing a second piece of information generated by a
second source, the first and second data having a specified degree
of correlation, the method comprising: receiving said first and
second data, decoding at least said first data in a first step,
determining an estimate of said degree of correlation on the basis
of said first data decoded in said first step and said second data,
and decoding at least said first data in a second step on the basis
of said estimate.
22. The method of claim 21, wherein decoding at least said first
data includes decoding said second data.
23. The method of claim 21, wherein determining an estimate of said
degree of correlation comprises determining a first correlation
value based on a comparison of the first and second data, the
method further comprising using said first correlation value to
readjust a decision criterion in said second step.
24. A communication network comprising: a first node including a
channel encoder configured to encode a first piece of information,
a second node including a channel decoder configured to decode said
channel coded first piece of information on the basis of an
estimated correlation between said first piece of information and a
second piece of information communicated over said network and
being available at the second node at the time of decoding said
first piece of information, and a correlation estimator configured
to provide a value indicating said estimated correlation to said
channel decoder, and a communication medium providing one or more
communication channels and being connected to the first and second
nodes and being configured to convey at least said channel coded
first piece of information to said second node.
25. The communication network of claim 24, wherein said channel
decoder comprises an iterative soft decision decoder.
26. The communication network of claim 24, further comprising a
third node including a channel encoder configured to encode said
second piece of information, said third node being connected to
said communication medium for conveying said second piece of
information to said second node.
27. The communication network of claim 24, wherein said channel
encoder of said first node is configured to encode said second
piece of information for transmission over said communication
medium.
28. The communication network of claim 24, wherein said first node
is one of a first plurality of nodes and said second node is one of
a second plurality of nodes, each of said first plurality including
a respective channel encoder configured to encode an associated
first piece of information, wherein said second node is one of a
second plurality of nodes, each of which includes a respective
channel decoder configured to decode said channel coded first
pieces of information on the basis of an estimated correlation
between said first pieces of information and a plurality of second
pieces of information including said second piece of information,
one of said second pieces of information being available at each of
said second nodes and wherein each of said second nodes includes a
respective correlation estimator.
29. The communication network of claim 24, further comprising a
platform configured to execute one or more applications that
produce said first and second pieces of information, said platform
at least being connected to said first node.
30. The communication network of claim 29, wherein said one or more
applications are associated with one or more hardware units, at
least one hardware unit being coupled with said first node.
31. The communication network of claim 30, wherein at least one
hardware unit is coupled with said third node.
32. The communication network of claim 29, wherein said one or more
hardware units each comprise a sensor element.
33. The communication network of claim 24, wherein power resources
and/or computational resources available at said first node are
less compared to power resources and/or computational resources
available at said second node.
34. A channel decoder comprising: an input section configured to
receive a first signal and a second signal and to demodulate said
first and second signals to produce first and second data
representing a first piece of information and a second piece of
information, respectively, at least said first signal being a
channel-coded signal, a correlation estimator configured to receive
said first data and said second date and to determine a correlation
value defining a degree of correlation between said first and
second data, and a decoder section connected to said input section
and said correlation estimator, said decoder section being
configured to decode at least said first data on the basis of said
correlation value.
35. The channel decoder of claim 34, wherein said decoder section
comprises an iterative soft decision decoder configured to adjust
at least one soft decision threshold on the basis of said
correlation value.
36. The channel decoder of claim 35, wherein said correlation
estimator is configured to receive a decoded version of said first
data after a first iterative step and to provide an updated
correlation value to said decoder for a subsequent iterative
step.
37. The channel decoder of claim 34, wherein said decoder section
is configured to decode said first data and said second data on the
basis of said correlation value.
38. The channel decoder of claim 34, further comprising a hardware
unit connectable to a network and being configured to process at
least said decoded first piece of information, wherein the channel
decoder and the hardware unit are components of a network node.
39. The channel decoder of claim 38, wherein said hardware unit is
further configured to assess a validity of said decoded first piece
of information and to transmit an instruction via said network to
resend at least said first piece of information.
Description
BACKGROUND
[0001] The applicants claim priority of European Patent Application
05006313.0, dated Mar. 22, 2005.
[0002] The present invention generally relates to methods and
systems used for generating and communicating information via a
network including wired and/or wireless transmission channels.
[0003] The rapid advance in the field of micro optical, micro
mechanical and micro electronic techniques brings about the
potential for enhanced generation of information, for instance in
the form of measurement data provided by sensor elements, and also
promotes the vast and efficient distribution of information over a
plurality of transmission channels, which are increasingly designed
as wireless channels, thereby frequently providing enhanced
connectivity along with improved user mobility. In this respect,
the term "network" is often used for describing a system that
allows data to be communicated between a plurality of network
nodes, which are connected to a communication medium including one
or more transmission channels so as to receive and/or transmit data
from one or more of the communication channels. The transmission
channels may represent wired and/or wireless communication lines,
such as cables, optical fibers, or any other electromagnetic fields
propagating in free space. Although the term "network" is sometimes
used in the context of systems including a high number of network
nodes, such as mobile `phone subscribers linked to a plurality of
base stations, computer devices linked to local and global
networks, and the like, a network is to be understood in this
application as a system comprising at least a first node and at
least a second node connected via at least one transmission
channel. Hereby, the first node and the second node may represent
different physical entities or may represent the same physical
entities at different states. For example, a hardware unit storing
data on a memory unit and reading the stored data at a later time
may also be considered to represent a first node, when storing the
data, and may be considered a second node, when retrieving the
data, while the memory unit may present the transmission
channel.
[0004] Generally, in network communication it is intended to
receive information provided as a stream of data bits transmitted
from a first node to a second node via the communication channel
with a minimal number of bit errors wherein, depending on the
specific application and the transmission channel characteristics,
more or less encoding and decoding efforts are necessary to
maintain a certain desired degree of data integrity. A measure for
quantitatively expressing the quality of the transmission channel
is the bit error rate (BER) representing the probability of
creating an erroneous bit during data transmission. In principle,
each transmission channel is subjected to environmental influences,
which may cause a disturbance of the initial signal fed into the
transmission channel. Moreover, other physical phenomena such as
noise, dispersion, and the like, may have a significant impact on
the probability of creating a bit error after reconverting an
analogue signal into its digital representation. Despite the
unavoidable probability for any bit errors created during the
transmission of a signal, the information may reliably be retrieved
from the signal as long as the information capacity of the
information source is less than the channel capacity and an
appropriate method of encoding the source information is used. In
this context, encoding source information so as to reduce the
probability of providing erroneous information after decoding the
transmitted signal at the receiver is referred to as channel
encoding. That is, channel encoding adds complexity to the original
information, for instance by providing a certain degree of
redundancy, so as to allow at the receiver side to retrieve, at
least to a certain desired degree, the original information
irrespective of any bit errors that may have occurred during the
transmission of the encoded information.
[0005] It should be appreciated that the degree of channel encoding
required for a reliable transmission of information depends on the
channel characteristics and the source capacity, wherein the
additional redundancy required for a perfectly reliable extraction
of the originally encoded information may frequently be not
acceptable for a plurality of applications. On the other hand many
applications, such as storing data on a storage medium,
transmitting "exe" files, and the like, require an extremely low
bit error rate so as to not jeopardize the operation of the
application when using stored data or running an "exe" file on a
corresponding computer platform. Thus, data communication is
frequently a compromise between information processing capabilities
and data reliability and/or application performance. Consequently,
information generation and information transmission may be
described as a process in which desired information is created and
is prepared for transmission in a first step that is typically
referred to as source encoding. In the process of source encoding,
the amount of information is reduced, for instance by removing
redundancy, removing portions of information that are considered
not essential for the application of interest, and the like, so as
to obtain a condensed form of the initially generated information.
Depending on the type of technique for compressing the information,
the reduced amount of information may reflect the initial
information with or without loss of data.
[0006] After this source encoding, the condensed information is
channel encoded. Additional redundancy is added to the condensed
information so as to allow the correction of transmission induced
bit errors or at least provide the potential to identify to a
certain degree the occurrence of bit errors at the receiver side.
Hereby, the efforts in channel encoding significantly determine the
reliability of the data transmission for a given transmission
channel and thus the feasibility or applicability in certain
applications. For instance, the great advances in construction of
low-cost, low-power and mass produced micro sensors and micro
electro mechanical systems has ushered a new era in system design
for a diverse range of applications. The advent of such devices has
indeed provided one key ingredient of what may be considered as a
sensory revolution. On the other hand, the ability to integrate,
extract and communicate useful information from a network of
distributed sensors renders the employment of distributed sensors
as an attractive solution for problems involved in a plurality of
applications. Consequently, research progress has been made in the
past decade on addressing several issues in connection with
enabling sensing nodes in a network to communicate with each other
and with the outside world.
[0007] Although many of the problems encountered with distributed
sensor networks are common problems also encountered in traditional
fields, such as the design of microprocessors and the like,
frequently more severe constraints are to be taken into
consideration. That is, compared to for instance traditional data
communication or terminal equipment, a significantly reduced
computational power and battery power is usually available at the
network nodes and thus requires a highly efficient channel encoding
of data communicated over the network so as to meet the
requirements with respect to computational power and battery power.
Hence, it is an important aspect to design the channel encoding and
the signal processing so as to reduce the transmit power for
severely power limited nodes for a given fidelity criterion. For
example, a network may be considered including a plurality of
sensor nodes that are hierarchically arranged in a tree structure
with collections of nodes at a given hierarchical level belonging
to different clusters each having a cluster head. The cluster heads
may be endowed with more signal processing capacity and available
power in comparison to the other cluster nodes. In such an example,
the cluster heads may represent aggregation nodes for data that
migrates from one level of the tree hierarchy to the next. In such
a configuration, the data communication from a lower rank network
node to a higher rank network node or to the cluster head may
suffer from reduced reliability owing to the severe constraints in
transmit power and/or computational power at the network nodes of
the lower rank.
[0008] In view of the situations described above, there exists a
need for improved techniques for information processing so as to
increase data reliability without unduly contributing to channel
encoding complexity.
SUMMARY
[0009] A method of information processing comprises: generating a
first piece of information and a second piece of information in a
timely-related manner and transmitting at least the first piece of
information from a first source to a second source over a first
transmission channel. Moreover, the method comprises decoding at
least the first piece of information at the second source by using
an estimated correlation of the transmitted first piece of
information and the second piece of information that is available
at the second source at the time of decoding at least the first
piece of information.
[0010] According to this aspect of the method, the presence of a
correlation between a first piece of information and a second piece
of information, which frequently is an inherent property of the
first and second pieces of information, may be exploited in
decoding at least one of the pieces of information that is
transmitted via the transmission channel. Hereby, the first and
second pieces of information are generated in a timely-related
fashion so that their time-relationship may be used in determining
a specified degree of correlation at the second source. Based on
the identified degree of correlation, which is highly robust with
respect to error-causing mechanisms in the transmission channel,
there is then in addition to the first and second pieces of
information further information available for more reliably
decoding at least the first piece of information at the receiver
side, thereby providing the potential for relaxing the constraints
with respect to channel encoding at the transmitter side or to
improve the data transmission reliability for a given configuration
of the first source, the second source and the transmission
channel. Consequently, due to the fact that the correlation
existing in the initially generated first and second pieces of
information is highly robust during transmission, any sources, such
as network nodes receiving the first and/or second piece of
information, may more reliably communicate information while
nevertheless meeting even highly severe constraints, for instance,
with respect to power availability and computational resources.
[0011] In a further preferred embodiment, decoding at least the
first piece of information comprises iteratively decoding the first
piece of information using a soft decision algorithm. As is
generally known, channel decoding on the basis of iterative
decoding techniques including soft decision criteria, as is
frequently used in conventional decoding schemes, may significantly
be enhanced by also exploiting the inherent cross-correlation
between the first and second pieces of information.
[0012] In one preferred embodiment, iteratively decoding at least
the first piece of information comprises partially decoding the
first piece of information in a first iteration step, estimating a
first correlation value relating the partially decoded first piece
of information to the second piece of information and finally using
the first correlation value in decoding the first piece of
information in a second iterative step.
[0013] Thus, by estimating the first correlation value on the basis
of the first piece of information as decoded in the first iterative
step, well-approved iterative decoding techniques may be used and
may thereafter be enhanced by providing the first correlation value
in a subsequent iteration step, wherein the additional information
conveyed by the correlation value may allow a more reliable
assessment of the correctness of the first piece of information.
Since the first correlation value is provided on the basis of the
preliminarily decoded first piece of information and the second
piece of information, no "side information" is required so as to
enhance the further decoding process, that is, neither the
transmission channel nor the first source is loaded with additional
information, while nevertheless providing for enhanced means in
deciding whether or not a bit of the first piece of information has
been correctly transmitted or not.
[0014] In a further embodiment, the first correlation value is used
to readjust at least one decision criterion of the soft decision
algorithm. Consequently, the first correlation value, obtained
without any side information with respect to the first source or
the transmission channel, may allow to readjust a decision
threshold in a subsequent iterative step, thereby reducing the
number of iterations required or enhance the data reliability for a
given number of iteration steps.
[0015] In a further embodiment, iteratively decoding the first
piece of information comprises partially decoding the first piece
of information as obtained after the second iterative step,
estimating a second correlation value relating the first piece of
information partially decoded twice to the second piece of
information, and using the second correlation value in decoding the
first piece of information in a third iterative step.
[0016] According to this embodiment, a further iterative step may
be performed on the basis of an updated correlation value, which is
calculated on the basis of the decoded first piece of information,
which is already based on a previously calculated correlation
value. Consequently, by using an updated correlation value the
further iteration process may even be more enhanced, since the
accuracy of the updated correlation value may improve, even though
the correlation between the first and second pieces of information
is of high reliability in the preceding iterative steps due to the
high robustness with respect to channel-induced errors.
[0017] In a further advantageous embodiment, the second piece of
information is transmitted to the second source via a second
transmission channel. In this arrangement, the second piece of
information may be conveyed in a similar fashion as the first piece
of information wherein, as explained with reference to the first
piece of information, the correlation initially present for the
first and second pieces of information is substantially maintained,
although the second transmission channel may also be subjected to
bit errors. Thus, the first and second pieces of information may be
generated by correlated information sources, wherein the robust
correlation between the first and second pieces of information may
be used in decoding the first and second pieces of information with
an enhanced degree of reliability.
[0018] In a further embodiment, the second piece of information is
transmitted via the first transmission channel. In this
configuration, the second piece of information may be made
available at the second source by means of the first transmission
channel, wherein the robustness of the correlation may assist in
decoding the first and/or the second piece of information. For
example, the first and second pieces of information may be
generated at disjoint information sources connected to the same
network node, or the first and second pieces of information may be
generated by one or more applications running at a specified
platform connected to a specified network node, or the first and
second pieces of information may represent respective portions of
information generated by a single information source.
[0019] In a further embodiment, the first piece of information is
generated at the first source and the second piece of information
is generated at the second source. In this configuration, the
second piece of information may not necessarily be transmitted via
a transmission channel but may instead be directly used without any
further encoding and decoding process. For instance, the first and
second sources may represent sensory network nodes connected by the
first transmission channel so that the second sensory network node
may receive information via the first transmission channel and may
be able to decode the information with enhanced reliability due to
exploiting the fact that a high degree of correlation, that is, a
high degree of similarity or dissimilarity, may be present between
the first and second pieces of information.
[0020] In another configuration, the first piece of information is
generated at the first source and the second piece of information
is generated at a third source. Thus, the first and second pieces
of information may be transmitted via respective transmission
channels so as to be received and decoded at the second source. As
an illustrative example, the first and third sources may be
considered as sensory network nodes communicating with the first
source, representing a further sensory network node that may have
increased computational power and supply power compared to the
first and third sources, which may be operated with severe
constraints regarding computational resources and supply power.
Hereby, despite the limited channel encoding and supply power
capabilities of the first and third sources data may be transferred
to the second source at high reliability, since transmission
induced errors may efficiently be identified due to the additional
information conveyed by the correlation and usable for
decoding.
[0021] In a further embodiment, the first piece of information is
one of a plurality of first pieces of information that are
transmitted via a plurality of first transmission channels, which
include the first transmission channel, to a plurality of second
sources including the second source, wherein each of the plurality
of first sources at least transmits at least one of the plurality
of first pieces of information and wherein each of the plurality of
second sources receives at least one of the plurality of first
pieces of information, wherein each of the plurality of second
sources has access to at least one of a plurality of second pieces
of information, which include the previously mentioned second piece
of information, and wherein the method further comprises decoding
the plurality of first pieces of information at the plurality of
second sources while using respective estimated correlations of the
plurality of first pieces of information with the plurality of
second pieces of information.
[0022] With this arrangement, a plurality of sources may transmit
respective information to a plurality of receiving sources, wherein
at the receiving side the possible correlation between one or more
received messages and at least one second piece of information
available at each of the receiving sources is used for an enhanced
channel decoding. Consequently, the above configuration is highly
advantageous in operating a network including a plurality of
transmitting network nodes and having a plurality of receiving
network nodes. Although not necessary for practicing the present
invention, this configuration may be highly advantageous if the
first sources may represent sources of reduced computational
resources and/or power supply compared to the receiving second
sources.
[0023] In one preferred embodiment, the method further comprises
transmitting the first piece of information without data
compression prior to any channel encoding of the first piece of
information. This embodiment is highly advantageous in applications
in which source encoding is a less attractive approach since source
encoding, although used for reducing the number of bits transferred
via a transmission channel, puts most of the signal processing
burden at the information source, thereby requiring highly advanced
computational resources and power supply. Furthermore, when the
data packet size is moderately small, as is often the case in
distributed sensor networks, source encoding may make no sense and
may in fact cause data expansion rather than compaction.
[0024] In a further preferred embodiment, the method additionally
comprises determining the estimated correlation by comparing first
data bits representing the first piece of information with second
data bits representing the second piece of information by a logic
operation. Thus, highly efficient means for assessing the degree of
correlation between the first and second pieces of information are
provided, thereby also reducing the amount of computational
resources at the second source (in this case, the receiving
source).
[0025] In a further embodiment, the method further comprises
obtaining the estimated correlation by determining a comparison
result on the basis of a number of agreements of the comparison and
by normalizing the comparison result. Consequently, according to
this embodiment the correlation may readily be determined by, for
instance, counting the number of agreements or the number of
disagreements between corresponding bits representing the first
piece of information and the second piece of information,
respectively, so that this comparison result may readily be used,
when appropriately normalized, for the further process of decoding
the data bits in a further iterative step.
[0026] In preferred embodiments, the first piece and the second
piece are iteratively decoded, advantageously in a common sequence
using the estimated correlation, obtained after a first step, in
evaluating a newly-decoded version of the first and second pieces
of information on the basis of the estimated correlation calculated
after the previous iteration step.
[0027] In further advantageous embodiments, at least the first
piece of information is channel-encoded, wherein in one embodiment
the channel encoding comprises a low density parity check for the
encoding of the first piece of information, while in another
embodiment the channel encoding comprises a serially concatenated
convolutional code.
[0028] In a further embodiment, the first and the second pieces of
information are both channel-encoded by the same encoding
method.
[0029] A method of channel decoding at least first data
representing a first piece of information generated by a first
source and second data representing a second piece of information
generated by a second source is provided, wherein the first and
second data have a specified degree of correlation. The method
comprises receiving the first and second data, decoding at least
the first data in a first step, determining an estimate of the
degree of correlation on the basis of the first data decoded in the
first step and the second data and decoding at least the first data
in a second step on the basis of the estimate of the degree of
correlation.
[0030] As previously already pointed out, the methods described
herein provide a novel technique for channel decoding data received
via a transmission channel, wherein the decoding is performed in at
least two steps while using the correlation between the first and
second data so as to enhance the reliability of the decoding
process. As previously discussed, in many applications requiring
the data transfer via transmission channels of a network, the
information received at a specified network node may include
correlated portions or information received from different network
nodes may bear a certain correlation, which is maintained to a high
degree irrespective of any bit errors occurring during the transfer
of information, as will be discussed in more detail later on. Thus,
by receiving the first and second data, wherein at least the first
data may be channel encoded and transmitted via a specified
transmission channel, the first data may be decoded on the basis of
additional information regarding the first and second data, i.e.,
their mutual correlation, without requiring additional resources at
the transmitter side and in the transmission channel. Thus, the
methods described herein are advantageous in network applications
having a hierarchical structure with severe constraints with
respect to computational resources and/or power supply at the
transmitting side. It should be emphasized, however, that these
methods are also applicable to any information processing of
information generated by correlated disjoint sources, wherein at
least a portion of the information is communicated via a
transmission channel. For example, the communication of slowly
changing measurement results over a noisy transmission channel may
significantly be improved by exploiting the presence of correlation
between two subsequent messages. Also, in other network
applications the transmission of subsequent similar data or
dissimilar data may provide the receiver side with additional
implicit information, that is, the correlation between subsequent
messages, so as to enhance the channel decoding process.
[0031] In preferred embodiments, the first data and the second data
may be decoded. In this configuration, both the first data and the
second data may be transmitted via one or more transmission
channels, wherein the decoding process exploits the inherent
correlation so as to enhance the decoding reliability for both the
first and second data irrespective of the error mechanisms acting
on the respective transmission channels.
[0032] According to a further embodiment, a communication network
is provided, which comprises a first node including a channel
encoder configured to encode a first piece of information. The
network further comprises a second node including a channel decoder
configured to decode the channel encoded first piece of information
on the basis of an estimated correlation between the first piece of
information and a second piece of information that is communicated
over the network and is available at the second node at the time of
decoding the first piece of information. The second node further
includes a correlation estimator that is configured to provide a
value indicating the estimated correlation to the channel decoder.
Additionally, the network comprises a communication medium
providing one or more communication channels and being connected to
the first and second nodes and being configured to convey at least
the channel encoded first piece of information to the second
node.
[0033] As previously discussed above, the concept of using
additional inherent information in the form of a correlation
existing between a first piece of information that may be
communicated via a communication channel, and a second piece of
information, which is available at the time of decoding the first
piece of information at the receiver side, may also advantageously
be applied to a communication network so as to enhance the decoding
reliability for a given configuration of the transmitter side and
the communication channel, or to lessen the burden at the
transmitter side and/or the communication channel for a desired
degree of quality of surface.
[0034] Moreover, the communication network specified above may be
provided in multiple configurations and embodiments, some of which
are described with respect to the method of information processing
and the method of channel decoding, wherein also a plurality of
advantages are provided that also apply to the inventive
communication network. In particular, the communication network
allows for improved communication between nodes in a network by
allowing to reduce or identify the errors caused by transmission
via the communication channel. Thus, a more efficient utilization
of the available band width of the communication channel is
accomplished thereby, for instance, reducing the number of times a
message may have to be retransmitted in order for it to be received
reliably at the destination node. Hereby, the first piece and the
second piece of information may be generated by disjoint sources,
wherein the term "disjoint sources" may include multiple segments
of the same message generated by a single source, segments of
messages generated by different sources, segments of messages
generated by multiple applications producing traffic at a single
network node or any combination of the preceding configurations.
Consequently, the inventive concept of a communication network may
be applicable to a wide variety of applications. Moreover, the
number of disjoint information sources that are processed by a
given network node may vary and may particularly include the
following combinations.
[0035] A disjoint node sends a piece of information or a message
that is relayed by another node, which in turn may have access to
information that is correlated to the information sent by the first
node. In this scenario, the receiving node may decode the message
sent by the former node while using the message available so as to
generate correlation information for enhanced decoding
reliability.
[0036] A plurality of nodes send correlated information, which may
be processed by a node, wherein the receiving node jointly decodes
the information from the plurality of transmitting nodes using the
mutually existing correlation of the information sent by the
plurality of nodes.
[0037] A plurality of nodes send correlated information to a
plurality of receiving nodes, wherein at each of the receiving
nodes the correlation is used in enhancing the decoding
process.
[0038] Moreover, any combination of the above-described scenarios
may be realized based on the network features discussed above.
[0039] A channel decoder may be provided, which comprises an input
section configured to receive a first signal and a second signal
and to demodulate the first and second signals to produce first and
second data representing a first piece of information and a second
piece of information, respectively, wherein at least the first
signal is a channel-encoded signal. Moreover, the channel decoder
comprises a correlation estimator configured to receive the first
data and the second data and to determine a correlation value
defining a degree of correlation between the first and the second
data. Finally, the channel decoder comprises a decoder section
connected to the input section and the correlation estimator,
wherein the decoder section is configured to decode at least the
first data on the basis of the correlation value.
[0040] As the channel decoder is based on the same principle as the
method and system described above, the same advantages may readily
be achieved by the channel decoder.
[0041] In a further embodiment, the decoder section comprises an
iterative soft decision decoder configured to adjust at least one
soft decision threshold on the basis of the correlation value.
Consequently, the iterative soft decision decoder imparts improved
efficiency to the channel decoder compared to conventional channel
decoders, without requiring any modifications at the transmitter
side or the transmission channel.
[0042] A network node unit may be provided, which comprises a
channel decoder as specified above and a hardware unit connectable
to a network and being configured to process at least the decoded
first piece of information.
[0043] In one embodiment, the hardware unit is further configured
to assess a validity of the decoded first piece of information and
to transmit an instruction via the network in order to instruct a
resending of at least the first piece of information. Thus, by
using an inherent correlation of pieces of information or messages
in decoding at least one of the pieces of information or messages,
a highly efficient network unit is provided that is especially
suited for sensor applications.
BRIEF DESCRIPTION OF THE DRAWINGS
[0044] FIG. 1 schematically depicts a communication network
including a channel decoder and a network node according to an
embodiment of the present invention;
[0045] FIGS. 2a-2c show graphs of results of simulation
calculations;
[0046] FIG. 3 schematically depicts a generic trellis diagram;
[0047] FIG. 4 schematically illustrates the architecture of a joint
channel decoder according to illustrative embodiments of the
present invention;
[0048] FIGS. 5a-5h depict graphs representing the results of
simulations for the bit error rate with respect to the
signal-to-noise ratio;
[0049] FIG. 6 schematically depicts the architecture of the encoder
and iterative decoder for conventional individual serially
concatenated convolutional codes (SCCC);
[0050] FIGS. 7a-7d represent graphs depicting the simulation
results of bit error rates, frame error rates, the estimated
correlation and the variance of the estimated correlation with
respect to the signal-to-noise ratio for an SCCC configuration;
[0051] FIG. 8 schematically shows the architecture of a joint
channel decoder of correlated sources according to an embodiment of
the present invention, wherein channel encoding is performed
according to a low density parity check (LDPC) coding method;
[0052] FIGS. 9a-9c represent graphs illustrating the bit error rate
with respect to the signal-to-noise ratio according to simulation
results; and
[0053] FIG. 9d schematically represents the empirical probability
mass functions of the LLR values according to some illustrative
embodiments of the present invention.
[0054] FIG. (9e) shows in a table the average number of local
iterations performed by the joint LDPC decoder at the end of a
given global iteration, for two values of correlation between the
sources.
DETAILED DESCRIPTION
[0055] The methods described herein exploit the potential
correlation existing between multiple information sources to
achieve additional coding gains from the channel codes used for
data protection. In this way, the existence of any channel side
information at the receiver is neither assumed nor is it used.
Rather, empirical estimates of the cross-correlation are used, in
particular embodiments, in partial decoding steps in an iterative
joint soft decoding paradigm.
[0056] FIG. (1) schematically shows a communication network 100,
which is configured so as to use an inherent correlation between
different pieces of information for channel decoding at least one
of these pieces of information. The network 100 comprises a first
information source 130, which may also represent a first network
node including necessary hardware units and equipment so as to
generate and provide a first piece of information, represented here
as first data 131, to a communication medium 120, which may include
one or more transmission channels. Thus, the first source 130 may
represent a platform for running one or more application routines,
one or more of which may produce the first data 131. The first data
131 may be provided to the communication medium 120 by any
well-known means, such as cable connections and the like. For
example, the first source 130 may represent a hardware unit
comprising micro optical, micro mechanical and/or micro electronic
components so as to generate data, channel encode the data and
provide the same to the communication medium 120. In particular
embodiments, the first source 130 may represent a sensor element
configured to generate and provide relevant data, such as
environmental data and the like. The communication medium 120 may
comprise a plurality of transmission channels provided as wired
and/or wireless transmission channels so that these transmission
channels, depending on the specific configuration, may suffer from
a certain unavoidable probability for creating channel-induced
errors when conveying the first data 131 through the communication
medium 120. The network 100 further comprises a second source 110,
which may represent a second network node connected to the
communication medium 120 so as to receive therefrom transmit data
132 that may differ from the first data 131 owing to
channel-induced errors. For receiving the transmit data 132, the
second source 110 may comprise an input section 111, which is
further configured to receive second data 133, which may inherently
be associated with the first data 131 by a specified degree of
correlation 134. The inherent correlation 134 may be caused by the
process of creating the first data 131 and the second data 133, for
instance when the second source 110 comprises a sensor element
placed in the vicinity of the first source 131 and detecting an
environmental property which may not significantly differ at the
locations of the first and second sources 130, 110. However, many
other applications may be contemplated in which an inherent
correlation between the first and second data 131, 133 may exist.
For instance, both the first and second data 131, 133 may be
created by the first source 130, therefore exhibiting a specified
degree of similarity or dissimilarity, and may be communicated via
the communication medium 120. In other embodiments, a plurality of
first sources 130 may be provided, each source generating a
respective set of first data 131, which may be communicated to the
second source 110. Also, a plurality of second sources 110 may be
provided, each of which receives first and second data having an
inherent correlation that may be exploited during the decoding
process for at least one of one or more sets of first data 131.
[0057] The second source or node 110 may further comprise a
detection section 112 that is configured to decode the data 132
with respect to a specified channel encoding technique used in the
first source 130 so as to enhance data protection during the
transmission through the communication medium 120. The first source
130 further comprises a correlation estimator 113 that is connected
to the input section 111 and the decoder section 112 and is
configured to determine an estimation of the inherent correlation
134 and provide the estimated correlation to the decoder section
112 which, in turn, may provide an enhanced decoded version of the
transmit data 132 received via the communication medium 120. As
will be shown in the following, the inherent correlation 134 is
quite robust with respect to any error mechanisms experienced data
communicated via the communication medium 120 so that the estimated
correlation provided by the estimator 113 represents a robust
criterion, which may be used in more reliably decoding the faulty
or error-prone transmit data 132, thereby providing the potential
for reducing the effort in channel encoding the first data 131
and/or reducing the constraints with respect to bandwidth of the
communication medium 120, or improving the quality of service (QoS)
for a given configuration of the first source 131 and the
communication medium 120. For instance, after receiving the
transmit data 132 and decoding the same on the basis of the
inherent correlation 134 in the second source 110, the number of
instructions for resending the first data 131 due to errors in the
transmit data 132 may be reduced.
[0058] During operation of the network 100, the first and second
data 131, 133 are generated in a timely-related manner,
irrespective of where the second data 133 are produced. According
to the time-correlation of the first and second data 131, 133 the
first and second data 131, 133 may correctly be assigned to each
other and therefore appropriately processed at the second source
110. Hereby, it should be appreciated that a respective
time-relationship between the first data 131 and the second data
133 may readily be established by, for instance, the sequence of
receipt at the second source 110, by the time of creation of the
respective data, wherein a corresponding time information may be
part of the data, or any other mechanisms. Thereafter, at least the
first data 131, which are to be transmitted via the communication
medium 120, are channel-encoded by any appropriate encoding
technique used for data protection for respective transmission
channels. Later in this specification, respective configurations
for convolutional coding techniques, low density parity check
(LDPC) encoding techniques and serially concatenated convolutional
codes (SCCC) will be described in more detail. It should be
appreciated, however, that the present invention may also be used
in any combination of block and convolutional coding regimes and
with any form of code concatenation (serial, parallel or
hybrid).
[0059] After passing through the communication medium 120, which
may also represent a storage medium, as is previously explained, a
certain degree of data corruption may have occurred, as is
well-known for data communication over wired and wireless
transmission channels, thereby creating the faulty data 132. After
receiving the faulty data 132 at the first node 110 and based on
the second data 133, which are available at the second source 110
at the time of decoding the faulty data 132, the decoder section
112 may provide a first estimate of a decoded version of the faulty
data 132 based on conventional decoding techniques. Thereafter, the
estimator 113 receiving the first estimate of the decoded data and
also receiving the second data 133 may calculate an estimation of
the inherent correlation 134 and may supply the estimated
correlation to the decoder section 112, which in turn may determine
a second estimate for the decoded faulty data 132 on the basis of
the estimated correlation. For instance, the decoder section 112
may include a soft decision algorithm, in which a decision
criterion may be adjusted by the estimated correlation provided by
the estimator 113. Due to the additional information contained in
the first data 131, 133 in the form of the inherent correlation
134, the decoding process in the second source 110 may provide a
decoding result of the faulty data 132 with enhanced
reliability.
[0060] In the following, the robustness of the inherent correlation
134 with respect to channel-induced error mechanisms will be
explained in more detail, wherein the following assumptions are
made to simplify the description. The invention, however, is not
limited to these simplifying assumptions. [0061] the data packets
to be transmitted by sources A and B, such as the first source 130,
are either correlated or very different (later it will be clarified
what is meant by this). The correlation may arise for instance if A
and B sample an environmental parameter that does not change
significantly at their locations. On the other hand, the data
generated by A and B may exhibit a large difference. The key point
is that the data packets generated by A and B cannot be assumed to
represent for instance two sequences of independent identically
distributed random variables; [0062] channel coding is indeed
feasible and of relatively low cost. In the following the use of
convolutional coding for data transmission is assumed; [0063]
relative timing synchronicity of the nodes engaged in this
communication is assumed; [0064] the severe power constraints at A
and B preclude options such as A sending a message to B and having
B relay A and B's message after some signal processing to a node C,
such as the second source 110, [0065] the nodes engaged in
communication are assumed to be stationary at least for the
duration of the transmission of the packet of data.
[0066] The fundamental question addressed here is as follows; how
can node C use the implicit source correlation between the encoded
data packets it receives from A and B to improve the Bit Error Rate
(BER) or Frame Error Rate (FER) for both data packets? If node C
can achieve an improvement, then the additional coding gain
obtained from the use of source correlation can be used to back-off
the power at the transmit nodes A and B to conserve power for the
same quality of service (i.e., a target BER or FER). It is noted
that the more complex signal processing required at C to use this
implicit correlation to improve performance, has a power penalty.
However, it is assumed that the decrease in transmit power from A
and B to C is more important and out-weighs this added signal
processing cost (i.e., communication power requirements out-weigh
signal processing power requirements, as is often the case).
[0067] Another scenario that could use the same process for
improving performance is when node A sends a packet to node B that
has a data correlated with the message sent from A. In this
scenario, node B is forwarding the packet generated from node A in
addition to sending its own packet. The potential correlation
between the packet at node B and the packet sent by A can be used
by the decoder at B that needs to decode A's message before
forwarding it to the next node along the chain. It is noted that in
a typical Distributed Sensor Networks (DSNs), 65% of the traffic at
nodes is forwarded packets. Of course, the previous scenarios can
be combined. The number of possibilities is large. In this example
the focus shall be on the first scenario.
[0068] It is to be noted that the first scenario just described
does not quite fit the conventional multiple access channel model
of network information theory, whereby the data transmitted from
multiple sources may interfere with each other. In particular, here
we assume that sufficient statistics associated with the
transmitted data from nodes A and B are both available at node C
and that there is no interference between the two sources. The dual
problem of Slepian-Wolf correlated source coding more closely fits
the scenario just described, although here it is dealt with channel
coding as opposed to source coding. Let us clarify; the result of
Slepian-Wolf theorem on correlated source coding is that rates
above the joint entropy are achievable even though the sources are
disparate. If C can improve its BER or FER (i.e., the Quality of
Service or QoS) at a fixed Signal to Noise Ratio (SNR) using the
knowledge of the implicit correlation between the messages of A and
B, then A and B can back-off their power levels for a fixed QoS
requirement. Alternatively, A and B can utilize higher rate
convolutional codes with reduced coding gains but use the same SNR
level needed to achieve the required QoS if A and B's messages were
independently decoded. Use of the higher rate codes at A and B
means fewer channel bits transmitted to C for the same QoS, which
is what the Slepian-Wolf theorem suggests is achievable. In
essence, with channel coding, correlated source compression can be
achieved without source encoding at A and B that may be too costly
or infeasible.
[0069] For this scenario, the sensitivity of the cross-correlation
to channel-induced errors may be estimated as follows.
[0070] In what follows, the relative robustness of the empirical
cross-correlation of the received data to channel induced errors
shall be demonstrated. To this end, let {right arrow over (X)} and
{right arrow over (Y)} be two binary vectors of length L. Let us
define Z.sub.n=X.sub.n.sym. Y.sub.n as the XOR of the n-th
component of the vectors {right arrow over (X)} and {right arrow
over (Y)}. Similarly, we define {right arrow over (Z)}={right arrow
over (X)}.sym. {right arrow over (Y)} whereby {right arrow over
(Z)} is obtained via componentwise XOR of the components of the
vectors {right arrow over (X)} and {right arrow over (Y)}.
[0071] Let the number of places in which {right arrow over (X)} and
{right arrow over (Y)} agree be r so that the empirical
cross-correlation between these two vectors is .rho.=r/L. Let us
suppose that what is available at the receiver are noisy versions
of {right arrow over (X)} and {right arrow over (Y)} denoted X
.fwdarw. .times. .times. and .times. .times. Y .fwdarw. ##EQU1##
respectively. For instance, X .fwdarw. .times. .times. and .times.
.times. Y .fwdarw. ##EQU2## could be erroneous versions of {right
arrow over (X)} and {right arrow over (Y)} obtained after
transmission through a noisy channel modeled as a Binary Symmetric
Channel (BSC) with transition probability p. We assume that the
error events inflicting the two sequences are independent
identically distributed (i.i.d.). The receiver generates an
empirical estimate of the cross-correlation based on the use of the
sequences X .fwdarw. .times. .times. and .times. .times. Y .fwdarw.
##EQU3## by forming the vector Z .fwdarw. = X .fwdarw. .sym. Y
.fwdarw. ##EQU4## and counting the number of places where Z
.fwdarw. ##EQU5## is zero. Let us denote this count as {circumflex
over (r)}. Clearly, {circumflex over (r)} is a random variable. The
question is, what is the Probability Mass Function (PMF) of
{circumflex over (r)}? Knowledge of this PMF allows us to assess
the sensitivity of our estimate of the cross-correlation to errors
in the original sequences.
[0072] It is relatively straightforward to find the probability
that ({circumflex over (z)}.sub.n=z.sub.n) Pr({circumflex over
(z)}.sub.n=z)=(1-p).sup.2+p.sup.2 (1) Pr({circumflex over
(z)}.sub.n.noteq.z)=2p(1-p), (2)
[0073] Consider applying a permutation to the sequences {right
arrow over (X)} and {right arrow over (Y)} so that the permuted
sequences agree in the first r locations, and disagree in the
remaining (L-r) locations. The permutation is applied to simplify
the explanation of how we may go about obtaining the PMF of
{circumflex over (r)} and by no means impacts the results. It is
evident that the permuted sequence .pi.({right arrow over (Z)})
contains r zeros in the first r locations and (L-r) ones in the
remaining locations. Now consider evaluation of the Pr({circumflex
over (r)}=r+k) for k=0, 1, . . . , (L-r). We define .pi.({right
arrow over (Z)}).sub.r to represent the first r bits of .pi.({right
arrow over (Z)}) and .pi.({right arrow over (Z)}).sub.L-r the
remaining (L-r) bits. Similarly we define .pi. .function. ( Z
.fwdarw. ) r .times. .times. and .times. .times. .pi. .function. (
Z .fwdarw. ) L - r . ##EQU6## For a fixed k, the event {{circumflex
over (r)}=r+k} corresponds to the union of the events of the type:
.pi. .function. ( Z .fwdarw. ) L - r ##EQU7## differs from
.pi.({right arrow over (Z)}).sub.L-r in (k+l) positions for some l
.di-elect cons.{0, 1, . . . , r}, .pi. .function. ( Z .fwdarw. ^ )
r ##EQU8## differs from .pi.({right arrow over (Z)}).sub.r in l
positions, and the remaining bits of .pi. .function. ( Z .fwdarw. ^
) ##EQU9## and .pi.({right arrow over (Z)}) are identical.
[0074] The probability of such elementary events are given by: ( r
l ) .times. ( L - r k + l ) .function. [ [ ( 1 - p ) 2 + p ] L - k
- 2 .times. l .function. [ 2 .times. p .function. ( 1 - p ) ] k + 2
.times. l ] ( 3 ) ##EQU10##
[0075] The probability of the event {{circumflex over (r)}=r+k} for
k=0, 1, . . . , (L-r) is given by: Pr .function. ( r ^ = r + k ) =
l = 0 r .times. ( r l ) .times. ( L - r k + l ) .times. .times. [ (
1 - p ) 2 + p 2 ] .times. L - k - 2 .times. l .function. [ 2
.times. p .function. ( 1 - p ) ] k + 2 .times. l ( 4 )
##EQU11##
[0076] Using similar arguments, for m=1, 2, . . . , r we have: Pr
.function. ( r ^ = r - m ) = l = m r .times. ( r l ) .times. ( L -
r l - m ) .times. .times. [ ( 1 - p ) 2 + p 2 ] .times. L - 2
.times. l + m .function. [ 2 .times. p .function. ( 1 - p ) ] 2
.times. l - m ( 5 ) ##EQU12##
[0077] Before looking at the PMF of the random variable {circumflex
over (r)} in detail, consider the behavior of the PMF for small p.
We consider .rho.={circumflex over (r)}/L which is the parameter of
real interest to us. Note that for sufficiently small p, the only
significant terms correspond to values k=0, 1 and m=1, and the only
significant contribution in the summations over l in the above
probability expressions is that due to l=0 in (4) and l=1 in (5).
Pr .function. ( .rho. ^ = .rho. ) .apprxeq. [ ( 1 - p ) 2 + p 2 ] L
.apprxeq. [ 1 - 2 .times. Lp ] ( 6 ) Pr .function. ( .rho. ^ =
.rho. - 1 L ) .apprxeq. r .function. [ ( 1 - p ) 2 + p 2 ] L - 1
.function. [ 2 .times. p .function. ( 1 - p ) ] .apprxeq. r
.function. [ 1 - 2 .times. ( L - 1 ) .times. p ] .function. [ 2
.times. p .function. ( 1 - p ) ] ( 7 ) Pr .function. ( .rho. ^ =
.rho. - 1 L ) .apprxeq. ( L - r ) .function. [ ( 1 - p ) 2 + p 2 ]
L - 1 .function. [ 2 .times. p .function. ( 1 - p ) ] .apprxeq. ( L
- r ) [ 1 - 2 .times. ( L - ( 8 ) ##EQU13##
[0078] The variance of the estimate {circumflex over (p)} based on
the above approximation is given by: .sigma. 2 .apprxeq. [ 1 - 2
.times. ( L - 1 ) .times. p ] .function. [ 2 .times. p .function. (
1 - p ) ] L ( 9 ) ##EQU14## with the obvious assumption that
p<1/[2(L-1)]. Finally, for small values of p, we have: .sigma. 2
.apprxeq. 2 .times. p L - 4 .times. p 2 ( 10 ) ##EQU15## where, now
we require p<1/(2L). Note that this variance diminishes rapidly
with decreasing p. To study the behavior of .sigma. as a function
of p, let p=1/(2sL) where s>1 is the parameter characterizing
both p and .sigma.. In particular, with simple manipulation we get:
pL = 1 2 .times. s ( 11 ) .sigma. .times. .times. L = 1 s - 1 s 2 (
12 ) ##EQU16##
[0079] FIG. (2a) depicts the .sigma.L product versus the pL product
as s varies from s=3 to s=40. The important observation is the
rather gradual increase in .sigma.L as pL is increased, which shows
that the variance of the estimate of .rho. tends to exhibit a
saturating behavior. As an example of the use of this figure, at
pL=0.1 we have .sigma.L=0.4. Hence, for a block length of L=100 we
get that at p=10.sup.-3, .sigma..apprxeq.4.times.10.sup.-3 which is
indeed very small for any reasonable value of .rho. encountered in
practice.
[0080] To confirm the general behavior observed above for larger
values of p, we have evaluated the PMF of {circumflex over (.rho.)}
for .rho. in the range from .rho.=0.1 to .rho.=0.9 for a block
length of L=100. Two key observations from the results of our
simulations are: [0081] the most probable value of {circumflex over
(.rho.)}, denoted M({circumflex over (.rho.)}) (i.e., the Mode),
obtained from evaluation of the empirical cross-correlation from
noisy received vectors is not necessarily the true value .rho..
This is particularly so at larger values of p and for small and
large values of .rho.. FIG. (2b) captures this behavior for two
values of p=0.1 and p=0.01 as a function of .rho.. In particular,
this figure shows the difference (.rho.-M({circumflex over
(.rho.)})) versus .rho. obtained from empirical evaluation of the
cross-correlation from noisy received vectors; [0082] the standard
deviation of {circumflex over (.rho.)} is independent of .rho. in
the range .rho.=0.1 to .rho.=0.9 for a fixed value of p as should
be suspected. However, this standard deviation is a strong function
of p itself. FIG. (2c) depicts the standard deviation of
{circumflex over (.rho.)} as a function of p for L=100. This figure
is essentially the extension of the results depicted in FIG. (2a)
to larger values of p and reconfirms our observation that the
standard deviation indeed increases slowly with increasing p. Note
that even at values of p as large as p=0.3 this standard deviation
is still relatively small for .rho. in the range .rho.=0.1 to
.rho.=0.9.
[0083] While the above analysis has focused on short block length
of L=100, our experimental results suggest that similar conclusions
also hold valid for larger values of L. The conclusion from the
above passage is that the computation of the empirical
cross-correlation between two received noisy vectors is relatively
insensitive to the errors inflicting the two sequences even at
rather large values of error probability p. Hence, the empirical
cross-correlation between two sequences is robust to channel
induced errors.
[0084] Next, a joint iterative decoding technique for decoding
correlated sources will now be described in more detail for an
illustrative example.
[0085] In this section we present the proposed joint decoding
algorithm for two correlated information sources. The extension to
more sources is omitted at this stage for clarity of the
presentation. In particular, as noted in the introduction, we
assume two nodes A and B, such as the source 130, in a
communication network have data to transmit to a given destination
node C, such as the source 110. Let the two data sequences be
represented by two packets of data which are correlated. The
prevailing example we have referred to in this example is a DSN
where the transmitting nodes are sensory nodes that are in close
proximity to each other and may sample some environmental parameter
of interest and wish to convey their information to their cluster
head which in our model, represents the receiving node. In this
scenario, it is relatively easy to envision the origin of
correlation between data generated at distinct nodes. It should be
appreciated that such correlation can indeed exist in a much
broader context within a communication network.
[0086] The individual source nodes A and B independently encode
their data using simple convolutional codes and transmit the
encoded data block over independent Additive White Gaussian Noise
(AWGN) channels. At the receiver, the sufficient statistics for
both sources are processed jointly. We note that aside from the
fact the receiver may a-priori presume some correlation between the
encoded received data might exist, no side information is
communicated to the receiver. For one thing, no such side
information can be generated by the individual sources without
mutual communication. The receiver uses an iterative soft decision
decoding technique for joint detection of the transmitted data
sequences. Hence, the starting point in our development shall be
the mathematical development behind joint soft decision
decoding.
[0087] Let Z be a random variable in Galois Field GF(2) assuming
values from the set {+1, -1 } with equal probability, where +1 is
the "null" element under the modulo-2 addition. As explained in
[1], the log-likelihood ratio of a binary random variable Z is
defined as L Z .function. ( z ) = log .function. [ P Z .function. (
z = + 1 ) P Z .function. ( z = - 1 ) ] ##EQU17## where P.sub.z(z)
is the probability that the random variable Z takes on the value z.
Under the modulo-2 addition, it is easy to prove that for
statistically independent random variables X and Y the following
relation is valid: P(X.sym.Y=+1)=P(X=+1)P(Y=+1)+(1-p(X=+1))
(1-p(Y=+1)) (13)
[0088] Hence, for Z=X.sym.Y: P Z .function. ( z = + 1 ) = e L Z
.function. ( z ) 1 + e L z .function. ( z ) ( 14 ) ##EQU18##
[0089] Furthermore, the following approximation holds: L Z
.function. ( z ) = log .function. [ 1 + e L X .function. ( x )
.times. e L Y .function. ( y ) e L X .function. ( x ) + e L Y
.function. ( y ) ] .times. .times. .apprxeq. sign .function. ( L X
.function. ( x ) ) sign .function. ( L Y .function. ( y ) ) min ( L
X .function. ( x ) , L ( 15 ) ##EQU19##
[0090] Soft decision joint iterative decoding of the received
signals can best be described using an elementary decoding module
denoted as the Soft-Input Soft-Output (SISO) decoder. The SISO
decoder works at the symbol level following the Maximum
A-posteriori Probability (MAP) decoding algorithm proposed by Bahl
et al with some modifications with the goal of making the SISO unit
operate on integer metrics (i.e., integer arithmetic as opposed to
floating point arithmetic implementation). The decoder operates on
the time-invariant trellis of a generic rate R o = p n ##EQU20##
convolutional encoder.
[0091] FIG. (3) schematically depicts a generic trellis section for
such a code. In this figure, the trellis edge is denoted by e, and
the information and code symbols associated with the edge e are
denoted by x(e) and c(e) respectively. The starting and ending
state of the edge e is identified by s.sup.s(e) and s.sup.E(e)
respectively.
[0092] The SISO operates on a block of encoded data at a time. In
order to simplify the notation, where not specified, x and y
indicate blocks of data bits. Sequence x is composed of the bits
x.sub.k,t for k=1, . . . , L and t=1, . . . , p, where
{X.sub.k,t}.sub.k=1.sup.l is the t-th input sequence of the rate
p/n code. A similar notation is used for the sequence y.
[0093] Furthermore, we shall formulate the metric evaluations for
the received data associated with the first source and denoted by x
only. This formulation obviously applies to the received data
associated with the other source y as well. Let us denote the
log-likelihood ratio associated with the information symbol x by
L(x). We use the following notation: [0094] L.sup.(i)(x;I) and
L.sup.(i)(y;I) denote the log-likelihood ratios of the extrinsic
information associated with the source symbols x and y at the input
of the SISO decoders at iteration i; [0095] L(c.sub.1; I) and
L(c.sub.2; I) denote the log-likelihood ratios of the encoded
symbols coming from the channel at the input of the SISO decoders;
[0096] L.sup.(i)(x;O) and L.sup.(i)(y;O) denote the extrinsic
log-likelihood ratios related to the information symbols x and y at
the output of the SISO decoders, evaluated under the code
constraints at iteration i; [0097] {circumflex over (x)}.sup.(i)
and y.sup.(i) represents the hard estimates of the source symbols x
and y at iteration i (i.e., the decoded symbols at iteration
i).
[0098] Consider the channel encoder at the source receiving an
input data block of L bits and generating an output data block of
LR.sub.0.sup.-1 bits, whereby R.sub.0 is the rate of the
convolutional encoder. Let the input symbol to the convolutional
encoder (for a generic rate p/n code) denoted x.sub.k(e) represent
the input bits x.sub.kj with j=1, . . . , p on a trellis edge at
time k (k=1, . . . , L), and let the corresponding output symbol of
the convolutional encoder c.sub.k(e) at time k be represented by
the output bits c.sub.k,j(e) with j=1, . . . , n and k=1, . . . ,
L. Based on these assumptions, the log-likelihood ratios of the
source bits x.sub.kj can be evaluated for any j=1, . . . , P by the
SISO decoder at iteration i as follows: L k ( i ) .function. ( x k
, j ; O ) = max e : x k , j .function. ( e ) = 1 * .times. {
.alpha. k - 1 .function. [ s S .function. ( e ) ] + t = 1 , t
.noteq. j p .times. x x , t .function. ( e ) .times. L k ( i )
.function. [ x k , t ; I ] + t = 1 n .times. c k , t .function. ( e
) .times. L k .function. [ c k , t ; I ] + .beta. k .function. [ s
E .function. ( e ) ] } + - max e : x k , j .function. ( e ) = 0 *
.times. { .alpha. k - 1 .function. [ s S .function. ( e ) ] + t = 1
, t .noteq. j p .times. x k , t .function. ( e ) .times. L k ( i )
.function. [ x k , t ; I ] + t = 1 n .times. c k , t .function. ( e
) .times. L k .function. [ c k , t ; I ] + .beta. k .function. [ s
E .function. ( e ) ] } , k = 1 , .times. , L - 1 ##EQU21## where,
the forward recursion at time k, .alpha..sub.k(.) [2], can be
evaluated through: .alpha. k .function. ( s ) = h .alpha. k + max *
e : s E .function. ( e ) = s .times. { .alpha. k - 1 .function. [ s
S .function. ( e ) ] + t = 1 p .times. x k , t .function. ( e )
.times. L k ( i ) .function. [ x k , t ; I ] + t = 1 n .times. c k
, t .function. ( e ) .times. L k .function. [ c k , t ; I ] } , k =
1 , .times. , L - 1 ( 16 ) ##EQU22## while, the backward recursion,
,.beta..sub.k(.), can be evaluated through: .beta. k .function. ( s
) = h .beta. k + max * e : s E .function. ( e ) = s .times. {
.beta. k - 1 .function. [ s S .function. ( e ) ] + t = 1 p .times.
x k + 1 , t .function. ( e ) .times. L k - 1 ( i ) .function. [ x k
+ 1 , t ; I ] + t = ! n .times. c k + 1 , t .function. ( e )
.times. L k + 1 .function. [ c k + 1 , t ; I ] } , k = L - 1 , L -
2 , .times. , 1 ( 17 ) ##EQU23## To initialize the above
recursions, the following are used: .alpha. 0 .function. ( s ) = {
0 if .times. .times. s = S 0 - .infin. otherwise .times. .times.
and ( 18 ) .beta. L .function. ( S i ) = { 0 if .times. .times. s =
S L - .infin. otherwise ( 19 ) ##EQU24## where, S.sub.0 and S.sub.L
are the initial and terminal states of the convolutional codes
(assumed to be the all-zero state). The SISO module operates in the
log-domain so that only summation of terms are needed. The operator
max* above signifies the following: max .function. ( a i ) i * =
log .function. [ i = 1 Q .times. e a i ] = max .function. ( a i ) +
.delta. .function. ( a 1 , .times. , a Q ) ( 20 ) ##EQU25## where,
.delta.(.alpha..sub.1, . . . , .alpha..sub.Q) is a correction term
that can be computed using a look-up table.
[0099] Finally, h.sub..alpha.k and h.sub..beta.k are two
normalization constants that for a hardware implementation of the
SISO are selected to prevent buffer overflows.
[0100] The bit decisions on the sequence {circumflex over
(x)}.sup.(i) at iteration i can be obtained from the log-likelihood
ratios of x.sub.k,t, .A-inverted.t=1, . . . , p, .A-inverted.k=1, .
. . , L by computing:
L.sub.x.sub.k,t.sup.(i)=L.sub.k.sup.(i)(x.sub.k,t;O) (21) and
making a hard decision on the sign of these metrics.
[0101] In the same way, the bit decisions on the sequence y.sup.(i)
at iteration i can be obtained from the log-likelihood ratios of
y.sub.k,t, .A-inverted.t=1, . . . , p, .A-inverted.k=1, . . . , L
by computing: L.sub.y.sub.k,t.sup.(i)=L.sub.k.sup.(i)(y.sub.k,t;O)
(22) and making a hard decision on the sign of these metrics.
[0102] The architecture of the joint channel decoder is depicted in
FIG. (4). Let us elaborate on the signal processing involved. In
particular, as before let X and Y be two correlated binary random
variables which can take on the values {+1, -1} and let Z=X.sym.Y.
Let us assume that random variable Z takes on the values {+1, -1}
with probabilities P.sub.z(z=+1)=p.sub.z and
P.sub.z(z=-1)=1-p.sub.z. Both sources independently from each
other, encode the binary sequences x and y with a rate-p/n
convolutional encoder having memory v. For simplicity, let us
consider a rate-1/2 convolutional encoder. Both encoded sequences
are transmitted over independent AWGN channels. The received
sequences are r.sub.x and r.sub.y which take on values in .sup.L (
is the set of real numbers) in the case the transmitted bits are
encoded in blocks of length L. Let N.sub.0/2 denote the
double-sided noise-power spectral density and recall that
.sigma..sup.2=N.sub.0/2. With this setup, the log-likelihood ratios
related to the observation samples r.sub.x at the output of the
matched filter can be evaluated as follows: L k .function. ( c 1 ,
k ; I ) = 2 .sigma. 2 .times. r x k , k = 1 , .times. , L - 1 ( 23
) ##EQU26##
[0103] In the same way, the log-likelihood ratios related to the
observation samples r.sub.y at the output of the matched filter can
be evaluated as follows: L k .function. ( c 2 , k ; I ) = 2 .sigma.
2 .times. r y k , k = 1 , .times. , L - 1 ( 24 ) ##EQU27##
[0104] The log-likelihood ratios L.sub.z.sup.(i)(z) at iteration
(i) are evaluated as follows: L z ( i ) .function. ( z ) = log
.function. ( 1 - p z ^ p z ^ ) ( 25 ) ##EQU28## by counting the
number of places in which {circumflex over (x)}.sup.(i) and
y.sup.(i) differ, or equivalently by evaluating the Hamming weight
w.sub.H(.) of the sequence {circumflex over
(z)}.sup.(i)={circumflex over (x)}.sup.(i).sym.y.sup.(i) whereby,
in the previous equation, p z ^ = w H .function. ( z ^ ( i ) ) L .
##EQU29## In the latter case, by assuming that the sequence {right
arrow over (Z)}={right arrow over (X)}.sym.{right arrow over (Y)}
is i.i.d., we have: L Z ( i ) .function. ( z ) = log .function. ( L
- w H .function. ( z ^ ( i ) ) w H .function. ( z ^ ( i ) ) ) ( 26
) ##EQU30## where L is the data block size.
[0105] Finally, applying equation (15) we can obtain an estimate of
the extrinsic information on the source bits for the next
iteration: L.sup.(i)(x;I)=L({circumflex over
(z)}.sup.(i-1).sym.y.sup.(i)) (27) and L.sup.(i)(y;I)=L({circumflex
over (z)}.sup.(i-1).sym.{circumflex over (x)}.sup.(i)) (28)
[0106] Note that as far as the LLR of the difference sequence
{right arrow over (Z)} is concerned, a correlation of for instance
10% or 90% between X and Y carry the same amount of information.
Hence, the performance gain of the iterative joint decoder in
either case is really the same (we have verified this
experimentally). Coding gains can be obtained if the two sequences
are either very similar (e.g., 90% correlated) or very different
(e.g., 10% correlated). From an information theoretic point of
view, all this says is that the entropy of the random variable Z is
symmetric about the 50% correlation point. Formally, the joint
decoding algorithm can be formalized as follows: [0107] 1. Set the
iteration index i=0 and set the log-likelihood ratios
L.sup.(0)(x;I) and L.sup.(0)(y;I) to zero (see FIG. (4)). Compute
the log-likelihood ratios for the channel outputs using equations
(23) and (24) for both received sequences r.sub.x and r.sub.y.
Conduct a preliminary MAP decoding in order to obtain an estimate
of both sequences {circumflex over (x)}.sup.(0) and y.sup.(0) and
evaluate w.sub.H({circumflex over (z)}.sup.(0))=w.sub.H({circumflex
over (x)}.sup.(0).sym.y.sup.(0)). Use w.sub.H({circumflex over
(z)}.sup.(0)) to evaluate L.sub.z.sup.(0)(z) in equation (26). Note
that if the receiver already has an estimate of the correlation
between the two transmitted sequences x and y (i.e., with side
information), it can directly evaluate equation (26). In our
simulations, we do not assume the availability of any side
information. [0108] 2. Set L.sup.(l)(x;I) and L.sup.(l)(y;I) to
zero. [0109] 3. For iteration i=1, . . . , q, perform the
following: [0110] a) Make a MAP decoding for both received
sequences r.sub.x and r.sub.y by using the log-likelihood ratios as
expressed in equations (23) and (24). [0111] b) Evaluate
L.sub.z.sup.i(z) using equations (26). [0112] c) Evaluate
L.sup.(i)(x;I) by using L.sub.z.sup.(i-1)(z) and L.sup.(i)(y;O).
Evaluate L.sup.(i)(y;I) by using L.sub.z.sup.(i-1)(z) and
L.sup.(i)(x;O). [0113] d) Go back to (a) and continue until the
last iteration q.
[0114] As it is possible to see from the algorithm, the joint
decoder at any stage i estimates the extrinsic log-likelihood
ratios L.sup.(i)(x;I) and L.sup.(i)(y;I) by using the new estimates
of the source bits {circumflex over (x)}.sup.(i) and y.sup.(i) and
the previous estimate on the difference sequence {circumflex over
(z)}(i-l).
[0115] Analytical Performance Bounds
[0116] This section develops analytical bounds on the performance
of the iterative joint channel decoder. If iterative joint channel
decoding is not performed, the performance of the individual links
between transmitter A and receiver C and transmitter B and receiver
C are essentially dominated by the performance of the individual
convolutional codes used for channel coding.
[0117] Upper-bounds on the performance of convolutional codes based
on well-known transfer functions or based on the knowledge of the
distance spectrum of the code are readily available. In practice,
we may use the knowledge of the first few lowest distance terms of
the distance spectrum of a given convolutional code to obtain a
reasonable approximation to the asymptotic performance of the code
using union bounding technique. This asymptotic performance is
achieved at sufficiently high SNR values.
[0118] It is known that for soft-decision Viterbi decoding, the BER
of a convolutional code of rate R 0 = p n ##EQU31## with BPSK or
QPSK modulation in AWGN, can be well upper-bounded by the following
expression: P b .ltoreq. 1 p .times. d = d free .infin. .times. w d
.times. Q ( 2 .times. .times. E b N 0 .times. R 0 .times. d ) ( 29
) ##EQU32## in which d.sub.free is the minimum non-zero Hamming
distance of the Convolutional Code (CC), w.sub.d is the cumulative
Hamming weight (for the information bits) associated with all the
paths that diverge from the correct path in the trellis of the
code, and re-emerge with it later and are at Hamming distance d
from the correct path, and finally Q(.) is the Gaussian integral
function, defined as Q .function. ( t 0 ) = 1 2 .times. .pi.
.times. .intg. 0 .infin. .times. e - t 2 2 .times. d t
##EQU33##
[0119] Similarly, it is possible to obtain an upper-bound on the
FER of the code as follows: P f .ltoreq. d = d free .infin. .times.
m d .times. Q ( 2 .times. E b N 0 .times. R 0 .times. d ) ( 30 )
##EQU34## where, m.sub.d is the multiplicity of all the paths that
diverge from the correct path in the trellis of the code and
re-emerge with it later and are at Hamming distance d from the
correct path.
[0120] A. Genie Aided Lower-Bound
[0121] A simple lower-bound on the performance of the iterative
joint channel decoder for correlated sources can be obtained by
shifting the BER or FER curve of the individual convolutional codes
to the left by an amount of 10 log(2)=3 dB. The justification for
the bound is simple. If a genie was available at the receiver that
would tell it simply in which locations the data transmitted by A
and B where identical and in which locations they were different
(assuming BPSK transmission for simplicity), then the receiver
prior to decoding, would combine the signals coherently and
effectively double the received SNR. This doubling of the receiver
input SNR corresponds to 3 dB of gain. In general for M correlated
sources, the Genie aided SNR gain would be 10 log(M).
[0122] The performance of the channel decoding technique described
above may be estimated as follows.
[0123] B. Performance Bound for Joint Channel Decoding of
Correlated Sources
[0124] Hagenauer provides the theoretical development for the
performance bound of the Viterbi Algorithm (VA) with a-priori soft
information. The performance of the Viterbi decoder with a-priori
soft information is essentially the same as the performance of the
SISO decoder employing one iteration of the forward-backward
algorithm with the same a-priori soft information. Hence, the
result can be directly used to provide an upper-bound on the
performance of the iterative joint channel decoder for correlated
sources.
[0125] We shall first provide a setup for using the Hagenauer bound
in the current context, and subsequently provide the bound itself.
In particular, suppose the receiver has the exact knowledge of the
correlation coefficient between the data transmitted by A and B
(note that this is much weaker than knowing where the two sequences
differ). As noted before, at sufficiently high SNR where union type
bounds have validity, the estimate of the cross correlation at the
decoder from the noisy received vectors is actually quite good.
Hence, the upper-bound to the performance of the decoder that knows
the actual value of the cross-correlation is reasonably close to
the upper-bound on the performance of the actual decoder. Assuming
that the SNR per link between A and C, and B and C are the same,
the independence of the channel noise inflicting the two
transmitted data packets suggests that to a first order
approximation, the error positions for the decoded data packet x
and for the decoded data packety are independent. This suggests
that the BER of the data sequence z is almost twice the BER of the
data sequences x and y.
[0126] Hence, an upper-bound on the BER of the sequence z provides
an upper-bound on the BER of sequences x and y. The exact knowledge
of the cross correlation coefficient is equivalent to knowing the
a-priori probability of the bits associated with the sequence z,
hence, the exact knowledge of the a-priori LLR on sequence z. Since
the CC is linear, the difference sequence z=x.sym.y when encoded,
produces a valid codeword that is in the code space. Hence, we can
envision the sequence z being encoded by the same CC that encodes
sequences x and y and subsequently find an upper-bound on the
performance of the Viterbi decoder with a-priori soft information
derived from the knowledge of the correlation coefficient. The
resulting upper-bound can then be used to provide an upper-bound on
the BER of the transmitted sequences x and y decoded by the actual
iterative joint channel decoder.
[0127] While it is anticipated that the BER of sequence z will be
twice the BER of sequences x and y, any error present in sequence z
corresponds to a frame error either in sequence a; or y or both. In
the worst case, a frame error on sequence z, corresponds to frame
errors on both sequences x and y. Hence, we can take as the
upper-bound on the FER of sequence x or y, the upper-bound on the
FER of the sequence z.
[0128] The Hagenauer bound with the LLR associated with sequence Z
denoted L .function. ( Z ) = log .function. ( .rho. 1 - .rho. )
##EQU35## is given by: P b .ltoreq. 1 p .times. d = d free .infin.
.times. w d .times. Q ( 2 .times. E b N 0 .times. R 0 .times. d
.function. ( 1 + w d m d .times. L .function. ( Z ) 4 .times. d
.times. .times. R o .times. E b / N 0 ) 2 ) ( 31 ) ##EQU36## where,
m.sub.d is the multiplicity of all the paths that diverge from the
correct path in the trellis of the code and re-emerge with it later
and are at Hamming distance d from the correct path and w.sub.d is
the cumulative Hamming weight (for the information bits) associated
with all the paths that diverge from the correct path in the
trellis of the code, and re-emerge with it later and are at Hamming
distance d from the correct path. This bound is essentially
identical to the bound expressed in prior art disclosures except
for the correction factor that accounts for the a-priori
information on Z.
[0129] In order to more clearly demonstrate the performance of the
joint soft decoding algorithm, the following simulations have been
performed.
[0130] We have simulated the performance of our proposed iterative
joint channel decoding of correlated sources assuming simple
convolutional encoding at the sources. We assume that our transmit
nodes use the same convolutional codes and the SNR of each of the
two received sequences are the same.
[0131] The convolutional codes used in our simulations are among
the best codes for a given memory and rate (and hence decoding
complexity), ever reported in the literature. The generator
matrices of the rate 1/2 encoders using the delay operator notation
D are: [0132] 4-state non-recursive, non-systematic encoder
G(D)=[1+D.sup.2,1+D+D.sup.2]; [0133] 8-state non-recursive,
non-systematic encoder G(D)=[1+D+D.sup.3,1+D+D.sup.2+D.sup.3];
[0134] 16-state non-recursive, non-systematic encoder
G(D)=[1+D.sup.3+D.sup.4,1+D+D.sup.2+D.sup.4].
[0135] We have verified by simulations that there is essentially no
difference between using systematic or non-systematic and recursive
or non-recursive encoders, hence, we opted for the codes listed
above and SISO decoders for these codes were generated.
[0136] The simulation results are reported as follows: [0137] 1)
FIGS. (5a), (5b) and (5c) show the BER of either the data sequence
x or y (the BER on these two sequences coincide; at a given
iteration, SISO.sub.1 refers to the BER of sequence x while
SISO.sub.2 refers to the BER of sequence y) encoded by the 4-state
code above as a function of SNR, for varying degrees of
cross-correlation p between the sources, when the sequences are
decoded by the proposed iterative joint channel decoder. Several
observations are in order: [0138] a) the performance curves for
cross-correlation .rho. and (1-.rho.) are identical. As noted
before, this symmetry is expected in light of the symmetry of the
entropy function about p=0.5; [0139] b) 3- or 4-iterations suffice
to get almost all that can be gained from the knowledge of the
cross-correlation. We note that our comparison of the simulation
results with the analytical performance bounds presented below,
reinforce this statement; [0140] c) since the estimates of the
cross-correlation are noisy at sufficiently low SNR levels,
decoding iterations are critical for improving performance,
otherwise 2-iterations are often sufficient to obtain most of the
achievable gain; [0141] d) as the cross-correlation approaches 0.5,
the achievable gains diminish as expected and reduces to zero at
.rho.=0.5. This implies that when the two sequences are totally
uncorrelated (according to our definition), the performance of the
iterative joint channel decoder is no better than the case each
received sequence is independently decoded. On the other hand, when
the cross-correlation level is nearly one or zero, the achievable
coding gain is a function of the operating BER and diminishes as
BER decreases. Note that the four state code with a value of .rho.
close to one achieves 2.1 dB of coding gain at 0 dB of SNR. This is
astonishing given that the gap between the performance of the
iterative joint decoder and the Genie aided lower-bound, is only
0.9 dB. [0142] 2) FIGS. (5d), (5e) and (5f) show the BER as a
function of SNR and .rho. at the end of four decoding iterations,
for the 4-state, 8-state and 16-state convolutional codes
respectively (the code generators are provided above). The main
observation from these figures is that the coding gain of the
iterative joint channel decoder does not seem to depend on the code
memory, rather, it is a strong function of the degree of
correlation between the sequences as should be expected.
[0143] Finally, FIGS. (5g) and (5h) provide a comparison of the
performance of the iterative joint channel decoder to the
analytical upper-bound derived above, for the 4-state and 8-state
codes at two values of L(z) specified in the figures. The value of
L(z)=0 corresponds to the case .rho.=0.5 and hence, there is no
a-priori information available to the joint decoder to improve
performance. In general, union type upper-bounds as reported in the
figures are loose at low values of SNR, and asymptotically tight at
sufficiently high SNR values. The gap between the simulated
performance and the upper-bounds at high SNR values is largely due
to the fact that we have implemented the SISO decoders using
integer arithmetic. This naturally results in some loss in
performance, otherwise the performance of the iterative joint
decoder almost coincides with the analytical upper-bound in high
SNR regimes, suggesting that at least asymptotically, the decoder
is close to optimal.
[0144] As a result, soft information may be generated at the
receiver associated with the cross-correlation between two
sequences during decoding iterations and this information may be
used to improve the decoder performance.
[0145] In an illustrative embodiment, serially concatenated
convolutional codes (SCCC) are used for channel encoding of
multiple correlated sources. In this embodiment, although the
present invention is applicable to any number of correlated
sources, two correlated sources are provided that transmit SCCC
encoded data to a single destination receiver. As before, any
channel side information is neither assumed nor used at the
receiver. As before, empirical cross-correlation measurements at
successive decoding iterations are employed to provide extrinsic
information to the outer codes of the SCCC configuration.
[0146] Two levels of soft metric iterative decoding are used at the
receiver: 1) iterative Maximum A-posteriori Probability (MAP)
decoding is used for efficient decoding of individual SCCC codes
(local iterations) and 2) iterative extrinsic information feedback
generated from the estimates of the empirical cross-correlation in
partial decoding steps is used to pass soft information to the
outer decoders of the global joint SCCC decoder (global
iterations). Later on, simulation results for iterative joint SCCC
decoding of correlated sources for a data packet size of L=320 will
be provided. Representative results associated with the estimation
of the correlation for this block length are as follows: [0147] 1)
The PMF of {circumflex over (.rho.)} for three different values of
raw error rate p when the true cross-correlation between the data
packets is .rho.=0.8 at an example data block length of L=320 has
been evaluated. The following observations are in order: a) there
is a bias in the empirical estimate of .rho. as measured between
the most probable value of {circumflex over (.rho.)} and the true
value of .rho. which is a strong function of p, b) at a value of
p=0.316 (representative of high raw error rate occurring at low
values of SNR), there is a non-zero probability that {circumflex
over (.rho.)}<0.5 implying that such cross-correlation
information when used by the iterative decoder may actually
increase the error rate. Fortunately, for the majority of the
received data packets {circumflex over (.rho.)}>0.5 and
cross-correlation feedback actually improves performance.
Simulation results shown later suggest that for the majority of the
data frames, the cross-correlation feedback reduces the error rate,
while for a very small number of data packets, the error rate
increases. The net effect is often such that the overall BER
actually decreases even at very low SNR values, and c) as expected,
the variance of the estimate diminishes rapidly and the bias is
reduced as p decreases. [0148] 2) As noted above, the most probable
value of {circumflex over (.rho.)}, denoted M({circumflex over
(.rho.)}) (i.e., the Mode), obtained from evaluation of the
empirical cross-correlation from noisy received vectors is not
necessarily the true value .rho.. This is particularly so at larger
values of .rho. and for small and large values of p. [0149] 3) The
standard deviation of {circumflex over (.rho.)} is independent of
.rho. in the range p=0.1 to .rho.=0.9 for a fixed value of p as
should be suspected. However, this standard deviation is a strong
function of p itself. The standard deviation of {circumflex over
(.rho.)} as a function of p for L=320 (a representative value) has
been evaluated. Analysis shows that the standard deviation indeed
increases slowly with increasing p. Note that even at values of p
as large as p=0.3 this standard deviation is still relatively small
for .rho. in the range .rho.=0.1 to .rho.=0.9.
[0150] While the above analysis has focused on short block length
of L=320, our experimental results suggest that similar conclusions
also hold valid for larger values of L. In assessing how large a
value of L can be used, what is more critical is the performance of
the iterative decoder. The conclusion from the above passage is
that the computation of the empirical cross-correlation between two
received noisy vectors is relatively insensitive to the errors
inflicting the two sequences even at rather large values of error
probability p. Hence, the empirical cross-correlation between two
sequences is robust to channel induced errors.
[0151] The joint iterative decoding of SCCC encoded correlated
sources may be performed in the following way.
[0152] Let the two data sequences be represented by two packets of
data x and y which are correlated. The individual source nodes A
and B independently encode their data using serially concatenated
convolutional codes and transmit the encoded data block over
independent Additive White Gaussian Noise (AWGN) channels. At the
receiver, the sufficient statistics for both sources are processed
jointly. We note that aside from the fact the receiver may a-priori
presume some correlation between the encoded received data might
exist, no side information is communicated to the receiver. The
receiver uses an iterative soft decision decoding technique for
joint detection of the transmitted data sequences. Hence, the
starting point in our development shall be the mathematical
equations needed for joint soft decision decoding.
[0153] Let Z be a random variable in Galois Field GF.sub.2 assuming
values from the set {+1, -1} with equal probability, where +1 is
the "null" element under the modulo-2 addition. As explained in
[1], the log-likelihood ratio of a binary random variable Z is
defined as L Z .function. ( z ) = log .function. [ P Z .function. (
z = + 1 ) P Z .function. ( z = - 1 ) ] , ##EQU37## where P.sub.z(z)
is the probability that the random variable Z takes on the value z.
Under the modulo-2 addition, it is easy to prove that for
statistically independent random variables X and Y the following
relation is valid: P .function. ( X .sym. Y = + 1 ) = P .function.
( X = + 1 ) .times. P .function. ( Y = + 1 ) + ( 1 - p .function. (
X = + 1 ) ) .times. ( 1 - p .function. ( Y = + 1 ) ) ( 32 )
##EQU38## Hence, for Z=X.sym.Y: P Z .function. ( z = + 1 ) = e L Z
.function. ( z ) 1 + e L Z ( z ) ( 33 ) ##EQU39##
[0154] Furthermore, the following approximation holds: L Z
.function. ( z ) = .times. log [ 1 + e L X .function. ( x ) .times.
e L Y .function. ( y ) e L X ( x ) + e L Y ( y ) ] .apprxeq.
.times. sign .function. ( L X .function. ( x ) ) sign .function. (
L Y .function. ( y ) ) min .function. ( L X .function. ( x ) , L Y
.function. ( y ) ) ( 34 ) ##EQU40##
[0155] Soft decision joint iterative decoding of the received
signals can best be described after having described the SCCC
decoder shown in FIG. (6). The SCCC decoder works at the bit level
employing Soft-In Soft-Out (SISO) elementary modules following the
decoding algorithm proposed with some modifications according to
known techniques to use integer arithmetic. In order to keep the
presentation concise, we will only deal with the modifications made
to the SCCC decoder in comparison to the standard decoder.
[0156] In the classic SCCC decoder, at any decoding iteration the
outer SISO decoding module receives the Log-Likelihood Ratios
(LLRs) L(c;I) of its code symbols from the inner SISO, while always
setting the extrinsic information L.sup.(i)(x;I) to zero because of
the assumption that the transmitted source information symbols are
equally likely. In our setup, the joint iterative decoding
algorithm is able to estimate the LLRs L.sup.(i)(x;I) using
cross-correlation information and to pass on this information to
the outer SISO decoding module during the iterations of the SCCC
decoder. Because of this fact, the outer SISO decoder should be
modified in order to account for the non-zero L.sup.(i)(x;I)
values. Let us focus only on these modifications, by omitting the
details of the inner SISO decoder for which the interested reader
can refer to prior art disclosures for additional details.
[0157] The outer SISO decoder operates on the time-invariant
trellis of a generic rate R.sub.0=1/2 convolutional encoder (the
code rate can be different, since in simulations we have used rate
1/2 codes, we make reference to this code rate). Again FIG. (3)
depicts a generic trellis section for such a code. In this figure,
the trellis edge is denoted by e, and the information and code
symbols associated with the edge e are denoted by x(e) and c(e)
respectively. The starting and ending states of the edge e are
identified by s.sup.S(e) and s.sup.E(e) respectively.
[0158] The SISO operates on a block of encoded bits at a time. In
order to simplify the notation, where not specified, x and y
indicate blocks of data bits. Sequence x is composed of the bits
X.sub.k for k=1, . . . , L. A similar notation is used for the
sequence y produced by the other source. Furthermore, we shall
formulate the metric evaluations for the received data associated
with the first source and denoted by x only. This formulation
obviously applies to the received data associated with the other
source y as well. Let us denote the log-likelihood ratio associated
with the information symbol x by L(x), We use the following
notation, as is illustrated in FIG. (6). FIG. (4) shows the
structure of the global decoder when the following modifications
are applied to the figure: a) replace L(c.sub.1;I) and L(c.sub.2;I)
by L(c.sub.1.sup.innI) and L(c.sub.2.sup.inn;I), and b) replace the
MAP decoder block, by SCCC decoder block whose internal structure
is shown in FIG. (6): [0159] L.sup.(i)(x;I) denotes the
log-likelihood ratios of the extrinsic information associated with
the source bits x at the input of the outer SISO decoder at
iteration i of the proposed joint decoding algorithm which shall be
presented shortly. Iteration index i is a global iteration index.
The decoding of each SCCC encoded sequence itself requires a number
of local iterations whose index is hidden for now for simplicity;
[0160] L(c;I) denotes the log-likelihood ratios of the code bits
coming from the inner SISO decoder after the application of the
inverse permutation .PI.; [0161] L(c.sup.inn;I) denotes the
log-likelihood ratios of the coded symbols c.sup.inn at the output
of the matched filter corresponding to the sufficient statistics
from the channel; [0162] L(x;0) denotes the extrinsic
log-likelihood ratios related to the information bits x at the
output of the outer SISO decoder, evaluated under the code
constraints imposed by the outer code; [0163] {circumflex over (x)}
represents the hard estimates of the source bits x (i.e., the
decoded bits after a predefined number of iterations at the output
of the SCCC decoders).
[0164] It is again referred to the SCCC decoder shown in FIG. (6).
The outer encoder at the source receives an input data block of L
bits and generates an output data block of LR.sub.0.sup.-1 bits,
whereby R.sub.0 is the rate of the outer convolutional encoder. It
is also evident that the product LR.sub.0.sup.-1 corresponds to the
size of the interleaver embedded in the SCCC (there is a small
difference in the actual size due to trellis termination of the
outer encoder).
[0165] Let the input bit to the convolutional encoder (for a rate
1/2 code) denoted x.sub.k (e) represent the input bits X.sub.k on a
trellis edge at time k (k=1, . . . , L), and let the corresponding
output symbol of the convolutional encoder C.sub.k(e) at time k be
represented by the output bits c.sub.k,t(e) with t=1,2 and k=1, . .
. , L. Based on these assumptions, the log-likelihood ratios of the
source bits X.sub.k can be evaluated by the outer SISO decoder at
local iteration j of the SCCC as follows: L k ( j ) .function. ( x
k ; O ) = max e : x k .function. ( e ) = 1 * .times. { .alpha. k -
1 .function. [ s S .function. ( e ) ] + t = 1 2 .times. c k , t
.function. ( e ) .times. L k .function. [ c k , t ; I ] + .beta. k
.function. [ s E .function. ( e ) ] } + .times. - max e : x k
.function. ( e ) = 0 * .times. { .alpha. k - 1 .function. [ s S
.function. ( e ) ] + t = 1 2 .times. c k , t .function. ( e )
.times. L k .function. [ c k ; I ] + .beta. k .function. [ s E
.function. ( e ) ] } , k = 1 , .times. , L ( 35 ) ##EQU41## where,
the forward recursion at time k, .alpha..sub.k(.) [2], can be
evaluated through: .alpha. k .function. ( s ) = max e : x k
.function. ( e ) = s * .times. { .alpha. k - 1 .function. [ s S
.function. ( e ) ] + x k .function. ( e ) .times. L k ( i )
.function. [ x k , t ; I ] + t = 1 2 .times. c k , t .function. ( e
) .times. L k .function. [ c k , t ; I ] } + h .alpha. k , k = 1 ,
.times. , L - 1 ( 36 ) ##EQU42## while, the backward recursion,
.beta..sub.k(.), can be evaluated through: .beta. k .function. ( s
) = max e : x k .function. ( e ) = s * .times. { .beta. k + 1
.function. [ s E .function. ( e ) ] + x k + 1 .function. ( e )
.times. L k + 1 ( i ) .function. [ x k + 1 , t ; I ] + t = 1 2
.times. c k + 1 , t .function. ( e ) .times. L k + 1 .function. [ c
k + 1 , t ; I ] } .times. h .beta. k , k = L - 1 , L - 2 , .times.
, 1 ( 37 ) ##EQU43##
[0166] To initialize the above recursions, the following are used:
.alpha. 0 .function. ( s ) = { 0 if .times. .times. s = S 0 -
.infin. otherwise .times. .times. and ( 38 ) .beta. L .function. (
S i ) = { 0 if .times. .times. s = S L - .infin. otherwise ( 39 )
##EQU44## where. S.sub.0 and S.sub.L are the initial and terminal
states of the convolutional codes (assumed to be the all-zero
state). The SISO module operates in the log-domain so that only
summation of terms are needed. The operator max* above signifies
the following: max .function. ( a i ) i * = log .function. [ i = 1
Q .times. e a i ] = max .function. ( a i ) i + .delta. .function. (
a 1 , .times. , a Q ) ( 40 ) ##EQU45## where,
.delta.(.alpha..sub.1, . . . , .alpha..sub.Q) is a correction term
that can be computed using a look-up table. Finally, h.sub.ak and
h.sub..beta.k are two normalization constants that for a hardware
implementation of the SISO are selected to prevent buffer
overflows.
[0167] The bit decisions on the sequence {circumflex over
(x)}.sup.(i) at local iteration j can be obtained from the
log-likelihood ratios of x.sub.k, .A-inverted.k=1, . . . , L by
computing:
L.sub.x.sub.k.sup.(j)=L.sub.k.sup.(j)(x.sub.k;O)+L.sub.k.sup.(i)(x.sub.k;-
I) (41) and making a hard decision on the sign of these metrics. In
the same way, the bit decisions on the sequence y.sup.(j) at
iteration j can be obtained from the log-likelihood ratios of
y.sub.k, .A-inverted.k=1, . . . , L by computing:
L.sub.y.sub.k.sup.(j)=L.sub.k.sup.(j)(y.sub.k;O)+L.sub.k.sup.(i)(y.sub.k;-
I) (42) and making a hard decision on the sign of these
metrics.
[0168] The architecture of the global joint channel decoder is
depicted in FIG. (4) where the following modifications should be
applied to the figure: a) replace L(c.sub.1;I) and L(c.sub.2;I) by
L(c.sup.inn;I) and L(c.sub.2.sup.inn;I), and b) replace the MAP
decoder block, by SCCC decoder block whose internal structure is
shown in FIG. (6). Let us elaborate on the signal processing
involved. In particular, as before let X and Y be two correlated
binary random variables which can take on the values {+1, -1} and
let Z=X.sym.Y. Let us assume that random variable Z takes on the
values {+1, -1} with probabilities P.sub.Z(z=+1)=p.sub.z and
P.sub.Z(z=-1)=1-p.sub.z.
[0169] Both sources independently from each other, encode the
binary sequences x and y with a rate-R.sub.s SCCC. For simplicity,
let us consider a rate-1/4 SCCC constituted by the serial
concatenation of two rate-1/2 convolutional codes. Both encoded
sequences are transmitted over independent AWGN channels. The
received sequences are r.sub.x and r.sub.y which take on values in
.sup.LR.sup.s.sup.-1 ( is the set of real numbers) in the case the
transmitted bits are encoded in blocks of length L. For each
sequence index k, there are R.sub.s.sup.-1 received statistics that
are processed by the decoder. Hence, to each information symbol
x.sub.k, we associate r.sub.x.sub.k,t, t=1, 2, . . . ,
R.sub.S.sup.-1 received statistics. Let N.sub.0/2 denote the
double-sided noise-power spectral density and recall that
.sigma..sub.n.sup.2=N.sub.0/2. With this setup, the log-likelihood
ratios related to the observation samples r.sub.x at the output of
the matched filter can be evaluated as follows: L k .function. ( c
1 , k inn ; I ) = 2 .sigma. n 2 .times. r x k , t , .times. k = 1 ,
.times. , L , .times. t = 1 , 2 , .times. , R S - 1 ( 43 )
##EQU46##
[0170] In the same way, the log-likelihood ratios related to the
observation samples r.sub.y at the output of the matched filter can
be evaluated as follows: L k .function. ( c 2 , k inn ; I ) = 2
.sigma. n 2 .times. r y k , t , .times. k = 1 , .times. , L ,
.times. t = 1 , 2 , .times. , R S - 1 ( 44 ) ##EQU47##
[0171] The log-likelihood ratios L.sub.z.sup.(i)(z) at iteration
(i) are evaluated as follows: L Z ( i ) .function. ( z ) = log
.function. ( 1 - p z ^ p z ^ ) ( 45 ) ##EQU48## by counting the
number of places in which {circumflex over (x)}.sup.(i) and
y.sup.(i) differ, or equivalently by evaluating the Hamming weight
w.sub.H (.) of the sequence {circumflex over
(z)}.sup.(i)={circumflex over (x)}.sup.(i).sym.y.sup.(i) whereby,
in the previous equation, p z ^ = w H .function. ( z ^ ( i ) ) L .
##EQU49## In the latter case, by assuming that the sequence {right
arrow over (Z)}={right arrow over (X)}.sym.{right arrow over (Y)}
is i.i.d., we have: L Z ( i ) .function. ( z ) = log .function. ( L
- w H .function. ( z ^ ( i ) ) w H .function. ( z ^ ( i ) ) ) ( 46
) ##EQU50## where L is the data block size. Finally, applying
equation (34) we can obtain an estimate of the extrinsic
information on the source bits for the next iteration:
L.sup.(i)(x;I)=L({circumflex over (z)}.sup.(i-1).sym.y.sup.(i))
(47) and L.sup.(i)(y;I)=L({circumflex over
(z)}.sup.(i-1).sym.{circumflex over (x)}.sup.(i)) (48)
[0172] Note that as far as the LLR of the difference sequence z is
concerned, a correlation of for instance 10% or 90% between x and y
carry the same amount of information. Hence, the performance gain
of the iterative joint decoder in either case is really the same
(we have verified this experimentally). From an information
theoretic point of view, all this says is that the entropy of the
random variable Z is symmetric about the 50% correlation point.
[0173] Formally, the joint decoding algorithm can be formalized as
follows: [0174] 1) Set the iteration index i=0 and set the
log-likelihood ratios L.sup.(0)(x;I) and L.sup.(0)(y;I) to zero
(see FIG. (6)). Compute the log-likelihood ratios for the channel
outputs using equations (43) and (44) for both received sequences
r.sub.x and r.sub.y. Conduct a preliminary set of iterations of
each SCCC decoder in order to obtain an estimate of both sequences
{circumflex over (x)}.sup.(0) and y.sup.(0) and evaluate w.sub.H
({circumflex over (z)}(0))=w.sub.H ({circumflex over
(x)}.sup.(0).sym.y.sup.(0)). Use w.sub.H ({circumflex over
(z)}.sup.(0)) to evaluate L.sub.z.sup.(0)(z) in equation (46). Note
that if the receiver already has an estimate of the correlation
between the two transmitted sequences x and y (i.e., with side
information), it can directly evaluate equation (46). In our
simulations, we do not assume the availability of any side
information. [0175] 4. Set L.sup.(l)(x;I) and L.sup.(l)(y;I) to
zero. [0176] 5. For iteration i=1, . . . , q, perform the
following: [0177] a) Make a predefined total number of iterations
of the SCCC decoder for both received sequences r.sub.x and r.sub.y
by using the log-likelihood ratios as expressed in equations (43)
and (44). [0178] b) Evaluate L.sub.z.sup.(i)(z) using equations
(46). [0179] c) Evaluate L.sub.(i)(x;I) by using
L.sub.z.sup.(i-1)(z) and L.sup.(i)(y;O). Evaluate L.sup.(i)(y;I) by
using L.sub.z.sup.(i-1)(z) and L.sup.(i)(x;O). [0180] d) Go back to
(a) and continue until the last iteration q.
[0181] As it is possible to see from the algorithm, the joint
decoder at any global iteration i estimates the extrinsic
log-likelihood ratios L.sup.(i) (x;I) and L.sup.(i) (y;I) by using
the new estimates of the source bits {circumflex over (x)}.sup.(i)
and y.sup.(i) and the previous estimate on the difference sequence
{circumflex over (z)}.sup.(i-1) (note that LLRs L.sup.(i) (x;I) and
L.sup.(i) (y;I) are supplied to the outer decoder in the respective
SCCCs). Note that there is no need for subtracting the available
a-priori information (e.g., from the previous iteration), from one
global iteration to the next. Looking at the SCCC decoder for one
of the two sources at a given global iteration, the updated
estimate of the cross-correlation is used to generate a-priori soft
information on the source bits that are combined with the intrinsic
information derived from the channel to restart a sequence of local
decoding iterations in the SCCC decoder. On the other hand,
extrinsic information generated by a given block at iteration (p-1)
within the SCCC iterative decoding loop must be subtracted at
iteration p for proper processing.
[0182] We have conducted simulations of the proposed iterative
joint channel decoder for correlated sources to verify
functionality and assess the potential gains of the approach. A
sample simulation result for a rate-1/4 SCCC obtained from a serial
concatenation of an outer encoder with generator matrix G
.function. ( D ) = [ 1 , 1 + D + D 3 + D 4 1 + D + D 4 ] ##EQU51##
and an inner encoder with generator matrix G .function. ( D ) = [ 1
, 1 - D 2 1 + D + D 2 ] ##EQU52## and employing a spread-25
Fragouli/Wesel interleaver of length 640 [3] is shown in FIG. (7a).
In the figure, we show the BER and FER performance on the
individual SCCCs (without joint decoding) after 35 iterations for
comparison purposes. We have verified that more than 35 iterations
did not yield further performance improvement of the individual
SCCC BER and FER. In the same figure we show the performance of the
proposed iterative joint channel decoder after 5 global iterations
of the proposed algorithm whereby during each global iteration 10
local iterations of the individual SCCCs have been conducted using
MAX*-Log-MAP algorithm with 3-bit quantization as specified in [4].
The simulation results reflect the performance of the iterative
joint decoder for various correlation coefficients between the two
sequences. All simulations have been conducted by counting 100
erroneous frames. The assumed modulation format is BPSK. The number
of the preliminary iterations to initialize the global iteration
was set to 12. To have an idea of the maximum achievable
performance of the proposed algorithm, we show the performance in
the case of 100% correlation existing between the two sequences
(i.e., the case in which the two sequences are identical).
[0183] To see the impact of global iterations, simulation results
shown in FIG. (7b) refer to the same rate-1/4 SCCC as above and
depict the performance of the iterative joint decoder as a function
of the number of global iterations. In the figure, we show the BER
of the individual SCCCs after 35 iterations without iterative joint
decoding, and with 2, 5, 9 and 13 global iterations of the proposed
decoder, during each one of which, 3 local iterations of
MAX*-Log-MAP algorithm have been applied for decoding of the
individual SCCC codes. The simulation results are for a reference
correlation coefficient of 70%. The number of the preliminary
iterations to initialize the global iterations was set to 12.
[0184] To verify some of the theoretical results in connection with
the estimation of the cross-correlation coefficient in the case of
real decoding (recall that we assumed the error sequences were
i.i.d in our analysis, and this is clearly not the case during
actual joint decoding of the SCCC codes), we have compiled data
from several simulation runs on the same SCCC codes as above
employing the iterative joint decoder and have generated several
empirical curves. In particular, FIG. (7c) shows the estimated
.rho. at the end of the final global decoding iteration as a
function of SNR E.sub.b/N.sub.0 for various block lengths and
various degrees of correlation between the data generated by the
correlated sources. Note the dependence of the estimate on SNR and
the data block length and the existence of the bias all of which
were predicted by the theoretical analysis. Finally, FIG. (7d)
depicts the variance of the estimate of .rho. at the and of the
final global decoding iteration as a function of SNR
E.sub.b/N.sub.0 for various block lengths and various degrees of
correlation between the data generated by the correlated sources.
Once again the dependence of the variance on SNR and the data block
length was correctly predicted by the simplified theoretical
analysis.
[0185] Joint Iterative LDPC-Decoding of Correlated Sources
[0186] Let the two data sequences be represented by two packets of
data which are correlated. The individual source nodes A and B
independently encode their data using LDPC codes and transmit the
encoded data block over independent Additive White Gaussian Noise
(AWGN) channels. At the receiver, the sufficient statistics for
both sources are processed jointly. We note that aside from the
fact the receiver may a-priori presume some correlation between the
encoded received data might exist, no side information is
communicated to the receiver.
[0187] In what follows, we shall briefly review the sum-product
algorithm in order to highlight the way in which the extrinsic
information can be exploited by the LDPC decoder in a joint
decoding paradigm. For the sake of exploiting the extrinsic
information in the LDPC decoder, the LDPC matrix for encoding each
source is considered as a systematic (n,k) code. Each codeword c is
composed of a systematic part u and a parity part p.sub.u, which
together form c=[u, p.sub.u]. With this setup and given parity
check matrix H.sup.n-k,n of the LDPC code, it is possible to
decompose H.sup.n-k,n as follows: H.sup.n-k,n=(H.sup.u,
H.sup.P.sup.u) (49) whereby, H.sup.u is a (n-k).times.(k) matrix
specifying the source bits participating in check equations, and
H.sup.P.sup.u is a (n-k).times.(n-k) matrix of the form: H p u = (
1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 ) ( 50 ) ##EQU53##
[0188] The choice of this structure for H has been motivated by the
fact that aside from being systematic, we obtain a LDPC code which
is encodable in linear time in the codeword length n. In
particular, with this structure, the encoding operation is as
follows: p u i = { [ j = 1 k .times. u j H i , j u ] .times. ( mod
.times. .times. 2 ) i = 1 [ p u i - 1 + j = 1 k .times. u j H i , j
u ] .times. ( mod .times. .times. 2 ) i = 2 , .times. , n - k ( 51
) ##EQU54## where, H.sub.j,k.sup.u represents the element (i,j) of
the matrix H.sup.u, and u.sub.j is the j-th bit of the source
sequence u.
[0189] The starting point in our development shall be the
mathematical development behind joint soft decision decoding of
LDPC codes.
[0190] Let Z be a random variable in Galois Field GF(2) assuming
values from the set {+1, -1} with equal probability, where +1 is
the "null" element under the modulo-2 addition. As explained in
[1], the log-likelihood ratio of a binary random variable Z is
defined as L Z .function. ( z ) = log .function. [ P Z .function. (
z = + 1 ) P Z .function. ( z = - 1 ) ] , ##EQU55## where P.sub.Z(z)
is the probability that the random variable Z takes on the value z.
Under the modulo-2 addition, it is easy to prove that for
statistically independent random variables X and Y the following
relation is valid: P .function. ( X .sym. Y = + 1 ) = P .function.
( X = + 1 ) .times. P .function. ( Y = + 1 ) ++ .times. ( 1 - p
.function. ( X = + 1 ) ) .times. ( 1 - p .function. ( Y = + 1 ) ) (
52 ) ##EQU56##
[0191] Hence, for Z=X.sym.Y: P Z .function. ( z = + 1 ) = e L Z
.function. ( z ) 1 + e L Z .function. ( z ) ( 53 ) ##EQU57##
[0192] Furthermore, the following approximation holds: L Z
.function. ( z ) = log .function. [ 1 + e L X .function. ( x )
.times. e L Y .function. ( y ) e L X .function. ( x ) + e L Y
.function. ( y ) ] .apprxeq. sign .function. ( L X .function. ( x )
) sign .function. ( L Y .function. ( y ) ) min ( L X .function. ( x
) , | L ( 54 ) ##EQU58##
[0193] The LDPC decoder operates on a block of encoded data at a
time. In order to simplify the notation, boldface u.sub.1 indicate
blocks of data bits, while u.sub.ij indicate the j-th bit in a
frame. Sequence u.sub.1 is composed of the bits u.sub.lj for j=1, .
. . , k. A similar notation is used for the sequence u.sub.2.
Furthermore, we shall formulate the metric evaluations for the
received data associated with other source u.sub.2 as well. Let us
denote the log-likelihood ratio associated with the information
bits u.sub.1 by L(u.sub.1), thus avoiding the use of a subscript
equal to the name of random variable. With reference to the
architecture of the joint decoder depicted in FIG. (8), we note
that there are two stages of iterative decoding. Index i denotes
global iteration whereby during each global iteration, the updated
estimate of the source correlation obtained during the previous
global iteration is passed on the sum-product decoder that performs
local iteration with a pre-defined stopping criterion and/or a
maximum number of local decoding iterations. With reference to such
an architecture, we use the following notation: [0194]
L.sub.ex.sup.(i-1)(u.sub.1) and L.sub.ex.sup.(i-1)(u.sub.2) denote
the log-likelihood ratios of the extrinsic information associated
with the estimated source bits u.sub.1 and u.sub.2 at the input of
the LDPC decoders; [0195] L.sub.c(r.sub.1) and L.sub.c(r.sub.2)
denote the log-likelihood ratios of the encoded bits coming from
the channel at the output of matched filter at the receiver; [0196]
L.sup.(i)(u.sub.1) and L.sup.(i)(u.sub.2 ) denote the
log-likelihood ratios related to the estimated information bits
u.sub.1 and u.sub.2 at the output of the LDPC decoders; [0197]
u.sub.1.sup.(i) and u.sub.2.sup.(i) represents the hard estimates
of the transmitted source bits u.sub.1 and u.sub.2.
[0198] Based on the notation above, we can now develop the
algorithm for exploiting the source correlation in the LDPC
decoder. Consider a (n,k)-LDPC identified by the matrix
H.sup.(n-k,n) as expressed in (49). Note that we only make
reference to maximum rank matrix H since the particular structure
assumed for H ensure this. In particular, the double diagonal on
the parity side of the H matrix always guarantees that the rank of
H is equal to the number of its rows, n-k.
[0199] It is well known that the parity check matrix H can be
described by a bipartite graph with two types of nodes: n bit-nodes
corresponding to the LDPC code bits, and n-k check nodes
corresponding to the parity cheeks as expressed by the rows of the
matrix H. Let B(m) denote the set of bit-nodes connected to the
m-th check node, and C(n) denote the set of the check-nodes
adjacent to the n-th bit-node. With this setup, B(m) corresponds to
the set of positions of the 1's in the m-th row of H, while C(n) is
the set of positions of the 1's in the n-th column of H. In
addition, let us use the notation C(n)m and B(m)n to mean the sets
C(n) and B(m) in which the m-th check-node and the n-th bit-node
respectively, are excluded. Furthermore, let us identify with
.lamda..sub.n,m(u.sub.n) the log-likelihood of the message that the
n-th bit-node sends to the m-th check-node, that is, the LLR of the
probability that n-th bit-node is 1 or 0 based on all checks
involving the n-th bit except the m-th check, and with
.LAMBDA.n,m(Un) the log-likelihood of the message that the m-th
check-node sends to the n-th bit-node, that is, the LLR of the
probability that n-th bit-node is 1 or 0 based on all the bit-nodes
checked by the m-th check except the information coming from the
n-th bit-node. With this setup, we have the following steps of the
sum-product algorithm:
[0200] Initialization Step: each bit-node is assigned an
a-posteriori LLR composed of the sum of the a-posteriori
probabilities L.sub.c(r) evaluated from the sufficient statistic
from the matched filter as follows: L c .function. ( r 1 , j ) =
log .function. ( P .function. ( u 1 , j = 1 | r 1 , j ) P
.function. ( u 1 , j = 0 | r 1 , j ) ) = 2 .sigma. n 2 .times. r 1
, j , .A-inverted. j = 1 , .times. , n ( 55 ) ##EQU59## plus an
extrinsic LLR added only to the systematic bit nodes, i.e., to the
bit nodes u.sub.i,j, j=1, . . . , k . In (55), .sigma..sub.n.sup.2
is the noise variance at the matched filter output due to the AWGN
channel. In summary, for any position (m,n) such that H.sub.m,n=1,
set: .lamda. m , n .function. ( u n ) = { L c .function. ( r 1 , j
) + L ex ( i - 1 ) .function. ( u ^ 1 , j ) j = 1 , .times. , k L c
.function. ( r 1 , j ) j = k + 1 , .times. , n .times. .times. and
( 56 ) .LAMBDA. m , n .function. ( u n ) = 0 ( 57 ) ##EQU60##
[0201] (1) Check-node update: for each m=1, . . . , n-k, and for
each n .di-elect cons.B(m), compute: .LAMBDA. m , n .function. ( u
1 , n ) = 2 .times. .times. tanh - 1 ( p .di-elect cons. B
.function. ( m ) .times. \ .times. .times. n .times. tanh
.function. ( .lamda. p , m .function. ( u 1 , p ) 2 ) ) ( 58 )
##EQU61## [0202] (2) Bit-node update: for each t=1, . . . , n, and
for each m .di-elect cons.C(t), compute: .lamda. t , m .function. (
u 1 , t ) = L c .function. ( r 1 , t ) + L ex ( i - 1 ) .function.
( u ^ 1 , t ) + p .di-elect cons. C .function. ( t ) .times. \
.times. .times. m .times. .LAMBDA. p , n .function. ( u 1 , t ) (
59 ) ##EQU62## [0203] (3) Decision: for each node u.sub.1,t with
t=1, . . . , n, compute: .lamda. t .function. ( u 1 , t ) = { L c
.function. ( r 1 , t ) + L ex ( i - 1 ) .function. ( u ^ 1 , t ) +
p .di-elect cons. C .function. ( t ) .times. .LAMBDA. p , n
.function. ( u 1 , t ) t = 1 , .times. , k L c .function. ( r 1 , t
) + p .di-elect cons. C .function. ( t ) .times. .LAMBDA. p , n
.function. ( u 1 , t ) t = k + 1 , .times. , n ( 60 ) ##EQU63##
[0204] and quantize the results such that u.sub.1,t=0 if
.lamda..sub.t(u.sub.1,t) <0, and u.sub.1,t=1 otherwise.
[0205] If Hu.sub.1.sup.T=0 then halt the algorithm and output
u.sub.1,t, t=1, . . . , k, as the estimate of the transmitted
source bits, u.sub.l, corresponding to the first source. Otherwise,
if the number of iterations is less than a predefined maximum
number, iterate the process starting from step (1). The
architecture of the joint channel decoder is depicted if FIG. (8).
Let us elaborate on the signal processing involved. In particular,
as before let u.sub.l and u.sub.2 be two correlated binary random
variables which can take on the values {+1, -1} and let
z=u.sub.l.sym.u.sub.2. Let us assume that random variable z takes
on the values {+1, -1 } with probabilities P.sub.Z(z=+1)=p.sub.z
and P.sub.Z(z=-1)=1-p.sub.z.
[0206] The log-likelihood ratios L.sub.z.sup.(i)(z) related at
global iteration (i) are evaluated as follows: L Z ( i ) .function.
( z ) = log .function. ( 1 - p z ^ p z ^ ) ( 61 ) ##EQU64## by
counting the number of places in which u.sub.1.sup.(i) and
u.sub.2.sup.(i) differ, or equivalently by evaluating the Hamming
weight w.sub.H(.) of the sequence {circumflex over
(z)}.sup.(i)=u.sub.1.sup.(i).sym.u.sub.2.sup.(i) whereby, in the
previous equation, p z ^ = w H .function. ( z ^ ( i ) ) L .
##EQU65## In the latter case, by assuming that the sequence
z=u.sub.1.sym.u.sub.2 is i.i.d., we have: L Z ( i ) .function. ( z
) = ( k - w H .function. ( z ^ ( i ) ) w H .function. ( z ^ ( i ) )
) ( 62 ) ##EQU66## where k is the data block size.
[0207] Finally, applying (54) we can obtain an estimate of the
extrinsic information on the source bits for the next iteration:
L.sub.ex.sup.(i)(u.sub.1)=L({circumflex over
(z)}.sup.(i).sym.u.sub.2.sup.(i)) (63) and
L.sub.ex.sup.(i)(u.sub.2)=L({circumflex over
(z)}.sup.(i).sym.u.sub.1.sup.(i)) (64)
[0208] Note that as far as the LLR of the difference sequence Z is
concerned, a correlation of for instance 10% or 90% between u.sub.1
and U.sup.2 carry the same amount of information. Hence, the
performance gain of the iterative joint decoder in either case is
really the same (we have verified this experimentally). From an
information theoretic point of view, all this says is that the
entropy of the random variable Z is symmetric about the 50%
correlation point.
[0209] Formally, the joint decoding algorithm can be formalized as
follows: [0210] 1) Set the log-likelihood ratios
L.sub.ex.sup.(0)(u.sub.1) and L.sub.ex.sup.(0)(u.sub.2) to zero
(see FIG. (8)). Compute the log-likelihood ratios for the channel
outputs using (55) for both received sequences r.sub.1 and r.sub.2.
[0211] 2) For iteration i=1, . . . , q, perform the following:
[0212] a) Perform a sum-product decoding for both received
sequences r.sub.1 and r.sub.2 by using a predefined maximum number
of iterations and extrinsic information L.sub.ex.sup.(i-1)(u.sub.1)
and L.sub.ex.sup.(i-1)(u.sub.2); [0213] b) Evaluate
L.sub.z.sup.(i)(z) using equation (62); [0214] c) Evaluate
L.sub.ex.sup.(i)(u.sub.1) and L.sub.ex.sup.(i)(u.sub.2) by using
(63) and (64); [0215] d) Go back to (a) and continue until the last
global iteration q.
[0216] Simulation Results
[0217] We have simulated the performance of our proposed iterative
joint channel decoder. We assume that our transmit nodes use the
same LDPC codes and the SNR of each of the two received sequences
are the same.
[0218] In the following, we provide sample simulation results
associated with a (n, k)=(504, 252) LDPC code designed in order to
reduce the number of length 4 and 6 cycles in the Tanner graph of
the code. In particular, the designed H has only six cycles of
length 6 and 12184 cycles of length 8. Average degree distributions
of the bit- and check-nodes of the considered LDPC are,
respectively, 3 and 6. Furthermore, sub-matrix H.sup.U has been
designed with an uniform bit-node degree distribution equal to 4.
For local decoding of the LDPC code, the maximum number of
iterations has been set to 80. We note that as far as the matrix H
of the LDPC is concerned, it is clear that any design criteria
already proposed in the literature can be employed, provided that
the LDPC considered is systematic.
[0219] The simulation results are reported as follows: [0220] 1)
FIG. (9a) shows the BER of the correlated sources for a correlation
coefficient p=0.99, and for various number of global iterations.
For comparison purposes, the curve labeled "LDPC(504,252)-80 it."
shows the performance of the LDPC without using the implicit
correlation information. Several observations are in order: [0221]
a) 3- or 4- global iterations suffice to get almost all that can be
gained from the knowledge of the cross-correlation. We note that
our comparison of the simulation results with the analytical
performance bounds presented below, reinforce this statement;
[0222] b) since the estimates of the cross-correlation are noisy at
sufficiently low SNR levels, decoding iterations are critical for
improving performance, otherwise 2-iterations are often sufficient
to obtain most of the achievable gain; [0223] c) as the
cross-correlation approaches 0.5, the achievable gains diminish as
expected and reduces to zero at p=0.5. This implies that when the
two sequences are totally uncorrelated, the performance of the
iterative joint channel decoder is no better than the case each
received sequence is independently decoded. On the other hand, when
the cross-correlation level is nearly one or zero, the achievable
coding gain is a function of the operating BER and diminishes as
BER decreases. [0224] 2) FIG. (9b) shows the BER as a function of
SNR and .rho. at the end of four decoding iterations.
[0225] FIG. (9c) shows simulation results and comparison to
upper-bound (denoted UB) at two values of .rho.. In FIG. (9d) we
show the empirical density functions of the LLR values that tend to
be Gaussian. In FIG. (9e) we show in a table the average number of
local iterations performed by the joint decoder at the end of a
given global iteration, for two values of correlation between the
sources. For comparison, we show the average number of local
iterations performed by the LDPC decoder without using extrinsic
information derived from source correlation. It is evident that
aside from the raw coding gain, there is a significant speed-up of
the sum-product decoder with increasing number of global
iterations.
[0226] As a result, the methods described herein provide a
technique for enhancing the decoding of channel encoded data by
exploiting an inherent correlation between individual data packets.
For instance, soft decision criteria are adjusted on the basis of a
value characterizing the inherent correlation.
REFERENCES
[0227] [1] J. Hagenauer, E. Offer, and L. Papke, "Iterative
decoding of binary block and convolutional codes," IEEE Trans. on
Inform. Theory, vol.42, no.2, pp.429-445, March 1996. [0228] [2] L.
R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of
linear codes for minimizing symbol error rate," IEEE Trans. On
Inform. Theory, vol.IT-20, pp.284-287, May 1974. [0229] [3] C.
Fragouli and R. D. Wesel, "Semi-random interleaver design
criteria," Proc. IEEE Globecom 1999, vol.5, pp.2352-2356, December
1999. [0230] [4] G. Montorsi and S. Benedetto, "Design of
fixed-point iterative decoders for concatenated codes with
interleavers," IEEE Journal on Selected Areas in Communications,
Vol.19, No.5, pp.871-882, May 2001.
* * * * *