U.S. patent application number 12/954152 was filed with the patent office on 2011-09-01 for method, device and system for the fusion of information originating from several sensors.
This patent application is currently assigned to THALES. Invention is credited to Frederic PICHON.
Application Number | 20110213749 12/954152 |
Document ID | / |
Family ID | 42224738 |
Filed Date | 2011-09-01 |
United States Patent
Application |
20110213749 |
Kind Code |
A1 |
PICHON; Frederic |
September 1, 2011 |
METHOD, DEVICE AND SYSTEM FOR THE FUSION OF INFORMATION ORIGINATING
FROM SEVERAL SENSORS
Abstract
The invention relates to a method, device and system for fusion
of information originating from several sensors. The invention
includes a mechanism for fusion of belief functions. To apply this
mechanism, various information, knowledge and operations are
modelled within the framework of the theory of belief functions:
information provided by the sensors, knowledge regarding the
propensity of the sensors to be in a given operating state, and
merge operators for each operating state considered.
Inventors: |
PICHON; Frederic; (Paris,
FR) |
Assignee: |
THALES
Neuilly-sur-Seine
FR
|
Family ID: |
42224738 |
Appl. No.: |
12/954152 |
Filed: |
November 24, 2010 |
Current U.S.
Class: |
706/59 |
Current CPC
Class: |
G06K 9/6288 20130101;
G06K 2009/6294 20130101 |
Class at
Publication: |
706/59 |
International
Class: |
G06N 5/02 20060101
G06N005/02 |
Foreign Application Data
Date |
Code |
Application Number |
Nov 27, 2009 |
FR |
09 05724 |
Claims
1. A method for the fusion of information originating from several
sensors (C.sub.1, C.sub.2) each producing a belief function
(m.sub.C1, m.sub.C2) representing knowledge on a reasoning space
(.OMEGA.), and comprising several states (E1, . . . , EN) each
representing an operating state of the set of sensors, said method
comprising: calculating a merged belief function (m.sub.F); and
calculating conditional belief functions on the basis of the belief
functions (m.sub.C1, m.sub.C2) produced by the sensors (C.sub.1,
C.sub.2) and of a predefined base of operators (105) comprising
associations between states of sensors and merge operators, wherein
a conditional belief function represents knowledge of a sensor on
the reasoning space (.OMEGA.) when the set of sensors is in a given
operating state (E1, . . . , EN), and wherein a step of calculating
the merged belief function (m.sub.F) uses the conditional belief
functions calculated and knowledge (m.sup.E) arising from a
knowledge base (104) regarding the operating state of the
sensors.
2. The method for fusion of information according to claim 1,
wherein the knowledge regarding the operating state of the sensors
is represented by a belief function (m.sup.E) indicating the
propensity of the sensors (C.sub.1, C.sub.2) to be in a given
operating state.
3. The method for fusion of information according to claim 2,
wherein the merged belief function (m.sub.F) follows the following
relation: m.sub.F=((.sym..sub.i=1, . . .
,N.sup.Dm.sup..OMEGA.[Ei.sup.E.times..OMEGA.).sym..sup.Dm.sup.E.uparw.E.t-
imes..OMEGA.).sup..dwnarw..OMEGA. where .sym..sup.D is an operator
called the unnormalized Dempster's rule, m.sup..OMEGA.[Ei], with
i=1, . . . , N, are conditional belief functions representing
knowledge on the reasoning space (.OMEGA.) when the sensors are in
various states (E1, . . . , EN), N being the number of operating
states, and m.sup.E is knowledge regarding the operating state of
the sensors, .uparw., .dwnarw. and being operations for
manipulating belief functions called respectively vacuous
extension, marginalization and deconditioning.
4. The method for fusion of information according to claim 1,
wherein the step of calculating the merged belief function
(m.sub.F) uses contextual information indicating that, if an object
observed by the sensors (C.sub.1, C.sub.2) is of type A, then the
sensors have a propensity p to be in a set of states E', where A is
a subset of the space of propositions (.OMEGA.) and E' is a subset
of a set E of the various states (E1, . . . , EN) of the
sensors.
5. The method for fusion of information according to claim 4,
characterized in that the merged belief function (m.sub.F) follows
the following relation: m.sub.F=((.sym..sub.i=1, . . .
,N.sup.Dm.sup..OMEGA.[Ei.sup.E.times..OMEGA.).sym..sup.D(.sym..sub.i=1,
. . . ,K.sup.Dm.sup.E[Aj.sup.E.times..OMEGA.)).sup..dwnarw..OMEGA.
where .sym..sup.D is an operator called the unnormalized Dempster's
rule, m.sup..OMEGA.[Ei], with i=1, . . . , N, are conditional
belief functions representing knowledge on the reasoning space
(.OMEGA.) when the sensors are in various states (E1, . . . , EN),
N being the number of operating states, and m.sup.E[Aj] is
contextual knowledge regarding the propensity of the sensors to be
in a given operating state when an object observed by the sensors
is of type A.sub.j, with j=1, . . . , K, A.sub.j being a subset of
.OMEGA. and such that A.sub.1, . . . , A.sub.K forms a partition of
.OMEGA., .uparw., .dwnarw. and being operations for manipulating
belief functions called respectively vacuous extension,
marginalization and deconditioning.
6. A device for fusion of information originating from several
sensors each producing a belief function representing knowledge on
a reasoning space and comprising several operating states, the
device comprising: means for the calculation of a merged belief
function; means for the calculation of conditional belief
functions, said means being linked to the sensors and to a
predefined base of operators comprising associations between states
of sensors and merge operators, a conditional belief function
representing knowledge of a sensor on the reasoning space when said
sensor is in a given operating state, and wherein the means for the
calculation of the merged belief function uses the conditional
belief functions calculated and knowledge regarding the operating
state of the sensors originating from a knowledge base regarding
the operating state of the sensors.
7. The device for fusion of information according to claim 6,
wherein the knowledge base regarding the operating state of the
sensors comprises belief functions indicating the propensity of the
sensors to be in a given operating state.
8. The device for fusion of information according to claim 6,
wherein the knowledge base regarding the operating state of the
sensors comprises contextual information indicating that, if an
object observed by the sensors is of type A then the sensors have a
propensity p to be in a set of states E', where A is a subset of
the space of propositions and E' is a subset of a set E of the
various states (E1, . . . , EN) of the sensors.
9. A system for fusion of information comprising: at least two
sensors each producing a belief function; and means for fusion of
the belief functions originating from said sensors, said fusion
means being linked to the sensors, wherein the means for fusion of
information comprises the device for fusion of information
according to claim 6.
10. A system for fusion of information comprising: at least two
sensors each producing a belief function; and means for fusion of
the belief functions originating from said sensors, said fusion
means being linked to the sensors, wherein the means for fusion of
information comprises the device for fusion of information
according to claim 7.
11. A system for fusion of information comprising: at least two
sensors each producing a belief function; and means for fusion of
the belief functions originating from said sensors, said fusion
means being linked to the sensors, wherein the means for fusion of
information comprises the device for fusion of information
according to claim 8.
Description
[0001] The invention relates to the fusion of information
originating from several sensors and more particularly to the
fusion of imperfect information originating from sensors whose
operating state is known.
[0002] Systems integrating several sensors are used in a great
variety of sectors such as site surveillance, maintenance,
robotics, medical diagnosis or meteorological forecasts. Such
systems carry out, for example, functions of classification,
identification and tracking in real time.
[0003] To best exploit multi-sensor systems, it is necessary to use
an effective scheme for fusion of information so as to combine the
data originating from the various sensors of the system and
generate a decision.
[0004] According to the known art, certain information fusion
schemes rely on Dempster-Shafer theory and thus use belief
functions, which is the basic tool for representing information in
this theory. Belief functions are known for their capacity to
faithfully represent imperfect information. Information comprising
an imprecision or an uncertainty or incomplete information is
called imperfect information. The sensors of a system are
considered to be imperfect information sources, notably because of
their imprecision. The term sensor is understood here in the broad
sense. It includes physical devices for data acquisition (camera,
radar, etc.) but also devices for processing these data. It is
possible to establish a belief function on the basis of the data
provided by most commercially available sensors. Schemes for
combining belief functions may be used. By dint of their nature,
these schemes are therefore particularly appropriate to the problem
of the fusion of imperfect information arising from sensors.
[0005] Let X be a variable with values in a finite set .OMEGA.. The
information held by a sensor as regards the value actually taken by
X may be quantified by a belief function. A belief function is
formally defined as a function, denoted bel, from the power set of
.OMEGA., denoted 2.sup..OMEGA., in the interval [0,1] and
satisfying certain mathematical properties. The quantity bel(A)
represents the total degree of belief allocated by the sensor to
the fact that the value of X is in A, where A is a part (also
called a subset) of .OMEGA. (this being written A.OR
right..OMEGA.). There exist various equivalent representations of a
belief function, which are useful in practice. In particular, the
mass function, denoted m, is a function from 2.sup..OMEGA. to [0,1]
which satisfies the condition:
A .OMEGA. m ( A ) = 1. ##EQU00001##
[0006] The quantity m(A), called the mass of A, represents the
degree of belief allocated to A (and to no strict subset). The
belief function bel associated with a mass function m is obtained
in the following manner:
bel(A)=.SIGMA..sub.O.noteq.B.OR right.Am(B),
for all A.OR right..OMEGA.. The mass function m associated with a
belief function bel is obtained in the following manner:
m(A)=.SIGMA..sub.O.noteq.B.OR right.A(-1).sup.|A|-|B|bel(B),
for all A.OR right..OMEGA.. By dint of the one-to-one
correspondence between a belief function bel and its associated
mass function m, we will use the name Belief Function (BF) in the
broad sense in this document to designate both a belief function
bel and also its associated mass function m (the context is
generally sufficient to determine whether the objects manipulated
are mass functions m or belief functions bel).
[0007] It is possible to consider by way of nonlimiting example, a
multi-sensor system of classifier type used for the optical
recognition of hand-written characters. It is assumed that the
system is intended to determine whether the character formed on an
image I (not represented) is one of the letters `a` or `b`. Let X
be the variable associated with the character. We therefore have a
set of values .OMEGA.={a, b} for the variable (the character) X.
Each of the sensors of the system is a classifier which itself
provides an item of information regarding the character to be
recognized in the form of a BF. It is assumed that there are two
sensors of this type in our example. The sensor C.sub.1 provides a
belief function m.sub.C1 defined as follows:
m.sub.C1(O)=0
m.sub.C1({a})=0.8
m.sub.C1({b})=0
m.sub.C1({a,b})=0.2
[0008] In this example, the sensor C.sub.1 considers that it is
highly probable (0.8) that the character to be recognized is {a}.
m.sub.C1({a,b})=m.sub.C1(.OMEGA.)=0.2 represents the sensor's
ignorance. The sensor C.sub.2 provides a belief function m.sub.C2
defined as follows:
m.sub.C2(O)=0
m.sub.C2({a})=0.2
m.sub.C2({b})=0
m.sub.C2({a,b})=0.8
[0009] The function m.sub.C2 conveys the fact that the sensor
C.sub.2 has fairly little information regarding the character
observed: the sensor ascribes a small degree of belief (0.2) to the
character {a} and a large degree of belief (0.8) to ignorance, that
is to say to the set .OMEGA.={a, b}.
[0010] Dempster-Shafer theory makes it possible to combine the
belief functions representing information arising from different
sources, so as to obtain a belief function taking into account the
influences of each of the sources. The belief function thus
obtained, called the merged belief function, represents the
combined knowledge of the various imperfect information sources
(the sensors).
[0011] In order to obtain a merged belief function representative
of reality, it is necessary to take into account knowledge
pertaining to the states of the sources.
[0012] The known information fusion schemes only make it possible
to take into account certain types of knowledge regarding the
states of the sources (sensors). Indeed, the existing solutions are
limited to the consideration of particular knowledge regarding the
dependency, the competence and the sincerity of the sources. For
example, a merge operator making it possible to merge information
arising from independent and competent sources is already known.
This operator is known in the literature by the name "unnormalized
Dempster's rule". We denote this operator by the symbol
.sym..sup.D. Thus, the BF resulting from the fusion by .sym..sup.D
of two BFs m.sub.C1 and m.sub.C2 is denoted
m.sub.C1.sym..sup.Dm.sub.C2. The definition of the operator
.sym..sup.D is as follows. Consider two belief functions m.sub.C1
and m.sub.C2, we have, for all A.OR right..OMEGA.:
m C 1 .sym. D m C 2 ( A ) = B C = A m C 1 ( B ) m C 2 ( C ) .
##EQU00002##
[0013] Applying the formula to the two belief functions m.sub.C1
and m.sub.C2 given as an example above, we obtain:
m C 1 .sym. D m C 2 ( { a } ) = m C 1 ( { a } ) m C 2 ( { a } ) + m
C 1 ( { a } ) m C 2 ( { a , b } ) + m C 1 ( { a , b } ) m C 2 ( { a
} ) = 0.8 * 0.2 + 0.8 * 0.8 + 0.2 * 0.2 = 0.84 ##EQU00003## m C 1
.sym. D m C 2 ( { a , b } ) = m C 1 ( { a , b } ) m C2 ( { a , b }
) = 0.8 * 0.2 = 0.16 ##EQU00003.2##
[0014] We also deduce m.sub.C1.sym..sup.Dm.sub.C2({b})=0 and
m.sub.C1.sym..sup.Dm.sub.C2(O)=0 through the condition
A .OMEGA. m ( A ) = 1 ##EQU00004##
for every BF m.
[0015] A merge operator making it possible to merge information
arising from independent sources at least one of which is competent
is also already known. This operator is known in the literature by
the name "disjunctive rule". We denote this operator by the symbol
.sym..sup.DP.
[0016] A more general operator than the operators .sym..sup.D and
.sym..sup.DP is also already known, making it possible to take into
consideration knowledge such as the propensity of the sensors to be
in some such state in regard to their competence and their
sincerity, rather than in some such other state. This operator,
however, requires that the sensors be independent. For example, if
we denote by E1 the state "the sources are competent and
independent" and by E2 the state "the sources are independent and
at least one is competent", this operator can take into account
knowledge of the type: "with a probability p, the sources are in
the state E1 (i.e. with a probability p, the sources are competent
and independent), and with a probability 1-p, they are in the state
E2 (i.e. with a probability 1-p the sources are independent and at
least one is competent)".
[0017] A merge operator making it possible to merge information
arising from competent and non-independent sources is also already
known. This operator is known in the literature by the name
"cautious rule". We denote this operator by the symbol .sym..sup.P.
Thus, the BF resulting from the fusion by .sym..sup.P of two BFs
m.sub.C1 and m.sub.C2 is denoted m.sub.C1.sym..sup.Pm.sub.C2. The
definition of the operator .sym..sup.P being complex in the general
case, but simple in the case of belief functions of the type of
those given as an example above, we are content here merely to give
the definition of the cautious rule for belief functions of the
type of those given as an example above. Consider two belief
functions m.sub.C1 and m.sub.C2 on .OMEGA.={a, b} such that:
m.sub.C1({a})=1-y, m.sub.C1({a,b})=y
and
m.sub.C2({a})=1-z, m.sub.C2({a,b})=z,
with y,z.epsilon.(0,1) (we therefore have m.sub.C1(O)=0,
m.sub.C1({b})=0, and m.sub.C2(O)=0, m.sub.C2({b})=0 through the
condition
A .OMEGA. m ( A ) = 1 ##EQU00005##
for every BF m). The result m.sub.C1.sym..sup.Pm.sub.C2 of the
fusion by the cautious rule of the BFs m.sub.C1 and m.sub.C2 is
then obtained with the formula:
m.sub.C1.sym..sup.Pm.sub.C2({a})=1-minimum(y,z),
m.sub.C1.sym..sup.Pm.sub.C2({a,b})=minimum(y,z).
[0018] Applying the formula to the two belief functions m.sub.C1
and m.sub.C2 given as an example above, we obtain:
m.sub.C1.sym..sup.Pm.sub.C2({a})=1-minimum(0.8,0.2)=1-0.2=0.8,
m.sub.C1.sym..sup.Pm.sub.C2({a,b})=minimum(0.8,0.2)=0.2.
Note that there also already exists an operator, called the "bold
rule", making it possible to merge information arising from
non-independent sources at least one of which is competent.
[0019] However, these known fusion schemes do not make it possible
to take into account all the types of knowledge regarding the
dependency, the competence and the sincerity of the sources. For
example, if we denote by E1 the state "the sources are competent
and non-independent" and by E2 the state "the sources are
independent and at least one is competent", the known schemes do
not make it possible to process the knowledge: "with a probability
p, the sources are in the state E1 and with a probability 1-p, they
are in the state E2", although the operators which correspond to
the states E1 and E2 (respectively, the cautious rule and the
disjunctive rule) are already known. More generally, it is possible
to define E={E1, . . . , EN} the set of possible operating states
of the sensors and .sym..sup.Ei the operator corresponding to the
state Ei where the states Ei, i=1, . . . N, considered do not
necessarily deal with the dependency, the competence and the
sincerity of the sources. In this case, no general scheme exists
making it possible to merge the imperfect information arising from
the sensors when we have knowledge regarding the states of the
sensors of the type "with a probability pi, the sensors are in the
state Ei, i=1, . . . N".
[0020] The invention is aimed at alleviating the problems cited
above by proposing a method, a device and a system for fusion of
information originating from several sensors making it possible to
calculate a merged belief function, by taking into account
knowledge regarding the operating states of the sensors.
[0021] For this purpose, the subject of the invention is a method
for fusion of information originating from several sensors each
producing a belief function representing knowledge on a reasoning
space, and comprising several operating states, the said method
comprising a step of calculating a merged belief function and being
characterized in that it comprises, furthermore, a step of
calculating conditional belief functions on the basis of the belief
functions produced by the sensors and of a predefined base of
operators comprising associations between states of sensors and
merge operators, a conditional belief function representing
knowledge of a sensor on the reasoning space when the said sensor
is in a given operating state, and in that the step of calculating
the merged belief function uses the conditional belief functions
calculated and knowledge arising from a knowledge base (104)
regarding the operating state of the sensors.
[0022] According to a first variant of the invention, the knowledge
regarding the operating state of the sensors is represented by a
belief function indicating the propensity of the sensors to be in a
given operating state.
[0023] According to the first variant of the invention, the merged
belief function (m.sub.F) follows the following relation:
m.sub.F=((.sym..sub.i=1, . . .
,N.sup.Dm.sup..OMEGA.[Ei.sup.E.times..OMEGA.).sym..sup.Dm.sup.E.uparw.E.t-
imes..OMEGA.).sup..dwnarw..OMEGA.
where .sym..sup.D is an operator called the unnormalized Dempster's
rule, m.sup..OMEGA.[Ei], with i=1, . . . , N, are conditional
belief functions representing knowledge on the reasoning space
(.OMEGA.) when the sensors are in various states, N being the
number of operating states, and m.sup.E is knowledge regarding the
operating state of the sensors, .uparw., .dwnarw. and being
operations for manipulating belief functions.
[0024] According to a second variant of the invention, the step
(106) of calculating the merged belief function uses contextual
information expressed in the following manner: if an object
observed by the sensors is of type A then the sensors have a
propensity p to be in a set of states E', where A is a subset of
the space of propositions (.OMEGA.) and E' is a subset of a set E
of the various states of the sensors.
[0025] According to the second variant of the invention, the merged
belief function follows the following relation:
m.sub.F=((.sym..sub.i=1, . . .
,N.sup.Dm.sup..OMEGA.[Ei.sup.E.times..OMEGA.).sym..sup.D(.sym..sub.j=1,
. . .
,K.sup.Dm.sup.E[Aj.sup.E.times..OMEGA.)).sup..dwnarw..OMEGA.
where .sym..sup.D is an operator called the unnormalized Dempster's
rule, m.sup..OMEGA.[Ei], with i=1, . . . , N, are conditional
belief functions representing knowledge on the reasoning space
(.OMEGA.) when the sensors are in various states (E1, . . . , EN),
N being the number of operating states, and m.sup.E[Aj] is
contextual knowledge regarding the propensity of the sensors to be
in a given operating state when an object observed by the sensor is
of type A.sub.j, with j=1, . . . , K, A.sub.j being a subset of
.OMEGA. and such that A.sub.1, . . . , A.sub.K forms a partition of
.OMEGA., .uparw., .dwnarw. and being operations for manipulating
belief functions.
[0026] The invention also relates to a device for fusion of
information originating from several sensors each producing a
belief function representing knowledge on a reasoning space, and
comprising several operating states, the device comprising means
for the calculation of a merged belief function and being
characterized in that it comprises, furthermore means for the
calculation of conditional belief functions, said means being
linked to the sensors and to a predefined base of operators
comprising associations between states of sensors and merge
operators, a conditional belief function representing knowledge of
a sensor on the reasoning space when the said sensor is in a given
operating state, and in that the means for the calculation of the
merged belief function use the conditional belief functions
calculated and knowledge regarding the operating state of the
sensors originating from a knowledge base regarding the operating
state of the sensors.
[0027] The invention also relates to a system for fusion of
information comprising: at least two sensors each producing a
belief function and means for fusion of the belief functions
originating from the said sensors, the said fusion means being
linked to the sensors, the said system being characterized in that
the means for fusion of information comprise the device for fusion
of information according to the invention.
[0028] The invention relies on a mechanism for fusion of belief
functions. To apply this mechanism, various information, knowledge
and operations must be modelled within the framework of the theory
of belief functions: information provided by the sensors, knowledge
regarding the propensity of the sensors to be in a given operating
state and merge operators for each operating state considered.
[0029] The advantage of the invention is to take into account the
knowledge regarding the operating states of the sources. Precisely,
the invention makes it possible to take into account knowledge of
the type: "with a probability pi, the sources are in the state Ei",
on the condition that there exists a belief function fusion
operator .sym..sup.Ei corresponding to the state Ei.
[0030] More generally, let E={E1, . . . , EN} be the set of states
of the sources, the advantage of the invention is to take into
account imperfect knowledge, represented by a belief function
m.sup.E defined on the set E, on the operating states of the
sources.
[0031] The provided solution also allows the consideration of
knowledge on the contextual behaviours of the sources. For example,
the invention allows the consideration of the following knowledge:
"knowing that the letter observed is of type `a`, the sensors are
independent and competent, and knowing that the letter is `b`, the
sensors are competent and are not independent".
[0032] The invention will be better understood and other advantages
will become apparent on reading the detailed description given by
way of nonlimiting example and with the aid of the figures among
which:
[0033] FIG. 1 represents a chart of the method according to the
invention.
[0034] FIG. 2 represents an exemplary implementation of the device
according to the invention.
[0035] Generally, a fusion system comprises at least two sensors
used to observe objects. These observations are used to class the
object among predetermined categories. The set of these categories
is called the reasoning space and is denoted .OMEGA.. When an
object is observed by these sensors, each of these sensors provides
information on this object in the form of a belief function.
[0036] We propose to illustrate the method according to the
invention, by way of nonlimiting example, with the previously
mentioned multi-sensor system of classifier type used for the
optical recognition of hand-written characters. It is assumed that
the system is intended to determine whether the character formed on
an image I (not represented) is one of the letters `a` or `b`.
[0037] The term sensor is understood here in the broad sense. It
includes physical devices for data acquisition (camera, micro,
etc.) but also devices for processing these data, in the example: a
classifier.
[0038] An exemplary embodiment of a system for fusion of
information according to the invention is illustrated by FIG. 1.
Such a system comprises a first classification system C.sub.1 101
and a second classification system C.sub.2 102. It is assumed that
each of the two classification systems comprises a device allowing
the acquisition of the image I such as a video camera for example,
and means for processing the signal comprising character
recognition on the basis of the captured image I and the creation
of a belief function m.sub.Ci representing the knowledge of the
sensor Ci on .OMEGA.={a, b}, with i=1,2.
[0039] In the example, the first sensor C.sub.1 provides a belief
function m.sub.C1 defined as follows:
m.sub.C1({a})=0.8
m.sub.C1({a,b})=0.2.
[0040] The sensor C.sub.2 provides a belief function m.sub.C2
defined as follows:
m.sub.C2({a})=0.2
m.sub.C2({a,b})=0.8
[0041] The method according to the invention makes it possible to
take into account knowledge regarding the operating state of the
sensors. E denotes the space of states of the sensors with E={E1, .
. . , En}. E1, . . . , En each represent an operating state of the
set of sensors, for example: "the sensors are independent and
competent". The method according to the invention uses a knowledge
base 104 on the states of the sensors.
[0042] In this example, we have knowledge regarding the states of
the two classification systems C.sub.1 and C.sub.2. The pattern
recognition schemes according to the known art rely on techniques
for automatic learning using predetermined learning bases. In the
example, it is assumed that the knowledge base 104 describes two
states of the sensors. A first state E1, corresponding to the case
where the classifiers are trained on different learning bases,
indicates that the two sensors are independent and competent. A
second state E2 corresponding to the case where the classifiers are
trained on the same learning base, indicates that the two sensors
are non-independent and competent.
[0043] According to a characteristic of the invention, the
knowledge regarding the operating state of the sensors are
represented by a belief function m.sup.E indicating the propensity
of the sensors to be in a given state.
[0044] In the example, the belief function m.sup.E is defined as
follows:
m.sup.E({E1})=0.8
m.sup.E({E2})=0.2
[0045] The belief function indicates that the sensors have a
propensity to be in the first state E1 (the two sensors are
independent and competent) greater than the propensity to be in the
second state E2 (the two sensors are non-independent and
competent).
[0046] The method according to the invention uses a base of
operators related to the states 105 in which the states E1, . . . ,
EN are associated with merge operators .sym..sup.E1, . . . ,
.sym..sup.EN. In the example, the base of operators 105 is produced
in the form of a table associating a merge operator with each of
the states E1 and E2:
TABLE-US-00001 States Operators E1 .sym..sup.D E2 .sym..sup.P
[0047] As may be seen in this table, the state E1 ("the two sensors
are independent and competent") is associated with the operator
.sym..sup.D (the unnormalized Dempster's rule), and the state E2
("the two sensors are non-independent and competent") is associated
with the operator .sym..sup.P (the cautious rule).
[0048] The method comprises a step 103 of calculating so-called
conditional belief functions m.sup..OMEGA.[Ei], i=1, . . . , N, on
the basis of the belief functions m.sub.C1 and m.sub.C2 produced by
the sensors C1,C2 and of the base of operators 105 relating the
states to operators. The conditional belief functions
m.sup..OMEGA.[Ei], i=1, . . . , N, represent the knowledge on
.OMEGA. when the sources are in the states Ei, i=1, . . . , N.
Knowing that the information provided by the sensors must be merged
by the operator .sym..sup.Ei when the sources are in the state Ei,
we therefore have
m.sup..OMEGA.[Ei](A)=m.sub.C1.sym..sup.Eim.sub.C2(A), for all A.OR
right..OMEGA. and i=1, . . . , N. In practice, conditional belief
functions are calculated only for certain states Ei.epsilon.E: all
the states Ei.epsilon.E'.OR right.E with E'={Ej:Ej.epsilon.E'',
m(E'').noteq.0, E''.OR right.E}.
[0049] A step 106 of calculating the merged belief function m.sub.F
is carried out by taking into account the knowledge regarding the
states of the sources, represented by the belief function m.sup.E,
and the conditional belief functions calculated in the previous
step 103. According to a first variant of the invention, the merged
belief function m.sub.F is calculated in the following manner:
m.sub.F=((.sym..sub.i=1, . . .
,N.sup.Dm.sup..OMEGA.[Ei.sup.E.times..OMEGA.).sym..sup.Dm.sup.E.uparw.E.t-
imes..OMEGA.).sup..dwnarw..OMEGA.
with .uparw., .dwnarw. and operations for manipulating belief
functions on product spaces known in the literature, respectively,
as vacuous extension, marginalization and deconditioning and
defined by the following formulae.
[0050] Let m.sup.E be a belief function on the space E. Its vacuous
extension to the product space E.times..OMEGA. is a belief function
denoted m.sup.E.uparw.E.times..OMEGA. and given by:
m E .uparw. E .times. .OMEGA. ( B ) = { m E ( A ) if B = A .times.
.OMEGA. , A E , 0 otherwise . ##EQU00006##
[0051] Let m.sup..OMEGA.[Ei] be a conditional belief function on
the space .OMEGA., with Ei.epsilon.E. Its deconditioning to the
product space E.times..OMEGA. is a belief function denoted
m.sup..OMEGA.[Ei.sup.E.times..OMEGA. and given by:
m .OMEGA. [ Ei ] E .times. .OMEGA. ( C ) = { m .OMEGA. [ Ei ] ( A )
if C = ( A .times. Ei ) ( .OMEGA. .times. ( E \ Ei ) , A .OMEGA. 0
otherwise . ##EQU00007##
[0052] Let m.sup.E.times..OMEGA. be a belief function on the
product space E.times..OMEGA.. Its marginalization on the space
.OMEGA. is a belief function denoted
m.sup.E.times..OMEGA..dwnarw..OMEGA. and given by:
m E .times. .OMEGA. .dwnarw. .OMEGA. ( A ) = { B E .times. .OMEGA.
Projection ( B .dwnarw. .OMEGA. ) = A } m E .times. .OMEGA. ( B ) ,
for all A .OMEGA. . ##EQU00008##
[0053] By applying to the example the formula making it possible to
calculate the merged belief function m.sub.F, we obtain:
m.sub.F=(m.sup..OMEGA.[E1.sup.E.times..OMEGA..sym..sup.Dm.sup..OMEGA.[E2-
.sup.E.times..OMEGA..sym..sup.Dm.sup.E.uparw.E.times..OMEGA.).sup..dwnarw.-
.OMEGA.
with
m.sup..OMEGA.[E1]=m.sub.C1.sym..sup.Dm.sub.C2,
m.sup.C1[E2]=m.sub.C1.sym..sup.Pm.sub.C2,
which signifies that when the sources are in the state E1, the
knowledge on S2 is obtained by fusion m.sub.C1 and m.sub.C2 with
the unnormalized Dempster's rule, and when the sources are in the
state E2, the knowledge on .OMEGA. is obtained by fusion m.sub.C1
and m.sub.C2 with the cautious rule.
[0054] Next, by applying the formulae for vacuous extension,
marginalization, deconditioning and Dempster's rule, the following
result known in the literature is obtained:
m.sub.F=m.sup.E({E1})*m.sub.C1.sym..sup.Dm.sub.C2+m.sup.E({E2})*m.sub.C1-
.sym..sup.Pm.sub.C2,
and in particular
m.sub.F({a})=m.sup.E({E1})*m.sub.C1.sym..sup.Dm.sub.C2({a})+m.sup.E({E2}-
)*m.sub.C1.sym..sup.Pm.sub.C2({a}),
m.sub.F({a,b})=m.sup.E({E1})*m.sub.C1.sym..sup.Dm.sub.C2({a,b})+m.sup.E(-
{E2})*m.sub.C1.sym..sup.Pm.sub.C2({a,b}).
[0055] By applying the formulae for Dempster's rule .sym..sup.D and
for the cautious rule .sym..sup.P, we then obtain
m F ( { a } ) = m E ( { E 1 } ) * ( m C 1 ( { a } ) * m C 2 ( { a }
) + m C 1 ( { a } ) * m C 2 ( { a , b } ) + m C 1 ( { a , b } ) * m
C 2 ( { a } ) ) + m E ( { E 2 } ) * ( 1 - minimum ( m C 1 ( { a , b
} ) , ##EQU00009## m C 2 ( { a , b } ) ) = 0.8 * ( 0.8 * 0.2 + 0.8
* 0.8 + 0.2 * 0.2 ) + 0.2 ( 1 - minimum ( 0.2 , 0.8 ) ) = 0.8 *
0.84 + 0.2 * 0.8 = 0.832 ##EQU00009.2## m F ( { a , b } ) = m E ( {
E 1 } ) * ( m C 1 ( { a , b } ) * m C 2 ( { a , b } ) ) + m E ( { E
2 } ) * ( minimum ( m C 1 ( { a , b } ) , m C 2 ( { a , b } ) ) =
0.8 * ( 0.2 * 0.8 ) + 0.2 * minimum ( 0.2 , 0.8 ) = 0.8 * 0.16 +
0.2 * 0.2 = 0.168 ##EQU00009.3##
[0056] The result of the merge indicates that it is highly probable
that the character observed is an {a}, although a slight degree of
ignorance persists.
[0057] According to a second variant of the invention, the step of
calculating the merged belief function uses a contextual item of
information. A contextual item of information specifies an
operating state of the sensor in a particular context. This item of
information is expressed in the following form: "knowing that the
object observed is of type A, the sensors have a propensity p to be
in the set of states E" where A is a subset of .OMEGA. and E' is a
subset of a set E of the various states (E1, . . . , EN) of the
sensors.
[0058] In the example, the classifiers C.sub.1 and C.sub.2 are
independent in respect of recognizing the letter `a` since they
have been trained on different learning bases for the recognition
of this letter. The same classifiers C.sub.1 and C.sub.2 are
non-independent in respect of recognizing the letter `b` since they
have been trained on the same learning base for the recognition of
this letter.
[0059] In this case, the knowledge regarding the operating states
of the sensors is no longer represented by a belief function
m.sup.E, but by a plurality of conditional belief functions,
denoted m.sup.E[A.sub.1], . . . , m.sup.E[A.sub.K], representing
the knowledge regarding the states of the sources when the value
taken by the variable X is in the part A.sub.j of .OMEGA., j=1, . .
. , K. Note that forms a partition of .OMEGA.. There are then
potentially |.OMEGA.| conditional belief functions if the
behaviours of the sources for every element of .OMEGA. are
known.
[0060] In the example, there is contextual knowledge expressed by
the following conditional belief functions:
[0061] m.sup.E[{a}](E.sub.1)=1 indicating that knowing that the
observed object is the letter {a} the sensors are competent and
independent,
[0062] m.sup.E[{b}](E.sub.2)=1 indicating that knowing that the
observed object is the letter b the sensors are competent and
non-independent.
[0063] According to a second variant embodiment of the invention,
the merged belief function is then calculated as follows:
m F = ( ( .sym. i = 1 , , N D m .OMEGA. [ Ei ] E .times. .OMEGA. )
.sym. D ( .sym. j = 1 , , K D m E [ Aj ] E .times. .OMEGA. ) )
.dwnarw. .OMEGA. ##EQU00010## with ##EQU00010.2## m E [ Aj ] E
.times. .OMEGA. ( A ) = { m E [ Aj ] ( C ) if A = ( C .times. Aj )
( E .times. ( .OMEGA. \ Aj ) , C E 0 otherwise . ##EQU00010.3##
m.sup.E[Aj] is contextual knowledge regarding the propensity of the
sensors to be in a given operating state, m.sup.E[Aj](E')
indicating the propensity of the sensor to be in the set of states
E'.OR right.E={E1, . . . , EN} knowing that the observed object is
in Aj.
[0064] By applying to the example the formula making it possible to
calculate the merged belief function m.sub.F, we obtain:
m.sub.F=(m.sup..OMEGA.[E1.sup.E.times..OMEGA..sym..sup.Dm.sup..OMEGA.[E2-
.sup.E.times..OMEGA..sym..sup.Dm.sup.E[{a}.sup.E.times..OMEGA..sym..sup.Dm-
.sup.E[{b}.sup.E.times..OMEGA.).sup..dwnarw..OMEGA.
[0065] In order to calculate m.sub.F on the basis of this formula,
it is therefore necessary [0066] 1. To calculate the
deconditionings on E.times..OMEGA. of m.sup..OMEGA.[E1],
m.sup..OMEGA.[E2], m.sup.E[{a}] and m.sup.E[{b}], that is to say
calculate m.sup..OMEGA.[E1.sup.E.times..OMEGA.,
m.sup..OMEGA.[E2.sup.E.times..OMEGA.,
m.sup.E[{a}.sup.E.times..OMEGA. and
[0066] m.sup.E[{b}.sup.E.times..OMEGA. [0067] 2. Then to combine
these deconditionings by Dempster's rule, that is to say
calculate
[0067]
m.sup..OMEGA.[E1.sup.E.times..OMEGA..sym..sup.Dm.sup..OMEGA.[E2.s-
up.E.times..OMEGA..sym..sup.Dm.sup.E[{a}.sup.E.times..OMEGA..sym..sup.Dm.s-
up.E[{b}.sup.E.times..OMEGA. [0068] 3. Then to marginalize on
.OMEGA. the result of this combination, that is to say
calculate
[0068]
(m.sup..OMEGA.[E1.sup.E.times..OMEGA..sym..sup.Dm.sup..OMEGA.[E2.-
sup.E.times..OMEGA..sym..sup.Dm.sup.E[{a}.sup.E.times..OMEGA..sym..sup.Dm.-
sup.E[{b}.sup.E.times..OMEGA.).sup..dwnarw..OMEGA.
[0069] By applying the deconditioning formula, we obtain:
m.sup.E[{a}.sup.E.times..OMEGA.(({a}.times.{E1}).orgate.({b}.times.{E}))-
=1,
m.sup.E[{b}.sup.E.times..OMEGA.(({b}.times.{E2}).orgate.({a}.times.{E}))-
=1,
m.sup..OMEGA.[E1.sup.E.times..OMEGA.(({a}.times.{E1}).orgate.(.OMEGA..ti-
mes.{E2}))=m.sub.C1.sym..sup.Dm.sub.C2({a})
m.sup..OMEGA.[E1.sup.E.times..OMEGA.(.OMEGA..times.E)=m.sub.C1.sym..sup.-
Dm.sub.C2({a,b})
m.sup..OMEGA.[E2.sup.E.times..OMEGA.(({a}.times.{E2}).orgate.(.OMEGA..ti-
mes.{E1}))=m.sub.C1.sym..sup.Pm.sub.C2({a})
m.sup..OMEGA.[E2.sup.E.times..OMEGA.(.OMEGA..times.E)=m.sub.C1.sym..sup.-
Pm.sub.C2({a,b})
[0070] By combining these deconditionings by Dempster's rule, we
obtain a belief function, denoted m.sub.F.sup.E.times..OMEGA., on
E.times..OMEGA. such that
m.sub.F.sup.E.times..OMEGA.({a}.times.{E1})=m.sub.C1.sym..sup.Dm.sub.C2(-
{a})*m.sub.C1.sym..sup.Pm.sub.C2({a})+m.sub.C1.sym..sup.Dm.sub.C2({a,b})*m-
.sub.C1.sym..sup.Pm.sub.C2({a})m.sub.F.sup.E.times..OMEGA.(({a}.times.{E1}-
).orgate.({b}.times.{E2}))=m.sub.C1.sym..sup.Dm.sub.C2({a})*m.sub.C1.sym..-
sup.Pm.sub.C2({a,b})+m.sub.C1.sym..sup.Dm.sub.C2({a,b})*m.sub.C1.sym..sup.-
Pm.sub.C2({a,b})
[0071] By marginalizing this belief function on .OMEGA., we
obtain:
m F ( { a } ) = m C 1 .sym. D m C 2 ( { a } ) * m C 1 .sym. P m C 2
( { a } ) + m C 1 .sym. D m C 2 ( { a , b } ) * m C 1 .sym. P m C 2
( { a } ) = 0.84 * 0.8 + 0.16 * 0.8 = 0.8 ##EQU00011## m F ( { a ,
b } ) = m C 1 .sym. D m C 2 ( { a } ) * m C 1 .sym. P m C 2 ( { a ,
b } ) + m C 1 .sym. D m C 2 ( { a , b } ) * m C 1 .sym. P m C 2 ( {
a , b } ) = 0.84 * 0.2 + 0.16 * 0.2 = 0.2 ##EQU00011.2##
[0072] The result of the merge indicates that it is highly probable
that the character observed is an {a}, although a slight degree of
ignorance persists.
[0073] The invention can also be applied to other sectors, such as
for example aerial target recognition. The sensors are then radars.
The fusion system makes it possible to classify targets into
predefined categories for example aircraft, helicopter and missile.
The reasoning framework is then .OMEGA.={aircraft, helicopter,
missile}.
[0074] The invention also relates to a device for fusion of
information 200 originating from several sensors C.sub.1, C.sub.2
each producing a belief function m.sub.C1, m.sub.C2 and comprising
several operating states E1, . . . , EN. The device 200 comprises
means 206 for the calculation of a merged belief function m.sub.F
on the basis of the belief functions m.sub.C1, m.sub.C2 arising
from the sensors C.sub.1, C.sub.2. The device comprises,
furthermore, means for the calculation 203 of conditional belief
functions, said means being linked to a predefined base of
operators 105 comprising associations between states of sensors and
merge operators, and in that the means for the calculation 206 of
the merged belief function m.sub.F use the conditional belief
functions calculated and knowledge regarding the operating state of
the sensors originating from a knowledge base 104 regarding the
operating state of the sensors.
[0075] According to a first variant embodiment of the device
according to the invention, the knowledge base 104 regarding the
operating state of the sensors comprises a belief function m.sup.E
indicating the propensity of the sensors C.sub.1, C.sub.2 to be in
a given operating state.
[0076] According to a second variant embodiment of the device
according to the invention, the belief functions arising from the
sensors being defined on a space of propositions .OMEGA., the
knowledge base 104 regarding the operating state of the sensors
comprises contextual information expressed in the following manner:
if the object observed is of type A then the sensors have a
propensity p to be in a set of states E', where A is a subset of
the space of propositions .OMEGA. and E' is a subset of a set E of
the various states E1, . . . , EN of the sensors.
[0077] The invention also relates to a system for fusion of
information comprising: at least two sensors C.sub.1, C.sub.2 each
producing a belief function m.sub.C1, m.sub.C2 and means for fusion
of the belief functions m.sub.C1, m.sub.C2 originating from the
said sensors C.sub.1, C.sub.2. The fusion means are linked to the
sensors. The means for fusion of information comprise the device
for fusion of information 200 according to the invention.
[0078] The invention can also be applied for example within the
framework of an aerial surveillance system charged with recognizing
aircraft as a function of their model (airliners, fighter aircraft,
helicopter) with the aid of a set of radars.
* * * * *