U.S. patent application number 14/575912 was filed with the patent office on 2015-04-16 for method and a system for determining the location of an object.
The applicant listed for this patent is Ecole Polytechnique Federale de Lausanne (EPFL). Invention is credited to Ivan DOKMANIC, Orhan OCAL, Martin VETTERLI.
Application Number | 20150106053 14/575912 |
Document ID | / |
Family ID | 50975639 |
Filed Date | 2015-04-16 |
United States Patent
Application |
20150106053 |
Kind Code |
A1 |
OCAL; Orhan ; et
al. |
April 16, 2015 |
METHOD AND A SYSTEM FOR DETERMINING THE LOCATION OF AN OBJECT
Abstract
A method for determining the location of a transmitter
(respectively a receiver) in a space defined by one or more
reflective surfaces, including the steps of sending a signal from
the transmitter (respectively from a set of transmitters);
receiving by a set of receivers (respectively by a receiver) the
transmitted signal and echoes of the transmitted signal reflected
by the reflective surfaces; finding by a first computing module the
location of the virtual sources (respectively virtual receivers) of
the echoes; mirroring by a second computing module the virtual
sources (respectively virtual receivers) into the space and
obtained mirrored virtual sources (respectively mirrored virtual
receivers); combining by a third computing module the mirrored
virtual sources (respectively mirrored virtual receivers) so as to
obtain location of the transmitter (respectively the receiver).
This method makes use of echoes for localizing the source
(respectively receiver) when there is no line of sight between the
transmitter(s) and the receiver(s).
Inventors: |
OCAL; Orhan; (Berkeley,
CA) ; DOKMANIC; Ivan; (Lausanne, CH) ;
VETTERLI; Martin; (Grandvaux, CH) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Ecole Polytechnique Federale de Lausanne (EPFL) |
Lausanne |
|
CH |
|
|
Family ID: |
50975639 |
Appl. No.: |
14/575912 |
Filed: |
December 18, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
PCT/EP2013/077694 |
Dec 20, 2013 |
|
|
|
14575912 |
|
|
|
|
61919316 |
Dec 20, 2013 |
|
|
|
Current U.S.
Class: |
702/150 |
Current CPC
Class: |
H04R 29/005 20130101;
H04R 1/08 20130101; G01C 15/00 20130101; G01S 15/46 20130101; G01S
15/876 20130101; G01S 2015/465 20130101; G01S 17/06 20130101; G01B
5/00 20130101; G01H 7/00 20130101; G01S 15/06 20130101; G01S 7/54
20130101; G01S 7/539 20130101; G01S 15/42 20130101; H04S 7/305
20130101 |
Class at
Publication: |
702/150 |
International
Class: |
G01S 15/46 20060101
G01S015/46 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 22, 2012 |
CH |
2935/12 |
Claims
1. A method for determining the location of a transmitter in a
space defined by one or more reflective surfaces, comprising the
steps of sending a signal from said transmitter; receiving by a set
of receivers the transmitted signal and echoes of the transmitted
signal reflected by said reflective surfaces; finding by a first
computing module the location of the virtual sources of said
echoes; mirroring by a second computing module the virtual sources
into the space and obtaining mirrored virtual sources; combining by
a third computing module the locations of said mirrored virtual
sources so as to obtain the location of the transmitter.
2. The method of claim 1, the location of the set of receivers and
the location and/or orientation of the reflective surfaces being
known.
3. The method of claim 1, the step of mirroring comprising applying
the method of images in reverse order.
4. The method of claim 1, the step of mirroring comprising: drawing
the lines connecting a virtual source to the set of receivers,
finding the reflective surface which intersects said lines,
reflecting the virtual source across said reflective surface and
generating a reflected virtual source, storing the points of
intersections on the reflective surface, checking if the reflected
virtual source is inside the space defined by the reflective
surfaces, repeating the previous steps if the reflected virtual
source is not inside the space defined by the reflective
surfaces.
5. The method of claim 1, comprising, if the reflected virtual
source is not inside the space defined by the reflective surfaces,
drawing the lines connecting the stored points of intersections and
the reflected virtual source, finding the new reflective surface
which intersects said lines, reflecting the reflected virtual
source across said reflective surface and generating a new
reflected virtual source, storing the points of intersections on
the new reflective surface, checking if the new reflected virtual
source is inside the space defined by the reflective surfaces,
repeating the previous steps if the new reflected virtual source is
not inside the space defined by the reflective surfaces.
6. The method of claim 1, comprising echoes' sorting.
7. The method of claim 6, said echo sorting comprising grouping the
echoes corresponding to a single virtual source.
8. The method of claim 7, wherein grouping the echoes corresponding
to a single virtual source comprising checking if i = 1 M ( s - m i
- r i ) 2 ##EQU00022## is less than a threshold, wherein M is the
number of receivers, s is the virtual source, m.sub.i is a receiver
and r.sub.i the distance between the transmitter and the m.sub.i
receiver.
9. The method of claim 1, comprising the optimization of the
location of the transmitter.
10. The method of claim 9, said optimization comprising a gradient
descent method.
11. The method of claim 10, comprising the tracking of the
transmitter by using said optimization method, e.g. by using the
gradient descent method.
12. The method of claim 1, the reflective surfaces defining a
non-convex space.
13. The method of claim 1, the transmitter being a mobile device,
the receivers being satellites.
14. A system for determining the location of a transmitter,
comprising: one or more reflective surfaces one transmitter for
sending a signal; a set of receivers for receiving the transmitted
signal and echoes of the transmitted signal reflected by said
reflective surfaces; a first computing module for finding the
location of the virtual sources of said echoes; a second computing
module for mirroring the virtual sources into the room and obtained
mirrored virtual sources; a third computing module for combining
the locations of said mirrored virtual sources so as to find the
location of the transmitter.
15. The system of claim 14, the first computing module, the second
computing module and the third computing module being the same
module.
16. The system of claim 14, the reflective surfaces defining a
non-convex space.
17. The system of claim 14, the transmitter being a mobile device,
the receivers being satellites.
18. The system of claim 14, the signal being a UWB signal, the
transmitter being a UWB transmitter, and the receivers being an
array of UWB receivers.
19. A computer program product, comprising: a tangible computer
usable medium including computer usable program code for
determining the location of a transmitter sending a signal received
by a set of receivers, the set of receivers receiving also echoes
of the transmitted signal reflected by one or more reflective
surfaces, the computer usable program code being used for finding
by a first computing module the location of the virtual sources of
said echoes; mirroring by a second computing module the virtual
sources into the space and obtained mirrored virtual sources;
combining by a third computing module the locations of said
mirrored virtual sources so as to obtain the location of the
transmitter.
20. A method for determining the location of a receiver in a space
defined by one or more reflective surfaces, comprising the steps of
sending a signal from a set of transmitters; receiving by said
receiver the transmitted signal and echoes of the transmitted
signal reflected by said reflective surfaces; finding by a first
computing module the location of the virtual receivers of said
echoes; mirroring by a second computing module the virtual
receivers into the space and obtained mirrored virtual receivers;
combining by a third computing module the locations of said
mirrored virtual receivers so as to obtain the location of the
receiver.
21. The method of claim 20, the location of the set of transmitters
and the location and/or orientation of the reflective surfaces
being known.
22. The method of claim 20, the step of mirroring comprising
applying the method of images in reverse order.
23. The method of claim 20, the step of mirroring comprising:
drawing the lines connecting a virtual receiver to the set of
transmitters, finding the reflective surface which intersects said
lines, reflecting the virtual receiver across said reflective
surface and generating a reflected virtual receiver, storing the
points of intersections on the reflective surface, checking if the
reflected virtual receiver is inside the space defined by the
reflective surfaces, repeating the previous steps if the reflected
virtual receiver is not inside the space defined by the reflective
surfaces.
24. The method of claim 20, comprising, if the reflected virtual
receiver is not inside the space defined by the reflective
surfaces, drawings the lines connecting the stored points of
intersections and the reflected virtual receiver, finding the new
reflective surface which intersects said lines, reflecting the
reflected virtual receiver across said reflective surface and
generating a new reflected virtual receiver, storing the points of
intersections on the new reflective surface, checking if the new
reflected virtual receiver is inside the space defined by the
reflective surfaces, repeating the previous steps if the new
reflected virtual receiver is not inside the space defined by the
reflective surfaces.
25. The method of claim 20, the mirroring comprising echoes'
sorting.
26. The method of claim 20, comprising the optimization of the
position estimate.
27. The method of claim 26, said optimization of the position
estimate comprising a gradient descent method.
28. The method of claim 27, comprising the tracking of the receiver
by using the optimization method, e.g. the gradient descent
method.
29. The method of claim 20, the reflective surfaces defining a
non-convex space.
30. A system for determining the location of a receiver,
comprising: one or more reflective surfaces a set of transmitters
for sending a signal; said receiver for receiving the transmitted
signal and echoes of the transmitted signal reflected by said
reflective surfaces; a first computing module for finding the
location of the virtual receivers of said echoes; a second
computing module for mirroring the virtual receivers into the room
and obtained mirrored virtual receivers; a third computing module
for combining the locations of said mirrored virtual receivers so
as to find the location of the receiver.
31. The system of claim 30, the first computing module, the second
computing module and the third computing module being the same
module.
32. The system of claim 30, the reflective surfaces defining a
non-convex space.
33. The system of claim 30, the receiver being a mobile device, the
transmitters being satellites.
34. The system of claim 30, the signal being a UWB signal, the
transmitters being an array of UWB transmitters and the receiver
being a UWB receiver.
35. A computer program product, comprising: a tangible computer
usable medium including computer usable program code for
determining the location of a receiver receiving a signal
transmitted by a set of transmitters, the receiver receiving also
echoes of the transmitted signal reflected by one or more
reflective surfaces, the computer usable program code being used
for finding by a first computing module the location of the virtual
receivers of said echoes; mirroring by a second computing module
the virtual receivers into the space and obtained mirrored virtual
receivers; combining by a third computing module the locations of
said mirrored virtual receivers so as to obtain the location of the
receiver.
Description
FIELD OF THE INVENTION
[0001] The present invention concerns a method and a system for
determining the location of an object as a receiver or a
transmitter.
DESCRIPTION OF RELATED ART
[0002] Indoor localization systems have become a popular research
area in the last years, and many universities and companies do
research in this field.
[0003] There are numerous applications of indoor localization,
covering a broad range of fields. One example is inventory
management: keeping track of the location and the amount of a
particular good in a warehouse. Another application is object
tracking, such as localizing medical personnel or equipment in a
hospital or tracking people with limited mobility to understand if
they need assistance. Yet another application that drew
considerable attention is location aware services. When you visit a
museum, you are given a device, and if you want to get information
about a specific piece of artwork, you can enter a code related to
that artwork to retrieve it. However, if the device was aware of
the location of the person, it could detect which artwork the
person is approaching and information about it can be directly
retrieved without the need of the user entering any information.
Another important fields where indoor localization are security and
rescue operations, such as detecting the location of police dogs
trained to find explosives, or localizing firemen in a building on
fire.
[0004] Positioning systems, containing both indoor and outdoor
localization, can be divided into three main topologies. The first
one is self-positioning system, where the receiver receives data
from distributed transmitters in order determine its own position
(e.g. GPS). Second topology is remote positioning, where receivers
located at possibly multiple locations measure the signal coming
from the object in order to localize it. Third topology that has
two subcategories is called indirect positioning: A data link is
used to transfer position information from the self-positioning
system to a remote site or vice-versa.
[0005] Besides different topologies of indoor localization, there
are also different modalities. One of the modalities is GPS.
Although GPS is one of the most accurate outdoor localization
systems, since the signals coming from the satellites have poor
indoor penetration due to shielding by the concrete, the
performance is poor indoors. Another modality is the
radio-frequency identification (RFID). This method has found
applications in tracking objects in warehouses or assembly lines.
There are two different RFID setups: passive and active RFID tags.
Passive RFID tags are inexpensive to manufacture, but the reading
ranges are short, typically 1-2 meters, and the tag readers are
expensive to manufacture. Active tags are more expensive to
produce, but they have increased range and the tag readers are less
expensive. Another modality is using the cellular network. This
enables indoor localization when the building is covered by
multiple base stations. However, the accuracy depends dramatically
on how well the location is covered and on the building's
structure. Wireless local area network (WEAN) is also used for
indoor localization. However, this approach needs preinstalled
infrastructure and the performance fluctuates since the channel
changes over time. Another approach is by using ultra-wideband
(UWE) radio, which is one of the most promising technologies for
indoor localization. First, UWB can send sharp pulses that enable
precise localization of time of arrivals in the receivers. Second,
UWB can penetrate walls, equipment and clothing, thus the signal
can be observed even behind obstructions. Finally, one can use
sound and ultrasound. The equipment needed for ultrasound
localization is inexpensive and easily accessible. Furthermore,
there are planar ultrasound transceivers, that is, transmitters and
receivers that emit and receive in the 2D plane. This reduced the
total amount of reflections we deal with and simplifies the
design.
[0006] Although indoor localization has been studied for twenty
years, the field is still an active research area and open for
further developments.
[0007] It is an aim of the present invention to obviate or mitigate
one or more of the disadvantages of the prior art.
[0008] It is another aim of the present invention to find an
alternative to the solutions of the prior art.
BRIEF SUMMARY OF THE INVENTION
[0009] According to the invention, these aims are achieved by means
of a method for determining the location of a transmitter in a
space defined by one or more reflective surfaces, comprising the
steps of
[0010] sending a signal from this transmitter;
[0011] receiving by a set of receivers the transmitted signal and
echoes of the transmitted signal reflected by these reflective
surfaces;
[0012] finding by a first computing module the location of the
virtual sources of the echoes;
[0013] mirroring by a second computing module the virtual sources
into the space and obtained mirrored virtual sources;
[0014] combining by a third computing module the locations of the
mirrored virtual sources so as to obtain the location of the
transmitter.
[0015] Advantageously the location of the set of receivers and the
location and/or orientation of the reflective surfaces are
known.
[0016] The method according to the invention locates a source or
transmitter in a room with general geometry bounded by reflective
surfaces using the measurements by the set of receivers, e.g. a
microphone array. In other words, the geometry of the space defined
by the reflective surfaces (e.g. a room) does not have to convex,
and a direct path between the transmitter and the receiver(s) is
not necessary. The method assumes the knowledge of the room
geometry and microphone positions.
[0017] The proposed method corresponds to a system architecture
fitting in remote localization topology. The method utilizes the
direct signal (if present) and early reflections to localize the
source. In this context the expression "direct signal" indicates
the signal sent by transmitter and directly, i.e. without
reflections, received by a receiver or the set of receivers.
[0018] The method according to the invention advantageously makes
use of early reflections or echoes, enabling to localize the source
even when there is no line of sight. The method uses method of
images to reduce the problem of indoor source localization to
multiple source localization in free field, and then finding the
source position. In other words, the method of images is used to
reduce the problem of localizing the source in indoor environment
to multiple source localization in free space and then estimating
the source position.
[0019] In order to localize the multiple virtual sources,
combinatorially selected echoes from each receiver of the set of
receivers are used and the resulting location is tested to check if
the echoes were corresponding to a single virtual source, for
example, the resulting location is tested to check if the echoes
uniquely localize a single virtual source.
[0020] After locating the virtual sources, the method according to
the invention finds the position or location inside the room that
results in the generation of such virtual source.
[0021] The step of mirroring of the method comprises applying the
method of images in reverse order ("inverse method of images", or
"inverse image source model"). This mirroring procedure could have
additional applications, whenever it is possible to observe
anything through echoes, and the geometry of the reflective
surfaces is known. A possible application is e.g. GPS in urban
environments, where the method according to the invention allows to
"see" the satellite only through (possibly multiple)
reflections.
[0022] The method according to the invention localizes an arbitrary
number of image sources and reflects them iteratively until they
are in the room. The main idea is then to use the knowledge of the
locations of the virtual sources for finding the (unknown) location
of the real and original source. Advantageously the inverse method
of images reflects the locations of the virtual sources back to the
real source location.
[0023] The virtual source could be of arbitrary order (1st, 2nd,
3rd, . . . ).
[0024] Advantageously the step of mirroring comprises:
[0025] drawing the lines connecting a virtual source to the set of
receivers,
[0026] finding the reflective surface which intersects these
lines,
[0027] reflecting the virtual source across this reflective surface
and generating a reflected virtual source,
[0028] storing the points of intersections on the reflective
surface,
[0029] checking if the reflected virtual source is inside the space
defined by the reflective surfaces,
[0030] repeating the previous steps if the reflected virtual source
is not inside the space defined by the reflective surfaces.
[0031] In other words, if the reflected virtual source is not
inside the space defined by the reflective surfaces, the following
steps are performed:
[0032] drawings the lines connecting the stored points of
intersections and the reflected virtual source,
[0033] finding the new reflective surface which intersects these
lines,
[0034] reflecting the reflected virtual source across this
reflective surface and generating a new reflected virtual
source,
[0035] storing the points of intersections on the new reflective
surface,
[0036] checking if the new reflected virtual source is inside the
space defined by the reflective surfaces,
[0037] repeating the previous steps if the new reflected virtual
source is not inside the space defined by the reflective
surfaces.
[0038] In one embodiment, the method according to the invention
comprises echoes' sorting, i.e. grouping the echoes corresponding
to a single virtual source.
[0039] In one embodiment grouping the echoes corresponding to a
single virtual source comprising checking if
i = 1 M ( s . - m i - r i ) 2 ##EQU00001##
is less than a threshold, wherein M is the number of receivers, s
is the estimated location of the virtual source with the current
selection of echoes, m.sub.i is a receiver and r.sub.i the distance
between the transmitter and the m.sub.i receiver.
[0040] Advantageously the method according to the invention
comprises optimization of the source location. A measure based on
the simulated room impulse response from the estimated source
location is defined to find the best estimate for the true source
position within the reflected virtual sources.
[0041] After choosing the estimate from the set of reflected
virtual sources, its position is optimized based on the difference
between simulated and recorded impulse responses.
[0042] In one embodiment the optimization of the source location
estimate comprises a gradient descent method, which is used for
optimizing the estimated location based on the simulated room
impulse response.
[0043] The method according to the invention could comprise the
tracking of the transmitter by using an optimization method. The
optimization algorithm can then be applied to the problem of source
tracking, where the new position of the source is estimated by
applying optimization based on the position estimated in the
previous time instance. In other words, the optimization method is
applied for tracking a moving source based on previous position
estimates.
[0044] By duality, in one particular embodiment, the method can be
applied to localizing a microphone using multiple transmitters. In
another particular embodiment, the method of the invention can be
applied to GPS, wherein the transmitter is a mobile device, and the
receivers are satellites (which, although is opposite to the
standard whereby mobile device are receivers and satellites are the
transmitters, is conceptually the same). It will be understood that
the method of the invention can equally be applied to GPS, wherein
the transmitter is a satellite, and the receiver is a mobile
device. The application of the method of the invention to GPS will
be described in more detail later.
[0045] The present invention concerns also a system for determining
the location of a transmitter, comprising:
[0046] one or more reflective surfaces
[0047] one transmitter for sending a signal;
[0048] a set of receivers for receiving the transmitted signal and
echoes of the transmitted signal reflected by these reflective
surfaces;
[0049] a first computing module for finding the location of the
virtual sources of said echoes;
[0050] a second computing module for mirroring the virtual sources
into the room and obtained mirrored virtual sources;
[0051] a third computing module for combining the locations of
these mirrored virtual sources so as to find the location of the
transmitter.
[0052] In one preferred embodiment, the first computing module, the
second computing module and the third computing module are the same
module.
[0053] In one preferred embodiment, the signal is a UWB signal, the
transmitter being a UWB transmitter and the receivers being UWB
receivers. However other kinds of signals (e.g. acoustic, RF,
light, etc.) can be used, as will be discussed.
[0054] The present invention concerns also a computer program
product, comprising:
a tangible computer usable medium including computer usable program
code for determining the location of a transmitter sending a signal
received by a set of receivers, the set of receivers receiving also
echoes of the transmitted signal reflected by one or more
reflective surfaces, the computer usable program code being used
for
[0055] finding by a first computing module the location of the
virtual sources of these echoes;
[0056] mirroring by a second computing module the virtual sources
into the space and obtained mirrored virtual sources;
[0057] combining by a third computing module the locations of these
mirrored virtual sources so as to obtain the location of the
transmitter.
[0058] By duality, the same procedure can be performed to localize
a receiver with several transmitters in a non-convex room.
[0059] The present invention concerns then also a method for
determining the location of a receiver in a space defined by one or
more reflective surfaces, comprising the steps of
[0060] sending a signal from a set of transmitters;
[0061] receiving by this receiver the transmitted signal and echoes
of the transmitted signal reflected by these reflective
surfaces;
[0062] finding by a first computing module the location of the
virtual receivers of the echoes;
[0063] mirroring by a second computing module the virtual receivers
into the space and obtained mirrored virtual receivers;
[0064] combining by a third computing module the locations of the
mirrored virtual receivers so as to obtain the location of the
receiver.
[0065] The present invention concerns then also a system for
determining the location of a receiver, comprising:
[0066] one or more reflective surfaces
[0067] a set of transmitters for sending a signal;
[0068] this receiver for receiving the transmitted signal and
echoes of the transmitted signal reflected by said reflective
surfaces;
[0069] a first computing module for finding the location of the
virtual receivers of the echoes;
[0070] a second computing module for mirroring the virtual
receivers into the room and obtained mirrored virtual
receivers;
[0071] a third computing module for combining the locations of the
mirrored virtual receivers so as to find the location of the
receiver.
[0072] The present invention concerns then also a computer program
product, comprising:
a tangible computer usable medium including computer usable program
code for determining the location of a receiver receiving a signal
transmitted by a set of transmitters, the receiver receiving also
echoes of the transmitted signal reflected by one or more
reflective surfaces, the computer usable program code being used
for
[0073] finding by a first computing module the location of the
virtual receivers of the echoes;
[0074] mirroring by a second computing module the virtual receivers
into the space and obtained mirrored virtual receivers;
[0075] combining by a third computing module the locations of the
mirrored virtual receivers so as to obtain the location of the
receiver.
[0076] Experiments performed by the applicant have demonstrated the
effectiveness, the accuracy and the robustness of the proposed
methods and systems.
[0077] The present invention concerns also a computer data carrier
storing presentation content created with the described
methods.
BRIEF DESCRIPTION OF THE DRAWINGS
[0078] The invention will be better understood with the aid of the
description of an embodiment given by way of example and
illustrated by the figures, in which:
[0079] FIG. 1A shows an example of trilateration as intersection of
circles without jitter.
[0080] FIG. 1B shows an example of trilateration as intersection of
circles with jitter.
[0081] FIG. 2 shows a room comprising a transmitter (source) and a
set of receivers, the transmitter being localized based on TOA.
[0082] FIG. 3 shows a room comprising a transmitter (source) and a
set of receivers, the transmitter being localized based on
TDOA.
[0083] FIG. 4 shows a room comprising a transmitter (source) and a
set of receivers, and an example of virtual sources of the first
and second order.
[0084] FIG. 5A shows a room comprising a transmitter (source), and
an example of a first order virtual source.
[0085] FIG. 5B shows a room comprising a transmitter (source), and
an example of a first order virtual source, and a second order
virtual source.
[0086] FIG. 6 illustrates an example of visibility of a second
order virtual source. Although v.sub.2;4 is visible by the
microphone in wall (4), it will not be heard unless wall (2)
extends until point a. With the illustrated room geometry, the
microphone can only hear v.sub.2;4 from intersection the two
illustrated shades in top left corner.
[0087] FIG. 7 illustrated an example of labelling obstructive walls
by using convex hull of the room. Two out of six represented walls
are inside the convex hull of the room, thus it is necessary to
check if the signal is blocked by only two walls.
[0088] FIG. 8A illustrates the times of arrivals of the echoes
coming from the real source and from the virtual sources of FIG.
8B.
[0089] FIG. 8B illustrates a room comprising the real source and a
set of receivers, and the virtual sources generated from the real
source.
[0090] FIG. 9 shows an example of how reflecting (mirroring) a
localized first order virtual source back into a room.
[0091] FIGS. 10A to 10C shows an example of three steps for
reflecting (mirroring) a localized second order virtual source into
the room using method of images in reverse order.
[0092] FIG. 11 illustrates recorded RIR by each microphone, and the
simulated RIR from the estimated location.
[0093] FIG. 12 illustrates an embodiment of the optimization of
position estimate.
[0094] FIG. 13 illustrates the dimensions of the L-shaped room used
in source localization simulations.
[0095] FIGS. 14A and 14B show the global view respectively vicinity
of the true source of the simulation results for localization in
L-shaped room without measurement jitter.
[0096] FIGS. 15A and 15B show the global view respectively vicinity
of the true source of the simulation results for localization in
L-shaped room with jitter being i.i.d. centered Gaussian with
.sigma.=0.05.
[0097] FIGS. 16A and 16B shows the global view respectively
vicinity of the true source of the simulation results for
localization in a room with complex geometry, with jitter being
i.i.d. centered Gaussian with .sigma.=0.1.
[0098] FIG. 17A shows simulation results for tracking (using all of
the steps except optimization) a moving source following the path
where s.sub.1(t)=1+3 cos.sup.3(2.pi.t/120) and s.sub.2(t)=6.5+2
sin.sup.3(2.pi.t/120) for t=0, 1, 2 . . . 120, the jitter being
i.i.d. with distribution .epsilon..sub.i.about.N(0.0.05).
[0099] FIG. 17B shows simulation results for tracking (using only
the optimization based on the previous estimate) a moving source
following the path where s.sub.1(t)=4+3 cos.sup.3(2.pi.t/120) and
s.sub.2(t)=6.5+2 sin.sup.3(2.pi.t/120) for t=0, 1, 2 . . . 120, the
jitter being i.i.d. with distribution
.epsilon..sub.i.about.N(0.0.05).
[0100] FIGS. 18A to 18D illustrate the contours of formula (11) in
8.times.8 square room for the source position s=(4, 5).sup.T, for
one microphone (top) and three microphones (bottom) using direct
and first order echoes. The contours on FIGS. 18A and 18C are for
jitter-free measurements, and the contours on FIGS. 18B and 18D are
for i.i.d centered Gaussian jitter with
.sigma.=0.05.
[0101] FIG. 19 illustrate an example of the l.sub.2 localization
error.
[0102] FIGS. 20A and 20B illustrate the contours of formula (10) in
8.times.8 square room for the source position s=(4, 5).sup.T, for
one microphone (FIG. 20A) and three microphones (FIG. 20B) using
direct and first order echoes. The contours are plotted for jitter
free measurements and the microphone positions are same with FIGS.
18A to 18D.
[0103] FIG. 21 illustrates an embodiment of a system according to
the invention.
[0104] FIG. 22 illustrates an embodiment of a data processing
system in which a method in accordance with an embodiment of the
present invention may be implemented.
DETAILED DESCRIPTION OF POSSIBLE EMBODIMENTS OF THE INVENTION
[0105] The present invention will be now described in more detail
in connection with its embodiment for determining the location of a
loudspeaker (or, in general, of a transmitter) by knowing the
geometry of a room, i.e. the location and the orientation of its
walls (or, in general, of its reflective surfaces), and the
location of a set of microphones (or, in general, of a receivers).
However the present invention finds applicability in connection
with many other fields, as will be discussed. For example, the
described method and system can be used for determining the
location of a receiver by knowing the geometry of a room, i.e. the
location and the orientation of its walls or reflective surfaces,
and the location of a set of transmitters.
[0106] The present invention will be now described in more detail
in connection with an ultrasonic signal. However the present
invention finds applicability of connection with other kinds of
signals, e.g. and in a non-limiting way a light signal, an RF
signal, etc.
[0107] The present invention will be now described in more detail
in connection with a room. The example described is with respect to
a 2D room for ease of understanding of the principles of the
present invention; however it will be understood that the invention
can be applied, using the same principles, to a 3D room. It will
also be understood that the present invention does not necessarily
need to be applied in a room.
Notation
[0108] Throughout this application vectors are denoted as boldface
letters, e.g. x. The position of the i.sup.th microphone is denoted
as m.sub.i. Without loss of generality, one corner of the room is
assumed to be at the origin, and the corners are numbered in
counter clockwise direction starting from `1` for the corner at the
origin. The coordinates of the i.sup.th corner of the room are
denoted by c.sub.i, and the unit outward normal vector of the
i.sup.th wall is denoted by n.sub.i. The inner product between two
vectors a and b is denoted as a, b=a.sup.Tb, and
.parallel.a.parallel. denotes the f.sub.2-norm of a defined as
.parallel.a.parallel.= {square root over (a, a)}. The coordinates
of the source are denoted by s.epsilon..sup.2. The virtual sources
(to be explained) generated by reflecting the source, in specified
order, by the walls, i, j, k is denoted as v.sub.i,j,k.
DEFINITIONS AND PROPERTIES
Times of Arrivals--TOA
[0109] Times of arrivals are the measurements of the absolute
propagation time for the signal to reach the microphones after
being emitted by the source. TOA can be measured if there is
synchronization between the microphones and the source, namely, if
the microphones have the information about when the signal was
emitted by the source.
Time Difference of Arrivals--TDOA
[0110] If the microphones do not know when the signal is emitted by
the source, the absolute propagation time for signal to reach the
microphones is unknown, thus TOA are not present. In this case one
can measure the time difference of arrivals between microphones if
they have a common time reference. This can be achieved by
designating one of the microphones as the reference microphone, and
finding the difference between the time when the signal reaches the
reference microphone and the other microphones.
Direction of Arrival--DOA
[0111] Direction of arrival is the angle with respect to a
predefined reference, which the signal is coming to a receiver.
Triangulation
[0112] Triangulation is determining the position of an object by
measuring the angles between the object and predefined fixed
anchors.
Trilateration
[0113] Trilateration is a method for finding the position of an
object by using distance measurements to at least three anchors; in
the case where one uses more than three anchors, the problem is
called multilateration.
[0114] It should be noted that in TOA, TDOA and DOA preferably at
least three receivers for unique localization in free space in 2D
are used, and four in 3D.
[0115] It should be understood that in the present application the
terms "virtual source" and "image source" are used interchangeably
and both terms stand for mirror images of a true source across one
or multiple walls. Image sources are used to model echoes. First
order echoes can be seen as coming from first order image sources,
and higher order echoes can be seen as coming from higher order
image sources.
DESCRIPTION OF EMBODIMENTS
Localization Based on TOA
[0116] If times of arrivals are known, the distance from the source
to the microphones can be found by using the speed of propagation
of sound in air C=343.2 m/s (precise value depends on other factors
such as the temperature). Then by trilateration, the position of
the source can be found by intersecting the distance circles 1a-d
as shown in FIG. 1a.
[0117] When the time of arrivals are present without measured
error, the circles will intersect at a single point and the
intersection gives the position of the source. However, if there is
jitter in the distance measurements, the circles 1a-d do not
intersect at a single point as shown in FIG. 1b. In that case one
can pose an optimization problem to estimate the location of the
source as is illustrated in FIG. 2.
[0118] Referring to FIG. 2, having measured distance r.sub.i of the
source 2 to the i.sup.th microphone m.sub.1-m.sub.4 (generalized as
m.sub.i) as:
r.sub.i=.parallel.x-m.sub.i.parallel.+.epsilon..sub.i,i=1,2, . . .
,M,
where .epsilon..sub.i is random measurement jitter and M is the
number of microphones, the position of the source 2 can be
estimated by finding the position in the 2D plane that minimizes
the sum of the squares of the differences between the measured
distances r.sub.1,r.sub.2 of the source 2 to the microphones and
the distances between the microphones m.sub.1-m.sub.4 (generalized
as m.sub.i) and the test position 7. This optimization problem can
be written as:
minimize x .di-elect cons. 2 i = 1 M ( r i - x - m i ) 2 . ( 1 )
##EQU00002##
[0119] The solution to this problem yields the maximum likelihood
estimator if errors follow a Gaussian distribution with covariance
matrix proportional to the identity matrix. However, this problem
is not convex, and there is no efficient algorithm to find the
globally optimal solution. There are methods for approximating the
solution, such as by semidefinite relaxation as disclosed in:
[0120] K. Cheung. W.-K. Ma, and H. So. "Accurate approximation
algorithm for toa-based maximum likelihood mobile location using
semidefinite programming," in Acoustics, Speech, and Signal
Processing. 2004, Proceedings. (ICASSP '04). IEEE International
Conference on, vol. 2, 2004, pp. ii 145 8 vol. 2. which is
incorporated herein with reference.
[0121] However, it is reported that although semi-definite
relaxation yields good results for some instances, it can perform
badly if the relaxation is not tight as disclosed in [0122] A.
Beck. P. Stoica, and J. Li. "Exact and Approximate Solutions of
Source Localization Problems." Signal Processing, IEEE Transactions
on, vol. 56, no. 5, pp. 1770 1778, 2008. which is incorporated
herein with reference.
[0123] Another optimization problem mentioned is the so called
`squared-range-based least squares` obtained by squaring the
distances in (1) defined as:
minimize x .di-elect cons. 2 i = 1 M ( x - m i 2 - r i 2 ) 2 ( 2 )
##EQU00003##
[0124] Although this is still a nonconvex problem, the solution can
be found efficiently and globally by the method described in the
publication "Exact and Approximate Solutions of Source Localization
Problems".
[0125] To formulate the problem so that the solution can be found
efficiently, first, we write it in constrained from as:
minimize x .di-elect cons. 2 i = 1 M ( .alpha. - 2 m i x + m i 2 -
r i 2 ) 2 ##EQU00004## subject to x 2 = .alpha. .
##EQU00004.2##
[0126] By using the substitution y=(x.sup.T,.alpha.).sup.T the
problem is written as:
minimize y .di-elect cons. 3 Ay - b 2 subject to y Dy + 2 f y = 0 ,
where A = ( - 2 m 1 - 1 - 2 m M 1 ) , b = ( r 1 2 - m 1 2 r M 2 - m
M 2 ) , And , D = ( I 2 0 2 .times. 1 0 1 .times. 2 0 ) , f = ( 0 2
.times. 1 - 0.5 ) . ( 3 ) ##EQU00005##
[0127] The resulting problem consists of minimization of a
quadratic objective subject to a single quadratic equality
constraint, which are called generalized trust region sub-problems
(GTRS) in optimization literature, as disclosed in: [0128] J. J.
More. "Generalizations of the trust region problem." Optimization
methods and Software, 1993. which is incorporated here by
reference.
[0129] It is shown that y.epsilon..sup.3 is an optimal solution if
and only if there exists .epsilon. such that .lamda..epsilon.:
(A.sup.TA+.lamda.D)y=A.sup.Tb-.lamda.f
y.sup.TDy+2f.sup.Ty=0
A.sup.TA+.lamda.D.gtoreq.0.
[0130] It follows that the optimal solution to (3) is given by:
{circumflex over
(y)}(.lamda.)=(A.sup.TA+.lamda.D).sup.T(A.sup.Tb-.lamda.f), (4)
[0131] Where .lamda. is the unique solution of:
{circumflex over (y)}(D).sup.TDy(.lamda.)-2f.sup.Ty(.lamda.)=0
(5)
over the interval where A.sup.TA+.lamda.D is positive definite.
Interval satisfying this property can be found by using congruence
transformations. By Sylvester's law of inertia, the matrix
C.sup.THC, where C is a non-singular matrix, has the same number of
positive eigenvalues, negative eigenvalues and zero eigenvalue as
matrix H, as is disclosed in: [0132] G. Strang. Linear Algebra and
Its Applications. 3rd. Brooks Cole. February 1988. which is
incorporated here by reference.
[0133] By saying that H is congruent to G if G=C.sup.THC for some
non-singular C and denoting this equivalence relation by H.about.G,
we have:
A.sup.TA+.lamda.D=(A.sup.TA).sup.1/2(I+.lamda.(A.sup.TA).sup.-1/2D(A.sup-
.TA).sup.-1/2)(A.sup.TA).sup.1/2.about.I+.lamda.(A.sup.TA).sup.-1/2D(A.sup-
.TA).sup.-1/2.
since all of the matrices on the right hand side of the equation
has nonnegative eigenvalues, it follows that:
= ( - 1 .lamda. max ( ( A A ) - 1 / 2 D ( A A ) - 1 / 2 ) , .infin.
) . ##EQU00006##
[0134] It is given in the publication "Linear Algebra and Its
Applications" that .lamda. can be found by simple bisection
algorithm since the function (5) is decreasing on the interval
.
Localization Based on TDOA
[0135] Referring to FIG. 3, setting one of the microphones
(m.sub.1-m.sub.4) (generalized as m.sub.i) as the reference
(denoting the reference microphone as `0`) and--without loss of
generality--setting it as the origin (0,0), we define the range
difference measurements between each microphone m.sub.i and
reference microphone 0 as:
d.sub.i=.parallel.s-m.sub.i.parallel.-.parallel.s.parallel.,i=1,2,
. . . ,M.
[0136] Geometrically, the points in the 2D plane that have a fixed
distance difference to two fixed anchors trace a hyperbola. Since
distance difference to two anchors yield one hyperbola, if we have
three or more microphones, we have multiple hyperbolas and in the
presence of precise range difference measurements, the intersection
of the hyperbolas yield the source position `s`. However, if there
is jitter in the measurements and we have more than two hyperbolas,
the hyperbolas will not intersect at a single point. In this case,
we can solve an optimization problem to find the best position
estimate for the source position `s`. Rewriting the range
difference equality we have:
-2d.sub.i.parallel.s.parallel.-2m.sub.i.sup.Ts=d.sub.i.sup.2-.parallel.m-
.sub.i.parallel..sup.2,i=1,2, . . . ,M,
which is satisfied in jitter-free measurements.
[0137] However when there is jitter, the equality does not hold,
but a reasonable estimate for the source position `s` can be found
by solving so called squared-range-difference-based least squares
problem:
minimize x .di-elect cons. 2 i = 1 M ( - 2 m i x - 2 d i x - g i )
2 , ( 6 ) ##EQU00007##
where y.sub.i=d.sub.i.sup.2-.parallel.m.sub.i.parallel..sup.2. as
is disclosed in [0138] J. O. Smith and J. S. Abel. "Closed-form
least-squares source location estimation from range-difference
measurements." Acoustics, Speech and Signal Processing. IEEE
Transactions on, vol. 35, no. 12, pp. 1661 1669, 1987. and [0139]
A. Beck, P. Stoica, and J. Li, "Exact and Approximate Solutions of
Source Localization Problems." Signal Processing. IEEE Transactions
on, vol. 56, no. 5, pp. 1770 1778, 2008. both of which are
incorporated herein by reference.
[0140] A closed form solution to this problem is derived in the
publication "Exact and Approximate Solutions of Source Localization
Problems". First, the problem is written in constraint form:
minimize y .di-elect cons. 3 B y - g 2 subject to y Cy = 0 y 3
.gtoreq. 0 , where y = ( x x ) and B = ( - 2 m 1 - 2 d 1 - 2 m M -
2 d m ) , C = ( I 2 0 2 .times. 1 0 1 .times. 2 - 1 ) . ( 7 )
##EQU00008##
[0141] It is shown in the publication "Exact and Approximate
Solutions of Source Localization Problems" that the sufficient
conditions for y to be the optimal point of the problem is there
exists .lamda..epsilon. such that:
(B.sup.TB+.lamda.C)y=B.sup.Tg
B.sup.TB+.lamda.C.gtoreq.0
y.sup.TCy=0
y.sub.3.gtoreq.0.
[0142] Using the optimality conditions a procedure prototype is
explained to find optimal solution as: [0143] 1. Find solution
.lamda.* to:
[0143] y(.lamda.).sup.TCy(.lamda.)=0,.lamda..epsilon.I.sub.1,
Where
y(.lamda.)=(B.sup.TB+.lamda.C).sup.-1B.sub.Tg, and I.sub.1 is the
interval where B.sup.TB+.lamda.C is positive definite. [0144] 2. If
the last entry of y(.lamda.*) is nonnegative, i.e. for
z=y(.lamda.*) we have z.sub.3.gtoreq.0, then z is the global
optimizer of the problem and position estimate can be found by
taking the first two entries of z. [0145] However, it might be the
case that the resulting vector does not satisfy the condition
z.sub.3.gtoreq.0. To find the global optimizer in this case the
necessary optimality conditions derived in the publication "Exact
and Approximate Solutions of Source Localization Problems" are
used, which states that the optimal solution to (7) is either y=0
or has the form:
[0145] t(.lamda.)=(B.sup.TB+.lamda.C).sup.-1B.sub.Tg, [0146] where
.lamda. satisfies:
[0146] y(.lamda.).sup.TCy(.lamda.)=0.
and B.sup.TB+.lamda.C has at most one negative eigenvalue. The
intervals of .lamda. corresponding to these settings can be found
by the congruence relation as before as:
B.sub.TB-.lamda.C=(B.sup.TB).sup.1/2(I+.lamda.(B.sup.TB).sup.-1/2C(B.sup-
.TB).sup.-1/2)(B.sup.TB).sup.1/2.about.I+.lamda.(B.sup.TB).sup.-1/2C(B.sup-
.TB).sup.-1/2.
[0147] Defining the matrix:
V=(B.sup.TB).sup.-1/2C(B.sup.TB).sup.-1/2
and denoting the i.sup.th eigenvalue of V as .lamda..sub.i(V) where
the eigenvalues are ordered in decreasing order as
.lamda..sub.1.gtoreq..lamda..sub.2.gtoreq..lamda..sub.3. Since have
B.sup.TB positive definite and C has 1 negative and 2 strictly
positive eigenvalues, we have
.lamda..sub.1.gtoreq..lamda..sub.2.gtoreq.0.gtoreq..lamda..sub.3.
From the congruence relation we see that signs of the eigenvalues
of B.sup.TB+.lamda.C are the same with I+.lamda.V which has
eigenvalues 1+.lamda..lamda..sub.i(V). Using these facts there are
three disjoint intervals where B.sub.TB+.lamda.C has at most 1
negative eigenvalue:
1. I 0 = ( - 1 .lamda. 3 ( V ) , ? ) : gives 2 positive 1 negative
eigenvalues ##EQU00009## 2. I 1 = ( - 1 .lamda. 1 ( V ) , - 1
.lamda. ? ( V ) ) : gives 3 positive eigenvalues ##EQU00009.2## 3.
I 2 = ( - 1 .lamda. 2 ( V ) , - 1 .lamda. ? ( V ) ) : gives 2
positive and 1 negative eigenvalues ##EQU00009.3## ? indicates text
missing or illegible when filed ##EQU00009.4##
[0148] Using these intervals, the full procedure is defined as:
[0149] 1. Find solution .lamda.* to
[0149] y(.lamda.).sub.TCy(.lamda.)=0,.lamda..epsilon.I.sub.1.
Where
y(.lamda.)=(B.sup.TB+.lamda.C).sup.-1B.sup.Tg. If the last entry of
y(.lamda.*) is nonnegative, i.e. for z=y(.lamda.*) we have
z.sub.3.gtoreq.0, then z is the global optimizer of the problem and
position estimate can be found by taking the first two entries of
z. If z.sub.3<0 then perform steps 2 and 3. [0150] 2. Find all
roots .lamda..sub.1, .lamda..sub.2, . . . , .lamda..sub.p of
[0150]
y(.lamda.).sup.TCy(.lamda.)=0,.lamda..epsilon.I.sub.0.orgate.I.su-
b.2, for which third entry of Ills nonnegative. [0151] 3. Set z as
the vector with smallest objective function among 0,
y(.lamda..sub.1), y(.lamda..sub.2), . . . , y(.lamda..sub.p). Take
position estimate as the first two entries of z.
Method of Images
[0152] Method of images (also known as image source model) provides
that reflections coming from walls can be viewed as direct signals
coming from virtual sources. These virtual sources are obtained by
mirroring the true source across the reflecting walls (possibly
across multiple walls) as disclosed in: [0153] J. Borish.
"Extension of the image model to arbitrary polyhedra." The Journal
of the Acoustical Society of America. 1984. which is incorporated
herein by reference.
[0154] FIG. 4 provides an illustration of 1.sup.st order virtual
source V.sub.1 and V.sub.2 (generalized as v.sub.1) generated from
walls 5, and 6 (generalized as i) respectively.
[0155] The positions of these virtual sources v.sub.i can be found
by:
v i = s - 2 n i ( s - p i ) n i = s - 2 N i ( s - p i ) . ( 8 )
##EQU00010##
where
N.sub.i:=n.sub.in.sub.i.sup.T
is the orthogonal projection operator onto the normal to wall i,
and p.sub.i is any point belonging to the i.sup.th wall.
[0156] To find higher order virtual sources one can reflect the
source across multiple walls, or equivalently reflect a virtual
source across a wall, as:
v.sub.i,j=v.sub.i-2N(v.sub.i-p.sub.j). (9)
[0157] By using the method of images, we are reducing the problem
of localizing the source in a room, to localization of multiple
sources in free space.
Problem Setup
[0158] One of the goals of the present invention is localization of
a source which transmits a signal (e.g. an ultrasonic source or
radio source) in a known reverberant room having general geometry,
not limited to convex, bounded by at least some planar walls, from
the measurements by a receiver (e.g. a microphone array). In the
present description an example in which the source is an ultrasonic
source and a receiver is a microphone array will be described,
however it will be understood that the present invention is not
limited to such an embodiment and other suitable types of sources
and receivers can be used.
[0159] When the room is convex, assuming point microphones so they
do not block the signals, the source is visible by all microphones
in the microphone array (i.e. each microphone in the microphone
array can receive a signal (such as an acoustic signal) which is
emitted by the source). When the source is visible, all microphones
(receivers) hear the direct signal, and the direct signal arrives
before any echo. Thus, in the convex room setting, these direct
signals can be used for the localization of the source, and it is
reported that the performance of the localization algorithms
decreases with reverberation, although there are notable
exceptions.
[0160] In an exemplary problem setting of the present invention,
the room is not assumed to be convex, thus there are positions in
the room where the source is partially visible or not visible by
the microphones in the microphone array (i.e. some microphone in
the microphone array cannot directly receive a signal (such as an
acoustic signal) which is emitted by the source), thus direct
signal may not be heard. In the context of the present invention a
"direct" signal is a signal which has not been reflected (e.g.
which has not been reflected by a wall or object). However, the
echoes reflecting from the walls are received by those microphones
in the microphone array which do not receive the direct signal. In
this setting, echoes are used, which in general makes the
performance worse in the convex room, to localize the source in
room with general geometry.
[0161] With reference to FIGS. 5A and 5B the proposed localization
algorithm is as follows. The signal emitted by an ultrasonic source
7 is recorded with a microphone array. The echoes coming from
different walls 5,6 may be received in different orders at each of
the microphones in the array, or put in context of the method of
images, signals coming from the virtual sources V.sub.5 V.sub.5,6
can be heard (i.e. received) in different orders by the
microphones. Since the virtual source localization algorithms
explained in above requires distances (or difference of distances
in TDOA localization) to a single source in order to estimate its
position, we need to find which echoes correspond to a single
virtual source. After finding such echoes and localizing the
virtual sources V.sub.5 V.sub.5,6, we know if they are located
inside or outside the room since the room geometry is known. In
case we find that a virtual source is located a position inside the
room we are done. However, if we find that a virtual source is
located a position outside the room, we need to find the position
inside the room that generates that virtual source, which we do by
using the method of images in the reverse order. From the multiple
localized and reflected sources, we find the position that best
estimates the source position and optimize the estimation by using
a measure based on the difference between the recorded and
simulated recordings which will be described in more detail
later.
Source Location Estimation
[0162] The building blocks of a method according to the present
invention will now be described: the forward model for generating
virtual sources given a source position inside the room,
localization of virtual sources from the recordings by the
microphone array, reflecting the localized virtual sources into the
room, estimating the source position from multiple reflected
sources and optimizing the position of the source location
estimate.
Forward Model
[0163] In an embodiment of the present invention a forward model is
used which generates the recordings by the microphone array given
the room geometry, source position and the microphone positions.
The approach used for the forward model in this exemplary
embodiment is based on the method disclosed in: [0164] J. B. Allen
and D. A. Berkley, "Image method for efficiently simulating
small-room acoustics." The Journal of the Acoustical Society of
America, 1979. and [0165] J. Borish, "Extension of the image model
to arbitrary polyhedra," The Journal of the Acoustical Society of
America. 1984. both of which are incorporated herein by
reference.
[0166] The positions of the virtual sources can be found by the
equations disclosed in the previous sections. For the generation of
virtual sources and checking if each virtual source is heard by a
microphone in a specific position, there are three aspects that
need to be tested: validity, visibility and obstruction.
[0167] Validity: The virtual source needs to correspond to valid
echoes. For a candidate virtual source to be valid, the generating
source needs to be directly adjacent to that wall. An example of an
invalid virtual source is reflecting a first generation virtual
source back in the room, across the same wall that generated it in
the first place. This would correspond to two consecutive bounces
off the same wall, which is physically infeasible. Visibility: With
reference to FIG. 6 in order for a virtual source V.sub.2 V.sub.2,4
to be heard by a microphone m.sub.1, virtual source needs to be
visible in the wall 4,2 it was generated from. A virtual source is
"visible" in a wall if a line joining the virtual source and
microphone intersects that wall. In other words, the line joining
the virtual V.sub.2 V.sub.2,4 source and the microphone m.sub.1
should intersect the wall 4,2 that generates the virtual source. As
can been seen in FIG. 6 the line joining virtual source V.sub.2 and
microphone m.sub.1 does not intersect the wall 2 that generates the
virtual source V.sub.2, while the line joining virtual source
V.sub.2,4 and microphone m.sub.1 does intersect the wall 4 that
generates the virtual source V.sub.2,4.
[0168] Although this is sufficient for a first order echo, for
higher order echoes, one needs to check visibility also in the
walls that were used to generate lower order virtual sources
generating it, i.e., point of intersection `a` of the generating
wall and the line drawn between the virtual source V.sub.2
V.sub.2,4 and microphone m.sub.1 needs to be visible from the
parent walls generating the virtual source. A parent wall is a wall
that is part of the sequence of walls (reflections) that lead to a
particular image source. Visibility of an image source from a
certain point means that the receiver at that point can hear the
signal from the image source. Conditions for visibility of higher
order image sources are also illustrated in FIG. 6. Although
V.sub.2,4 is visible by the microphone in wall (4), it will not be
heard by the microphone unless wall (2) extends to point a. With
the illustrated room geometry, the microphone can only hear
V.sub.2,4 from the hatched region in the top left corner.
[0169] Obstructions: In a convex room, since convex combinations of
any set of points belonging to the room is also inside the room
there is no obstruction of the source. However, as shown in FIG. 7
when the room is not convex, there are walls 7,8 that may obstruct
the line of sight between the microphones and the source. Hence, in
non convex room, it is necessary to check if there is line of sight
between the virtual source and the microphones. To this end, one
can note that only the walls that violate the convexity can
obstruct the line of sight, thus we may label these walls as
`obstructive` and check only if they are obstructing the line of
sight in each iteration to reduce the number of tests. The
`obstructive` walls can be found by finding the convex hull of the
room, and labelling which walls intersect its interior as shown in
FIG. 7.
[0170] Localization of Virtual Sources
[0171] In this section we discuss the localization of virtual
sources in two settings, where we have TOA and TDOA. First we
consider the case where we have the signals recorded by M
microphones containing the TOA. At the receiver, we do not know
whether the signal is coming directly from the source or through a
reflection from a wall. In particular, if the signal is reflected,
we do not know which wall(s) generate the reflection.
[0172] In order to localize the virtual sources, we take one
arrival time from each microphone combinatorially and we calculate
the range by multiplying it with the speed of sound, to obtain the
distance r.sub.i between the virtual source and microphone where
i=1, 2, . . . , M. FIG. 8B shows the virtual sources
V.sub.1-V.sub.4 generated by real source s and also shows
microphones mic1-4 FIG. 8A shows the time of arrival at microphones
mic1-4 of echoes coming from virtual sources V.sub.1-V.sub.4. From
these ranges, we localize the position by using squared-range-based
least squares algorithm to have a position estimate s If the
selected echoes correspond to a correct combination, i.e. they are
generated by a single virtual source, the algorithm will produce a
location whose distances to microphones match r.sub.i with high
precision. However, if the echo combination used for localization
does not come from a single virtual source, with very high
probability, there will be no point in the 2D plane that will have
these distances to the microphones. Based on this idea we define
the localization score, G.sub.LOC that measures on how well the
resulting distances and the input distances match as:
G LOC ( s ^ ) := i = 1 M ( s ^ - m i - r i ) 2 . ##EQU00011##
[0173] Using this measure, we say that a particular combination of
echoes corresponds to a single virtual source if the score is less
than a chosen threshold.
[0174] In the case where we do not have TOA but TDOA, we use a
similar approach to find correct echo combinations corresponding to
a single virtual source. We designate one microphone as the
reference microphone and--without loss of generality--assume it to
be at the origin. Then we go through each pulse recorded in that
reference microphone and we combinatorially take pulses one from
other microphones and multiply the times by the speed of sound to
obtain distances. Before using the chosen echo combination in the
squared-range-difference-based least squares optimization, we shift
the pulses so that the distance difference in the reference
microphone equals to 0 (to have it indeed become the reference).
Formally, if t.sub.i is the time instances of the selected pulses
from microphones i=0,1, . . . , M-1 where microphone 0 is the
reference, we define range-differences as
d.sub.i=c(t.sub.i-t.sub.0), where c is the speed of propagation of
sound. Then we localize the virtual source using
squared-range-difference-based least squares algorithm using
distances d.sub.1, . . . , D.sub.M-1 and check if the echo
combination was correct by evaluating range-difference localization
score, G.sub.RDL, defined as:
G RDL ( s ) := i = 1 M - 1 ( s - m i - ? - d i ) 2 . ? indicates
text missing or illegible when filed ##EQU00012##
[0175] Again, we accept the chosen echo combination as coming from
a single virtual source if the score is less than a threshold.
[0176] Reflecting Localized Virtual Sources
[0177] After finding the location of the virtual source, since the
room geometry is known, one can use the method of images in reverse
order to find the real source position that would have generated
that virtual source.
[0178] Referring to FIG. 9 and FIG. 10A-C, in order to find the
real source location, we draw the lines connecting the virtual
source 10 to the microphones 11a-d and find the wall 12 that
intersects these lines. Then the virtual source is reflected across
that wall as shown in FIG. 10b, and the points of intersections on
the wall are remembered. If the reflected source is inside the
room, as shown in FIG. 9, we are done. If not, as is the case in
FIG. 10B, lines connecting the stored intersection points and the
new virtual source 15 are drawn, as shown in FIG. 10B, and the
virtual source is reflected across the new wall 13 of intersection,
as is illustrated in FIG. 10C. This procedure is iterated until
position inside room is found, as is shown in FIG. 10C. Thus
illustration of this procedure is depicted in FIGS. 10A-C, where a
second order virtual source is reflected into the room.
[0179] One problem that might occur while applying the inverse
method of images is that the lines drawn to multiple microphones
may intersect multiple walls. This may happen due to errors in
virtual source localization or jitter. In that case one may choose
to drop that localized virtual source or reflect across the wall
with the highest number of intersections.
[0180] Estimating the Source Position
[0181] So far we have multiple localized virtual sources that are
reflected inside the room. The remaining questions is how to pick
the estimate position for the true source position. To this end,
one may make use of the localization scores, G.sub.LOC (or
G.sub.RDL for TDOA), of the virtual sources where the less
G.sub.LOC is the closer the estimated virtual source is to the true
one. However, as the measurement jitter increases, incorrect echo
combinations start to mimic correct echo combinations coming from
false virtual sources and give low localization scores. Hence, for
stable localization in case of high measurement jitter, a different
measure (score) of how good/accurate an position estimate is may be
used:
[0182] We derive the score based on the following idea when we have
TOA recordings. If the estimated source position is close to the
true source, the simulated room impulse response from the estimated
source will be close to the recorded one as shown in FIG. 11. Using
this idea we define a score measuring how close the simulated and
the recorded impulse responses are by taking echo by echo recorded
by each microphone and finding the closest peak in the simulated
response. We then sum the squares of differences between these two.
Mathematically, we define the RIR score, G.sub.RIR, as:
G RIR ( s ^ ) := i = 1 M j = 1 n m e i , j 2 , where ##EQU00013## e
i , j = min k r ^ i , k - r i , j , ##EQU00013.2##
and r.sub.i,j is the j.sup.th pulse recorded by i.sup.th
microphone, and {circumflex over (r)}.sub.i,k is the k.sup.th pulse
that would have been recorded by the i.sup.th microphone if the
source was at s.
[0183] Using this measure, we pick the reflected source that gives
the least G.sub.RIR as the estimate position.
[0184] Optimizing the Position Estimate
[0185] Above we have defined a score based on RIR to choose among
the localized and reflected virtual sources the virtual source that
accurately estimates the true source position. We can further
improve the estimate by moving it inside the room so that the RIR
score, G.sub.RIR, is minimized. FIG. 12 provides an illustration
depicting the notation of optimization of position measurement.
[0186] Towards this end, we define the virtual source that
simulates the echo closest in time to the j.sup.th echo recorded in
i.sup.th microphone by {circumflex over (v)}(i,j) as:
v ^ ( i , j ) ( s ^ ) = arg min k r i , j - .nu. ^ k ( s ^ ) - m i
, ##EQU00014##
[0187] where we denote virtual sources of any order by single
subscript. With this notation we define the RIR score again, this
time as an explicit function of the estimated source position:
G RIR ( s ^ ) = i = 1 M j ( r i , j - .nu. ^ ( i , j ) ( s ^ ) - m
i ) 2 . ##EQU00015##
[0188] To minimize score one may solve the optimization:
minimize s .di-elect cons. ? 2 i = 1 M j ( r i , j - v ^ ( i , j )
( s ^ ) - m i ) 2 . ( 10 ) ? indicates text missing or illegible
when filed ##EQU00016##
[0189] Although this problem is again non-convex, if the initial
position estimate is good enough we can find the minimizer
iteratively by using gradient descent, or any other local search
technique. The gradient of the RIR with respect to the source
position is calculated as:
.gradient. G RIR ( s ^ ) = i = 1 M j 2 ( w .di-elect cons. v ^ ? (
I 2 - 2 n w n w ) ) ( v ^ ( i , j ) ( s ^ ) - m i - r i , j ) v ^ (
i , j ) ( s ^ ) - m i v ^ ( i , j ) ( s ^ ) - m i , ? indicates
text missing or illegible when filed ##EQU00017##
[0190] where the product is over the wall sequence that generates
the virtual source {circumflex over (v)}(i,j). Then the position is
optimized iteratively by setting:
s.rarw.s-.eta..gradient.G.sub.RIR({circumflex over (s)}),
[0191] where .eta..gtoreq.0 is the learning rate. The algorithm may
be stopped when the l.sub.2 norm of the update in the source
position is smaller than a predefined positive threshold.
[0192] Given the measurements with jitter, .epsilon..sub.i,k
resulting from the virtual source v.sub.k obtained from true source
position in microphone i as:
r.sub.i,k=.parallel.v.sub.k-m.sub.i.parallel.+.epsilon..sub.i,k
we solve the optimization problem:
minimize x i = 1 M k ( r i , k - v ^ k - m i ) 2 , ( 11 )
##EQU00018##
equivalently,
minimize x i = 1 M k ( v k - m i + i , k - v ^ k - m i ) 2 .
##EQU00019##
[0193] If the jitter is i.i.d. Gaussian, the optimization problem
gives the source position that will generate the echoes with
maximum likelihood. Denoting the likelihood of obtaining the set of
recorded echoes as:
p ( r s ^ ) = i = 1 M k p ( r i , k v ^ k ) , ##EQU00020##
[0194] and taking negative logarithm to get the negative
log-likelihood, we have:
- log ( i = 1 M k p ( r i , k v ^ k ) ) = - i = 1 M k log ( p ( r i
, k v ^ k ) ) = - i = 1 M k log ( 1 2 .pi..sigma. 2 exp ( - ( r i ,
k - v ^ k - m i ) 2 / 2 .sigma. 2 ) ) = i = 1 M k const ( r i , k -
v ^ k - m i ) 2 , ##EQU00021##
where const is a constant that depends on .sigma.. Maximizing the
likelihood is equivalent to minimizing the negative log-likelihood,
which yields (11). Hence, by solving (11) we obtain the maximum
likelihood estimator for the position that generates the set of
recorded pulses.
[0195] However, since the signals coming from the virtual sources
are unlabeled, we do not have direct access to r.sub.i,k, thus we
do not know:
r.sub.i,k-.parallel.{circumflex over
(v)}.sub.k-m.sub.i.parallel..
[0196] The minimization problem (10) can be viewed as a heuristic
method aiming to solve (11), that estimates
r.sub.i,k-.parallel.{circumflex over
(v)}.sub.k-m.sub.i.parallel.
by taking the virtual source that gives the closest time difference
to r.sub.i,k.
Simulations
[0197] We will now present simulation results of the present
invention for source localization with TOA measurements. First the
results of localization in an L-shaped room with and without
measurement jitter will be shown. Then it will be shown how the
present invention performs for room with complex geometry that has
no parallel walls and conclude by applying the present invention to
tracking a moving source.
[0198] It should be noted that in all of the FIGS. 13-21, blank
circles denote microphone positions and black dots depicts the true
source position. Squares are localized and reflected virtual source
ordered by their localization score where smaller index denotes
better localization score. The striped dot marks the reflected
virtual source that has the best RIR score, and the crossed dot is
the result of optimization algorithm based on the position of
striped dot.
Source Localization
[0199] For testing the developed indoor localization algorithm, we
take a typical nonconvex room having L-shaped geometry shown in
FIG. 13.
[0200] Referring to FIGS. 14A-16B, to test the performance of the
localization algorithm without jitter we position the source at
(6,7) and use four microphones m.sub.1-m.sub.4 positioned randomly
by uniform distribution over the square region with bottom left
corner at (1,1) and upper right corner at (3,3) to record the
signal up to third order echoes. It is important to note that with
this setting, there is no line of sight between the source and any
of the microphones.
[0201] An outcome of the localization from jitter-free measurements
can be seen in FIGS. 14A,B. It can be seen that the present
invention finds estimates close to the true source position, and
the localization scores mark positions close to the true source as
begin good. It is also seen that the RIR score chooses within
reflected virtual sources the closest one to the true source
position. The crossed dot that depicts the outcome of the
optimization algorithm based on the position of the striped dot is
seen to perfectly localize the source.
[0202] FIGS. 15A,B shows an outcome of localization when there is
measurement jitter drawn i.i.d. from centered Gaussian with
.sigma.=0.05. It is seen that although there are reflected sources
in the vicinity of the true source position, the best reflected
sources in terms of localization scores are away from it. Thus
picking the reflected source having best localization score as the
estimate of the true source position is not a valid option.
However, also here, the reflected source having the best RIR score
is the one closest to the true source position and the estimate is
further improved by applying the optimization step based on that
position.
[0203] As the last localization simulation we show the result of
using the present invention for a room with very complex geometry
with measurement jitter drawn i.i.d. from centered Gaussian with
.sigma.=0.1. As can be seen in FIGS. 16A,B although the reflected
sources are distributed in a broad range, the vicinity of the true
source position is still dense. Furthermore, although the positions
having best localization scores are distributed, the RIR score
picks the one that is closest to the true source position and
optimization algorithm gives an even closer estimate.
Source Tracking
[0204] One approach for source tracking is by taking measurements
at distinct time instances and localizing the position
independently of the previous ones. However, since the position of
the source depends on its history, one can leverage the previous
positions estimates in localizing the source.
[0205] In this simulation we compare the performance of tracking a
moving source with two approaches. First approach is by going
through all of the steps of the algorithm by recording the signal,
finding echoes corresponding to virtual sources and localizing
them, reflecting the localized virtual sources and taking the one
giving the highest RIR score but not applying the optimization
algorithm. The second method is by localizing first position by
using first method and in addition applying optimization algorithm
and for other time instances, applying only optimization algorithm
based on the position estimate of the previous time instance.
[0206] FIGS. 17A,B show the results of the two approaches where the
source traces the curve:
s.epsilon..sup.2,
where,
s.sub.1(t)=4-3 cos.sup.3(2.pi.t/120)
[0207] and s.sub.2(t)=6.5+2 sin.sup.3(2.pi.t/120) for t=0, 1, 2, .
. . , 120, and the jitter is drawn i.i.d. from centered Gaussian
with .sigma.=0.05.
[0208] FIG. 17A shows the result of the first approach and FIG. 17B
shows the result of the second approach. As can be seen the second
approach outperforms the first approach while being computationally
lighter.
[0209] We will now discuss the objective functions behind the
optimization step and plot average localization error for special
case of square room through simulations.
[0210] FIG. 18A-D show the contours of the function (11) which the
present invention aims to minimize by solving the heuristic
optimization (10). The contours are drawn for a square room of size
8.times.8 for the source position of:
s=(4,5).sup.T
[0211] and first order virtual sources generated from this
position.
[0212] FIGS. 18A and 18B show the contours for one microphone for
measurements without jitter (FIG. 18A) and with jitter (FIG. 18B).
As can be seen, when there is no jitter in the measurements, the
contours are smooth and the optimal position can be found, e.g., by
gradient descent algorithm. With jitter, the contours gets
distorted and local minima can occur, hence gradient descent
algorithm may get stuck in local minima. However, as seen from
FIGS. 18A and 18B as the number of microphones increase the cost
curves becomes smoother with for same jitter level.
[0213] FIG. 19 shows the average l.sub.2 localization error defined
as the Euclidean distance between the true source and the estimated
source positions, i.e.
E.sub.l.sub.2:=.parallel.s-s.parallel..
[0214] In order to find the optimal position, the algorithm is
started from the vicinity of true source position and gradient
descent algorithm is used. The plot shows that localization error
based on minimization of (11) increases smoothly as the jitter is
increased.
[0215] However, as explained in earlier above, since we do not have
the labels for the echoes, we cannot minimize (11) so we estimate
its solution by using the heuristic method (10). The contours for
the (10) are plotted in FIG. 10A for jitter free measurements using
one microphone, and are plotted in FIG. 10E for jitter free
measurements using three microphones. As can be seen, the resulting
function has local minima, hence gradient descent algorithm may get
stuck at local minima. However, as also can be seen by comparing
the two plots, as the number of microphones increases the contours
become smoother, and if the starting point of the algorithm is
close enough to the optimal position, one can find the global
minimizer by the gradient descent algorithm.
FURTHER ASPECTS OF THE PRESENT INVENTION
[0216] Finally, we will now describe FIG. 21 which illustrates an
embodiment of a system according to the invention and FIG. 22 which
illustrates an embodiment of a data processing system in which a
method in accordance with an embodiment of the present invention
may be implemented.
[0217] FIG. 22 is an embodiment of a data processing system 300 in
which an embodiment of a method of the present invention may be
implemented. The data processing system 300 of FIG. 22 may be
located and/or otherwise operate at any node of a computer network,
that may exemplarily comprise clients, servers, etc., and it is not
illustrated in the Figure. In the embodiment illustrated in FIG.
22, data processing system 300 includes communications fabric 302,
which provides communications between processor unit 304, memory
306, persistent storage 308, communications unit 310, input/output
(I/O) unit 312, and display 314.
[0218] Processor unit 304 serves to execute instructions for
software that may be loaded into memory 306. Processor unit 304 may
be a set of one or more processors or may be a multi-processor
core, depending on the particular implementation. Further,
processor unit 304 may be implemented using one or more
heterogeneous processor systems in which a main processor is
present with secondary processors on a single chip. As another
illustrative example, the processor unit 304 may be a symmetric
multi-processor system containing multiple processors of the same
type.
[0219] In some embodiments, the memory 306 shown in FIG. 22 may be
a random access memory or any other suitable volatile or
non-volatile storage device. The persistent storage 308 may take
various forms depending on the particular implementation. For
example, the persistent storage 308 may contain one or more
components or devices. The persistent storage 308 may be a hard
drive, a flash memory, a rewritable optical disk, a rewritable
magnetic tape, or some combination of the above. The media used by
the persistent storage 308 also may be removable such as, but not
limited to, a removable hard drive.
[0220] The communications unit 310 shown in FIG. 22 provides for
communications with other data processing systems or devices. In
these examples, communications unit 310 is a network interface
card. Modems, cable modem and Ethernet cards are just a few of the
currently available types of network interface adapters.
Communications unit 310 may provide communications through the use
of either or both physical and wireless communications links.
[0221] The input/output unit 312 shown in FIG. 22 enables input and
output of data with other devices that may be connected to data
processing system 300. In some embodiments, input/output unit 312
may provide a connection for user input through a keyboard and
mouse. Further, input/output unit 312 may send output to a printer.
Display 314 provides a mechanism to display information to a
user.
[0222] Instructions for the operating system and applications or
programs are located on the persistent storage 308. These
instructions may be loaded into the memory 306 for execution by
processor unit 304. The processes of the different embodiments may
be performed by processor unit 304 using computer implemented
instructions, which may be located in a memory, such as memory 306.
These instructions are referred to as program code, computer usable
program code, or computer readable program code that may be read
and executed by a processor in processor unit 304. The program code
in the different embodiments may be embodied on different physical
or tangible computer readable media, such as memory 306 or
persistent storage 308.
[0223] Program code 316 is located in a functional form on the
computer readable media 318 that is selectively removable and may
be loaded onto or transferred to data processing system 300 for
execution by processor unit 304. Program code 316 and computer
readable media 318 form a computer program product 320 in these
examples. In one example, the computer readable media 318 may be in
a tangible form, such as, for example, an optical or magnetic disc
that is inserted or placed into a drive or other device that is
part of persistent storage 308 for transfer onto a storage device,
such as a hard drive that is part of persistent storage 308. In a
tangible form, the computer readable media 318 also may take the
form of a persistent storage, such as a hard drive, a thumb drive,
or a flash memory that is connected to data processing system 300.
The tangible form of computer readable media 318 is also referred
to as computer recordable storage media. In some instances,
computer readable media 318 may not be removable.
[0224] Alternatively, the program code 316 may be transferred to
data processing system 300 from computer readable media 318 through
a communications link to communications unit 310 and/or through a
connection to input/output unit 312. The communications link and/or
the connection may be physical or wireless in the illustrative
examples. The computer readable media also may take the form of
non-tangible media, such as communications links or wireless
transmissions containing the program code.
[0225] The different components illustrated for data processing
system 300 are not meant to provide architectural limitations to
the manner in which different embodiments may be implemented. The
different illustrative embodiments may be implemented in a data
processing system including components in addition to or in place
of those illustrated for data processing system 300. Other
components shown in FIG. 22 can be varied from the illustrative
examples shown. For example, a storage device in data processing
system 300 is any hardware apparatus that may store data. Memory
306, persistent storage 308, and computer readable media 318 are
examples of storage devices in a tangible form.
[0226] Therefore, as explained at least in connection with FIG. 22
the present invention is as well directed to a system for
determining location of an object, a computer program product for
determining location of an object and a computer data carrier.
[0227] In accordance with a further embodiment of the present
invention is provided for a computer data carrier storing
presentation content created while employing the methods of the
present invention.
[0228] Although the present invention has been described in more
detail in connection with its embodiment for determining the
location of a loudspeaker or a microphone, the present invention
finds applicability of connection with many other fields.
[0229] The present invention can be used for determining the exact
position of a receiver r, which is a person in the FIG. 21. In the
case a satellite, e.g. a GPS satellite is the source s of a radio
signal which can be reflected by some buildings B1, B2. If the echo
e1 is not used, the localisation of a mobile device r of a person
can be computed incorrectly (the mobile device r will be considered
located in correspondence of {tilde over (e)}).
[0230] Knowing the position of the satellite s, the position of the
buildings B1, B2, etc. (this is possible e.g. by using an
electronic map) and applying the method according to the invention,
it is possible to accurately locate the mobile device r and then
the person, without any error.
* * * * *