U.S. patent application number 16/517120 was filed with the patent office on 2020-01-23 for method and device for increasing the accuracy of localization.
The applicant listed for this patent is Continental Automotive GmbH. Invention is credited to Henning Hamer, Helmut Hamperl, Karn Holger, Karsten Mattmuller.
Application Number | 20200025945 16/517120 |
Document ID | / |
Family ID | 66647181 |
Filed Date | 2020-01-23 |
United States Patent
Application |
20200025945 |
Kind Code |
A1 |
Hamer; Henning ; et
al. |
January 23, 2020 |
METHOD AND DEVICE FOR INCREASING THE ACCURACY OF LOCALIZATION
Abstract
A method and a device for increasing the accuracy of
localization. The method includes the following steps: a) detecting
a selected object in an environment, using a camera of a first
vehicle; b) estimating a first position of the selected object,
wherein a first program uses first location coordinates of the
first vehicle and a first distance estimate to the selected object;
c) transferring an attribute of the selected object, the first
location coordinates and the first distance estimate to a server;
d) carrying out the steps a) to c) for a second vehicle; e)
calculating a third position of the selected object by the server,
wherein a third program performs an averaging of the plurality of
location coordinates and distance estimates; f) using the third
position of the selected object as a reference position for
increasing the accuracy of a localization of a third vehicle.
Inventors: |
Hamer; Henning; (Munchen,
DE) ; Mattmuller; Karsten; (Burglengenfeld, DE)
; Hamperl; Helmut; (Cham, DE) ; Holger; Karn;
(Tournefeuille, FR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Continental Automotive GmbH |
Hannover |
|
DE |
|
|
Family ID: |
66647181 |
Appl. No.: |
16/517120 |
Filed: |
July 19, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G01C 21/30 20130101;
G01C 21/3602 20130101; G06T 2207/30252 20130101; G01S 19/48
20130101; G01S 5/0252 20130101; G01S 5/0036 20130101; G06T 7/70
20170101; G01C 11/06 20130101 |
International
Class: |
G01S 19/48 20060101
G01S019/48; G06T 7/70 20060101 G06T007/70; G01C 21/36 20060101
G01C021/36; G01C 11/06 20060101 G01C011/06 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 20, 2018 |
DE |
10 2018 212 133.8 |
Claims
1. A device for increasing the accuracy of localization in a
vehicle, which comprises a computation and control device, which is
configured to perform operations comprising: a) detecting a
selected object in an environment, using a camera of a first
vehicle; b) estimating a first position of the selected object, by
means of a first program for localization, in the first vehicle,
wherein the first program uses first location coordinates of the
first vehicle and a first distance estimate to the selected object;
c) transferring an attribute of the selected object, the first
location coordinates and the first distance estimate to a server;
d) carrying out the steps a) to c) for the selected object, to
estimate a second position of the selected object, using a camera
of a second vehicle, by means of a second program for localization,
using second location coordinates and a second distance estimate;
e) calculating a third position of the selected object by the
server, by means of a third program, using an attribute of the
selected object, the first and the second location coordinates and
the first and the second distance estimate, wherein the third
program performs an averaging of the plurality of location
coordinates and distance estimates; f) using the third position of
the selected object as a reference position for increasing the
accuracy of a localization of a third vehicle.
2. The device as claimed in claim 1, wherein step c*) is inserted
after step c), with c*), replacement, on the server, of the first
location coordinates and the second location coordinates, by means
of a fourth program, using corrected first location coordinates and
corrected second location coordinates.
3. (canceled)
4. (canceled)
5. The device as claimed in claim 2, wherein the selected object is
a static landmark or a traffic sign.
6. The device as claimed in claim 5, wherein the location
coordinates are determined using raw GPS.
7. The device as claimed in claim 6, wherein the position is
specified in WGS 94 coordinates.
8. (canceled)
9. (canceled)
10. (canceled)
11. (canceled)
12. (canceled)
13. A device for increasing the accuracy of localization in a
vehicle, which comprises a computation and control device, which is
configured to perform operations comprising: a) detecting a
selected object in an environment, using a camera of a first
vehicle; b) estimating a first position of the selected object, by
means of a first program for localization, in the first vehicle,
wherein the first program uses first location coordinates of the
first vehicle and a first distance estimate to the selected object;
c) transferring an attribute of the selected object, the first
location coordinates and the first distance estimate to a server;
d) carrying out the steps a) to c) for the selected object, to
estimate a second position of the selected object, using a camera
of a second vehicle, by means of a second program for localization,
using second location coordinates and a second distance estimate;
e*) calculation of a third position of the selected object, wherein
the third program calculates a center of gravity of the plurality
of location coordinates and distance estimates, wherein the third
program performs an averaging of the plurality of location
coordinates and distance estimates; f) using the third position of
the selected object as a reference position for increasing the
accuracy of a localization of a third vehicle.
14. The device as claimed in claim 13, wherein the selected object
is a static landmark or a traffic sign.
15. The device as claimed in claim 14, wherein the location
coordinates are determined using raw GPS.
16. The device as claimed in claim 15, wherein the position is
specified in WGS 94 coordinates.
17. A device for increasing the accuracy of localization in a
vehicle, which comprises a computation and control device, which is
configured to perform operations comprising: a) detecting a
selected object in an environment, using a camera of a first
vehicle; b) estimating a first position of the selected object, by
means of a first program for localization, in the first vehicle,
wherein the first program uses first location coordinates of the
first vehicle and a first distance estimate to the selected object;
c) transferring an attribute of the selected object, the first
location coordinates and the first distance estimate to a server;
d) carrying out the steps a) to c) for the selected object, to
estimate a second position of the selected object, using a camera
of a second vehicle, by means of a second program for localization,
using second location coordinates and a second distance estimate;
e) calculating a third position of the selected object by the
server, by means of a third program, using an attribute of the
selected object, the first and the second location coordinates and
the first and the second distance estimate, wherein the third
program performs an averaging of the plurality of location
coordinates and distance estimates; f*) correction of the first
location coordinates of the first vehicle and the second location
coordinates of the second vehicle using a fourth position of the
selected object, wherein the fourth position of the selected object
is retrieved from a database.
18. The device as claimed in claim 17, wherein the selected object
is a static landmark or a traffic sign.
19. The device as claimed in claim 18, wherein the location
coordinates are determined using raw GPS.
20. The device as claimed in claim 19, wherein the position is
specified in WGS 94 coordinates.
Description
BACKGROUND
[0001] The invention relates to a method and a device for
localization, in particular for increasing the accuracy of
localization, as well as a usage, a program element and a
computer-readable medium.
[0002] In many cases, navigation systems are used to determine
position. These are available either as individual devices
(so-called "handhelds") or they are installed in a vehicle. In many
cases, navigation systems use information sent by satellites. A
family of navigation systems uses the so-called "GPS" system (GPS:
Global Positioning System). It should be pointed out that, within
the context of the present invention, GPS is representative of all
global navigation satellite systems (GNSS), such as e.g. GPS,
Galileo, GLONASS (Russia), Compass (China), IRNSS (India), Beidou
or others.
[0003] For various reasons the information received from the
satellites is in many cases inaccurate. This inaccuracy leads in
many cases, under exclusive use of the signals from the satellites,
to a deviation of several meters in the position determined by the
navigation systems from the actual position of an object and/or its
own position. This accuracy is inadequate for a range of
applications.
BRIEF SUMMARY
[0004] The object of the present invention is to improve the
accuracy of localization, at least in part.
[0005] Said object is achieved by means of the subject matter of
the independent patent claims. Further refinements of the invention
will emerge from the dependent claims and the following
description.
[0006] A method for increasing the accuracy of localization
comprises the following steps:
[0007] a) detecting a selected object in an environment, using a
camera of a first vehicle.
[0008] An environment is understood to mean a real environment, in
which objects are arranged and in which, for example, persons or
vehicles can move freely. Certain objects can be selected. These
selected objects can be, e.g., static landmarks, such as churches,
towers, castles, bridges (e.g. for boats), or other topographical
objects. Selected objects can also be, for example, traffic signs
or traffic islands.
[0009] The recording of a selected object is carried out using a
camera. For this, for example, mono, stereo or surround-view
cameras can be used. The camera or cameras can be positioned on the
vehicle. The vehicle is, for example, a motor vehicle, such as an
automobile, a bus or a truck, or else a rail vehicle, a ship, an
aircraft, such as a helicopter or an airplane, or for example a
bicycle. In one embodiment it can also be a pedestrian or, in
another embodiment, a stationary support for a camera.
[0010] b) Estimating a first position of the selected object, by
means of a first program for localization, in the first vehicle.
The first program uses first location coordinates of the first
vehicle and a first distance estimate to the selected object.
[0011] The positions are described in particular in WGS-94
coordinates (WGS-94: World Geodetic System 1894). The first
position in this step is estimated by means of a first program for
localization. The first program can be an existing program, which
in some vehicles--for different uses--may already be available as
standard in the vehicle, or else it can be retrofitted and/or
adjusted on a regular basis. The program can consist of a
vehicle-specific algorithm. To estimate the first position of the
selected object the first program determines, for example in an
intermediate step, at least first location coordinates of the first
vehicle and a first distance estimate. The first location
coordinates may have been calculated, for example, using standard
GPS, e.g. if information from the GPS system is used. These
location coordinates can be affected by serious inaccuracy, so that
for many purposes they are not usable directly. The distance
estimate to the selected object can be carried out by the first
program, for example by means of triangulation and using a stereo
camera. The calculation by means of the first program can be
carried out in a computing device in the vehicle, but also on a
server that receives the data from the first vehicle. The location
coordinates refer to a defined point on or in the first vehicle,
e.g. to the wing mirror on the driver's side.
[0012] Therefore, after this step at least the following items of
information are available: [0013] One or more attributes of the
selected object, e.g., "STOP sign in the XY-road" or "eastern side
of the middle bridge pillar". [0014] Location coordinates of the
first vehicle, such as the "Raw GPS" data of the first vehicle at
the time of measurement. [0015] A first distance estimate between
the first vehicle and the selected object. Either two- or
three-dimensional data are provided. These data can be, for
example, Cartesian or polar coordinates.
[0016] c) Transferring an attribute of the selected object, the
first location coordinates and the first distance estimate to a
server.
[0017] The transfer of these data can be carried out via wireless
means, but they can also, for example, be read out from a memory of
the vehicle via a service interface. Suitable technologies for the
transfer are direct communication links, such as WLAN (e.g. WLAN
802.11a/b/g/n or WLAN 802.11p), ZigBee or WiMax, or cellular radio
systems such as GPRS, UMTS or LTE. It is also possible to use other
transmission protocols. The specified protocols provide the
advantage of the standardization which has already taken place.
[0018] d) Carrying out the steps a) to c) for the selected object,
to estimate a second position of the selected object, using a
camera of a second vehicle, by means of a second program for
localization, using second location coordinates and a second
distance estimate.
[0019] The steps a) to c) are also carried out for the selected
object, mutatis mutandis, by a second vehicle. The second vehicle
can be, for example, a different second vehicle, it may be the
first vehicle that is newly located at a specific position, from
which the camera can capture the selected object; the second
vehicle can also be a plurality of vehicles, which capture the
selected object by means of a camera. In one embodiment, the camera
can be arranged to be stationary at a defined distance from the
selected object. The second program for localization can be
identical to the first program, or else it can be a completely
different program, which supplies the same kind of results as the
first program. After the step d) therefore, the data of an
attribute of the selected object, the second location coordinates
and the second distance estimate are available, and these are
transferred to the server.
[0020] e) Calculating a third position of the selected object by
the server, by means of a third program, using an attribute of the
selected object, the first location coordinates and the first
distance estimate, the second location coordinates and the second
distance estimate. The third program also performs an averaging of
the plurality of location coordinates and distance estimates.
[0021] The averaging of the plurality of location coordinates can
use, for example, the arithmetic mean or the weighted arithmetic
mean. In many cases the first program and the second program are
not very time-consuming or computationally intensive. This may be
attributable to the fact that these programs must at least satisfy
soft real-time requirements. This can be attributable to the fact
that in many cases the computing capacity in vehicles, e.g. due to
energy or cost reasons, is limited. The third program, on the other
hand, can be significantly time- and/or computation-intensive,
because for this program it is not so important that its results
are delivered quickly; instead, a high accuracy of the results is
sought. The result of the third program includes at least the third
position of the selected object.
[0022] f) Using the third position of the selected object as a
reference position for increasing the accuracy of a localization of
a third vehicle.
[0023] If the third position is available, then this can be used as
a reference position by the third vehicle (or else e.g. by handheld
devices) to increase the accuracy of localization. This is an
advantageous feature, in particular because using the method
described above a larger number of reference positions are
available, so that a correction of the localization--for example,
on the basis of "raw GPS" data--can be performed in a relatively
fine-grained manner, so that at any point of an area prepared in
such a way the deviation is not very great. The reference positions
also do not need to be defined and marked on site involving a high
degree of effort, but can be, for example, determined or calculated
at a central location, by using the third program. Furthermore, the
number of the reference positions can be continuously extended, so
that the correction can become more fine-grained over time. In
particular because of the small amount of preparation effort and
because the computing power is not time-critical--and therefore can
be distributed over a longer time span--this method is also quite
cost-effective.
[0024] In one embodiment, step c*) is inserted after step c), with:
c*) Replacement, on the server, of the first location coordinates
and the second location coordinates, by means of a fourth program,
using corrected first location coordinates and corrected second
location coordinates. The fourth program can use, for example, raw
GPS data, so as to determine more accurate data for the corrected
first and corrected second location coordinates. In this embodiment
the steps a) to c*) are carried out in step d).
[0025] In one embodiment, step e) is replaced by e*), with: e*)
Calculation of a third position of the selected object, wherein the
third program calculates a centre of gravity of the plurality of
location coordinates and distance estimates.
[0026] The centre of gravity of the plurality of location
coordinates and distance estimates is thus the center of a point
cloud, containing either the total number of location coordinates
and distance estimates or the total number of location coordinates
and the total number of distance estimates.
[0027] In one embodiment, step f) is replaced by f*), with f*)
correction of the first location coordinates of the first vehicle
and the second location coordinates of the second vehicle using a
fourth position of the selected object. In doing so, the fourth
position of the selected object is retrieved from a database.
[0028] If the fourth position of the selected object is available
with high accuracy, i.e., for example, it can be obtained from a
database, then the first location coordinates of the first vehicle
and the second location coordinates of the second vehicle can be
corrected in such a way that they form, for example, the difference
between the position of the selected object calculated in the
vehicles and the fourth position of the selected object. This
difference can be calculated at a point where the camera captures
the selected object.
[0029] In one embodiment the selected object is, for example, a
static landmark, such as a church, a tower, a castle, a bridge
(e.g. for boats), or another topographical object. In one
embodiment the selected object is a traffic sign or a traffic
island.
[0030] These selected objects can also be exchanged between
different mapping and/or navigation systems.
[0031] In one embodiment, the location coordinates are determined
using raw GPS. In particular, this can take place in an environment
in which GPS is particularly suitable for use in navigation
systems.
[0032] In one embodiment, the position is specified in WGS-94
coordinates. This is based on the "World Geodetic System 1894"
platform.
[0033] The invention also comprises a device for increasing the
accuracy of localization in a vehicle, which comprises a
computation and control device which is configured to carry out one
of the methods described above.
[0034] The invention also comprises a program element, which, if it
is executed on a computation and control device, is suitable for
carrying out the method described above.
[0035] The invention further comprises a computer-readable medium
on which the program element is stored.
[0036] The invention comprises the use of a device or a method as
described above for increasing the accuracy of localization in a
navigation system, in particular in a vehicle.
BRIEF DESCRIPTION OF THE DRAWINGS
[0037] For further clarification, the invention will now be
described on the basis of an embodiments shown in the figures.
These embodiments are to be understood merely as examples and not
as limitations.
[0038] The figures show:
[0039] FIG. 1: an example scenario, in which a vehicle detects a
selected object;
[0040] FIG. 2: a method for increasing the accuracy of
localization.
DETAILED DESCRIPTION
[0041] FIG. 1 shows a vehicle 31, 32, which detects a selected
object 15 in an environment 10. The vehicle can be a first vehicle
31 or a second vehicle 32. The vehicle has two cameras: 21 and 21'
are arranged on the first vehicle, 22 and 22' on the second
vehicle. These can be configured as a stereo camera. Each of the
vehicles 31, 32 detects the selected object 15, using its own
location coordinates 61 or 62. The reference point can be, for
example, the left-hand wing mirror or else the camera 21' or 22'.
Further, the first vehicle 31 performs a triangulation. In doing so
the location 71' determined by the camera 21', is coordinated with
the location 71'' determined by the camera 21, and, by means of a
first program, a first distance estimate 71 is calculated. The
second vehicle 32 calculates a second distance estimate 72, by
means of the cameras 22 and 22' and a second program. In one
embodiment the first and the second program can be the same
program. The co-ordination of GPS and cameras takes place via the
computation and control device 80, which is connected via the
interfaces 86 and 82 to the GPS navigation system and the cameras
(for the sake of simplicity, only one symbol and one reference sign
is used, which are intended to apply to each vehicle). The
computation device 80 can also execute the first or the second
program.
[0042] Each of the vehicles 31, 32 (and, in reality, also a
plurality of other vehicles) transmits these data, together with an
attribute of the selected object 15, to the server 90, over the
wireless interfaces 89 and 99. The server 90 can be arranged in a
cloud 95. The server 90 calculates, on this basis, a third position
53 of the selected object 15, which it stores in the database 92.
In one embodiment a fourth position 54 is already stored in the
database 92. Other vehicles can then use the position 53 or 54 as a
reference position for increasing the accuracy of their determined
location. In one embodiment, the positions 53 or 54 are reviewed
continuously or at selectable intervals, so that changes--for
example, removal or movement of the selected object 15--can be
dynamically taken into account.
[0043] FIG. 2 illustrates a method 100 for increasing the accuracy
of localization. In Step 101 a selected object 15 is detected by
means of a camera 21 of a first vehicle 31. In step 102, a first
position 51 of the selected object 15 is calculated, by means of
first location coordinates 61 and a first distance estimate 71,
using a first program. In step 103, the first location coordinates
61 and the first distance estimate 71 are transmitted or sent to
the server 90.
[0044] In steps 101', 102' and 103', similar actions to those in
steps 101, 102 and 103 are carried out, but for a second vehicle 32
or for a plurality of vehicles 32. In step 104, the location
coordinates 61, 62 and distance estimates 71, 72 in the server 90
are merged and the result is used to calculate the third position
53 of the selected object 15. In step 104, the third position 53 is
used as a reference position. This can be carried out by any of the
vehicles which use a list of the selected objects 15 and the method
described here.
LIST OF REFERENCE NUMERALS
[0045] 10 environment
[0046] 15 selected object
[0047] 21, 22 camera
[0048] 31, 32 first and second vehicle
[0049] 61, 62 first and second location coordinates
[0050] 61', 62' corrected first and second location coordinates
[0051] 71, 72 first and second distance estimate
[0052] 71', 71'', 72', 72'' determined locations
[0053] 80 computation and control device
[0054] 82 interface to the camera
[0055] 86 interface to the position device
[0056] 89 wireless interface of the vehicle
[0057] 90 server
[0058] 92 database
[0059] 95 cloud
[0060] 99 wireless interface of the server
[0061] 100 method
[0062] 101 . . . 105 steps
* * * * *