U.S. patent number 7,372,030 [Application Number 11/178,735] was granted by the patent office on 2008-05-13 for method for representing a front field of vision from a motor vehicle.
This patent grant is currently assigned to Siemens Aktiengesellschaft. Invention is credited to Hubert Adamietz, Manfred Hahl.
United States Patent |
7,372,030 |
Adamietz , et al. |
May 13, 2008 |
Method for representing a front field of vision from a motor
vehicle
Abstract
A method for representing the front field of vision from a motor
vehicle on a display panel of a display includes sensing, by an
infrared camera system, the field of vision and feeding
corresponding infrared data to a data processing unit. The infrared
data is modified in the data processing unit on the basis of
further information which is fed to the data processing unit and
supplied as display data for actuating the display. The infrared
camera system is a near-infrared camera system which senses the
light from an infrared light source which is reflected from the
field of vision and supplies it as corresponding near-infrared data
to the data processing unit. A further infrared camera system is a
far-infrared camera system which senses the thermal radiation from
the field of vision which is fed as corresponding far-infrared data
to the data processing unit. Furthermore, the physical variables
from the field of vision which are sensed by one or more sensors
are fed as corresponding sensor data to the data processing
unit.
Inventors: |
Adamietz; Hubert (Kleinostheim,
DE), Hahl; Manfred (Muhlheim, DE) |
Assignee: |
Siemens Aktiengesellschaft
(Munchen, DE)
|
Family
ID: |
35540334 |
Appl.
No.: |
11/178,735 |
Filed: |
July 11, 2005 |
Prior Publication Data
|
|
|
|
Document
Identifier |
Publication Date |
|
US 20060006331 A1 |
Jan 12, 2006 |
|
Foreign Application Priority Data
|
|
|
|
|
Jul 12, 2004 [DE] |
|
|
10 2004 033 625 |
|
Current U.S.
Class: |
250/330;
348/E5.09 |
Current CPC
Class: |
B60R
1/00 (20130101); G02B 27/01 (20130101); G06T
5/50 (20130101); B60R 2300/105 (20130101); B60R
2300/106 (20130101); B60R 2300/205 (20130101); B60R
2300/30 (20130101); B60R 2300/304 (20130101); B60R
2300/8053 (20130101); G02B 2027/0138 (20130101); G02B
2027/014 (20130101); G06T 2207/10048 (20130101); G06T
2207/20221 (20130101); G06T 2207/30252 (20130101); H04N
5/33 (20130101) |
Current International
Class: |
G02F
1/01 (20060101) |
Field of
Search: |
;250/330 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
40 32 927 |
|
Apr 1992 |
|
DE |
|
100 16 184 |
|
Oct 2001 |
|
DE |
|
102 18 175 |
|
Nov 2003 |
|
DE |
|
WO 02/36389 |
|
May 2002 |
|
WO |
|
WO 2004/047449 |
|
Jun 2004 |
|
WO |
|
Other References
German Office Action dated Jan. 21, 2005. cited by other .
2006Q235474DE ITWissen Das gro.beta.e Online-Lexikon fur
Informationstechnologie; Oct. 16, 2006,
http://www.itwissen.info/index.php?id=31&aoid=9514. cited by
other .
Search Report dated Oct. 16, 2006 issued for the corresponding
German Application No. 10 2004 033 625.3-31. cited by
other.
|
Primary Examiner: Porta; David
Assistant Examiner: Vu; Mindy
Attorney, Agent or Firm: Cohen Pontani Lieberman &
Pavane LLP
Claims
What is claimed is:
1. A method for representing the front field of vision from a motor
vehicle on a display panel of a display, comprising the steps of:
sensing, by an infrared camera system, the field of vision and
feeding corresponding infrared data to a data processing unit;
modifying the sensed infrared data in the data processing unit on
the basis of further information fed to the data processing unit;
supplying the modified infrared data as display data for actuating
the display, wherein the infrared camera system includes a
near-infrared camera system and a far infrared camera system, the
near-infrared camera system senses the light from an infrared light
source which is reflected from the field of vision and supplies
corresponding near-infrared data to the data processing unit, the
far-infrared camera system senses the thermal radiation from the
field of vision and supplies corresponding far-infrared data to the
data processing unit, and physical variables from the field of
vision are sensed by at least one sensor and fed as corresponding
sensor data to the data processing unit; and restricting the field
of vision in the data processing unit when processing the
near-infrared data and identifying objects in the restricted field
of vision when processing the far-infrared data.
2. The method of claim 1, further comprising the step of modifying,
in the data processing unit, the near-infrared data by
superimposing the near-infrared data with the far-infrared data to
form display data.
3. The method of claim 1, further comprising the step of sensing
physical variables from the field of vision by the at least one
sensor and feeding the corresponding sensor data to the data
processing unit, wherein the at least one sensor include at least
one of a radar sensor, an ultrasonic sensor, and ultraviolet
sensor.
4. The method of claim 1, further comprising the step of feeding at
least one of the near-infrared data, the far-infrared data, and the
sensor data through an optimization stage in the data processing
unit.
5. The method of claim 4, further including the step of passing the
at least one of the near-infrared data, the far-infrared data, and
the sensor data through a noise filter in the optimization
stage.
6. The method of claim 5, further including the step of passing the
at least one of the near-infrared data, the far-infrared data, and
the sensor data through an edge filter in the optimization
stage.
7. The method of claim 6, further including the step of passing the
at least one of the near-infrared data, the far-infrared data, and
the sensor data through a contrast improving stage in the
optimization stage.
8. The method of claim 5, further including the step of passing the
at least one of the near-infrared data, the far-infrared data, and
the sensor data through a contrast improving stage in the
optimization stage.
9. The method of claim 4, further including the step of passing the
at least one of the near-infrared data, the far-infrared data, and
the sensor data through a contrast improving stage in the
optimization stage.
10. The method of claim 4, further including the step of passing
the at least one of the near-infrared data, the far-infrared data,
and the sensor data through an edge filter in the optimization
stage.
11. The method of claim 1, further comprising the step of
extracting areas of the image information represented by the data
in the data processing unit when processing at least one of the
near-infrared data, the far-infrared data, and the sensor data.
12. The method as claimed in claim 1, further comprising the step
of superimposing a virtual lattice on the display data defined by
the near-infrared data.
13. The method as claimed in claim 12, wherein one side of the
lattice corresponds to the boundary of the roadway.
14. The method of claim 13, further comprising the step of
generating one of a visual and audible warning signal by the data
processing unit when an object is identified in the restricted
field of vision.
15. The method of claim 12, further comprising the step of
generating one of a visual and audible warning signal by the data
processing unit when an object is identified in the restricted
field of vision.
16. The method of claim 1, further comprising the step of
generating one of a visual and audible warning signal by the data
processing unit when an object is identified in the restricted
field of vision.
17. The method of claim 1, further comprising the step of adding at
least a portion of the near-infrared data to the far-infrared data
by the data processing unit to form complete display data items of
an object.
Description
BACKGROUND OF THE INVENTION
The invention relates to a method for representing a field of
vision, in particular, a front field of vision from a motor vehicle
on a display panel of a display, the method including sensing with
an infrared camera system the field of vision and feeding
corresponding infrared data to a data processing unit, modifying
infrared data in the data processing unit on the basis of further
information fed to the data processing unit, and supplying the
infrared data as display data for actuating the display.
In such a method for providing night sight assistance it is known
to correct the image recorded by a near-infrared camera. To do
this, the intensity of the pixel of the display is increased or
reduced.
SUMMARY OF THE INVENTION
An object of the invention is to provide a method for representing,
in particular, a front field of vision from a motor vehicle on a
display panel such that objects in the scenery in the field of
vision from the motor vehicle are highly recognizable even at night
or in poor visibility conditions.
This object is achieved according to the invention in that the
infrared camera system is a near-infrared camera system which
senses the light from an infrared light source which is reflected
from the field of vision and supplies it as corresponding
near-infrared data to the data processing unit, and/or in that the
infrared camera system is a far-infrared camera system which senses
the thermal radiation from the field of vision which is fed as
corresponding far-infrared data to the data processing unit, and/or
in that the physical variables from the field of vision which are
sensed by one or more sensors are fed as corresponding sensor data
to the data processing unit.
The reflected light of the infrared light source is sensed by the
near-infrared camera and the corresponding near-infrared data is
processed in order to present images. This alone gives rise to a
familiar form of representation which does not require any
familiarization time for the viewer of the display since it
corresponds approximately to the black/white representation of a
normal driving situation at night using headlights.
However, in this way it is not possible to sense any heat-emitting
objects such as humans or animals located behind an object, for
example a bush, sensed by the near-infrared camera system. Such
heat-emitting objects are sensed by the far-infrared camera system
which picks up the thermal radiation in the field of vision. The
presentation in accordance with the far-infrared data is similar to
a photonegative image and is therefore not suitable for
representation on a display. An observer requires a relatively long
familiarization phase to be able to identify relevant objects.
However, the recording of thermal images is very homogeneous and
facilitates the improvement of long-range vision.
The modification of the near-infrared data by the far-infrared data
gives rise to a combination of the spectral ranges of these two
systems and thus to an improved direct representation and to a more
efficient evaluation of images and identification of objects.
The sensor data may also be processed either for further refinement
of the representation or as a replacement for the near-infrared
data or the far-infrared data.
In the data processing unit, the near-infrared data is preferably
modified by the far-infrared data by superimposition to form
display data.
The sensors may be radar sensors and/or ultrasonic sensors and/or
ultraviolet sensors. More distant objects may be satisfactorily
sensed by radar sensors and closer objects may be sensed by
ultrasonic sensors. The advantage of ultraviolet sensors is that
ultraviolet signals may also pass satisfactorily through rain and
supply signals.
To match the near-infrared data and/or the far-infrared data and/or
the sensor data and to optimize such data in the data processing
unit, the data may be supplied to an optimization stage such as,
for example, a noise filter. Especially near-infrared data may
include a high level of information noise of the near-infrared data
which results from the large quantity of sensed information and
gives rise to representations which are onerous for the viewer. If
the near-infrared data and/or the far-infrared data and/or the
sensor data is passed through a noise filter, it is possible, for
example, to filter the high level of information noise.
The optimization stage may additionally or alternatively have an
edge filter through which near-infrared data and/or the
far-infrared data and/or the sensor data is passed.
To improve the representation, the optimization stage may
additionally or alternatively have a contrast improving stage
through which near-infrared data and/or the far-infrared data
and/or the sensor data are passed.
If areas of the image information represented by the data are
extracted in the data processing unit when processing the
near-infrared data and/or the far-infrared data and/or the sensor
data, the data of just one area of the overall image may be
combined with the data of the complete overall image by the various
systems. As a result, a combination may be made of, for example,
the data of the complete image for a system which essentially
contains the road in front of the motor vehicle and only the data
of the area of the lane by another system.
To produce an intensive representation in a specific part of the
field of vision and a reduced representation outside the specific
part of the field of vision, it is possible to restrict the field
of vision in the data processing unit when processing the
near-infrared data and to identify objects in the restricted field
of vision when processing the far-infrared data.
For this purpose, a virtual lattice can be superimposed on the
display data defined by the near-infrared data.
According to a preferred embodiment, one side of the lattice
preferably corresponds to a boundary of the roadway.
When an object is identified in the restricted field of vision by
the data processing unit it is possible to generate a visual and/or
audible warning signal either automatically or in a manually
selectable fashion.
In the data processing unit, the near-infrared data may be added to
the far-infrared data to form complete display data items of an
object.
A pedestrian wearing clothes which have good thermal insulation is
recognizable in a representation of the far-infrared data only from
his head and hands. The remainder of the pedestrian would be
difficult or even impossible to recognize and display using
far-infrared data since the clothing prevents thermal emissions. By
adding the near-infrared data to the far-infrared data the body of
the pedestrian in the direct vicinity of the head and the hands may
be completed so that from this compiled information the image
processing system can quickly identify a pedestrian.
The display panel of the display may be arranged, for example, as a
screen in the motor vehicle.
Alternatively, the representation may be included in a display of a
head-up display device. In such a case, the viewer has the direct
possibility of comparing the representation with the original
image.
Other objects and features of the present invention will become
apparent from the following detailed description considered in
conjunction with the accompanying drawings. It is to be understood,
however, that the drawings are designed solely for purposes of
illustration and not as a definition of the limits of the
invention, for which reference should be made to the appended
claims. It should be further understood that the drawings are not
necessarily drawn to scale and that, unless otherwise indicated,
they are merely intended to conceptually illustrate the structures
and procedures described herein.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings, wherein like reference characters denote similar
items out the several views:
FIG. 1 is a block circuit diagram of a device for carrying out the
method ng to the present invention;
FIG. 2 is a first flowchart of the method according the present
invention;
FIG. 3 is a second flowchart of the method.
DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS
The device illustrated in FIG. 1 has a near-infrared camera system
1, a far-infrared camera system 2 and a sensor system 3,
collectively referred to as the systems 1, 2, 3. Near-infrared
data, far-infrared data and sensor data are fed to a controller 4
by the systems 1, 2, 3 for processing this data. The controller 4
may also be referred to as a data processing unit.
Display data for representation on a display panel of a display are
then fed via the output of the controller 4. The display may
comprise a display device 5 for displaying the field of vision of
the driver on the information panel of a motor vehicle. The display
may alternatively or additionally include a display device of a
head-up display means 6 and/or a display device 7 in the vicinity
of a front seat passenger.
According to the flowchart in FIG. 2, the data that is generated in
the near-infrared camera system 1, the far-infrared camera system 2
and the sensor system 3 is conditioned in conditioning step 8 and
read in by the near-infrared camera system 1 in step 9. The data is
then fed to an optimization stage 10.
The data is passed through a noise filter, an edge filter and/or a
contrast improving stage in the optimization stage 10.
Manually selected information which is to be used for modifying the
data supplied by the system may be fed from a user input means 12
to the output of the optimization stage 10.
In step 11 it is determined whether the near-infrared data and
far-infrared data is to be superimposed.
If a superimposition is to be carried out, the image areas of the
two camera systems are adapted to one another, and if appropriate
also restricted, in a step 13.
In step 14, the data is superimposed. Display data is generated in
step 15 and a display panel of a display is actuated by the
generated display data.
If the far-infrared data is determined not to be superimposed on
the near-infrared data in step 11, the sensor data is selected,
step 16, in accordance with the prescribed values of the user unit
12 which are then conditioned as display data and passed on for the
actuation of the display panel of the display in step 15.
Using the user input means 12 it is possible to select the
combination of the systems 1, 2, 3 which supply the best data for
the respective situation.
FIG. 3 shows a flow chart having a branch A in which the
near-infrared data which is generated by a near-infrared camera
system 1' and is fed to an optimization stage 10' and conditioned
therein in an optimized way. The optimized near-infrared data is
then fed to a feature extraction stage 17 in which the data for
sensed objects is extracted and passed on to a combination stage
18.
According to the same procedure, in a branch B the far-infrared
data is generated by a far-infrared camera system 2' and is fed to
the combination stage 18 via an optimization stage 10'' and a
feature extraction stage 17'.
The procedure in a branch C also corresponds to this. In branch C,
the sensor data generated by a sensor system 3' is fed to the
combination stage 18 via an optimization stage 10''' and a feature
extraction stage 17''.
In the combination stage 18, display data which is further
processed in an object detection stage 19 is generated from the
data from all, or from some, of the branches A, B and C, for
example by superimposition. If an object which presents a danger is
recognized in this process, a visual or audible warning is issued
in a warning stage 20.
Thus, while there have shown and described and pointed out
fundamental novel features of the invention as applied to a
preferred embodiment thereof, it will be understood that various
omissions and substitutions and changes in the form and details of
the devices illustrated, and in their operation, may be made by
those skilled in the art without departing from the spirit of the
invention. For example, it is expressly intended that all
combinations of those elements and/or method steps which perform
substantially the same function in substantially the same way to
achieve the same results are within the scope of the invention.
Moreover, it should be recognized that structures and/or elements
and/or method steps shown and/or described in connection with any
disclosed form or embodiment of the invention may be incorporated
in any other disclosed or described or suggested form or embodiment
as a general matter of design choice. It is the intention,
therefore, to be limited only as indicated by the scope of the
claims appended hereto.
* * * * *
References