U.S. patent application number 13/726281 was filed with the patent office on 2013-05-02 for driver assistance system for a vehicle.
This patent application is currently assigned to DONNELLY CORPORATION. The applicant listed for this patent is Donnelly Corporation. Invention is credited to Mark L. Larson, Kenneth Schofield, Keith J. Vadas.
Application Number | 20130106993 13/726281 |
Document ID | / |
Family ID | 23769265 |
Filed Date | 2013-05-02 |
United States Patent
Application |
20130106993 |
Kind Code |
A1 |
Schofield; Kenneth ; et
al. |
May 2, 2013 |
DRIVER ASSISTANCE SYSTEM FOR A VEHICLE
Abstract
A driver assistance system suitable for use in a vehicle
includes an image capture device having a field of view rearward of
the vehicle. The image capture device comprises a CMOS image
capture device and is operable to capture image data. A display
system displays video images output by the image capture device.
The display system includes a video display screen. A graphic
overlay is superimposed on the displayed images, and, at least when
the wheels of the vehicle are straight, the graphic overlay
includes a pair of converging lines extending outward relative to
the rear of the vehicle. At least one horizontal line at least
partially extends between the pair of converging lines in order to
enhance the ability of a driver of the vehicle to judge relative
distance to an object located rearward of the vehicle.
Inventors: |
Schofield; Kenneth;
(Holland, MI) ; Larson; Mark L.; (Grand Haven,
MI) ; Vadas; Keith J.; (Coopersville, MI) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Donnelly Corporation; |
Holland |
MI |
US |
|
|
Assignee: |
DONNELLY CORPORATION
Holland
MI
|
Family ID: |
23769265 |
Appl. No.: |
13/726281 |
Filed: |
December 24, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
11626529 |
Jan 24, 2007 |
|
|
|
13726281 |
|
|
|
|
10643602 |
Aug 19, 2003 |
7859565 |
|
|
11626529 |
|
|
|
|
09776625 |
Feb 5, 2001 |
6611202 |
|
|
10643602 |
|
|
|
|
09313139 |
May 17, 1999 |
6222447 |
|
|
09776625 |
|
|
|
|
08935336 |
Sep 22, 1997 |
5949331 |
|
|
09313139 |
|
|
|
|
08445527 |
May 22, 1995 |
5670935 |
|
|
08935336 |
|
|
|
|
Current U.S.
Class: |
348/36 ;
348/148 |
Current CPC
Class: |
B60N 2/002 20130101;
B60R 11/04 20130101; B60Q 2300/056 20130101; B60R 2300/105
20130101; B60R 2300/205 20130101; B60R 2300/30 20130101; B60Q
2300/42 20130101; B60R 2300/303 20130101; B60R 2300/304 20130101;
B60R 21/01538 20141001; B60R 2300/107 20130101; B60R 2300/8066
20130101; B60R 2001/1253 20130101; B60Q 2300/3321 20130101; B60Q
2300/41 20130101; B60R 2300/50 20130101; B60R 1/12 20130101; B60R
2300/101 20130101; B60R 2300/103 20130101; B60R 2300/207 20130101;
B60S 1/0844 20130101; H04N 7/18 20130101; B60R 2300/302 20130101;
B60Q 9/008 20130101; B60R 2300/106 20130101; B60R 2300/802
20130101; B60R 1/088 20130101; B60S 1/0885 20130101; B60R 1/00
20130101; B60S 1/0822 20130101; B60Q 1/1423 20130101; B60R
2300/8053 20130101; H04N 13/239 20180501; G01J 1/4228 20130101;
B60R 2300/8026 20130101; B60R 2300/305 20130101; B60R 2300/804
20130101; B60R 2300/8086 20130101; G01J 1/4204 20130101; B60R
2300/307 20130101; B60R 2300/8093 20130101; G08G 1/167 20130101;
H04N 7/181 20130101; H04N 2013/0081 20130101; B60R 2001/1223
20130101; B60Q 2300/054 20130101; B60Q 2300/21 20130101; B60Q
2300/314 20130101; B60Q 1/14 20130101; B60R 2300/806 20130101; B60R
2300/404 20130101 |
Class at
Publication: |
348/36 ;
348/148 |
International
Class: |
H04N 7/18 20060101
H04N007/18 |
Claims
1. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said image capture device
comprises an array of photosensing elements operable to capture
image data; wherein said image capture device comprises a CMOS
image capture device; wherein said image capture device comprises a
wide angle optic; wherein at least one of (i) said wide angle optic
comprises optical distortion correction and (ii) distortion in
captured image data is reduced using electronic distortion
correction; a display system that displays video images output by
said image capture device; wherein said display system comprises a
back-lit liquid crystal video display screen; wherein a graphic
overlay is superimposed on the displayed images, and wherein, at
least when the wheels of the vehicle are straight, said graphic
overlay comprises a pair of converging lines extending outward
relative to the rear of the vehicle; and wherein at least one
horizontal line at least partially extends between said pair of
converging lines in order to enhance the ability of a driver of the
vehicle to judge relative distance to an object located rearward of
the vehicle.
2. The driver assistance system of claim 1, wherein said pair of
converging lines extend from adjacent the rear of the vehicle.
3. The driver assistance system of claim 1, wherein said pair of
converging lines comprise a pair of substantially straight
converging lines.
4. The driver assistance system of claim 1, wherein said graphic
overlay adjusts responsive to a steering system of the vehicle.
5. The driver assistance system of claim 4, wherein, when the
wheels of the vehicle are turned in a first steering direction,
said graphic overlay curves in a first direction indicative of the
anticipated rearward travel of the vehicle and, when the wheels of
the vehicle are turned in a second steering direction opposite to
said first steering direction, said graphic overlay curves in a
second direction opposite to said first direction.
6. The driver assistance system of claim 1, wherein said graphic
overlay is indicative of the anticipated rearward travel of the
vehicle.
7. The driver assistance system of claim 1, wherein said graphic
overlay enhances the driver's sense of perspective when executing a
reversing maneuver.
8. The driver assistance system of claim 1, wherein said graphic
overlay enhances the driver's judgment of the location of an object
at least one of (i) behind the vehicle and (ii) alongside the
vehicle.
9. The driver assistance system of claim 1, wherein said graphic
overlay assists the driver when reversing into a parking space.
10. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen displays images that are
viewable by the driver of the vehicle at least one of (i) when the
driver is looking forward of the vehicle and (ii) when the driver
is looking generally toward the windshield of the vehicle.
11. The driver assistance system of claim 1, wherein image data
captured by said array of photosensing elements is displayed in
reverse at said back-lit liquid crystal video display screen.
12. The driver assistance system of claim 11, wherein reversal of
image data occurs at said image capture device.
13. The driver assistance system of claim 11, wherein reversal of
image data occurs at said display system.
14. The driver assistance system of claim 1, wherein said at least
one horizontal line comprises a plurality of spaced apart
horizontal lines.
15. The driver assistance system of claim 14, wherein said spaced
apart horizontal lines are superimposed on displayed video images
at predetermined intervals.
16. The driver assistance system of claim 15, wherein said spaced
apart horizontal lines comprise spaced apart short horizontal
lines.
17. The driver assistance system of claim 1, wherein said driver
assistance system includes a distance-sensing system.
18. The driver assistance system of claim 17, wherein said
distance-sensing system is selected from the group consisting of a
radar system, an ultrasonic system and an infrared system.
19. The driver assistance system of claim 1, wherein said image
capture device is disposed generally at the centerline of the
vehicle.
20. The driver assistance system of claim 1, wherein said field of
view of said image capture device is symmetrical about the
longitudinal axis of the vehicle.
21. The driver assistance system of claim 1, wherein said field of
view of said image capture device encompasses a ground area at the
rear of the vehicle.
22. The driver assistance system of claim 1, wherein said field of
view of said image capture device encompasses a ground area at the
immediate rear of the vehicle.
23. The driver assistance system of claim 1, wherein said field of
view of said image capture device encompasses a portion of the rear
of the vehicle.
24. The driver assistance system of claim 1, wherein said wide
angle optic has a wide angle view at least at a lower portion of
said field of view of said image capture device.
25. The driver assistance system of claim 1, wherein video images
derived from captured image data are displayed by said display
system substantially distortion-free.
26. The driver assistance system of claim 1, wherein said wide
angle optic comprises non-symmetrical optics.
27. The driver assistance system of claim 1, wherein said image
capture device is fixedly mounted at the rear of the vehicle.
28. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen comprises a back-lit flat panel
liquid crystal video display screen.
29. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen is positioned in the vehicle
cabin without obstructing the driver's view through the windshield
of the vehicle.
30. The driver assistance system of claim 29, wherein said back-lit
liquid crystal video display screen is positioned at a lower
portion of the driver's forward field of view.
31. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen is mounted at one of a
dashboard of the vehicle, a fascia of the vehicle, a header of the
vehicle and a windshield of the vehicle.
32. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen is mounted at a dashboard of
the vehicle.
33. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen is mounted at a fascia of the
vehicle.
34. The driver assistance system of claim 1, wherein said back-lit
liquid crystal video display screen is mounted at a position
conventionally occupied by an interior rearview mirror of the
vehicle.
35. The driver assistance system of claim 1, wherein video images
output by said image capture device are output as one of (i) analog
data and (ii) digital data.
36. The driver assistance system of claim 1, wherein said image
capture device comprises part of a multi-camera system of the
vehicle and wherein said multi-camera system comprises an image
capture device disposed at the left side of the vehicle and an
image capture device disposed at the right side of the vehicle.
37. The driver assistance system of claim 1, wherein video images
output by said image capture device are conveyed to said display
system via electrically conductive leads.
38. The driver assistance system of claim 1, wherein video images
output by said image capture device are conveyed to said display
system via a fiber optic cable.
39. The driver assistance system of claim 1, wherein said array of
photosensing elements is operable to capture frames of image data
at multiple exposures.
40. The driver assistance system of claim 39, wherein said multiple
exposures, at least in part, highlight dark areas in displayed
images.
41. The driver assistance system of claim 1, wherein said array of
photosensing elements is operable to capture frames of image data
at a rate of up to 30 frames per second.
42. The driver assistance system of claim 1, wherein said array of
photosensing elements comprises at least 160 columns of photosensor
elements and at least 40 rows of photosensor elements.
43. The driver assistance system of claim 1, comprising circuitry
responsive to detection of light by said photosensing elements of
said image capture device.
44. The driver assistance system of claim 1, wherein said array of
photosensing elements and circuitry associated therewith are
constructed on a common semiconductor substrate.
45. The driver assistance system of claim 44, wherein said
circuitry comprises a digital-to-analog converter.
46. The driver assistance system of claim 44, wherein said
circuitry comprises a logic circuit.
47. The driver assistance system of claim 44, wherein said
circuitry comprises a clock.
48. The driver assistance system of claim 44, wherein said
circuitry comprises random-access-memory.
49. The driver assistance system of claim 1, wherein said image
capture device has an extended dynamic range.
50. The driver assistance system of claim 1, wherein said driver
assistance system compensates for at least one of a bright spot and
a dark spot in captured image data.
51. The driver assistance system of claim 1, wherein said driver
assistance system comprises image morphing.
52. The driver assistance system of claim 1, wherein said driver
assistance system comprises warping compensation.
53. The driver assistance system of claim 1, wherein said driver
assistance system comprises anti-blooming to mitigate the effect of
charge leakage from at least one of said photosensing elements to
at least an adjacent one of said photosensing elements.
54. The driver assistance system of claim 1, wherein said graphic
overlay is superimposed on the displayed images responsive to the
driver of the vehicle placing a gear selector device of the vehicle
in a reverse gear position.
55. The driver assistance system of claim 54, wherein said graphic
overlay is not superimposed on the displayed images responsive to
the driver of the vehicle placing the gear selector device in a
position other than in the reverse gear position.
56. The driver assistance system of claim 1, wherein the vehicle
comprises an automobile.
57. The driver assistance system of claim 1, wherein the vehicle
comprises a light truck.
58. The driver assistance system of claim 1, wherein the vehicle
comprises a sport utility vehicle.
59. The driver assistance system of claim 1, wherein said wide
angle optic comprises optical distortion correction and wherein
distortion in captured image data is reduced using electronic
distortion correction.
60. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said image capture device is
fixedly mounted at the rear of the vehicle; wherein said image
capture device comprises an array of photosensing elements operable
to capture image data; wherein said image capture device comprises
a CMOS image capture device; wherein said image capture device
comprises a wide angle optic; a display system that displays video
images output by said image capture device; wherein said display
system comprises a video display screen; wherein image data
captured by said array of photosensing elements is displayed in
reverse at said video display screen; wherein a graphic overlay is
superimposed on the displayed images, and wherein, at least when
the wheels of the vehicle are straight, said graphic overlay
comprises a pair of converging lines extending outward relative to
the rear of the vehicle; and wherein at least one horizontal line
at least partially extends between said pair of converging lines in
order to enhance the ability of a driver of the vehicle to judge
relative distance to an object located rearward of the vehicle.
61. The driver assistance system of claim 60, wherein said video
display screen comprises a back-lit liquid crystal video display
screen.
62. The driver assistance system of claim 60, wherein said wide
angle optic comprises optical distortion correction.
63. The driver assistance system of claim 60, wherein distortion in
captured image data is reduced using electronic distortion
correction.
64. The driver assistance system of claim 60, wherein said graphic
overlay is indicative of the anticipated rearward travel of the
vehicle.
65. The driver assistance system of claim 60, wherein at least one
of (a) said pair of converging lines comprise a pair of
substantially straight converging lines, (b) said graphic overlay
enhances the driver's sense of perspective when executing a
reversing maneuver, (c) said graphic overlay assists the driver
when reversing into a parking space and (d) said graphic overlay
enhances the driver's judgment of the location of an object at
least one of (i) behind the vehicle and (ii) alongside the
vehicle.
66. The driver assistance system of claim 60, wherein said graphic
overlay adjusts responsive to a steering system of the vehicle, and
wherein, when the wheels of the vehicle are turned in a first
steering direction, said graphic overlay curves in a first
direction indicative of the anticipated rearward travel of the
vehicle and, when the wheels of the vehicle are turned in a second
steering direction opposite to said first steering direction, said
graphic overlay curves in a second direction opposite to said first
direction.
67. The driver assistance system of claim 60, wherein said video
display screen displays images that are viewable by the driver of
the vehicle at least one of (i) when the driver is looking forward
of the vehicle and (ii) when the driver is looking generally toward
the windshield of the vehicle.
68. The driver assistance system of claim 60, wherein reversal of
image data occurs at said image capture device.
69. The driver assistance system of claim 60, wherein reversal of
image data occurs at said display system.
70. The driver assistance system of claim 60, wherein said at least
one horizontal line comprises a plurality of spaced apart
horizontal lines, and wherein at least one of (i) said spaced apart
horizontal lines are superimposed on displayed video images at
predetermined intervals, and (ii) said spaced apart horizontal
lines comprise spaced apart short horizontal lines superimposed on
displayed video images at predetermined intervals.
71. The driver assistance system of claim 60, wherein said driver
assistance system includes a distance-sensing system, and wherein
said distance-sensing system is selected from the group consisting
of a radar system, an ultrasonic system and an infrared system.
72. The driver assistance system of claim 60, wherein at least one
of (i) said image capture device is disposed generally at the
centerline of the vehicle, (ii) said field of view of said image
capture device is symmetrical about the longitudinal axis of the
vehicle, (iii) said field of view of said image capture device
encompasses a ground area at the rear of the vehicle and (iv) said
field of view of said image capture device encompasses a ground
area at the immediate rear of the vehicle.
73. The driver assistance system of claim 60, wherein said field of
view of said image capture device encompasses a portion of the rear
of the vehicle.
74. The driver assistance system of claim 60, wherein at least one
of (i) said wide angle optic has a wide angle view at least at a
lower portion of said field of view of said image capture device,
(ii) video images derived from captured image data are displayed by
said display system substantially distortion-free and (iii) said
wide angle optic comprises non-symmetrical optics.
75. The driver assistance system of claim 60, wherein at least one
of (i) said video display screen comprises a flat panel video
display screen, (ii) said video display screen is positioned in the
vehicle cabin without obstructing the driver's view through the
windshield of the vehicle, (iii) said video display screen is
positioned in the vehicle cabin without obstructing the driver's
view through the windshield of the vehicle and said video display
screen is positioned at a lower portion of the driver's forward
field of view, and (iv) said video display screen is mounted at a
position conventionally occupied by an interior rearview mirror of
the vehicle.
76. The driver assistance system of claim 60, wherein at least one
of (a) video images output by said image capture device are
conveyed to said display system via electrically conductive leads,
(b) video images output by said image capture device are conveyed
to said display system via a fiber optic cable, and (c) video
images output by said image capture device are output as one of (i)
analog data and (ii) digital data.
77. The driver assistance system of claim 60, wherein at least one
of (i) said array of photosensing elements is operable to capture
frames of image data at multiple exposures, (ii) said array of
photosensing elements is operable to capture frames of image data
at multiple exposures and wherein said multiple exposures, at least
in part, highlight dark areas in displayed images, and (iii) said
array of photosensing elements is operable to capture frames of
image data at a rate of up to 30 frames per second.
78. The driver assistance system of claim 60, wherein said array of
photosensing elements comprises at least 160 columns of photosensor
elements and at least 40 rows of photosensor elements.
79. The driver assistance system of claim 60, wherein said array of
photosensing elements and circuitry associated therewith are
constructed on a common semiconductor substrate, and wherein said
circuitry comprises random-access-memory.
80. The driver assistance system of claim 60, wherein said image
capture device has an extended dynamic range.
81. The driver assistance system of claim 60, wherein said graphic
overlay is superimposed on the displayed images responsive to the
driver of the vehicle placing a gear selector device of the vehicle
in a reverse gear position.
82. The driver assistance system of claim 81, wherein said graphic
overlay is not superimposed on the displayed images responsive to
the driver of the vehicle placing the gear selector device in a
position other than in the reverse gear position.
83. The driver assistance system of claim 60, wherein at least one
of (i) the vehicle comprises an automobile, (ii) the vehicle
comprises a light truck and (iii) the vehicle comprises a sport
utility vehicle.
84. The driver assistance system of claim 60, wherein said video
display screen is mounted at one of a dashboard of the vehicle and
a fascia of the vehicle.
85. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said image capture device
comprises an array of photosensing elements operable to capture
image data; wherein said image capture device comprises a CMOS
image capture device; wherein said image capture device comprises a
wide angle optic; a display system that displays video images
output by said image capture device; wherein said display system
comprises a video display screen; wherein image data captured by
said array of photosensing elements is displayed in reverse at said
video display screen; wherein a graphic overlay is superimposed on
the displayed images, and wherein, at least when the wheels of the
vehicle are straight, said graphic overlay comprises a pair of
converging lines extending outward relative to the rear of the
vehicle; wherein a plurality of spaced apart horizontal lines at
least partially extend between said pair of converging lines in
order to enhance the ability of a driver of the vehicle to judge
relative distance to an object located rearward of the vehicle, and
wherein said spaced apart horizontal lines are superimposed on
displayed video images at predetermined intervals; wherein at least
one of (a) said pair of converging lines comprise a pair of
substantially straight converging lines, (b) said graphic overlay
enhances the driver's sense of perspective when executing a
reversing maneuver, (c) said graphic overlay assists the driver
when reversing into a parking space and (d) said graphic overlay
enhances the driver's judgment of the location of an object at
least one of (i) behind the vehicle and (ii) alongside the vehicle;
and wherein said graphic overlay is superimposed on the displayed
images responsive to the driver of the vehicle placing a gear
selector device of the vehicle in a reverse gear position.
86. The driver assistance system of claim 85, wherein said field of
view of said image capture device encompasses a ground area at the
immediate rear of the vehicle.
87. The driver assistance system of claim 85, wherein said wide
angle optic comprises optical distortion correction.
88. The driver assistance system of claim 85, wherein distortion in
captured image data is reduced using electronic distortion
correction.
89. The driver assistance system of claim 85, wherein said wide
angle optic comprises optical distortion correction and wherein
distortion in captured image data is reduced using electronic
distortion correction.
90. The driver assistance system of claim 85, wherein said graphic
overlay is indicative of the anticipated rearward travel of the
vehicle.
91. The driver assistance system of claim 85, wherein said graphic
overlay adjusts responsive to a steering system of the vehicle, and
wherein, when the wheels of the vehicle are turned in a first
steering direction, said graphic overlay curves in a first
direction indicative of the anticipated rearward travel of the
vehicle and, when the wheels of the vehicle are turned in a second
steering direction opposite to said first steering direction, said
graphic overlay curves in a second direction opposite to said first
direction.
92. The driver assistance system of claim 85, wherein said spaced
apart horizontal lines comprise spaced apart short horizontal lines
superimposed on displayed video images at predetermined
intervals.
93. The driver assistance system of claim 85, wherein video images
derived from captured image data are displayed by said display
system substantially distortion-free.
94. The driver assistance system of claim 85, wherein said video
display screen comprises a back-lit liquid crystal video display
screen and wherein at least one of (i) said back-lit liquid crystal
video display screen comprises a back-lit flat panel liquid crystal
video display screen, (ii) said back-lit liquid crystal video
display screen is positioned in the vehicle cabin without
obstructing the driver's view through the windshield of the
vehicle, (iii) said back-lit liquid crystal video display screen is
positioned in the vehicle cabin without obstructing the driver's
view through the windshield of the vehicle and said back-lit liquid
crystal video display screen is positioned at a lower portion of
the driver's forward field of view, and (iv) said back-lit liquid
crystal video display screen is mounted at a position
conventionally occupied by an interior rearview mirror of the
vehicle.
95. The driver assistance system of claim 85, wherein at least one
of (a) video images output by said image capture device are
conveyed to said display system via electrically conductive leads,
(b) video images output by said image capture device are conveyed
to said display system via a fiber optic cable, and (c) video
images output by said image capture device are output as one of (i)
analog data and (ii) digital data.
96. The driver assistance system of claim 85, wherein said array of
photosensing elements comprises at least 160 columns of photosensor
elements and at least 40 rows of photosensor elements.
97. The driver assistance system of claim 85, wherein said graphic
overlay is not superimposed on the displayed images responsive to
the driver of the vehicle placing the gear selector device in a
position other than in the reverse gear position.
98. The driver assistance system of claim 85, wherein said image
capture device is fixedly mounted at the rear of the vehicle.
99. The driver assistance system of claim 85, wherein said field of
view of said image capture device encompasses a portion of the rear
of the vehicle.
100. The driver assistance system of claim 85, wherein said video
display screen is mounted at one of a dashboard of the vehicle and
a fascia of the vehicle.
101. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said field of view of said
image capture device encompasses a ground area at the immediate
rear of the vehicle; wherein said image capture device comprises an
array of photosensing elements operable to capture image data;
wherein said image capture device comprises a CMOS image capture
device; wherein said image capture device comprises a wide angle
optic; wherein distortion in captured image data is reduced using
electronic distortion correction; a display system that displays
video images output by said image capture device; wherein at least
one of (a) video images output by said image capture device are
conveyed to said display system via electrically conductive leads,
(b) video images output by said image capture device are conveyed
to said display system via a fiber optic cable, and (c) video
images output by said image capture device are output as one of (i)
analog data and (ii) digital data; wherein said display system
comprises a video display screen; wherein image data captured by
said array of photosensing elements is displayed in reverse at said
video display screen; wherein a graphic overlay is superimposed on
the displayed images, and wherein, at least when the wheels of the
vehicle are straight, said graphic overlay comprises a pair of
converging lines extending outward relative to the rear of the
vehicle; wherein at least one horizontal line at least partially
extend between said pair of converging lines in order to enhance
the ability of a driver of the vehicle to judge relative distance
to an object located rearward of the vehicle; and wherein said
graphic overlay is superimposed on the displayed images responsive
to the driver of the vehicle placing a gear selector device of the
vehicle in a reverse gear position.
102. The driver assistance system of claim 101, wherein said
graphic overlay is indicative of the anticipated rearward travel of
the vehicle.
103. The driver assistance system of claim 101, wherein at least
one of (a) said pair of converging lines comprise a pair of
substantially straight converging lines, (b) said graphic overlay
enhances the driver's sense of perspective when executing a
reversing maneuver, (c) said graphic overlay assists the driver
when reversing into a parking space and (d) said graphic overlay
enhances the driver's judgment of the location of an object at
least one of (i) behind the vehicle and (ii) alongside the
vehicle.
104. The driver assistance system of claim 101, wherein said at
least one horizontal line comprises a plurality of spaced apart
horizontal lines, and wherein at least one of (i) said spaced apart
horizontal lines are superimposed on displayed video images at
predetermined intervals, and (ii) said spaced apart horizontal
lines comprise spaced apart short horizontal lines superimposed on
displayed video images at predetermined intervals.
105. The driver assistance system of claim 101, wherein said
graphic overlay adjusts responsive to a steering system of the
vehicle, and wherein, when the wheels of the vehicle are turned in
a first steering direction, said graphic overlay curves in a first
direction indicative of the anticipated rearward travel of the
vehicle and, when the wheels of the vehicle are turned in a second
steering direction opposite to said first steering direction, said
graphic overlay curves in a second direction opposite to said first
direction.
106. The driver assistance system of claim 101, wherein said video
display screen comprises a back-lit liquid crystal video display
screen and wherein at least one of (i) said back-lit liquid crystal
video display screen comprises a back-lit flat panel liquid crystal
video display screen, (ii) said back-lit liquid crystal video
display screen is positioned in the vehicle cabin without
obstructing the driver's view through the windshield of the
vehicle, (iii) said back-lit liquid crystal video display screen is
positioned in the vehicle cabin without obstructing the driver's
view through the windshield of the vehicle and said back-lit liquid
crystal video display screen is positioned at a lower portion of
the driver's forward field of view, and (iv) said back-lit liquid
crystal video display screen is mounted at a position
conventionally occupied by an interior rearview mirror of the
vehicle.
107. The driver assistance system of claim 101, wherein said array
of photosensing elements comprises at least 160 columns of
photosensor elements and at least 40 rows of photosensor
elements.
108. The driver assistance system of claim 101, wherein said
graphic overlay is not superimposed on the displayed images
responsive to the driver of the vehicle placing the gear selector
device in a position other than in the reverse gear position.
109. The driver assistance system of claim 101, wherein said field
of view of said image capture device encompasses a portion of the
rear of the vehicle.
110. The driver assistance system of claim 101, wherein said video
display screen is mounted at one of a dashboard of the vehicle and
a fascia of the vehicle.
111. The driver assistance system of claim 101, wherein reversal of
image data occurs at said image capture device.
112. The driver assistance system of claim 101, wherein reversal of
image data occurs at said display system.
113. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said field of view of said
image capture device encompasses a portion of the rear of the
vehicle; wherein said image capture device comprises an array of
photosensing elements operable to capture image data; wherein said
array of photosensing elements comprises at least 160 columns of
photosensor elements and at least 40 rows of photosensor elements;
wherein said image capture device comprises a CMOS image capture
device; a display system that displays video images output by said
image capture device; wherein said display system comprises a video
display screen; wherein image data captured by said array of
photosensing elements is displayed in reverse at said video display
screen; wherein a graphic overlay is superimposed on the displayed
images, and wherein said graphic overlay comprises a pair of
substantially straight converging lines extending outward from
adjacent the rear of the vehicle; wherein at least one horizontal
line at least partially extends between said pair of converging
lines in order to enhance the ability of a driver of the vehicle to
judge relative distance to an object located rearward of the
vehicle; and wherein said graphic overlay is superimposed on the
displayed images responsive to the driver of the vehicle placing a
gear selector device of the vehicle in a reverse gear position.
114. The driver assistance system of claim 113, wherein said
graphic overlay is indicative of the anticipated rearward travel of
the vehicle.
115. The driver assistance system of claim 113, wherein at least
one of (a) said graphic overlay enhances the driver's sense of
perspective when executing a reversing maneuver, (b) said graphic
overlay assists the driver when reversing into a parking space and
(c) said graphic overlay enhances the driver's judgment of the
location of an object at least one of (i) behind the vehicle and
(ii) alongside the vehicle.
116. The driver assistance system of claim 113, wherein said at
least one horizontal line comprises a plurality of spaced apart
horizontal lines, and wherein at least one of (i) said spaced apart
horizontal lines are superimposed on displayed video images at
predetermined intervals, and (ii) said spaced apart horizontal
lines comprise spaced apart short horizontal lines superimposed on
displayed video images at predetermined intervals.
117. The driver assistance system of claim 113, wherein said video
display screen comprises a back-lit liquid crystal video display
screen and wherein at least one of (i) said back-lit liquid crystal
video display screen comprises a back-lit flat panel liquid crystal
video display screen, (ii) said back-lit liquid crystal video
display screen is positioned in the vehicle cabin without
obstructing the driver's view through the windshield of the
vehicle, (iii) said back-lit liquid crystal video display screen is
positioned in the vehicle cabin without obstructing the driver's
view through the windshield of the vehicle and said back-lit liquid
crystal video display screen is positioned at a lower portion of
the driver's forward field of view, and (iv) said back-lit liquid
crystal video display screen is mounted at a position
conventionally occupied by an interior rearview mirror of the
vehicle.
118. The driver assistance system of claim 113, wherein said
graphic overlay is not superimposed on the displayed images
responsive to the driver of the vehicle placing the gear selector
device in a position other than in the reverse gear position.
119. The driver assistance system of claim 113, wherein reversal of
image data occurs at said image capture device.
120. The driver assistance system of claim 113, wherein reversal of
image data occurs at said display system.
121. The driver assistance system of claim 113, wherein said video
display screen is mounted at one of a dashboard of the vehicle and
a fascia of the vehicle.
122. The driver assistance system of claim 113, wherein said image
capture device comprises a wide angle optic, and wherein said wide
angle optic comprises optical distortion correction.
123. The driver assistance system of claim 113, wherein distortion
in captured image data is reduced using electronic distortion
correction.
124. A driver assistance system suitable for use in a vehicle, said
driver assistance system comprising: an image capture device
positioned at a rear portion of a vehicle and having a field of
view rearward of the vehicle; wherein said image capture device is
fixedly mounted at the rear of the vehicle; wherein said field of
view of said image capture device encompasses a ground area at the
immediate rear of the vehicle, and wherein said field of view of
said image capture device encompasses a portion of the rear of the
vehicle; wherein said image capture device comprises an array of
photosensing elements operable to capture image data; wherein said
array of photosensing elements comprises at least 160 columns of
photosensor elements and at least 40 rows of photosensor elements;
wherein said image capture device comprises a CMOS image capture
device; a display system that displays video images output by said
image capture device; wherein said display system comprises a video
display screen; wherein image data captured by said array of
photosensing elements is displayed in reverse at said video display
screen; wherein a graphic overlay is superimposed on the displayed
images, and wherein, at least when the wheels of the vehicle are
straight, said graphic overlay comprises a pair of converging lines
extending outward relative to the rear of the vehicle; wherein a
plurality of spaced apart horizontal lines at least partially
extend between said pair of converging lines in order to enhance
the ability of a driver of the vehicle to judge relative distance
to an object located rearward of the vehicle, and wherein said
spaced apart horizontal lines are superimposed on displayed video
images at predetermined intervals; and wherein said graphic overlay
is superimposed on the displayed images responsive to the driver of
the vehicle placing a gear selector device of the vehicle in a
reverse gear position.
125. The driver assistance system of claim 124, wherein said pair
of converging lines comprise a pair of substantially straight
converging lines.
126. The driver assistance system of claim 124, wherein, when the
wheels of the vehicle are turned in a first steering direction,
said graphic overlay curves in a first direction indicative of the
anticipated rearward travel of the vehicle and, when the wheels of
the vehicle are turned in a second steering direction opposite to
said first steering direction, said graphic overlay curves in a
second direction opposite to said first direction.
127. The driver assistance system of claim 124, wherein said
graphic overlay is indicative of the anticipated rearward travel of
the vehicle.
128. The driver assistance system of claim 124, wherein said
graphic overlay enhances the driver's sense of perspective when
executing a reversing maneuver.
129. The driver assistance system of claim 124, wherein video
images derived from captured image data are displayed by said
display system substantially distortion-free.
130. The driver assistance system of claim 124, wherein said video
display screen is positioned in the vehicle cabin without
obstructing the driver's view through the windshield of the
vehicle.
131. The driver assistance system of claim 130, wherein said video
display screen is positioned at a lower portion of the driver's
forward field of view.
132. The driver assistance system of claim 124, wherein said video
display screen is mounted at a dashboard of the vehicle.
133. The driver assistance system of claim 124, wherein said array
of photosensing elements is operable to capture frames of image
data at a rate of up to 30 frames per second.
134. The driver assistance system of claim 124, wherein said array
of photosensing elements and circuitry associated therewith are
constructed on a common semiconductor substrate.
135. The driver assistance system of claim 124, wherein said driver
assistance system comprises anti-blooming to mitigate the effect of
charge leakage from at least one of said photosensing elements to
at least an adjacent one of said photosensing elements.
136. The driver assistance system of claim 124, wherein said
graphic overlay is not superimposed on the displayed images
responsive to the driver of the vehicle placing the gear selector
device in a position other than in the reverse gear position.
137. The driver assistance system of claim 124, wherein at least
one of (i) said image capture device comprises a wide angle optic
and said wide angle optic comprises optical distortion correction
and (ii) distortion in captured image data is reduced using
electronic distortion correction.
138. The driver assistance system of claim 124, wherein said video
display screen comprises a back-lit liquid crystal video display
screen and wherein at least one of (i) said back-lit liquid crystal
video display screen comprises a back-lit flat panel liquid crystal
video display screen, (ii) said back-lit liquid crystal video
display screen is positioned in the vehicle cabin without
obstructing the driver's view through the windshield of the
vehicle, (iii) said back-lit liquid crystal video display screen is
positioned in the vehicle cabin without obstructing the driver's
view through the windshield of the vehicle and said back-lit liquid
crystal video display screen is positioned at a lower portion of
the driver's forward field of view, and (iv) said back-lit liquid
crystal video display screen is mounted at a position
conventionally occupied by an interior rearview mirror of the
vehicle.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] The present application is a continuation of U.S. patent
application Ser. No. 11/626,529, filed Jan. 24, 2007 (Attorney
Docket MAG04 P-1329), which is a continuation of U.S. patent
application Ser. No. 10/643,602, filed Aug. 19, 2003, now U.S. Pat.
No. 7,859,565, which is a continuation of application Ser. No.
09/776,625, filed on Feb. 5, 2001, now U.S. Pat. No. 6,611,202,
which is a continuation of application Ser. No. 09/313,139, filed
on May 17, 1999, now U.S. Pat. No. 6,222,447, which is a
continuation of application Ser. No. 08/935,336, filed on Sep. 22,
1997, now U.S. Pat. No. 5,949,331, which is a continuation of
application Ser. No. 08/445,527, filed on May 22, 1995, now U.S.
Pat. No. 5,670,935.
BACKGROUND OF THE INVENTION
[0002] This invention relates generally to vision systems for
vehicles and, more particularly, to rearview vision systems which
provide the vehicle operator with scenic information in the
direction rearward of the vehicle. More particularly, the invention
relates to a rearview vision system utilizing image capture
devices, such as CMOS imaging arrays and the like.
[0003] A long-felt need in the art of vehicle rearview vision
systems has been to eliminate exterior rearview mirrors by
utilizing image capture devices, such as cameras, in combination
with dashboard displays. This would be beneficial because it would
reduce wind drag on the vehicle, wind noise and vehicle weight.
Furthermore, rearview mirrors protrude a substantial distance from
the side of the vehicle, which makes maneuvering in tight spaces
more difficult. Image capture devices are capable of positioning in
a greater variety of locations on the vehicle, providing more
flexibility of vehicle styling. It is further expected that camera
systems would greatly reduce the blind spots to the sides and rear
of the vehicle common with vehicles equipped with conventional
rearview mirror systems. The driver cannot perceive vehicles,
objects, or other road users in such blind spots without turning
his or her body, which interferes with forward-looking visual
activities.
[0004] Camera-based rearview vision systems for vehicles have not
obtained commercial acceptance. One difficulty with proposed
systems has been that they present a large amount of visual
information in a manner which is difficult to comprehend. This
difficulty arises from many factors. In order to significantly
reduce blind spots, multiple image capture devices are typically
positioned at various locations on the vehicle. The image of an
object behind the equipped vehicle is usually captured by more than
one image capture device at a time and displayed in multiple
images. This may confuse the driver as to whether more than one
object is present. When multiple image capture devices are
positioned at different longitudinal locations on the vehicle,
objects behind the vehicle are at different distances from the
image capture devices. This results in different image sizes for
the same object. This effect is especially noticeable for laterally
extending images, such as a bridge, highway crosswalk markings, the
earth's horizon, and the like. Such images are at different
vertical angles with respect to the image capture devices. This
results in different vertical positions on the display causing the
elongated image to appear disjointed.
[0005] A camera system provides a monocular view of the scene,
compared to the binocular, or stereoscopic, view obtained when the
scene is viewed through a rearview mirror. This makes the ability
to judge distances in a camera system a problem. This effect is
most noticeable at distances close to the vehicle where
stereoscopic imaging is relied upon extensively by the driver in
judging relative locations of objects. Therefore, known camera
systems fail to provide to the driver important information where
that information is most needed at small separation distances from
surrounding objects.
[0006] Another difficulty with camera systems is that, in order to
provide a sufficient amount of information, the camera system
typically presents the driver with a greatly increased field of
view. This improves performance by further reducing blind spots at
the side and rear of the vehicle. However, an increased field of
view is often obtained by utilizing a wide-angle lens which
introduces distortion of the scene and further impairs the ability
of the driver to judge distances of objects displayed. The problem
with such distortion of the scene is that the driver must
concentrate more on the display and take a longer time to interpret
and extract the necessary information. This further distracts the
driver from the primary visual task of maintaining awareness of
vehicles and other objects in the vicinity of the driven
vehicle.
SUMMARY OF THE INVENTION
[0007] The present invention is directed towards enhancing the
interpretation of visual information in a rearview vision system by
presenting information in a manner which does not require
significant concentration of the driver or present distractions to
the driver. This is accomplished according to the invention in a
rearview vision system having at least two image capture devices
positioned on the vehicle and directed rearwardly with respect to
the direction of travel of the vehicle. A display is provided for
images captured by the image capture devices. The display combines
the captured images into an image that would be achieved by a
single rearward-looking camera having a view unobstructed by the
vehicle. In order to obtain all of the necessary information of
activity, not only behind but also along side of the vehicle, the
virtual camera should be positioned forward of the driver. The
image synthesized from the multiple image capture devices may have
a dead space which corresponds with the area occupied by the
vehicle. This dead space is useable by the driver's sense of
perspective in judging the location of vehicles behind and along
side of the equipped vehicle.
[0008] The present invention provides techniques for synthesizing
images captured by individual, spatially separated, image capture
devices into such ideal image, displayed on the display device.
This may be accomplished according to an aspect of the invention by
providing at least three image capture devices. At least two of the
image capture devices are side image capture devices mounted on
opposite sides of the vehicle. At least one of the image capture
devices is a center image capture device mounted laterally between
the side image capture devices. A display system displays an image
synthesized from outputs of the image capture devices. The
displayed image includes an image portion from each of the image
capture devices. The image portion from the center image capture
device is vertically compressed.
[0009] It has been discovered that such vertical compression
substantially eliminates distortion resulting from the spatial
separation between the cameras and can be readily accomplished. In
an illustrated embodiment, the image compression is carried out by
removing selective ones of the scan lines making up the image
portion. A greater number of lines are removed further away from
the vertical center of the image.
[0010] The compression of the central image portion produces a dead
space in the displayed image which may be made to correspond with
the area that would be occupied by the vehicle in the view from the
single virtual camera. Preferably, perspective lines are included
at lateral edges of the dead space which are aligned with the
direction of travel of the vehicle and, therefore, appear in
parallel with lane markings. This provides visual clues to the
driver's sense of perspective in order to assist in judging
distances of objects around the vehicle.
[0011] According to another aspect of the invention, image
enhancement means are provided for enhancing the displayed image.
Such means may be in the form of graphic overlays superimposed on
the displayed image. Such graphic overlap may include indicia of
the anticipated path of travel of the vehicle which is useful in
assisting the driver in guiding the vehicle in reverse directions.
Such graphic overlay may include a distance grid indicating
distances behind the vehicle of objects juxtaposed with the
grid.
[0012] These and other objects, advantages, and features of this
invention will become apparent by review of the following
specification in conjunction with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] FIG. 1 is a top plan view of a vehicle having a rearview
vision system according to the invention;
[0014] FIG. 2 is a side elevation of the vehicle in FIG. 1;
[0015] FIG. 3 is a front elevation of a display according to the
invention;
[0016] FIG. 4 is the same view as FIG. 1 illustrating an
alternative embodiment of the invention;
[0017] FIG. 5 is a block diagram of an electronic system according
to the invention;
[0018] FIG. 6 is the same view as FIG. 3 illustrating an alternate
mode of operation of the system;
[0019] FIG. 7 is the same view as FIG. 2 illustrating an
alternative embodiment of the invention;
[0020] FIG. 8 is the same view as FIG. 3 illustrating an
alternative embodiment of the invention;
[0021] FIG. 9 is the same view as FIGS. 1 and 4 illustrating an
alternative embodiment of the invention;
[0022] FIG. 10 is the same view as FIGS. 3 and 8 illustrating an
alternative embodiment of the invention; and
[0023] FIG. 11 is a chart illustrating the horizontal row of pixels
(n1, n2) on which an object will be imaged from two longitudinally
separated image capture devices as that object is spaced at
different longitudinal distances from the image capture
devices.
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0024] Referring now specifically to the drawings, and the
illustrative embodiments depicted therein, a vehicle 10, which may
be an automobile, a light truck, a sport utility vehicle, a van, a
bus, a large truck, or the like includes a rearview vision system,
generally illustrated at 12, for providing a driver of the vehicle
with a view rearwardly of the vehicle with respect to the direction
of travel D of the vehicle (FIG. 1). Vision system 12 includes at
least two side image capture devices 14 positioned, respectively,
on opposite sides of vehicle 10 and a center image capture device
16 positioned on the lateral centerline of the vehicle. All of the
image capture devices are directed generally rearwardly of the
vehicle. Rearview vision system 12 additionally includes an image
processor 18 for receiving data signals from image capture devices
14, 16 and synthesizing, from the data signals, a composite image
42 which is displayed on a display 20.
[0025] As will be set forth in more detail below, the images
captured by image capture devices 14, 16 are juxtaposed on display
20 by image processor 18 in a manner which approximates the view
from a single virtual image capture device positioned forwardly of
the vehicle at a location C and facing rearwardly of the vehicle,
with the vehicle being transparent to the view of the virtual image
capture device. Vision system 12 provides a substantially seamless
panoramic view rearwardly of the vehicle without duplicate or
redundant images of objects. Furthermore, elongated,
laterally-extending, objects, such as the earth's horizon, appear
uniform and straight across the entire displayed image. The
displayed image provides a sense of perspective, which enhances the
ability of the driver to judge location and speed of adjacent
trailing vehicles.
[0026] Each of side image capture devices 14 has a field of view 22
and is aimed rearwardly with respect to the vehicle about an axis
24 which is at an angle, with respect to the vehicle, that is half
of the horizontal field of view of the image capture device. In
this manner, each of the image capture devices 14 covers an area
bounded by the side of the vehicle and extending outwardly at an
angle defined by the horizontal field of view of the respective
side image capture device. Center image capture device 16 has a
horizontal field of view 26, which is symmetrical about the
longitudinal axis of the vehicle. The field of view of each side
image capture device 14 intersect the field of view of center image
capture device 16 at a point P which is located a distance Q behind
vehicle 10.
[0027] Rear blind zones 30 are located symmetrically behind vehicle
10 extending from the rear of the vehicle to point P. Side blind
zones 25 located laterally on respective sides of the vehicle
extend rearwardly of the forward field of view 36 of the driver to
the field of view 22 of the respective side image capture device
14. An object will not be captured by side image capture devices 14
or center image capture devices 16 if the object is entirely within
one of the blind zones 25, 30. In order for an object, such as
another vehicle V or other road user travelling to the side of
vehicle 10, to be observed by an operator of vehicle 10, the object
must be either at least partially within the forward field of view
36 of the driver or be captured by image capture devices 14, 16 and
displayed on display 20. FIG. 4 illustrates vehicle 10 travelling
on a three-lane highway having lanes L1, L2, and L3 with the
vehicle in lane L2. Another vehicle V is shown positioned mostly
within one of the blind zones 25, but with the rearmost portion of
the vehicle V extending into field of view 22 where the vehicle
image will be captured by one of side image capture devices 14. In
the illustrated embodiment, vehicle V is a motorcycle travelling in
the center of lanes L1 or L3 and represents a worst case for
observing a vehicle travelling at least partially within one of the
blind zones 25. In order for a portion of vehicle V to be extending
either forwardly or rearwardly of the respective blind zone 25,
where the vehicle V may be observed by either the forward field of
view 36 of the driver or by the rearview vision system 12, the
field of view 22 of side image capture devices 14 must be
sufficiently wide to capture a portion of vehicle V as illustrated
in FIG. 4. Preferably, the horizontal field of view 22 of side
image capture devices 14 is no greater than that required to
provide sufficient coverage which would be in the range of between
approximately 55 degrees and approximately 70 degrees. In the
illustrated embodiment, the horizontal field of view 22 is 61
degrees. In order for a portion of vehicle V to be within a
vertical field of view 40 of one of side image capture devices 14,
the field of view should extend to the pavement at a plane M which
intersects vehicle V (FIG. 2). Preferably, vertical field of view
40 is between approximately 60 degrees and approximately 75
degrees. In the illustrated embodiment, vertical field of view 40
is 66 degrees. A left overlap zone 32 and a right overlap zone 34
extend rearward from respective points P where the horizontal
fields of view of the side image capture devices intersect the
field of view of center image capture device 16. Overlap zones 32,
34 define areas within which an object will be captured both by
center image capture device 16 and one of the side image capture
devices 14. An object in an overlap zone 32, 34 will appear on
display 20 in multiple image portions in a redundant or duplicative
fashion. In order to avoid the presentation of redundant
information to the driver, and thereby avoid confusion and simplify
the task of extracting information from the multiple images or
combined images on display 20, the object should avoid overlapping
zones 32, 34. In practice, this may be accomplished to a
satisfactory extent by moving points P away from the vehicle and
thereby increasing distance Q. It is desirable to increase distance
Q to a length that will exclude vehicles travelling at a typical
separation distance behind vehicle 10 from overlapping zones 32,
34. This separation distance is usually a function of the speed at
which the vehicles on the highway are travelling. The faster the
vehicles are travelling, the further Q should be moved behind
vehicle 10 to keep overlap zones 32 and 34 outside of the
recommended vehicle spacing. If, however, the vehicles are
travelling at a slower speed, then the generally accepted
recommendation for vehicle spacing decreases and it is more likely
that a vehicle will be within overlap zone 32, 34. Therefore, the
distance Q may be selected to accommodate expected vehicle spacing
for an average driving speed of vehicle 10.
[0028] Distance Q is a function of the effective horizontal field
of view 26 of center image capture device 16. As field of view 26
decreases, points P move further rearward of the vehicle from a
distance Q.sub.1, to a distance Q.sub.2, as best illustrated in
FIG. 4. In order to increase distance Q to eliminate redundant and
duplicative information displayed on display 20 for most driving
conditions of vehicle 10, field of view 26 is preferably less than
12 degrees. In the illustrated embodiment, field of view 26 is
between 6 and 8 degrees. Alternatively, distance Q may be
dynamically adjusted according to some parameter, such as the speed
of vehicle 10. This would allow Q to be greater when the vehicle is
travelling at a faster speed, where vehicle separation tends to be
larger, and vice versa. Field of view 26 may be adjusted by
utilizing a selective presentation of pixels of the captured image
in the displayed image.
[0029] Referring to FIG. 3, image display device 20 displays a
composite image 42 made up of a left image portion 44, a right
image portion 46, and a center image portion 48. Each image portion
44-48 is reversed from the image as captured by the respective
image capture device 14, 16 utilizing conventional techniques.
These techniques include reading the image in reverse with the
image capture device, writing the image in reverse to display
device 20, or reversing the image in image processor 18. Left image
portion 44 is joined with central image portion 48 at a boundary
50. Central image portion 48 is joined with right image portion 46
at a boundary 52. As may best be seen in FIG. 3, the image portions
at boundaries 50 and 52 are continuous whereby composite image 42
is a seamless panoramic view rearwardly of the vehicle. As also is
apparent from FIG. 3, central image portion 48 is narrower than
either left image portion 44 or right image portion 46. This is a
result of reducing the horizontal field of view 26 of center image
capture device 16 sufficiently to move points P, and thus overlap
zones 32 and 34, a sufficient distance behind vehicle 10 to reduce
redundant and duplicative images between image portions 44-48.
Composite image 42 provides a clear image, which avoids confusion
and simplifies the task of extracting information from the multiple
image portions 44-48. As also may be seen by reference to FIG. 3,
display 20 may additionally include indicia such as the readout of
a compass 54, vehicle speed 56, turn signals 58, and the like as
well as other graphical or video displays, such as a navigation
display, a map display, and a forward-facing vision system. In this
manner, rearview vision system 12 may be a compass vision system or
an information vision system.
[0030] In the embodiment of rearview vision system 12 having a
dynamically adjusted value of distance Q, the spacing between
boundaries 50 and 52 will dynamically adjust in sequence with the
adjustment of distance Q. Thus, as overlap zones 32, 34 move
further away from the vehicle; for example, in response to an
increase in speed of the vehicle, boundary lines 50 and 52 will
move closer together and vice versa. In this manner, composite
image 42 is dynamic, having image portions of dynamically adaptive
sizes.
[0031] Display 20 is of a size to be as natural as possible to the
driver. This is a function of the size of the display and the
distance between the display and the driver. Preferably, the
displayed image simulates an image reflected by a rearview mirror.
As such, the size of display 20 is approximately the combined areas
of the three rearview mirrors (one interior mirror and two exterior
mirrors) conventionally used with vehicles. As best seen by
reference to FIG. 2, display 20 is preferably positioned within the
driver's physiological field of view without obstructing the view
through the windshield. It is known that the driver's field of
view, with the head and eyes fixed forward, extends further in a
downward direction than in an upward direction. Display 20 could be
located above the vertical view through the windshield wherein the
display may be observed at the upward portion of the driver's field
of view. However, the position for the display illustrated in FIG.
2 is preferred wherein the display is within the lower portion of
the driver's field of view.
[0032] Display 20, in the illustrated embodiment, is a flat panel
display, such as a back-lit liquid crystal display, a plasma
display, a field emission display, or a cathode ray tube. However,
the synthesized image could be displayed using other display
techniques such as to provide a projected or virtual image. One
such virtual display is a heads-up display. The display may be
mounted/attached to the dashboard, fascia or header, or to the
windshield at a position conventionally occupied by an interior
rearview mirror.
[0033] Although various camera devices may be utilized for image
capture devices 14, 16, an electro-optic, pixelated imaging array,
located in the focal plane of an optical system, is preferred. Such
imaging array allows the number of pixels to be selected to meet
the requirements of rearview vision system 12. The pixel
requirements are related to the imaging aspect ratio of the
respective image capture devices, which, in turn, are a function of
the ratio of the vertical-to-horizontal field of view of the
devices, as is well known in the art. In the illustrated
embodiment, the imaging aspect ratio of side image capture devices
14 is 2:1 and the image aspect ratio of central image capture
device 16 is variable down to 0.1:1. Such aspect ratio will produce
images which will not typically match that of commercially
available displays. A commercially available display may be used,
however, by leaving a horizontal band of the display for displaying
alpha-numeric data, such as portions of an instrument cluster,
compass display, or the like, as illustrated in FIG. 3.
[0034] In the illustrated embodiment, image capture devices 14, 16
are CMOS imaging arrays of the type manufactured by VLSI Vision
Ltd. of Edinburgh, Scotland, which are described in more detail in
U.S. patent application Ser. No. 08/023,918, filed Feb. 26, 1993,
by Kenneth Schofield and Mark Larson for an AUTOMATIC REARVIEW
MIRROR SYSTEM USING A PHOTOSENSOR ARRAY, now U.S. Pat. No.
5,550,677, the disclosure of which is hereby incorporated herein by
reference. However, other pixelated focal plane image-array
devices, which are sensitive to visible or invisible
electromagnetic radiation, could be used. The devices could be
sensitive to either color or monochromatic visible radiation or
near or far infrared radiation of the type used in night-vision
systems. Each image capture device could be a combination of
different types of devices, such as one sensitive to visible
radiation combined with one sensitive to infrared radiation.
Examples of other devices known in the art include charge couple
devices and the like.
[0035] As disclosed in U.S. patent application Ser. No. 08/023,918,
filed Feb. 26, 1993, by Kenneth Schofield and Mark Larson for an
AUTOMATIC REARVIEW MIRROR SYSTEM USING A PHOTOSENSOR ARRAY, now
U.S. Pat. No. 5,550,677, incorporated above, the logic and control
circuit comprises a logic circuit, a clock, a random-access-memory
(RAM), or other appropriate memory, and a digital-to-analog
converter. The logic circuit is preferably a dedicated
configuration of digital logic elements constructed on the same
semiconductor substrate as the photosensor array. Alternatively,
the logic circuit may also be a microprocessor comprising a central
processing unit (CPU) and a read-only-memory (ROM). The logic
circuit may also be implemented using gate array technology or any
other appropriate hardwired logic circuit technology. The logic
circuit interfaces with the clock, provides array control signals
to the photosensor array, manages data flow to and from the RAM and
converters and, and performs all computations for determining, for
example, a digital mirror control signal for causing the variable
reflectance mirror element to assume a desired reflectance level.
The analog-to-digital converter converts the analog photosensor
element signals to the digital photosensor element signals
processed by the logic circuit. It has been found that an eight-bit
analog-to-digital converter provides adequate data resolution for
controlling the mirrors. Preferably, the analog-to-digital
converter is constructed on the same semiconductor substrate as the
photosensor array. The digital photosensor element signals output
to the logic and control circuit are generally stored in the RAM
for processing. The values of the digital photosensor element
signals for the photosensor array are correspondingly stored in an
array in the RAM. The logic circuit processes the values of each of
the digital photosensor element signals to determine, for example,
an instantaneous or substantially real-time background light signal
for a time period t and at least one peak light signal. The logic
circuit uses these signals, which may also be temporarily stored in
the RAM, to determine a digital control signal to, for example,
cause at least one mirror or mirror segment to assume a desired
reflectance level. The digital mirror control signal may be output
to the digital-to-analog converter, which may output a
corresponding analog mirror control signal to a mirror drive
circuit. Alternatively, the digital-to-analog converter need not be
used if the logic circuit generates a pulse-width-modulated (PWM)
mirror control signal to control the mirror drive circuit. In one
embodiment of the invention, the logic circuit determines the
background light signal by calculating the average value of the
photosensor element signals, previously stored in RAM, for the
photosensor elements in a lowest row or rows of the photosensor
array corresponding to an area below the rear window. This means
that the background light signal may be determined from photosensor
element signals generated by the photosensor elements located in
one row of the photosensor matrix array. The logic circuit may then
output to the RAM for later processing. The logic circuit may also
determine a background light level by calculating an average value
of all of the photosensor element signals in the entire photosensor
array. More generally, the background light signal for the rearward
scene may be determined by calculating the average value of X
percent of the lowest photosensor element signal values in the RAM
array, where X is preferably 75, but typically may be in the range
of 5 to 100.
[0036] Preferably, image capture devices 14 and 16 are all mounted
at the same vertical height on vehicle 10, although compromise may
be required in order to accommodate styling features of the
vehicle. The horizontal aim of image capture devices 14 and 16 is
preferably horizontal. However, the portion of the image displayed
is preferably biased toward the downward portion of the captured
image because significantly less useful information is obtained
above the horizontal position of the image capture devices.
[0037] Each image-capturing device 14, 16 is controlled by
appropriate supporting electronics (not shown) located in the
vicinity of the imaging array such that, when operating power is
supplied, either an analog or a digital data stream is generated on
an output signal line supplied to image processor 18. The support
electronics may be provided partially on the image chip and
partially on associated electronic devices. For each exposure
period, a value indicative of the quantity of light incident on
each pixel of the imaging array during the exposure period is
sequentially outputted in a predetermined sequence, typically
row-by-row. The sequence may conform to video signal standards
which support a direct view such that, when a scene is viewed by an
image-capturing device, the image presented on a display represents
directly the scene viewed by the image-capturing devices. However,
when looking forward and observing a displayed image of a rearward
scene, the driver will interpret the image as if it were a
reflection of the scene as viewed through a mirror. Objects to the
left and rearward of the vehicle, as viewed by the rearward-looking
camera, are presented on the left-hand side of the display and vice
versa. If this reversal is effected in image processor 18, it may
be by the use of a data storage device, or buffer, capable of
storing all of the pixel values from one exposure period. The data
is read out of the data storage device in a reversed row sequence.
Alternatively, the imaging array electronics could be constructed
to provide the above-described reversal at the image-capturing
device or at the display.
[0038] Data transmission between image capture devices 14, 16 and
image processor 18 and/or between image processor 18 and display 20
may be by electrically conductive leads or fiber-optic cable. It is
possible, for particular applications, to eliminate image processor
18 and direct drive display 20 from image capture devices 14, 16 at
the pixel level.
[0039] The data streams from image-capturing devices 14, 16 are
combined in image processor 18 and directly mapped to the pixel
array of display 20. This process is repeated preferably at a rate
of at least 30 times per second in order to present an essentially
real time video image. The image captured by side image capture
device 14 on the right side of the vehicle is presented in right
image portion 46 and the image from side image capture device 14 on
the left side of the vehicle is displayed on left image portion 44.
The image from center image capture device 16 is displayed on
central image portion 48. The three image portions 44-48 are
presented in horizontal alignment and adjacent to each other.
However, the composite image may be positioned at any desired
vertical position in the display 20. It is also possible to display
image portions 44-48 on separate image devices which are adjacent
each other.
[0040] In vision system 12, side image capture devices 14 are
positioned preferably at a forward longitudinal position on vehicle
10 and center image capture device 16 is positioned at a rearward
longitudinal position on the vehicle. As best seen by reference to
FIG. 7, this positioning creates a difference in the vertical angle
between each side image capture device 14 and center image capture
device 16 with respect to a fixed location P1 that is a distance D1
behind the vehicle. This difference in sensing angle will cause
each side image capture device 14 to image an object located at P1
on a horizontal row of pixels that is different from the horizontal
row of pixels that center image capture device 16 will image the
same object. If the image is below the horizontal centerline of the
image capture device, it will be imaged on a lower row of pixels by
center image capture device 16 than the row of pixels it will be
imaged by the side image capture devices 14, as illustrated in FIG.
7. This mismatch between horizontal pixel rows of the captured
image is furthermore a function of the distance of the captured
image from the rear of the vehicle. This can be understood by
reference to FIG. 11 which presents a chart 90 having a first
column 92 of pixel lines n1, measured from the array centerline, at
which an object will be imaged by side image capture device 14 and
a second column 94 of pixel lines n2, measured from the array
vertical centerline, at which the same object will be imaged by
center image capture device 16. The result is that an object, which
is captured by both side and center image capture devices 14, 16,
will be vertically disjointed at the boundary of the displayed
image, if the object is captured by more than one image capture
device. The amount of disjointment will be greater closer to the
vehicle and less at further distances. If the object is elongated
in the horizontal direction, such as earth's horizon, bridges, or
cross-markings on highways, then the object will appear to be
either broken or crooked.
[0041] In order to provide uniform display of laterally elongated
images, a rearview vision system 12' is provided having a central
image portion 48' which is processed differently from the image
display portions 44' and 46' produced by the side image capture
devices (FIG. 8). Central image portion 48' is reduced vertically,
or compressed, by removing specified scan lines, or pixel rows,
from the image captured by center image capture device 16 in a
graduated fashion. The difference in the pixel line at which an
object will be imaged by each of the side and center image capture
devices is a function of the distance D of the object from the rear
of the vehicle, with a greater variation occurring at shorter
distances and the variation reducing to zero for infinite
distances. Therefore, the compression of central image portion 48'
is non-linear, with substantially no compression at the vertical
center of the image and greater compression at greater distances
above and below the vertical center point of the image. This is
accomplished by removing specific lines from the center display in
a graduated fashion with a greater number of lines removed further
from the vertical center of the image. The removed lines may be
merely discarded in order to vertically reduce the image.
Alternatively, the data contained in the removed lines may be
utilized to modify the value of adjacent pixels above and below the
removed line in order to enhance the quality of the compressed
image. Averaging, median filtering, or other such known techniques
may also be used.
[0042] Each of right image portion 46' and left image portion 44'
includes an upper portion 64 which extends above the compressed
upper portion of the central image portion 48'. In the illustrated
embodiment, upper portions 64 are deleted in order to present a
uniform upper horizontal boundary for display 20'. In the
illustrated embodiment, the mismatch between the lower horizontal
boundary of central image portion 48' and each of the left and
right image portions provides a dead space 66 which provides a
visual prompt to the user of the approximate location of the
rearward corners S of vehicle 10. This dead space 66 in the image
displayed on display 20' approximates the footprint occupied by
vehicle 10 when viewed from point C. This is particularly useful
because it provides a visual indication to the driver that a
vehicle passing vehicle 10, as viewed in either left image portion
44' or right image portion 46', is at least partially adjacent
vehicle 10 if the image of the approaching vehicle is partially
adjacent to dead space 66.
[0043] In an alternative embodiment, the vertical compression
technique may be applied to only a lower vertical portion of
central image portion 48'. In most driving situations, objects
imaged by rearward-facing image capture devices above the horizon
are at a long distance from the vehicle while those below the
horizon get progressively closer to the vehicle in relation to the
distance below the horizon in the displayed image. Therefore,
compression of the upper vertical portion of the central image
portion may be eliminated without significant reduction in
performance.
[0044] Compression of the central image portion may also
advantageously be provided horizontally, as well as vertically.
Spatial separation of center image capture device 16 from side
image capture devices 14 causes similar distortion, as that
described above, in the horizontal direction. This effect is
spherical in nature and would require a more complex corrective
action, such as compressing the image based upon the removal of
pixels from an approximation to concentric circles centered on the
center of the imaging array, or other techniques which would be
apparent to those skilled in the art.
[0045] A rearview vision system 12'' includes an image display 20''
having a compressed central image portion 48'' and left and right
image portions 44'' and 46'', respectively (FIG. 10). A border 50'
between left side image 44'' and central image 48'' includes a
vertical central border portion 50a', an upper border portion 50b',
and a lower border portion 50c'. Upper border portion 50b' and
lower border portion 50c' diverge laterally outwardly, vertically
away from central portion 50a'. A border 52' between central image
portion 48'' and right image portion 46'' includes a central
boundary portion 52a', an upper boundary portion 52b', and a lower
boundary portion 52c'. Upper boundary portion 52b' and lower
boundary portion 52c' diverge laterally outwardly vertically away
from central portion 52a'. This creates an upper portion of central
image portion 48'' and a lower portion of central image portion
48'' which extend beyond the center portion thereof. This
configuration is based upon the realization that the surface of the
road immediately behind the vehicle is captured by central image
capture device 16. Likewise, the horizontal plane above the
vehicle, which is symmetrical with the road surface, is captured by
the center image capture device. This may be seen by referring to
point P in FIG. 10, which illustrate the points where the effective
radius 68 of the virtual image capture device intersects dead zones
30 and by referring to point S in FIG. 10 which illustrates the
corners or the rear of the vehicle (S).
[0046] The image displayed on display 20'' includes a dead space
66' having diverging lateral sides 68a, 68b. Diverging sides 68a
and 68b are configured in order to extend in the direction of
travel of vehicle 10 which is parallel to lane markings of a
highway on which vehicle 10 is travelling. This further enhances
the visual perception of the driver by providing a visual clue of
the location of images appearing on display 20'' with respect to
the vehicle 10. Side portions 68a, 68b, in the illustrated
embodiment, are natural extensions of lower boundary portions 50c'
and 52c' and extend from point S on each respective side of the
vehicle to point R, which represents the intersection of the lower
extent of the vertical field of view 40 of each side image capture
device 14 with the pavement (FIG. 7).
[0047] Rearview vision systems 12' and 12'' utilize a displayed
synthesized image which takes into account the use of perspective
in enhancing the driver's understanding of what is occurring in the
area surrounding the vehicle. The images produced on displays 20'
and 20'' effectively remove the vehicle bodywork and replace the
bodywork with a vehicle footprint as would be viewed by virtual
camera C. The image displayed on display 20'' further includes
perspective lines which further enhance the roll of perspective in
the driver's understanding of what is occurring.
[0048] In order to further enhance the driver's understanding of
what is occurring in the area surrounding the vehicle, a rearview
vision system 12'' includes a display 20'' having image
enhancements (FIG. 6). In the illustrative embodiment, such image
enhancements include graphic overlays 70a, 70b which are hash marks
intended to illustrate to the driver the anticipated path of
movement of vehicle 10. In the illustrated embodiment, the
anticipated vehicle motion is a function of the vehicle direction
of travel as well as the rate of turn of the vehicle. The forward
or rearward direction of vehicle travel is determined in response
to the operator placing the gear selection device (not shown) in
the reverse gear position. The degree of turn of the vehicle may be
determined by monitoring the movement of the vehicle steering
system, monitoring the output of an electronic compass, or
monitoring the vehicle differential drive system. In the embodiment
illustrated in FIG. 6, the configuration of graphic overlays 70a,
70b indicates that the vehicle is in reverse gear and that the
wheels are turned in a manner that will cause the vehicle to travel
toward the driver's side of the vehicle. If the wheels were turned
in the opposite direction, graphic overlays 70a, 70b would curve
clockwise toward the right as viewed in FIG. 6. If the vehicle's
wheels were straight, graphic overlays 70a, 70b would be
substantially straight converging lines. If the vehicle is not in
reverse gear position, graphic overlays 70a, 70b are not presented.
Other types of graphic overlays of the displayed image are
comprehended by the invention.
[0049] Horizontal grid markings on the display may be provided to
indicate distances behind the vehicle at particular markings. Such
grid would allow the driver to judge the relative position of
vehicles behind the equipped vehicle. In one embodiment, short
horizontal lines are superimposed on the displayed image at regular
rearward intervals in horizontal positions which correspond to the
boundaries of the lane in which the vehicle is travelling. In order
to avoid confusion when the vehicle is travelling in a curved path,
from a lack of correspondence between the graphic overlay and the
road, a signal indicative of the vehicle's rate of turn may be
taken into account when generating the graphic overlay. In this
manner, the distance indications may be moved laterally, with
reduced horizontal separation, to correspond to the positions of
the curved lane boundaries and vertically on the image to
compensate for the difference between distances along a straight
and curved path.
[0050] Another image enhancement is to alter the appearance of an
object in a particular zone surrounding the vehicle in order to
provide an indication, such as a warning, to the driver. As an
example, a vehicle that is too close to the equipped vehicle for
safe-lane change, may be displayed in a particular color, such as
red, may flash, or otherwise be distinguishable from other images
on the display. Preferably, the speed of the equipped vehicle 10,
which may be obtained from known speed transducers, may be provided
as an input to the rearview vision system in order to cause such
warning to be a function of the vehicle speed which, in turn,
affects the safe separation distance of vehicles. The operation of
the turn signal may also be used to activate such highlighting of
other road users or to modify the scope of the image displayed. In
order to determine the distance of objects behind vehicle 10, a
separate distance-measuring system may be used. Such separate
system may include radar, ultrasonic sensing, infrared detection,
and other known distance-measuring systems. Alternatively,
stereoscopic distance-sensing capabilities of side image capture
devices 14 may be utilized to determine the separation distance
from trailing objects utilizing known techniques.
[0051] Thus, it is seen that the image displayed on display
20-20''' may be different under different circumstances. Such
different circumstances may relate to the vehicle's direction of
travel, speed, rate of turn, separation from adjacent objects, and
the like.
[0052] Various other forms of image processing may be utilized with
rearview vision system 12-12'''. Luminant and chrominant blending
may be applied to the images captured by image capture devices 14,
16 in order to produce equality of the image data whereby the image
portions appear as if they were produced by one image capture
device. The dynamic range of the image capture devices may be
extended in order to provide high quality images under all lighting
conditions. Furthermore, individual pixel groups may be controlled
in order to selectively compensate for bright or dark spots. For
example, anti-blooming techniques may be applied for bright spots.
Multiple exposure techniques may be applied to highlight dark
areas. Image morphing and warping compensation techniques may
additionally be applied. Resolution of the image capture devices
and display may be selected in order to provide sufficient image
quality for the particular application.
[0053] A heater may be applied to each image capture device in
order to remove dew and frost that may collect on the optics of the
device. Although, in the illustrative embodiment, the optical
centerline of the camera coincides with the field of view,
particular applications may result in the centerline of the camera
pointing in a direction other than the centerline of the field of
view. Although, in the illustrative embodiment, the image capture
devices are fixed, it may be desirable to provide selective
adjustability to the image capture devices or optical paths in
particular applications. This is particularly desirable when the
system is used on articulated vehicles where automated and
coordinated camera aim may be utilized to maintain completeness of
the synthesized image.
[0054] When operating the vehicle in the reverse direction, it may
be desirable to provide additional data concerning the area
surrounding the immediate rear of the vehicle. This may be
accomplished by utilizing non-symmetrical optics for the center
image capture device in order to provide a wide angle view at a
lower portion of the field of view. Alternatively, a wide angle
optical system could be utilized with the electronic system
selectively correcting distortion of the captured image. Such
system would provide a distortion-free image while obtaining more
data, particularly in the area surrounding the back of the
vehicle.
[0055] The invention additionally comprehends the use of more than
three image capture devices. In addition to side image capture
devices positioned at the front sides of the vehicle and a center
image capture device positioned at the center rear of the vehicle,
additional image capture devices may be useful at the rear corners
of the vehicle in order to further eliminate blind spots. It may
additionally be desirable to provide an additional center image
capture device at a higher elevation in order to obtain data
immediately behind the vehicle and thereby fill in the road surface
detail immediately behind the vehicle. Such additional detail is
particularly useful when operating the vehicle in the reverse
direction. Of course, each of the image capture devices could be a
combination of two or more image capture devices.
[0056] Although the present invention is illustrated as used in a
rearview vision system, it may find utility in other applications.
For example, the invention may be useful for providing security
surveillance in an area where a building or other object obstructs
the view of the area under surveillance. Additionally, the
invention may find application in night-vision systems and the
like. For example, the invention may be applied to forward-facing
night-vision systems, or other vision enhancement systems such as
may be used in adverse weather or atmospheric conditions such as
fog, applied to provide an enhanced display of a synthesized image,
which approximates a forward-facing view from a single virtual
camera located rearwardly of the driver, taking advantage of the
perspective features of the image.
[0057] Thus, it is seen that the present invention enhances the
relationship between the driver's primary view and the image
presented on the rearview vision system. This is accomplished in a
manner which provides ease of interpretation while avoiding
confusion so that the driver does not have to concentrate or look
closely at the image. In this manner, information presented on the
display is naturally assimilated. This is accomplished while
reducing blind spots so that other vehicles or objects of interest
to the driver will likely be displayed to the driver. Additionally,
the use of perspective allows distances to be more accurately
determined.
[0058] Changes and modifications in the specifically described
embodiments can be carried out without departing from the
principles of the invention, which is intended to be limited only
by the scope of the appended claims, as interpreted according to
the principles of patent law including the doctrine of
equivalents.
* * * * *