U.S. patent number 7,079,668 [Application Number 09/902,768] was granted by the patent office on 2006-07-18 for vehicle front-view monitoring system.
This patent grant is currently assigned to Fuji Jukogyo Kabushiki Kaisha. Invention is credited to Yutaka Hiwatashi, Masaru Kogure.
United States Patent |
7,079,668 |
Kogure , et al. |
July 18, 2006 |
Vehicle front-view monitoring system
Abstract
A vehicle front-view monitoring system takes fail-safe measures
when a fail-safe measure-interruption requirement using first
parameter are met on a monitored image for a predetermined first
period. The functions interrupted by the fail-safe measure is
resumed when a fail-safe measure-release requirement using second
parameter which is different from the first parameter is met within
a predetermined second period after the fail-safe
measure-interruption requirement has been met. The first period is
variable in accordance with how accurately lane marking on a road
in the monitored image is recognized.
Inventors: |
Kogure; Masaru (Mitaka,
JP), Hiwatashi; Yutaka (Mitaka, JP) |
Assignee: |
Fuji Jukogyo Kabushiki Kaisha
(JP)
|
Family
ID: |
18708032 |
Appl.
No.: |
09/902,768 |
Filed: |
July 12, 2001 |
Prior Publication Data
|
|
|
|
Document
Identifier |
Publication Date |
|
US 20020018120 A1 |
Feb 14, 2002 |
|
Foreign Application Priority Data
|
|
|
|
|
Jul 12, 2000 [JP] |
|
|
2000-212064 |
|
Current U.S.
Class: |
382/104 |
Current CPC
Class: |
B60Q
9/008 (20130101); G01S 11/12 (20130101) |
Current International
Class: |
G06K
9/00 (20060101) |
Field of
Search: |
;382/104 ;348/148 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
05-265547 |
|
Oct 1993 |
|
JP |
|
2001-092970 |
|
Apr 2001 |
|
JP |
|
Primary Examiner: Werner; Brian
Assistant Examiner: Lavin; Christopher
Attorney, Agent or Firm: Smith, Gambrell & Russell
Claims
What is claimed is:
1. A vehicle monitoring system for taking a fail-safe measure for a
fault monitoring condition on a monitored image, comprising: a
camera device provided on a vehicle for taking the monitored image;
and a controller for judging an optical irregularity occurred on
the monitored image by taking the fail-safe measure when a
fail-safe measure requirement using a first parameter is met on the
monitored image for a predetermined first period, and for
interrupting the fail-safe measure when a fail-safe measure-release
requirement using a second parameter is met within a predetermined
second period after the fail-safe measure-interruption requirement
has been met, wherein the second parameter is different from the
first parameter, and the first parameter includes at least a value
obtained by normalizing an addition of a luminance-characteristic
value on the monitored image by a shutter speed of the camera
device and a specific number of data related to luminance edges on
the monitored image and the second parameter includes at least the
shutter speed and the addition of a luminance-characteristic
value.
2. A vehicle monitoring system, comprising: a camera device
provided on a vehicle for taking an image to be monitored as the
monitored image; a processor for analyzing the image and providing
a warning to a driver and/or automatically controlling one or more
vehicle systems in response to the analyzing; and a controller for
judging an optical irregularity is interfering with an acceptable
quality of image capture based on the occurrence of a first
parameter and inhibiting the warning and/or vehicle system
automatic control in response thereto, and judging when the optical
irregularity interfering with an acceptable quality of image
capture is no longer present based on a second parameter different
from the first parameter, after such inhibiting and restoring the
ability of the processor to warn and/or automatically control one
or more vehicle systems in response to the analyzing, wherein the
first parameter includes a function of a camera shutter speed
normalized luminance-characteristic of the monitored image and a
data related to luminance edges on the monitored image and the
second parameter includes a function of shutter speed and a
luminance-characteristic.
3. The vehicle monitoring system according to claim 2, wherein each
of the first and the second parameters includes at least a
parameter related to luminance-distribution characteristics on the
monitored image but different from each other.
4. The vehicle monitoring system according to claim 3, wherein a
parameter related to the luminance-distribution characteristics and
involved in the first parameter is a value obtained by normalizing
a luminance-addition variance or the maximum addition of a
luminance on the monitored image by the shutter speed of the camera
device whereas a parameter related to the luminance-distribution
characteristics and involved in the second parameter is the
luminance-addition variance.
5. A vehicle monitoring system comprising: a camera device provided
on a vehicle for taking an image to be monitored; and a controller
for taking fail-safe measures when a fail-safe measure-interruption
requirement using a first parameter is met on a monitored image for
a predetermined first period and resuming a function interrupted by
the fail-safe measures when a fail-safe measure-release requirement
using a second parameter different from the first parameter is met
within a predetermined second period after the fail-safe
measure-interruption requirement has been met, wherein the first
parameter includes at least a value obtained by normalizing an
addition of a luminance-characteristic value on the monitored image
by a shutter speed of the camera device and a specific number of
luminance edges on the monitored image and the second parameter
includes at least the shutter speed and the addition of a
luminance-characteristic value.
6. The vehicle monitoring system according to claim 5, wherein the
first period is variable in accordance with how accurately lane
markings on a road in the monitored image are recognized.
7. A vehicle monitoring system comprising: a camera device provided
on a vehicle for taking an image to be monitored; and a controller
for taking fail-safe measures when a fail-safe measure-interruption
requirement using a first parameter is met on a monitored image for
a predetermined first period and resuming a function interrupted by
the fail-safe measures when a fail-safe measure-release requirement
using a second parameter different from the first parameter is met
within a predetermined second period after the fail-safe
measure-interruption requirement has been met, wherein the first
and the second parameters include parameters related to
luminance-distribution characteristics on the monitored image but
different from each other and a parameter related to the
luminance-distribution characteristics and involved in the first
parameter is a value obtained by normalizing a luminance-addition
variance or the maximum addition of a luminance on the monitored
image by a shutter speed of the camera device whereas a parameter
related to the luminance-distribution characteristics and involved
in the second parameter is the luminance-addition variance.
8. The vehicle monitoring system according to claim 7, wherein the
first period is variable in accordance with how accurately lane
markings on a road in the monitored image are recognized.
9. A vehicle monitoring system, comprising: a camera device
provided on a vehicle for taking an image to be monitored; a
processor for analyzing the image and providing a warning to a
driver and/or automatically controlling one or more vehicle systems
in response to the analyzing; and a controller for judging whether
an optical irregularity is interfering with an acceptable quality
of image capture based on the occurrence of a first parameter and
inhibiting the warning and/or vehicle system automatic control in
response thereto, and judging when the optical irregularity
interfering with an acceptable quality of image capture is no
longer present based on a second parameter after such inhibiting
and restoring the ability of the processor to warn and/or
automatically control one or more vehicle systems in response to
the analyzing, wherein the first period is variable in accordance
with how accurately a lane making on a road in the monitored image
is recognized; and the first parameter includes a function of a
camera shutter speed normalized luminance-characteristic of the
monitored image and a data related to luminance edges on the
monitored image and the second parameter includes a function of
shutter speed and a luminance-characteristic value.
Description
BACKGROUND OF THE INVENTION
The present invention relates to a vehicle front-view monitoring
system having a function of taking fail-safe measures against
images irregularly monitored due to sunlight interfering.
The vehicle front-view monitoring system having CCD cameras has
recently attracted much attention. The system recognizes driving
conditions such as a distance to a vehicle travelling ahead based
on images taken by the cameras and warns a driver of danger or
controls the vehicle such as by downshifts.
Recognition of the driving conditions using a stereo-distance
measuring technique obtains a positional difference (parallax) on
an object in a pair of the images and calculates the distance to
the vehicle travelling ahead by trigonometrical survey.
Such a vehicle front-view monitoring system requires fail-safe
measures for insuring safe driving. A failure to be detected by the
system is a fault monitoring condition such as when the vehicle in
front is beyond recognition while monitoring cameras are facing
weak sunlight.
Image monitoring failures are discussed with reference to FIGS. 1
to 3.
When monitoring vehicles that are traveling ahead (FIG. 1) while
facing weak sunlight, disc-like diffusion patterns could appear on
the monitored image, as illustrated on the left-upper side in FIG.
2, due to saturation of luminance. Such a disc-like diffusion
pattern appearing on the monitored image while the vehicle is
facing the sunlight is called a sunlight-diffusion disc
hereinafter. These sunlight-diffusion discs cause the saturation of
pixel luminance on the disc areas and the surrounding areas. A
correct image data therefore can not be obtained, or vehicles in
front are masked by the sunlight-diffusion discs, which results
FIG. 3 illustrates distance-data calculation based on stereo images
(a reference image and an image to be compared such as shown in
FIG. 1).
In FIG. 3, black sections have a big luminance change between the
adjacent pixels in the horizontal direction. These sections carry
distance data (depth). The intensities of most pixels in the
sunlight-diffusion disc area have been saturated, so that no
distance data is calculated in this area (a lost state). Moreover,
the disc edges have a big luminance change in the horizontal
direction, so that the incorrect distance data could be
calculated.
Insuring safe driving in such a fault condition due to a generation
of the sunlight-diffusion discs discussed above requires the
fail-safe measures to temporarily halt the monitoring function.
SUMMARY OF THE INVENTION
A purpose of the present invention is to provide a vehicle
front-view monitoring system for monitoring optical irregularities
occurring on monitored images and swiftly taking fail-safe
measures.
Another purpose of the present invention is to provide a vehicle
front-view monitoring system taking appropriate fail-safe measures
while avoiding repetition of unnecessary fail-safe measures.
The present invention provides the vehicle front-view monitoring
system for taking the fail-safe measures when a fail-safe
measure-interruption requirement using a first parameter is met on
a monitored image for a predetermined first period and resuming a
function interrupted by the fail-safe measures when a fail-safe
measure-release requirement using a second parameter different from
the first parameter is met within a predetermined second period
after the fail-safe measure-interruption requirement has been
met.
Moreover, the present invention provides a vehicle front-view
monitoring system for taking a fail-safe measure when a fail-safe
measure-interruption requirement is met on a monitored image for a
predetermined first period and resuming a function interrupted by
the fail-safe measures when a fail-safe measure-release requirement
is met within a predetermined second period after the fail-safe
measure-interruption requirement has been met, the first period
being variable in accordance with an accuracy of a lane marking on
a road in the monitored image is recognized.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 illustrates a normal image monitored on a monitor
screen;
FIG. 2 illustrates an irregular image monitored on the monitor
screen in which sunlight-diffusion discs are shown;
FIG. 3 illustrates the irregular image monitored on the monitor
screen, including inaccurate distance data due to generation of
sunlight-diffusion discs;
FIG. 4 is a block diagram of a preferred embodiment of the vehicle
front-view monitoring system according to the present
invention;
FIG. 5 shows a flow chart of a weak-sunlight-facing driving
determination routine;
FIG. 6 shows a flow chart of a weak-sunlight-facing driving
determination routine;
FIG. 7 illustrates the upper and lower areas set on the monitor
screen;
FIG. 8 illustrates an area in which a vehicle travelling ahead is
monitored;
FIG. 9 illustrates a monitoring area;
FIG. 10 illustrates measured luminance-distribution characteristic
parameters;
FIG. 11 illustrates normalized luminance-distribution
characteristic parameters;
FIGS. 12A and 12B illustrate luminance addition characteristics in
normal driving;
FIGS. 13A and 13B illustrate luminance addition characteristics in
weak-sunlight-facing driving;
FIGS. 14A to 14C illustrate luminance center;
FIG. 15 shows a flow chart of a fail determination routine;
FIG. 16 shows a flow chart of a count-up amount setting
routine;
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
Preferred embodiments according to the present invention will be
disclosed with reference to the attached drawings.
FIG. 4 shows a block diagram of a preferred embodiment of a vehicle
front-view monitoring system according to the present
invention.
A pair of cameras 1 and 2 having image sensors such as CCDs are
attached to a vehicle such as an automobile in the vicinity of
tires with a predetermined distance (corresponding to
camera-baseline length) in the direction of vehicle width, for
taking images ahead.
The main camera 1 (right camera) takes a reference image required
for stereo processing whereas the sub-camera 2 (left camera) takes
an image to be compared.
Analog images taken by the cameras 1 and 2 while the main camera 1
and sub-camera 2 are synchronized with each other are converted
into a specific luminance gradation (for example, gray scale at 256
gradations).
The digitized images are processed by an image correcting section 5
for correction of luminance, image geometrical conversion, etc. An
inevitable misalignment on the cameras 1 and 2 will affect left and
right images. Such effect can be cancelled by geometrical
conversion such as an image rotation and a parallel shift with
Affine conversion, etc. Luminance data on the stereo images (a
reference image and an image to be compared) for one frame
corrected by the image correcting section 5 are stored in an image
memory 8.
The luminance data are further supplied to a stereo-image processor
6 for calculating distance data (parallax for each object). Each
distance data corresponds to a positional deviation on the same
object in the stereo images. An actual distance to the object is
defined on the basis of parallax by a trigonometrical survey. The
distance data for one frame thus obtained are stored in a
distance-data memory 7.
The luminance data and the distance data are supplied to a
microcomputer 9.
The microcomputer 9 includes a road-recognizing section 10, a
three-dimensional (3-D) object-recognizing section 11, a
fail-determining section 12 and a processor 13.
In response to the luminance and distance data, the
road-recognizing section 10 recognizes the condition of a road
ahead, and the 3-D object-recognizing section 11 recognizes
three-dimensional objects (vehicles travelling ahead).
Based on the data from the sections 10 and 11, the processor 13
warns the driver of a danger via a warming device 19 such as a
display or a speaker when it determines that the warning is
necessary.
Moreover, the processor 13 controls several controllers 14 to 18 if
necessary. For instance, it sends a command signal to an automatic
transmission (AT) controller 14 for downshifts or to an engine
controller 18 for an engine power-down. The processor 13 may
further sends the command signals to an antilock-braking system
(ABS) controller 15, a traction control system (TCS) controller 16
and a vehicle controller 17 for controlling an engine speed and a
torque distribution to tires, and etc.
The failure-determining section 12 determines whether the cameras 1
and 2 are taking the images into the sun and a failure is
occurring, based on the luminance and the distance data stored in
the image memory 8. The vehicle and the warning controls are
interrupted by the fail-safe measures for prevention of malfunction
which could otherwise occur due to lowering in monitoring accuracy
during a period of determination in that a failure is occurring, or
of "1" set for a failure flag NG which will be described later.
FIGS. 5 and 6 show flowcharts for indicating a routine of
determining whether the cameras 1 and 2 are taking images into the
sun while the vehicle is facing the sunlight, which is called a
sunlight-facing driving hereinafter. The routine is repeated by the
failure-determining section 12 by a predetermined interval (a cycle
of 100 ms for example).
The requirements for taking the fail-safe measures (called
fail-safe measure-interruption requirements hereinafter) are
discussed before explaining the routine in detail.
Requirements for steps S1 to S6 are necessary for taking the
fail-safe measures in the present invention. It is determined that
the cameras 1 and 2 are taking the images into the sun only when
all the fail-safe measure-interruption requirements are met in
steps S1 to S6 and the routine proceeds to step S12 whereas to step
S7 if not met in any one of steps S1 to S6.
As disclosed later, the failure determining section 12 determines
whether the fail-safe measure-interruption requirement or a
fail-safe measure-release requirement is met and performs counter
increment or decrement in accordance with a result of
determination. The fail-safe measure-release requirements are the
requirements for resuming functions that have been interrupted by
taking the fail-safe measures.
The interruption requirements are required for determining whether
the cameras 1 and 2 are taking images into the sun, so that the
fail-safe measures should be taken. A counter increment is
performed when an interruption requirement is met. The fail-safe
measures will never be taken before the counter reaches a
predetermined value, even if the interruption requirements are
met.
The release requirements are required for determining whether to
halt the counter increment. In the present invention, the counter
starts a count-up when an interruption requirement is met and
continues until a release requirement is met even though monitoring
situation is changing and no interruption requirement is met
anymore.
The present invention deploys the following six interruption
requirements for taking fail-safe measures, however, any
combination of at least two of the requirements or addition of
other requirements are available.
First Requirement: No vehicle is recognized ahead, or no distance
to a vehicle in front is calculated (step S1).
The fail-safe measures will be taken in the present invention when
the cameras 1 and 2 are taking the images into the sun
(sunlight-facing driving condition) so that the vehicle that is
actually traveling ahead is lost on a monitor screen of the
monitoring system. In other words, the fail-safe measures will not
be taken even though the sunlight-diffusion discs shown in FIG. 2
are appearing when the vehicle traveling ahead is recognized.
The present invention therefore deploys a decision in that no
vehicles are recognized ahead as a fail-safe measure-interruption
requirement. This decision step is executed even though no vehicles
actually exist ahead.
When the requirement in step S1 (FIG. 5) is met, the routine
proceeds to step S2 to determine whether the cameras 1 and 2 are
taking the images into the sun (sunlight-facing driving
condition).
Calculation of the distance to the vehicle in front in step S1 is
explained before disclosing the step S2.
The distance to the vehicle in front is calculated in combination
of a stereo-distance measuring technique and an image-recognition
technique.
Calculated first is a lane model that indicates a three-dimensional
shape of a road ahead. Lane markings painted on the left and right
sides of a lane and displayed on the monitor screen have the
luminance higher than that of the road surface. The markings are
extracted as areas having luminance edges on the borders between
that and the road surface.
The parallax on the lane thus extracted is calculated for each
(4.times.4)-pixel block, for example. A two-dimensional position
and the parallax in the monitored image for one pixel block in
which the lane is displayed provide a three-dimensional position
and a distance of the lane portion by using known coordinate
conversion formulas.
The calculated three-dimensional lane distribution is approximated
into a straight line for a predetermined segment. The straight
lines are connected into a polygonal line for calculating the lane
model.
The vehicle travelling ahead is recognized as a three-dimensional
object existing on the area (road surface) surrounded by left and
right lane models. The distance to the vehicle in front is
calculated based on the parallax of the object.
These techniques are disclosed in detail in Japanese Patent
Application No. 11-269578 (road recognition) and Japanese
Unexamined Patent Publication No. 5-265547 (recognition of vehicle
traveling ahead).
Second Requirement: A shutter speed "a" is equal to or lower than a
predetermined reference speed (step S2).
The shutter speed tends to be higher in a sunlight-facing driving
condition where the vehicle in front is not visible on the monitor
screen than that in a usual driving condition. In view of this
tendency, an appropriate reference shutter speed such as 2000
.mu.sec is set.
The present invention deploys a decision in that the shutter speed
"a" is lower than this reference shutter speed as a fail-safe
measure-interruption requirement.
The shutter speed, however, tends to be extremely higher against a
strong sunlight in mid-summer, for example 200 .mu.sec. or lower.
In view of this tendency, the routine proceeds to step S3 (FIG. 5)
when the requirements in steps S1 and S2 are both met.
Third Requirement: The upper luminance saturation factor LR1 is
larger than the lower luminance saturation factor LR2 (step
S3).
Sunlight-diffusion discs images shown in FIG. 2 will be generated
on the upper side of the monitor screen when a vehicle in front is
not visible during sunlight-facing driving. The tendency for this
phenomenon is because the luminance saturates on the upper
side.
In view of this tendency, as illustrated in FIG. 7, the upper and
the lower areas are provided in a reference image.
The present invention deploys the decision in that the upper
luminance saturation factor LR1 in the upper area is larger than
that of the lower luminance saturation factor LR2 in the lower area
as a fail-safe measure-interruption requirement.
The luminance saturation factors LR1 and LR2 correspond to a ratio
of the number of pixels for which luminance has saturated (for
example, the luminance 240 or more) to the total number of pixels
in the upper and the lower areas, respectively.
The requirement in step S3 could be met even though no
sunlight-diffusion discs (FIG. 2) are generated when there is no
shielding on a fair day or while travelling on a bridge. In view of
this, the routine proceeds to step S4 (FIG. 5) when the
requirements in steps S1, S2 and S3 are all met.
No reference values are set for the upper and lower luminance
saturation factors LR1 and LR2 in this embodiment. This is because
factors tend to vary very largely according to the driving
conditions. Particularly, the lower factor LR2 often becomes zero
in a normal driving condition.
Fourth Requirement: The number of luminance edges EN for an area of
the vehicle in front is smaller than a predetermined reference
number (step S4).
While the vehicle travelling ahead is not visible on the monitor
screen during the sunlight-facing driving, the number of the
luminance edges EN on the sunlight-diffusion disc area (FIG. 2)
tends to become extremely small.
The number of the luminance edges EN indicates the number of a pair
of adjacent pixels in the horizontal direction on the monitor
screen for which variation in luminance is more than a
predetermined reference value.
As illustrated in FIG. 8, a vehicle in front area is set in the
upper-mid section of the monitor screen, in which the vehicle
travelling ahead would be displayed.
The present invention deploys the decision in that the number of
the luminance edges EN in the vehicle in front area (FIG. 5) is
smaller than an appropriate reference number as a fail-safe
measure-interruption requirement.
This requirement could, however, be met in driving at night, in
tunnel, mist, smoke or on a snowy road. In view of this, the
routine proceeds to step S5 (FIG. 5) when the requirements in steps
S1, S2, S3 and S4 are all met.
The number of the calculation of the distance data in the vehicle
in front area can be used instead of the number of the luminance
edges EN. This is because the number of the luminance edges EN and
the number of the calculation of the distance data have a strong
correlation in a stereo-matching, or the larger the number of the
luminance edges EN, the larger the number of the calculation of the
distance data.
Fifth Requirement: The luminance-distribution-characteristics
parameter normalized by using a shutter speed "a" meets a
requirement.
As shown in FIG. 9, several slit-like monitoring areas Ni are
arranged on the upper side of the monitor screen in the horizontal
direction. The reason for the arrangement of monitoring areas Ni on
the upper side is that the luminance mostly varies on the upper
side when the vehicle is facing a weak sunlight.
The monitoring areas Ni are provided for evaluating the
luminance-distribution characteristics for a monitored image in the
horizontal direction.
For example, a reference image is divided into blocks of 16 pixels
each in the horizontal direction when the reference image is
composed of 200 and 512 pixels in the vertical and horizontal
directions, respectively. Therefore, 32 monitoring areas Ni of
(200.times.16) pixels are set.
Calculated next is addition of luminance Ai for each monitoring
area Ni. The addition of luminance Ai for a monitoring area Ni can
be calculated by adding the luminance of sample pixels that are
uniformly distributed over the monitoring area Ni. The added
luminance of the sample pixels may further be averaged.
The addition of luminance. Ai is the value, for example, obtained
by adding the luminance of 4 pixels sampled in the horizontal and
vertical directions each. Pixels uniformly distributed over the
monitoring area Ni can be used for calculating the luminance over
the area Ni with less amount of calculation. The addition of
luminance Ai can be calculated for all pixels involved in the area
Ni if the amount of calculation is out of question.
The Ai-distribution characteristics for the 32 monitoring areas
corresponds to the image distribution characteristics in the
horizontal (lateral) direction.
The parameter (luminance-distribution characteristic value) that
indicates luminance-distribution characteristics on the addition of
luminance Ai is normalized by a shutter speed "a". Used here as the
parameter is luminance-addition variance VAR that indicates a
variance in the addition of luminance Ai. Luminance-addition
variance VAR' normalized by the shutter speed "a", called a
normalized luminance-addition variance VAR', is calculated by the
following equations: VAR'=VAR/a
VAR.sup.2={(A1-A.sub.ave.mu.).sup.2+(A2-A.sub.ave).sup.2+ . . .
(An-A.sub.ave).sup.2}/n=.SIGMA.(Ai.sup.2)/n-(.SIGMA.Ai/n).sup.2
(1)
The normalized luminance-addition variance VAR' as the
luminance-distribution-characteristics parameter requires a
decision in that the VAR' is larger than an appropriate reference
value as of fail-safe measure-interruption requirement.
Instead of the VAR', the maximum addition of luminance Amax that
indicates the maximum (peak) of the addition of luminance Ai can be
used as the luminance-distribution-characteristics parameter.
The Amax requires a decision in that the maximum addition of
luminance Amax' normalized by the shutter speed "a", called
normalized maximum addition of luminance Amax', is larger than an
appropriate reference value as a fail-safe measure-interruption
requirement.
Another decision in that both the normalized VAR' and the
normalized Amax' are larger than the corresponding appropriate
reference values can be used as a fail-safe measure-interruption
requirement.
The reason for using the normalized VAR' and Amax' for
determination of whether the vehicle is facing the weak sunlight is
discussed with reference to FIGS. 10 and 11.
FIG. 10 shows the un-normalized characteristics of the
luminance-distribution-characteristics parameters (VAR and Amax)
that were measured in an actual driving test. Images were taken
into the weak sunlight from about 300 sec. to about 340 sec. after
the start of measurement.
The measurement of Amax teaches that Amax was measured in a fixed
range through the test and hence it is difficult to find out an
extreme tendency on a weak-sunlight-facing driving.
The measurement of VAR shows a peak on about 270 sec. after the
start of measurement, which appeared while the vehicle was not
facing the sun, and hence it is also difficult to find out an
extreme tendency on the weak-sunlight-facing driving condition.
FIG. 11 shows the normalized characteristics of the
luminance-distribution characteristics parameters (VAR' and
Amax').
Compared to the test results shown in FIG. 10, the results in FIG.
11 show an extreme tendency in which both VAR' and Amax' are
extremely increased while the vehicle is facing the weak
sunlight.
It is therefore apparent that these parameters VAR' and Amax' can
be used effectively for determining whether the vehicle is facing
the weak sunlight.
Accordingly, the present invention employs the parameters Amax' and
VAR' that are obtained by normalizing the parameters Amax and VAR
with a shutter speed "a" for determination of the weak
sunlight-facing driving in view of the fact that Amax and VAR are
the functions of the shutter speed "a".
The decision in that both Amax' and VAR' are larger than the
corresponding reference values as a fail-safe measure-interruption
requirement further enhances accuracy of determination on fail-safe
measurements.
The shutter speed, however, tends to be higher on a fair day, and
hence the routine proceeds to step S6 (FIG. 5) when the
requirements in steps S1, S2, S3, S4 and S5 are all met.
Six requirement: Division of the luminance moment by the total
addition of the luminance is equal to or lower than a reference
value (step S6). Step S6 evaluates the distribution of addition of
luminance Ai for each monitoring area Ni from the point of view
which is different from step S5.
FIGS. 12A and 12B show the distribution characteristics of the
addition of luminance Ai in regular conditions.
In detail, FIG. 12A shows an example of the distribution
characteristics for a driving condition such that a vehicle is
entering a freeway, so that the sky is displayed all over the upper
section of the monitor screen.
FIG. 12B shows another example of the distribution characteristics
for a driving condition such that the vehicle is travelling on a
road uncovered with snow, which is displayed in the mid section of
the monitoring screen, while the ground besides the road has been
covered with snow.
On the other hand, FIGS. 10A and 10B show the distribution
characteristics of the addition of luminance Ai while the cameras 1
and 2 is facing a weak sunlight during the sunlight-facing
driving.
The weak-sunlight-facing driving results in a tendency in that high
luminance portions being converged on one section corresponding to
the sunlight-diffusion disc (FIG. 2) of a monitored image in the
horizontal direction. This is shown in FIG. 13A in which the high
luminance portions are converged on the mid section of the monitor
screen. FIG. 13B shows high luminance portions converged on the
right section of the monitor screen.
Step S6 determines whether the present driving condition is a
regular one such as shown in FIG. 12A or 12B or the
weak-sunlight-facing driving such as shown in FIG. 13A or 13B.
For this determination, a luminance center LC and a luminance
moment LM are introduced as defined in the following equations:
LC=.SIGMA.(i.times.Ai)/.SIGMA.Ai LM=.SIGMA.(|i-LC|.times.Ai)
(2)
where "i" indicates a monitoring area Ni-number (0 to 31) and Ai
indicates an addition of the luminance Ai in the i-th monitoring
area Ni, like in step S5.
The luminance center LC is defined as a horizontal position at
which the addition of luminance Ai in the horizontal direction are
converged. The LC is obtained by dividing the total of the product
of Ai.times."i"(=horizontal position) by the total of Ai.
For example, when all of the addition of luminance Ai are the same
value as shown in FIG. 14A, the luminance center LC is 15.5 that
lies between 0 and 31. Moreover, when the addition of luminance A0
to A15 are all zero and A16 to A31 are the same value (.noteq.0),
the luminance center LC is 23.5.
The luminance moment LM is defined as the total of the product of a
distance from the luminance center LC to a horizontal position "i"
times the addition of luminance Ai. The moment LM indicates the
distribution of the addition of luminance Ai in the horizontal
direction on the basis of the luminance center LC. The luminance
moment LM becomes small when the addition of luminance Ai is small
in overall or the luminance is converged in the vicinity of the
luminance center LC.
Calculated next is the value obtained by dividing the luminance
moment LC by the total of addition of luminance
.SIGMA. Ai, or luminance moment LM' that is normalized by .SIGMA.
Ai as indicated by the following equation:
.times.'.times..times..times. ##EQU00001##
In normal driving at the distribution characteristics such as shown
in FIG. 14A, the luminance moment LM tends to become large because
luminance are distributed all over in the horizontal direction.
On the other hand, in the sunlight-facing driving at the
distribution characteristics such as shown in FIG. 14C, the
luminance moment LM tends to become small because the luminances
are converged in the vicinity of the luminance center LC.
However, it is sometimes difficult to discriminate the
sunlight-facing driving condition from a normal driving condition
because the luminance moment LM becomes large for a bright image
even in sunlight-facing driving.
In view of this, the normalized luminance moment LM' that is
obtained by dividing the luminance moment LM by the total of
addition of luminance Ai and that depends only on luminance
distribution is used for the determination in step S6.
The normalized luminance moment LM' has a tendency of becoming
large for the distribution characteristics shown in FIGS. 12A and
12B but small in FIGS. 13A and 13B.
In view of this, step S6 compares the normalized luminance moment
the LM' with an appropriate reference value for detecting the
weak-sunlight-facing driving condition that is shown in FIGS. 13A
and 13B.
The equation (3) can be modified for a large weight of the addition
of luminance Ai for evaluation of Ai distribution as follows:
LM'=.SIGMA.(|i-LC|.times.Ai.sup.2)/.SIGMA.Ai
LC=.SIGMA.(i.times.Ai.sup.2)/.SIGMA.Ai (4)
In FIG. 5, when all the requirements in steps S1 to S6 are met, or
all the fail-safe measure-interruption requirements are met, it is
determined that the vehicle is facing a weak sunlight. A count flag
Fcnt is then set at "1" (step S12). The count flag Fcnt is set at
"0" when a counter K reaches the lower limit value 0 whereas "1"
while the counter K is counting up or down (K.noteq.0).
The counter K is set at a count value a (>0) in the following
step S13.
The counter K is used for a failure determining routine such as
shown in FIG. 15 for determining whether to take fail-safe
measures.
When the counter K reaches the upper limit value Klim (step S30)
due to increment in the weak-sunlight-facing driving condition
determining routine, a failure flag NG is set at "1" (step S31). On
the other hand, when the counter K reaches the lower limit value 0
(step S32), the count flag Fcnt and the failure flag NG are both
reset at "0" (steps S33 and S34).
Moreover, when the counter K is in the range from the lower limit
value 0 to the upper limit value Klim, the routine ends via steps
S30 and S32 with no change in the count flag Fcnt and the failure
flag NG.
While a front-view monitoring is interrupted by the fail-safe
measures (NG=1), it will not be resumed (fail-safe measures will
not be released) unless the counter K reaches the lower limit value
0.
On the other hand, while the front-view monitoring is performed
(NG=0), it will not be interrupted by the fail-safe measures unless
the counter K reaches the upper lower limit value Kim.
The counter K starts the count-up when a fail-safe
measure-interruption requirement is met. The failure flag NG is set
at "1" when such a driving condition continues for a predetermined
period (decided based on the upper limit value Klim and the
count-up amount .alpha.) for taking the fail-safe measures.
Monitoring is thus interrupted for avoiding a decrease in
monitoring control accuracy in the weak-sunlight-facing
driving.
The count-up amount .alpha. is decided according to a count-up
amount setting routine shown in FIG. 6. The amount .alpha. is
decided in accordance with a distance variable and a difference in
luminance in left and right cameras (step S20).
Usually, cameras are facing the sunlight in the sunlight-facing
driving condition and hence the variation in calculation of the
distance tends to be larger compared to a normal driving
condition.
In view of such a tendency, the present invention obtains the
distance variable in a predetermined area in a monitored image, for
example, an area extending in the vertical direction in the image.
The count-up amount .alpha. is set at a large one when the variable
is large. It is also set at a large amount in the sunlight-facing
driving condition due to a larger luminance difference between the
left and right cameras compared to the normal driving
condition.
On the other hand, it is determined that the vehicle (the cameras 1
and 2) is not facing the sunlight if any one (or more) of the
fail-safe measure-interruption requirements is (or are) not met
through steps S1 to S6 in FIG. 5.
The routine then proceeds to step S3 in FIG. 6 to determine whether
the count flag Fcnt has been set at "1", or the counter K has been
counting up or down in step S7. If it is determined as K=0, the
routines will not proceed anymore and ends.
As disclosed, monitoring is continued without taking any fail-safe
measurements during the normal driving condition without facing
sunlight and the count K having reached the lower limit value 0, or
the failure flag NG having been set at "0".
On the contrary, when any one (or more) of the fail-safe
measure-interruption requirements is (or are) not met through steps
S1 to S6, but it is "YES" (Fcnt=1, K.noteq.0) in step S7, the
routine proceeds to step S8.
Steps S8 to S10 define fail-safe measure-release requirements for
the counter K to count up or down to resume functions that have
been interrupted by the fail-safe measures. The counter K will
count down when a fail-safe measure-release requirement is met but
will count up when a requirement is not met.
The fail-safe measure-release requirements are not so severe than
the fail-safe measure-interruption requirements for avoiding
repetition of the fail-safe measures when it is repeated that the
vehicle travelling ahead is lost and appeared on the monitoring
screen.
As one of the fail-safe measure-release requirements, it is
determined in step S8 whether the shutter speed "a" is 3000
.mu.sec. or lower. If not, or this fail-safe measure-release
requirement is met, the routine proceeds to step S11 in which the
counter K counts down by a count-down amount .beta..
On the other hand, if "YES" in step S8, the routine proceeds to
step S9 to determine whether the luminance-addition variance VAR is
equal to or larger than the reference value.
The VAR tends to be large in the sunlight-facing driving condition
because, particularly, when facing a strong sunlight,
luminance-saturated sections appear on the monitor screen due to
smears or sunlight-diffusion discs (FIG. 2) and also blackout
sections.
However, VAR tends to be large not only in the sunlight-facing
driving, but also, for instance, in a sunny day, due to a strong
contrast between sunny and shade places. It tends to be large,
particularly, there is the shade of guard rails or roadside trees
along the road.
In view of such a tendency, the present invention deploys the
luminance-addition variance VAR that will not vary so much in the
weak-sunlight-facing driving compared to the normalized
luminance-addition variance VAR', for the fail-safe measure-release
requirements less severer than the fail-safe measure-interruption
requirements, to avoid the repetition of fail-safe measures
discussed above.
If "YES" in step S9 (FIG. 6), the routine proceeds to step S13
(FIG. 5) in which the counter K starts count-up. On the other hand,
if "NO" in step S9, the routine proceeds to step S10 to determine
whether lane markings have been accurately recognized. Accuracy on
a lane marking recognition can be used as a fail-safe
measure-release requirement for avoiding the repetition of the
fail-safe measures discussed above. This is because the road
surface will be mostly correctly displayed on the monitor screen
and hence the lane marking recognition accuracy will not be lowered
so much even in the weak-sunlight-facing driving condition.
If the lane marking recognition accuracy has been lowered in step
S10, the routine proceeds to step S13 (FIG. 5) in which the counter
K starts count-up.
On the other hand, if the lane marking recognition accuracy is not
lowered in step S10, the routine proceeds to step S11 in which the
counter K starts count-down.
The lane marking recognition accuracy can be determined as
disclosed in Japanese Patent Application No. 11-269578. The
accuracy of lane markings detected on a monitored image is
determined based on the number of edges of luminance on the lane
markings, the positional relationship (continuity) to the lane
markings detected previously and parallelism between left and right
lane markings. High lane marking recognition accuracy is achieved
as the number of edges of luminance is larger, the lane markings
detected in the present image more continues to the previously
detected markings and the detected left and right markings are more
parallel to each other.
As clearly understood from the foregoing disclosure, the fail-safe
measure-release requirements (count-down requirements for the
counter K) are as follows:
(1) the shutter speed "a" is higher than 3000 .mu.se., or
(2) if not, the luminance-addition variance VAR is smaller than the
reference value and lane marking recognition is accurate.
As disclosed, the present invention monitors parameters such as the
luminance-distribution characteristics parameter, the number of
edges of luminance, the upper and lower luminance saturation
factors and the luminance moment for detection of a
weak-sunlight-facing driving condition condition.
The fail-safe measures can therefore be taken exactly during the
weak-sunlight-facing driving condition, thus offering a highly
reliable vehicle front-view monitoring system.
Particularly, in the present invention, several monitoring areas
are set on the monitor screen, which are slit-like areas extending
vertically on a monitored image for obtaining the
luminance-distribution characteristic values (the
luminance-addition variance VAR and the maximum addition of
luminance Amax) in the horizontal direction of the image.
The luminance-distribution characteristic values are normalized
using a shutter speed "a" for obtaining parameters (the
luminance-addition variance VAR' and the normalized maximum
addition of luminance Amax') that are highly related to
weak-sunlight-facing driving condition.
The normalized parameters VAR' and Amax' can be used for detection
of a weak-sunlight-facing condition, which is more accurate than
using the un-normalized parameters VAR and Amax as the fail-safe
measure-interruption requirements.
The fail-safe measure-release requirements are set as including
several conditions in addition to a weak-sunlight-facing driving
condition so that it is not as severe as those of the fail-safe
measure-interruption requirements.
This requirement setting avoids the repetition of unnecessary
fail-safe measures such as the warning which could irritate the
driver while it is repeated that the vehicle travelling ahead is
not visible and appeared on the monitoring screen.
The fail-safe measure-interruption requirements include the
normalized Luminance-addition characteristic values VAR' and Amax'
whereas the fail-safe measure-release requirements include VAR and
Amax, the parameters different from VAR' and Amax because those are
not normalized.
Moreover, the fail-safe measure-release requirements include the
accuracy on the lane marking recognition that is not deployed for
the fail-safe measure-interruption requirements.
Deploying different parameters for the fail-safe
measure-interruption/release requirements offers flexible fail-safe
requirement settings that match an actual driving condition
compared to setting in which reference values are changed between
the interruption/release requirements for the same parameters.
The count-up amount .alpha. is adjusted based on the luminance
difference between the left and right cameras and distance data
variable.
This adjustment offers variable settings for a fail-safe
determination time from a moment at which the fail-safe
measure-interruption requirements are met to a moment of taking
fail-safe measures in accordance with actual driving
conditions.
Disclosed so far is the vehicle front-view monitoring system that
takes fail-safe measures against a weak-sunlight-driving condition
in which the vehicle is travelling facing the weak sunlight so that
sunlight-diffusion discs appear on the monitor screen.
However, the present invention is applicable to not only such a
weak-sunlight-driving condition but also another optical phenomenon
such as smear occurring on a monitoring screen.
As disclosed, according to the present invention, optical
irregularities occurring on the monitored images due to optical
interfering can be accurately detected, thus, the fail-safe
measures can be taken swiftly in response to such an optical
phenomenon for insuring safe driving.
Moreover, different parameters for the fail-safe
measure-interruption/release requirements offer flexible fail-safe
requirement settings that match actual driving conditions.
It is further understood by those skilled in the art that the
foregoing description is a preferred embodiment of the disclosed
device and that various change and modification may be made in the
invention without departing from the spirit and scope thereof.
* * * * *