U.S. patent application number 16/662442 was filed with the patent office on 2020-02-20 for vehicular image-display system.
The applicant listed for this patent is Kenji KAGAMI, Tomonori KIMURA, Takuroh NAITOH, Yasuhiro NOMURA, Kenji NOZAKI, Hiromitsu SHIMIZU, Kenji SHIRAISHI. Invention is credited to Kenji KAGAMI, Tomonori KIMURA, Takuroh NAITOH, Yasuhiro NOMURA, Kenji NOZAKI, Hiromitsu SHIMIZU, Kenji SHIRAISHI.
Application Number | 20200055455 16/662442 |
Document ID | / |
Family ID | 55746800 |
Filed Date | 2020-02-20 |
![](/patent/app/20200055455/US20200055455A1-20200220-D00000.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00001.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00002.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00003.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00004.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00005.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00006.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00007.png)
![](/patent/app/20200055455/US20200055455A1-20200220-D00008.png)
United States Patent
Application |
20200055455 |
Kind Code |
A1 |
NAITOH; Takuroh ; et
al. |
February 20, 2020 |
VEHICULAR IMAGE-DISPLAY SYSTEM
Abstract
A vehicular image-display system includes an image capturing
device, a control device, and an image display device. The image
capturing device includes an imaging element configured to capture
an image of surroundings of a vehicle, and an image acquirer
configured to acquire, based on the captured image captured by the
imaging element, an image that is smaller in area than the captured
image. The control device includes an image generator configured to
generate an image that is smaller in area than the acquired image
based on the acquired image acquired by the image acquirer. The
image display device is configured to display the image to be
displayed generated by the image generator. The acquired image is
transmitted from the image capturing device to the control
device.
Inventors: |
NAITOH; Takuroh; (Kanagawa,
JP) ; SHIRAISHI; Kenji; (Kanagawa, JP) ;
KAGAMI; Kenji; (Kanagawa, JP) ; KIMURA; Tomonori;
(Kanagawa, JP) ; NOZAKI; Kenji; (Kanagawa, JP)
; NOMURA; Yasuhiro; (Kanagawa, JP) ; SHIMIZU;
Hiromitsu; (Kanagawa, JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
NAITOH; Takuroh
SHIRAISHI; Kenji
KAGAMI; Kenji
KIMURA; Tomonori
NOZAKI; Kenji
NOMURA; Yasuhiro
SHIMIZU; Hiromitsu |
Kanagawa
Kanagawa
Kanagawa
Kanagawa
Kanagawa
Kanagawa
Kanagawa |
|
JP
JP
JP
JP
JP
JP
JP |
|
|
Family ID: |
55746800 |
Appl. No.: |
16/662442 |
Filed: |
October 24, 2019 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15486332 |
Apr 13, 2017 |
10486598 |
|
|
16662442 |
|
|
|
|
PCT/JP2015/079377 |
Oct 16, 2015 |
|
|
|
15486332 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 7/18 20130101; H04N
5/23293 20130101; H04N 7/183 20130101; B60R 1/006 20130101; G06T
3/40 20130101; B60R 1/00 20130101; B60R 1/04 20130101; G06F 3/013
20130101; G06K 9/00845 20130101 |
International
Class: |
B60R 1/00 20060101
B60R001/00; H04N 7/18 20060101 H04N007/18; H04N 5/232 20060101
H04N005/232; G06T 3/40 20060101 G06T003/40; G06K 9/00 20060101
G06K009/00; G06F 3/01 20060101 G06F003/01 |
Foreign Application Data
Date |
Code |
Application Number |
Oct 17, 2014 |
JP |
2014-212828 |
Claims
1. A vehicular image-display system, comprising: an imaging device
configured to capture an image of surroundings of a vehicle; first
processing circuitry configured to generate an display image that
is smaller in area than the acquired image, the display image being
generated based on motion of a driver who is driving the vehicle;
and an image display configured to display the generated display
image.
2. The vehicular image-display system according to claim 1, further
comprising second processing circuitry configured to calculate an
image position and an area of the display image, based on the
motion of the driver.
3. The vehicular image-display system according to claim 2, wherein
the second processing circuitry is included in the image capturing
device.
4. The vehicular image-display system according to claim 2, wherein
the second processing circuitry is included in the control
device
5. The vehicular image-display system according to claim 2, wherein
when the image position and the area of the display image are
determined not to fall within an area of the acquired image, the
second processing circuitry is further configured to calculate the
image position and the area of the image to fall within an area of
the acquired image.
6. The vehicular image-display system according to claim 2, wherein
the second processing circuitry is configured to analyze a
line-of-sight position of the driver, and calculate the image
position and the area of the display image device based on a result
of analysis of the line-of-sight position.
7. The vehicular image-display system according to claim 1, wherein
the imaging device is provided in the vehicle.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] The present application a continuation Application of U.S.
application Ser. No. 15/486,332, filed Apr. 13, 2017, which is a
continuation application of International Application No.
PCT/JP2015/079377, filed Oct. 16, 2015, which claims priority to
Japanese Patent Application No. 2014-212828, filed Oct. 17, 2014.
The contents of these applications are incorporated herein by
reference in their entirety.
BACKGROUND OF THE INVENTION
1. Field of the Invention
[0002] The present invention relates to a vehicular image-display
system.
2. Description of the Related Art
[0003] Vehicle-mounted cameras are increasingly widespread with
widespread of drive recorders and around-vehicle monitoring
cameras.
[0004] Meanwhile, although electronic mirrors that differ from
side-view mirrors and rear-view mirrors are being developed, due to
high technical obstacles to be overcome in terms of safety,
reliability, and performance, electronic mirrors have not gained
widespread use yet.
[0005] What is required of an electronic mirror is to continue
supplying images around a vehicle stably. To achieve this, a system
including from a camera to a display is required to perform
processing at a high resolution (high pixel density), high
contrast, high frame rate, and high speed.
[0006] A technique that, even though a captured image is used,
enables more quick and intuitive awareness of an object to be
noticed in the surroundings of own vehicle is known from, for
example, Japanese Patent No. 4941760.
SUMMARY OF THE INVENTION
[0007] According to one aspect of the present invention, a
vehicular image-display system includes an image capturing device,
a control device, and an image display device. The image capturing
device includes an imaging element configured to capture an image
of surroundings of a vehicle, and an image acquirer configured to
acquire, based on the captured image captured by the imaging
element, an image that is smaller in area than the captured image.
The control device includes an image generator configured to
generate an image that is smaller in area than the acquired image
based on the acquired image acquired by the image acquirer. The
image display device is configured to display the image to be
displayed generated by the image generator. The acquired image is
transmitted from the image capturing device to the control
device.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] FIG. 1 is an overall architecture diagram of a vehicular
image-display system according to an embodiment;
[0009] FIG. 2 is a system architecture diagram of the vehicular
image-display system according to the embodiment;
[0010] FIG. 3 is a system architecture diagram of a vehicular
image-display system as a reference comparative example for the
vehicular image-display system according to the embodiment;
[0011] FIG. 4 is a diagram illustrating areas of images used in the
vehicular image-display system according to the embodiment;
[0012] FIG. 5 is a diagram illustrating a captured-image-and-mirror
relationship in the vehicular image-display system according to the
embodiment;
[0013] FIG. 6 is a diagram describing an example of an image
position and area that are responsive to motion of a driver in the
vehicular image-display system according to the embodiment;
[0014] FIG. 7 is a diagram describing the example of the image
position and area that are responsive to the motion of the driver
in the vehicular image-display system according to the
embodiment;
[0015] FIG. 8 is a diagram describing another example of an image
position and area that are responsive to motion of the driver in
the vehicular image-display system according to the embodiment;
[0016] FIG. 9 is a diagram describing the other example of the
image position and area that are responsive to the motion of the
driver in the vehicular image-display system according to the
embodiment;
[0017] FIG. 10 is a diagram describing still another example of an
image position and area that are responsive to motion of the driver
in the vehicular image-display system according to the embodiment;
and
[0018] FIG. 11 is a diagram describing the still other example of
the image position and area that are responsive to the motion of
the driver in the vehicular image-display system according to the
embodiment.
[0019] The accompanying drawings are intended to depict exemplary
embodiments of the present invention and should not be interpreted
to limit the scope thereof. Identical or similar reference numerals
designate identical or similar components throughout the various
drawings.
DESCRIPTION OF THE EMBODIMENTS
[0020] The terminology used herein is for the purpose of describing
particular embodiments only and is not intended to be limiting of
the present invention.
[0021] As used herein, the singular forms "a", "an" and "the" are
intended to include the plural forms as well, unless the context
clearly indicates otherwise.
[0022] In describing preferred embodiments illustrated in the
drawings, specific terminology may be employed for the sake of
clarity. However, the disclosure of this patent specification is
not intended to be limited to the specific terminology so selected,
and it is to be understood that each specific element includes all
technical equivalents that have the same function, operate in a
similar manner, and achieve a similar result.
[0023] An embodiment of the present invention will be described in
detail below with reference to the drawings.
[0024] FIG. 1 is an overall architecture diagram of a vehicular
image-display system 7 according to an embodiment of the present
invention. The vehicular image-display system 7 is installed in a
vehicle 101. The vehicular image-display system 7 includes an image
capturing device 3 that captures an image of surroundings of the
own vehicle and acquires an image of a necessary area from the
thus-captured image, a control device 5 to which the acquired image
acquired by the image capturing device 3 is to be transmitted from
the image capturing device 3 via an acquired-image transmission
part 9, and an image display device 6 attached to the vehicle 101
to display a display image generated by the control device 5.
[0025] An in-vehicle camera 11 is attached to inside of the vehicle
101 and continuously captures an image of the face of a driver 10.
The image capturing device 3 analyzes information about
line-of-sight position of the driver 10 based on image data
captured by the in-vehicle camera 11 and acquires, from the
captured image, an image that is smaller in area than the captured
image based on a result of the analysis.
[0026] FIG. 2 is a system architecture diagram of the vehicular
image-display system 7 of the present embodiment.
[0027] The image capturing device 3 includes an imaging element 1
that captures an image of the surroundings of the own vehicle, an
image calculator 8 that calculates an image of a necessary area
from the captured image captured by the imaging element 1, and an
image acquirer 2 that acquires an image of the area calculated by
the image calculator 8.
[0028] The image calculator 8 analyzes information about
line-of-sight position of the driver 10 from image data captured by
the in-vehicle camera 11, calculates position and area of a
required image based on a result of the analysis, and transmits
information about the position and area to the imaging element 1
and the image acquirer 2.
[0029] The imaging element 1 captures a rear-view image of the
vehicle and transmits thus-captured image data to the image
acquirer 2.
[0030] The imaging element 1 captures an image of only a partial
image region of an entire capturable image region based on the
information about the required image position and area received
from the image calculator 8 and transmits the image to the image
acquirer 2. It is preferable that the partial image region to be
captured by the imaging element 1 covers across the entire
capturable image region in the horizontal direction but covers only
a partial image region of the entire capturable image region in the
vertical direction.
[0031] The image acquirer 2 performs trimming on the captured image
received from the imaging element 1 in accordance with the
information about the position and area of the required acquired
image received from the image calculator 8, thereby acquiring an
image that is smaller in area than the captured image. Hence, only
the image of the required position and area is transmitted via the
acquired-image transmission part 9 to the control device 5. At this
time, the "information about the position and area" calculated by
the image calculator 8 is transmitted via the acquired-image
transmission part 9 to the control device 5 together with the
image.
[0032] The control device 5 includes an image generator 4 that
generates, based on the acquired image transmitted from the image
capturing device 3, a display image to be displayed on the image
display device 6 in accordance with the information about the
required position and area of the acquired image received from the
image calculator 8.
[0033] In the vehicular image-display system 7 according to the
present embodiment, because only the acquired image, which is a
part of the entire area of the captured image captured by the image
capturing device 3, of the required position and area, which are
received from the image calculator 8, is transmitted to the control
device 5, volume of image data to be transmitted can be reduced.
Accordingly, image processing load can be reduced, and the
surroundings of the own vehicle can be displayed quickly.
[0034] FIG. 3 is a system architecture diagram of a vehicular
image-display system 27 as a reference comparative example for
describing an advantage of the present embodiment having the
configuration illustrated in FIG. 2.
[0035] In the vehicular image-display system 27, an image capturing
device 23 includes an imaging element 21 that captures surroundings
of own vehicle. Image data representing a captured image captured
by the image capturing device 23 of an entire capturable image area
is transmitted via an image transmission part 12 to a control
device 25.
[0036] The control device 25 includes an image calculator 28 and an
image generator 24. The image calculator 28 analyzes information
about line-of-sight position of the driver 10 from image data
captured by the in-vehicle camera 11, calculates a required image
position and area based on a result of the analysis, and transmits
the position and area to the image generator 24.
[0037] The image generator 24 generates, based on the image data of
the entire area transmitted from the image capturing device 23, an
image to be displayed on the image display device 6 using
information about the required position and area calculated by the
image calculator 28.
[0038] In the reference comparative example illustrated in FIG. 3,
image trimming or, more specifically, processing of acquiring the
image that is smaller in area than the captured image, is performed
by the image generator 24 on the side of the control device 25.
Accordingly, the image data captured by the imaging element 21,
inclusive of image data that is not to be used for display on the
image display device 6, is to be transmitted using the image
transmission part 12 from the image capturing device 23 to the
control device 25.
[0039] By contrast, in the present embodiment, processing of
acquiring an image that is smaller in area than a captured image is
performed by the image acquirer 2 on the side of the image
capturing device 3, by which image data representing the acquired
image of only a position and area required by the image generator 4
is obtained. Accordingly, image data transmitted via the
acquired-image transmission part 9 from the image capturing device
3 to the control device 5 can be minimized.
[0040] A camera used in a general vehicle typically uses, as an
image-data transmission part, a long harness that is several meters
long. Because noise originating from the image-data transmission
part can matter considerably, reducing volume of image data
transmitted via the image-data transmission part can lead to
considerable effectiveness. Furthermore, in the reference
comparative example of FIG. 3, image data of the entire area
captured by the image capturing device 23 is transmitted via the
image transmission part 12 to the control device 25. Accordingly,
transmission load for an image region that is not to be displayed
is wasted for no use, which inhibits quick display of the
surroundings of the own vehicle.
[0041] FIG. 4 is a diagram illustrating areas of images used in the
vehicular image-display system 7 of the present embodiment.
[0042] A captured image 15 is an image captured by the imaging
element 1 included in the image capturing device 3.
[0043] An acquired image 14 is an image of an area calculated by
the image calculator 8 included in the image capturing device 3
acquired by the image acquirer 2.
[0044] A calculated image 13 is an image position and area to be
displayed on a display unit calculated by the image calculator
8.
[0045] The image generator 4 included in the control device 5
generates, as a display image to be displayed on the image display
device 6, an image position and area corresponding to the
calculated image 13 based on the acquired image 14.
[0046] The acquired image 14 is an image that is smaller in area
than the captured image 15 and equal to or larger in area than the
calculated image 13 calculated by the image calculator 8. The
calculated image 13 is equal to or smaller in area than the
acquired image 14.
[0047] Because the acquired image 14 is smaller in area than the
captured image 15 of the entire area, the acquired image 14 can
achieve data volume reduction as compared with the captured image
15. Accordingly, transmission load for transmitting the acquired
image 14 to the control device 5 can be reduced.
[0048] Because the acquired image 14 is equal to or larger in area
than the calculated image 13 that is to be actually displayed, when
determining an appropriate display area of a display image to be
displayed on the image display device 6, the image generator 4 can
select the display area from an area having a certain extent. As a
result, the need for re-capturing an image using the image
capturing device 3 can be obviated.
[0049] FIG. 5 is a diagram illustrating a captured-image-and-mirror
relationship. When the captured image 15 captured by the image
capturing device 3 is used in place of a mirror, as illustrated in
FIG. 5, an area A defined by and with reference to a vertical
position of the captured image 15 is larger than an area B defined
by and with reference to a vertical position of an image in the
otherwise-used mirror.
[0050] FIG. 6 and FIG. 7 are diagrams describing an example of an
image position and area that are responsive to motion of a driver
in the vehicular image-display system 7 of the present embodiment.
When the driver 10 desires to see a subject indicated at a position
C in FIG. 6 and FIG. 7, the driver 10 typically takes an action D
of shifting his/her line-of-sight position as does a driver 10a as
an action of seeing rearward using the mirror. Accordingly, there
will arise a desire to see the subject indicated at the position C
in FIG. 6 and FIG. 7 by taking the action D even when the captured
image 15 captured by the image capturing device 3 is used in place
of the mirror.
[0051] When the captured image 15 captured by the image capturing
device 3 is used in place of the mirror, if an image of a position
and a large area, such as the area A defined by and with reference
to the vertical position of the image of FIG. 5, is displayed on
the image display device 6, unnecessary information having adverse
effect on safe driving can be given to the driver 10. Accordingly,
an image of an area E defined by and with reference to the position
of the image of FIG. 6 is preferably displayed on the image display
device 6 when line of sight extends from the position of the driver
10, while an image of an area F defined by and with reference to
the position of the image of FIG. 6 is preferably displayed on the
image display device 6 when line of sight extends from the position
of the driver 10a.
[0052] When line of sight extends from the position of the driver
10, the image calculator 8 analyzes information about the
line-of-sight position of the driver 10 based on image data
captured by the in-vehicle camera 11 and calculates, as a required
image area, the area E based on a result of the analysis. The image
acquirer 2 acquires the acquired image 14 containing the calculated
area E from the captured image 15. The image generator 4 generates
a display image of the area E as an image to be displayed on the
image display device 6 based on the acquired image 14 and area
information indicating the area E. Thus, when the line of sight
extends from the position of the driver 10, the image of the area E
is displayed on the image display device 6.
[0053] Thereafter, when the driver 10 shifts the line-of-sight
position as does the driver 10a, the image calculator 8 analyzes
information about the line-of-sight position of the driver 10 based
on image data captured by the in-vehicle camera 11 and calculates,
as a required image area, the area F based on a result of the
analysis. The image acquirer 2 acquires the acquired image 14
containing the calculated area F from the captured image 15. The
image generator 4 generates a display image of the area F as an
image to be displayed on the image display device 6 based on the
acquired image 14 and area information indicating the area F. Thus,
when the driver 10 shifts the line-of-sight position to the
position of the driver 10a, the image of the area F is displayed on
the image display device 6.
[0054] FIG. 8 and FIG. 9 are diagrams describing another example of
an image position and area that are responsive to motion of the
driver in the vehicular image-display system 7 of the present
embodiment. When the driver 10 desires to see a subject indicated
at a position G in FIG. 8 and FIG. 9, the driver 10 typically takes
an action I of shifting his/her line-of-sight position as does a
driver 10b as an action of seeing rearward using the mirror.
Accordingly, there will arise a desire to see the subject indicated
at the position G in FIG. 8 and FIG. 9 by taking the action I even
when the captured image 15 captured by the image capturing device 3
is used in place of the mirror.
[0055] In this case, an image of the area E defined by and with
reference to the position of the image of FIG. 8 is preferably
displayed on the image display device 6 when the line of sight
extends from the position of the driver 10, while an image of an
area H defined by and with reference to the position of the image
of FIG. 8 is preferably displayed on the image display device 6
when the line of sight extends from the position of the driver 10b.
According to the vehicular image-display system 7 of the present
embodiment, it is possible to cause the image of the area E to be
displayed on the image display device 6 when line of sight extends
from the position of the driver 10 but, when the driver 10 shifts
his/her line-of-sight position to the position of the driver 10b,
cause the image of the area H to be displayed on the image display
device 6.
[0056] FIG. 10 and FIG. 11 are diagrams describing still another
example of an image position and area that are responsive to motion
of the driver in the vehicular image-display system 7 of the
present embodiment. When the driver 10 desires to see the mirror
indicated by the area E of FIG. 10 from a closer position, the
driver 10 illustrated in FIG. 11 typically takes an action K of
shifting his/her line-of-sight position as does a driver 10c.
Accordingly, there will arise a desire to see the mirror in a
close-up manner by taking the action K even when the captured image
15 captured by the image capturing device 3 is used in place of the
mirror.
[0057] In this case, an image of the area E defined by and with
reference to the position of the image of FIG. 10 is preferably
displayed on the image display device 6 when line of sight extends
from the position of the driver 10, while an image of an area J
defined by and with reference to the position of the image of FIG.
10 is preferably displayed on the image display device 6 when line
of sight extends from the position of the driver 10c. According to
the vehicular image-display system 7 of the present embodiment, it
is possible to cause the image of the area E to be displayed on the
image display device 6 when line of sight extends from the position
of the driver 10 but, when the driver 10 shifts his/her
line-of-sight position to the position of the driver 10c, cause the
image of the area J to be displayed on the image display device
6.
[0058] It will be important that the position and area of an image
to be displayed on the image display device 6 are changed depending
on line-of-sight position of a driver in this manner when the
captured image 15 captured by the image capturing device 3 is used
in place of a mirror.
[0059] According to an embodiment, it is possible to reduce
transmission volume.
[0060] The above-described embodiments are illustrative and do not
limit the present invention. Thus, numerous additional
modifications and variations are possible in light of the above
teachings. For example, at least one element of different
illustrative and exemplary embodiments herein may be combined with
each other or substituted for each other within the scope of this
disclosure and appended claims. Further, features of components of
the embodiments, such as the number, the position, and the shape
are not limited the embodiments and thus may be preferably set. It
is therefore to be understood that within the scope of the appended
claims, the disclosure of the present invention may be practiced
otherwise than as specifically described herein.
[0061] Each of the functions of the described embodiments may be
implemented by one or more processing circuits or circuitry.
Processing circuitry includes a programmed processor, as a
processor includes circuitry. A processing circuit also includes
devices such as an application specific integrated circuit (ASIC),
digital signal processor (DSP), field programmable gate array
(FPGA) and conventional circuit components arranged to perform the
recited functions.
* * * * *