U.S. patent application number 15/751327 was filed with the patent office on 2018-08-16 for billboard display and method for selectively displaying advertisements by sensing demographic information of occupants of vehicles.
The applicant listed for this patent is Robert Bosch GmbH. Invention is credited to Patrick GRAF, James Stephen MILLER, Frank RIGGI.
Application Number | 20180232770 15/751327 |
Document ID | / |
Family ID | 56926335 |
Filed Date | 2018-08-16 |
United States Patent
Application |
20180232770 |
Kind Code |
A1 |
MILLER; James Stephen ; et
al. |
August 16, 2018 |
BILLBOARD DISPLAY AND METHOD FOR SELECTIVELY DISPLAYING
ADVERTISEMENTS BY SENSING DEMOGRAPHIC INFORMATION OF OCCUPANTS OF
VEHICLES
Abstract
A billboard display system selectively displays advertising to
occupants in one or more vehicles approaching a billboard display
located near a roadway. The billboard display system receives each
of: a number of occupants, demographic information for occupants,
and vehicle position information from interior occupant sensing
systems of one or more vehicles approaching the billboard display.
A display controller of the billboard display system processes the
demographic information and number of occupants to determine
advertising relevant to occupants, and selectively displays
relevant advertising with the billboard display to occupants of
approaching vehicles.
Inventors: |
MILLER; James Stephen;
(Dexter, MI) ; GRAF; Patrick; (Southfield, MI)
; RIGGI; Frank; (Plymouth, MI) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Robert Bosch GmbH |
Stuttgart |
|
DE |
|
|
Family ID: |
56926335 |
Appl. No.: |
15/751327 |
Filed: |
September 2, 2016 |
PCT Filed: |
September 2, 2016 |
PCT NO: |
PCT/US2016/050106 |
371 Date: |
February 8, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62214416 |
Sep 4, 2015 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06Q 30/0266 20130101;
G06Q 30/0645 20130101; G06Q 30/0242 20130101; G05D 1/0214 20130101;
G06K 9/00228 20130101; G05D 1/0088 20130101; G06K 9/00838 20130101;
G06K 2009/00322 20130101; G06Q 30/0269 20130101 |
International
Class: |
G06Q 30/02 20060101
G06Q030/02 |
Claims
1. A method for selectively displaying advertisements with a
billboard display system to occupants in one or more vehicles
approaching a billboard display located near a roadway, comprising:
receiving a number of occupants, demographic information of
occupants, and vehicle position information from an interior
occupant sensing system for one or more vehicles approaching the
billboard display; processing the demographic information and the
number of occupants to determine advertising relevant to occupants;
and selectively displaying relevant advertising with the billboard
display to one or more vehicles approaching the billboard
display.
2. The method according to claim 1, wherein the receiving of the
number of occupants, the demographic information, and the vehicle
position information is received by a communication unit of the
billboard display system by wireless transmission from the interior
occupant sensing system of one or more vehicles.
3. The method according to claim 2, wherein the number of occupants
and the demographic information is provided for each vehicle, by
the interior occupant sensing system: operating at least one video
camera disposed in the vehicle to sense video data including faces
of occupants in the vehicle; providing the video data to an
electronic control unit disposed in the vehicle; detecting the
number of occupants disposed in the vehicle from the video data;
determining the demographic information from the video data of each
of the faces; tracking the faces to determine emotions of
occupants; and storing the number of occupants, the demographic
information and the emotions of occupants in the electronic control
unit.
4. The method according to claim 3, including for the interior
occupant sensing system for each of one or more vehicles: receiving
a GPS signal for determining the vehicle position information of a
vehicle; determining the vehicle position information with the GPS
signal; and wherein the wireless transmission includes transmitting
the vehicle position information, the number of occupants, the
demographic information, the emotions of occupants and the vehicle
position information for each of the one or more vehicles to the
billboard display system via vehicle-to-everything (V2X)
communications.
5. The method according to claim 3, including sensing the emotions
of occupants in response to the advertisement displayed on the
billboard display to determine reactions to the advertising, and
storing the reactions to the advertising.
6. The method according to claim 3, including, with the interior
occupant sensing system for each of one or more vehicles: detecting
sound in a vehicle interior with a microphone as audio data;
providing the audio data to the electronic control unit; processing
the audio data to further determine the emotions of occupants and
storing the emotions; wirelessly transmitting the emotions for
occupants to the display controller, and analyzing effectiveness of
the selectively controlled advertising in response to the emotions
of occupants.
7. The method according to claim 1, wherein the demographic
information and the number of occupants is stored as metadata.
8. A billboard display system for providing advertising on a
billboard display to occupants in one or more vehicles in response
to operation of an interior occupant sensing system in one or more
vehicles to obtain demographic information of occupants,
comprising: the billboard display located near a roadway; a display
controller including a processor and a memory; a communication unit
for providing communication between the display controller and the
interior occupant sensing system of one or more vehicles on a
roadway; wherein the processor of the display controller is
configured to: receive via the communication unit, a number of
occupants disposed in one or more vehicles, the demographic
information for each occupant disposed in one or more vehicles, and
vehicle position information from an interior occupant sensing
system that includes a video camera disposed in each of the one or
more vehicles; and in response to at least the demographic
information and the vehicle position information, provide an output
to the billboard display to selectively display advertising that is
targeted to the demographic information of occupants of the one or
more vehicles.
9. The billboard display system according to claim 8, wherein the
processor of the display controller is configured to: receive via
the communication unit, emotive responses of occupants, and store
the number of occupants, the demographic information, and the
emotive responses of occupants, wherein the vehicle position
information indicates where the corresponding vehicle is located on
a roadway relative to the billboard display.
10. The billboard display system according to claim 9, wherein the
number of occupants, the demographic information, and the emotive
responses of occupants are stored as metadata.
11. The billboard display system according to claim 9, wherein the
number of occupants, the demographic information and the emotive
responses of occupants are determined by the video camera that
tracks faces of occupants and an electronic control unit of the
interior occupant sensing system that processes video data from the
video camera.
12. The billboard display system according to claim 9, wherein the
display controller is configured to communicate the demographic
information and the emotive responses to a cloud server for
analyzing effectiveness of the selectively controlled
advertising.
13. A method for selectively displaying advertisements on a
billboard display of a billboard display system to occupants in one
or more vehicles approaching the billboard display located near a
roadway, comprising: in an interior occupant sensing system of a
vehicle approaching the billboard display, operating a video camera
to obtain video data including faces of occupants in a vehicle;
providing the video data to an electronic control unit; sensing a
GPS signal and determining a vehicle position information of a
vehicle; detecting a number of occupants disposed in a vehicle from
the video data; determining demographic information from the video
data of the faces of occupants; storing the vehicle position
information, the number of occupants, and the demographic
information of occupants in a memory of the interior occupant
sensing system; and wirelessly transmitting the number of
occupants, the demographic information, and the vehicle position
information; and in a display controller of the billboard display
system, receiving the number of occupants, the demographic
information of occupants, and the vehicle position information
wirelessly transmitted by the interior occupant sensing system for
a vehicle approaching the billboard display; processing the
demographic information and the number of occupants to determine
advertising relevant to occupants; and selectively displaying
relevant advertising on the billboard display to occupants in an
approaching vehicle.
14. The method according to claim 13, further including the
interior occupant sensing system of a vehicle approaching the
billboard display: providing trip destination and/or trip duration
information to the electronic control unit; storing the trip
destination and/or trip duration information; and transmitting the
trip destination and/or trip duration information via at least one
or more from the group consisting of: cellular data,
vehicle-to-everything (V2X) communications, and Wi-Fi, and in the
display controller of the billboard display system, receiving the
trip destination and/or trip duration information; processing the
trip destination and/or the trip duration information, in
combination with the demographic information and the number of
occupants, to determine advertising relevant to occupants; and
selectively displaying relevant advertising on the billboard
display to occupants in an approaching vehicle in response to the
demographic information, the number of occupants, and the trip
destination and/or the trip duration information.
15. The method according to claim 13, wherein the billboard display
system is configured to receive at least one of impression counts
and emotive responses transmitted from the interior occupant
sensing system, and to provide the impression counts and the
emotive responses to a remote cloud server.
16. The method according to claim 14, further including: a second
interior occupant sensing system of a second vehicle approaching
the billboard display that operates a second video camera to sense
video data of occupants in a second vehicle; transmitting the
number of occupants and the demographic information via at least
one or more from the group consisting of cellular data,
vehicle-to-everything (V2X) communications, and Wi-Fi; and in the
display controller of the billboard display system, receiving the
number of occupants, the demographic information of occupants, and
the vehicle position information from the second interior occupant
sensing system for a second vehicle approaching the billboard
display; processing the demographic information and the number of
occupants in a second vehicle and the number of occupants in a
first vehicle to determine advertising most relevant to a
population of occupants located in a first vehicle and a second
vehicle; and selectively displaying relevant advertising with the
billboard display to the population in a first vehicle and a second
vehicle approaching the billboard display.
17. The method according to claim 13, wherein the processing of the
demographic information and the number of occupants to determine
advertising relevant to occupants includes determining the
advertising based on an emotional state of the occupants.
Description
RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional
Patent Application No. 62/214,416 filed Sep. 4, 2015, the entire
contents of which are incorporated herein by reference.
BACKGROUND
[0002] Embodiments of the invention relate to a billboard display
system that receives data from interior occupant sensing systems of
vehicles and analyzes the data to provide advertising on a
billboard display that is directed toward the occupants of the
vehicles.
[0003] The concept of electronic billboard displays is old and
known. The electronic billboard displays provide various
advertisements that change periodically. The advertisements,
however, are simply selected based on payments by advertisers and
displayed without any knowledge of, or relationship to, the persons
in vehicles that are positioned to view the billboard displays.
SUMMARY
[0004] Increasingly, products, including vehicles, are being
customized for specific customer demographics. Many applications
can benefit from improved knowledge of the demographic profile of
each occupant in a vehicle. For example, with demographic knowledge
of the occupants, vehicles can be designed for the needs of their
users, automotive systems can respond and adjust based on the needs
and actions of the occupants. In addition to determining the number
and profile of occupants in the vehicle, it would also be useful to
analyze each occupant's reactions to external stimuli, such as the
reactions to advertisements (e.g., billboards). Thus, advertisers
can better target advertisements to occupants.
[0005] Through data mining and machine learning, dense profiles are
modeled and analyzed to assist marketers and advertisers with
targeted out-of-vehicle electronic displays.
[0006] In one embodiment, the invention provides a method for
selectively displaying advertisements with a billboard display
system to occupants in one or more vehicles approaching a billboard
display located near a roadway. The method includes receiving a
number of occupants, demographic information of occupants, and
vehicle position information from an interior occupant sensing
system for one or more vehicles approaching the billboard display,
processing the demographic information and the number of occupants
to determine advertising relevant to occupants, and selectively
displaying relevant advertising with the billboard display to one
or more vehicles approaching the billboard display.
[0007] In another embodiment, a billboard display system provides
advertising on a billboard display to occupants in one or more
vehicles in response to operation of an interior occupant sensing
system in one or more vehicles to obtain demographic information of
occupants. Besides the billboard display located near a roadway,
the billboard display system includes a display controller
including a processor and a memory, along with a communication unit
for providing communication between the display controller and the
interior occupant sensing system of one or more vehicles. The
processor is configured to receive via the communication unit, a
number of occupants disposed in one or more vehicles, the
demographic information for each occupant disposed in one or more
vehicles, and vehicle position information from an interior
occupant sensing system that includes a video camera disposed in
each of the one or more vehicles. In response to at least the
demographic information and the vehicle position information, the
processor provides an output to the billboard display to
selectively display advertising that is targeted to the demographic
information of occupants of the one or more vehicles.
[0008] In another embodiment, the invention provides a method of
selectively displaying advertisements on a billboard display of a
billboard display system to occupants in one or more vehicles
approaching the billboard display located near a roadway. The
method includes, in an interior occupant sensing system of a
vehicle approaching the billboard display, operating a video camera
to obtain video data including faces of occupants in a vehicle. The
video data is provided to an electronic control unit. The method
includes sensing a GPS signal and determining a vehicle position
information of a vehicle, detecting a number of occupants disposed
in a vehicle from the video data, and determining demographic
information from the video data of the faces of occupants. The
method further includes storing the vehicle position information,
the number of occupants, and the demographic information of
occupants in a memory of the interior occupant sensing system, and
wirelessly transmitting the number of occupants, the demographic
information, and the vehicle position information. Further, the
method includes, in a display controller of the billboard display
system, receiving the number of occupants, the demographic
information of occupants, and the vehicle position information
wirelessly transmitted by the interior occupant sensing system for
a vehicle approaching the billboard display. The billboard display
system processes the demographic information and the number of
occupants to determine advertising relevant to occupants; and
selectively displays relevant advertising on the billboard display
to occupants in an approaching vehicle.
[0009] Other aspects of the invention will become apparent by
consideration of the detailed description and accompanying
drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is a diagram of a roadway and a billboard display
system, according to some embodiments.
[0011] FIG. 2 is a block diagram of an interior occupant sensing
system, according to some embodiments.
[0012] FIG. 3 is a block diagram of an electronic controller,
according to some embodiments.
[0013] FIG. 4 is a front view of a dashboard and a windshield of a
vehicle, according to some embodiments.
[0014] FIG. 5 is a flow chart illustrating a method for detecting
data for vehicle occupants, according to some embodiments.
[0015] FIG. 6 is a flow chart illustrating a method for detecting
marketing metadata for vehicle occupants that includes speech
recognition, according to some embodiments.
[0016] FIG. 7 is a flow chart illustrating a method for interacting
with a billboard display system using occupant metadata, according
to some embodiments.
DETAILED DESCRIPTION
[0017] Before any embodiments of the invention are explained in
detail, it is to be understood that the invention is not limited in
its application to the details of construction and the arrangement
of components set forth in the following description or illustrated
in the following drawings. The invention is capable of other
embodiments and of being practiced or of being carried out in
various ways.
[0018] Also, it is to be understood that the phraseology and
terminology used herein is for the purpose of description and
should not be regarded as limiting. The use of "including,"
"comprising" or "having" and variations thereof herein is meant to
encompass the items listed thereafter and equivalents thereof as
well as additional items. The terms "mounted," "connected" and
"coupled" are used broadly and encompass both direct and indirect
mounting, connecting and coupling. Further, "connected" and
"coupled" are not restricted to physical or mechanical connections
or couplings, and can include electrical connections or couplings,
whether direct or indirect. Also, electronic communications and
notifications may be performed using any known means including
wired connections, wireless connections, etc.
[0019] It should also be noted that a plurality of hardware and
software based devices, as well as a plurality of different
structural components may be used to implement the embodiments. In
addition, it should be understood that embodiments may include
hardware, software, and electronic components or modules that, for
purposes of discussion, may be illustrated and described as if the
majority of the components were implemented solely in hardware.
However, one of ordinary skill in the art, and based on a reading
of this detailed description, would recognize that, in at least one
embodiment, the electronic based aspects of the embodiments may be
implemented in software (e.g., stored on non-transitory
computer-readable medium) executable by one or more processors. As
such, it should be noted that a plurality of hardware and software
based devices, as well as a plurality of different structural
components may be utilized to implement the embodiments. For
example, "processing units" and "controllers" described in the
specification can include standard processing components, such as
one or more processors, one or more memory modules including
non-transitory computer-readable medium, one or more input/output
interfaces, and various connections (e.g., a system bus) connecting
the components.
[0020] External Billboard Display System
[0021] FIG. 1 illustrates a roadway 10 with vehicles 12, and a
nearby billboard display system 20. The billboard display system 20
near the roadway 10 includes a display controller 22 that includes
a processor 24 in communication with a memory 26 and a display
driver 28. The display driver 28 connects to or communicates with a
billboard display 30 located adjacent the roadway 10. The processor
24 is linked to a communication unit 34 for communicating with
vehicles 12, etc. The billboard display 30 selectively displays
various advertising images. In one embodiment, the billboard
display 30 is a digital electronic display for providing various
advertising images stored in memory 26. In some embodiments, the
selected advertising images include video images, such that the
image displayed on the billboard display 30 changes visually to
attract attention to the billboard display 30.
[0022] The processor 24 provides a digital image signal for
advertising to the display driver 28, which provides the image to
the billboard display 30 for viewing thereon. The processor 24
obtains the digital image signal from the memory 26 or from a
remote cloud server 36 shown in FIG. 1 or other electronic storage
device via an internet connection or via the communication unit
34.
[0023] Interior Occupant Sensing System
[0024] FIG. 2 illustrates an interior occupant sensing system 40.
The interior occupant sensing system 40 is integrated into each
vehicle 12, and includes an electronic controller 42, a video
camera 44, a microphone 46, a database 48, a communications system
50, a human-machine interface (HMI) 52, a global positioning signal
(GPS) navigation system 54, an entertainment system 56, and other
vehicle systems 58. As illustrated in FIG. 2, the components of the
interior occupant sensing system 40 are in communication with the
electronic controller 42.
[0025] The electronic controller 42 shown in FIG. 2 may be
dedicated to the interior occupant sensing system 40, or may also
be part of another vehicle operating system. In some embodiments,
the electronic controller 42 is integrated with the video camera 44
and microphone 46 into a single device. In other embodiments, the
electronic controller 42 is part of a multi-camera system that
includes interior and exterior cameras. In some embodiments, the
interior occupant sensing system 40 includes more than one camera.
The video camera 44 is a digital video camera in one embodiment.
The video camera 44 is positioned to view the interior of the
vehicle and the occupants (e.g., the driver and one or more
passengers) of the vehicle. The electronic controller 42 is
configured to receive and process images or video data from the
video camera 44. The microphone 46 is positioned in the interior of
the vehicle and is configured to sense or detect sound (including
voices), convert the sound or audio signal to audio data, and
provide the audio data to the electronic controller 42. The
electronic controller 42 is configured to receive and process the
audio data from the microphone 46. The microphone 46 may be stand
alone or it may be part of another vehicle system (e.g., a
hands-free cellular system).
[0026] The database 48 shown in FIG. 2 electronically stores
information regarding the interior occupant sensing system 40,
including metadata of occupants, according to the methods described
herein. The electronic controller 42 is configured to read and
write such information to and from the database 48. In the
illustrated embodiment, the database 48 is accessible by the
electronic controller 42. In alternative embodiments, the database
48 may be located in the electronic controller 42, or housed on a
suitable database server or other system external to the interior
occupant sensing system 40 and accessible over one or more
intervening networks.
[0027] The communications system 50 shown in FIG. 2 includes
hardware and software components that allow the electronic
controller 42 to communicate wirelessly using one or more
modalities from the group consisting of: cellular data,
vehicle-to-everything (V2X) communications, and Wi-Fi. Other
communication arrangements and protocols are also contemplated. The
communications system 50 enables the electronic controller 42 to
communicate with other systems using public data networks (e.g.,
the Internet).
[0028] The HMI 52 shown in FIG. 2 provides an interface between the
interior occupant sensing system 40 and the occupants of the
vehicle 12. The HMI 52 is coupled to the electronic controller 42
and is configured to receive inputs from the occupants, receive
data from the controller, and provide warnings or other information
to the occupants based on the data. The HMI 52 includes suitable
input and output mechanisms, including, for example, buttons, a
touch-screen display having a graphical user interface (GUI), voice
recognition, etc. The GPS navigation system 54 includes hardware
and software for locating the vehicle using GPS signals, the global
positioning system and plotting routes based for the driver. The
entertainment system 56 includes hardware (e.g., a display screen)
and software configured to provide video and audio entertainment
content to occupants of the vehicle. In some embodiments, the
entertainment content includes video content streamed through the
communications system 50. The electronic controller 42 is also
coupled to other vehicle systems 58, including, for example,
steering, braking, and automated vehicle control systems.
[0029] As illustrated in FIG. 3, in one embodiment, the electronic
controller 42 includes an electronic processing unit 60 (e.g., a
microprocessor or another suitable programmable device), a
non-transitory memory 64 (e.g., a computer-readable storage
medium), and an input/output interface 68. In one embodiment, the
input/output interface 68 includes one or more control or data
buses enabling the electronic processing unit 60 to communicate
with the various devices illustrated in FIG. 2. The input/output
interface 68 provides an electrical connection over a data bus or a
wired, wireless, or optical connection that enables the devices
shown in FIG. 2 to communicate using network communications
protocols, for example, the Ethernet, CAN or FLEXRAY protocol.
[0030] The memory 64 can include a program storage area (e.g., read
only memory (ROM) and a data storage area (e.g., random access
memory (RAM), and another non-transitory computer readable medium.
The electronic processing unit 60 executes software stored in the
memory 64. The software may include instructions and algorithms for
performing methods as described herein.
[0031] The input/output interface 68 shown in FIG. 3 receives
inputs and provides outputs to and from systems external to the
electronic controller 42, including the devices and systems shown
in FIG. 2. In some embodiments, the electronic controller 42
includes a plurality of electrical and electronic components that
provide power, operational control, and protection to the
components and modules within the electronic controller 42. It
should be understood that the electronic controller 42 may include
additional, fewer, or different components.
[0032] In one embodiment, the electronic controller 42 is
configured to perform machine learning functions. The memory 64
stores one or more learning engines executable by the electronic
processing unit 60 to process data received from the video camera
44 and the microphone 46, and develop demographic metadata on the
occupants of the vehicle. Machine learning generally refers to the
ability of a computer application to learn without being explicitly
programmed. In particular, a computer application performing
machine learning (sometimes referred to as a learning engine) is
configured to develop an algorithm based on training data. For
example, to perform supervised learning, the training data includes
example inputs and corresponding desired (e.g., actual) outputs,
and the learning engine progressively develops a model that maps
inputs to the outputs included in the training data. Machine
learning can be performed using various types of methods and
mechanisms including, but not limited to, decision tree learning,
association rule learning, artificial neural networks, inductive
logic programming, support vector machines, clustering, Bayesian
networks, reinforcement learning, representation learning,
similarity and metric learning, sparse dictionary learning, and
genetic algorithms.
[0033] In some embodiments, the electronic controller 42 accesses
one or more sources of training data, predictive models, or
learning engines through one or more communication networks, such
as the Internet and other public and private networks.
Alternatively or in addition, the electronic controller 42 stores
training data in the database 48 or in the memory 64.
[0034] FIG. 4 is a partial perspective view of a front interior
portion 70 of a vehicle 12 that includes a dashboard 72, a
windshield 74 and a roof 76. Further, a video camera 44 is mounted
to the roof 76 near the rear view mirror 78 to obtain video data
within the vehicle. Further, FIG. 4 shows the human-machine
interface 52, such as a touch screen.
[0035] Operation of Interior Occupant Sensing System
[0036] For the billboard display system 20 to operate properly,
information from at least one of the vehicles 12 shown in FIG. 1
must be provided to the billboard display system. Thus, the
interior occupant sensing system 40 for each vehicle operates as
follows.
[0037] As shown in the images and flow chart 100 of FIG. 5, four
faces 104 of occupants are sensed by video camera 44 disposed in a
vehicle 12 and provided as video data to the electronic processing
unit 60. Using machine learning occupant detection algorithms, the
electronic processing unit 60 determines the number, pose and
location of each occupant that is detected (step 102). For each
face, demographic information and markers are estimated or
determined (step 106) by the electronic processing unit 60
executing classifier algorithms. Determining demographic
information includes utilizing the video data or video images for a
given occupant's face to 1) classify age, 2) classify gender and 3)
classify ethnicity and/or race. In some embodiments, other
classification information is obtained. The demographic information
is determined for each individual occupant.
[0038] Upon determining the demographic information from video
images of faces of occupants, the electronic processing unit 60
further determines emotive responses (step 110). The emotive
responses include 1) gaze estimation, including where the face is
looking and 2) current emotions, such as a smile, frown or laugh.
Changes in the occupants' emotive response to stimuli may be caused
by changes in operation of the vehicle itself or externally (i.e.,
current emotion). For instance, an occupant might display happiness
when the vehicle accelerates, or display anxiety when a warning is
issued by the HMI 52. Besides determining emotions, the electronic
processing unit 60 is configured for storing the emotions and the
exact time that the emotions or emotive responses occur in the
memory 64.
[0039] Thereafter, the face 104 of an occupant is tracked or
followed within the video image (step 114). Tracking faces 104 of
occupants enables the interior occupant sensing system 40 to
determine emotions of occupants and their emotive responses to
stimuli, such as advertising. That way, a face 104 that has been
classified is followed and reclassification is not necessarily
required.
[0040] The electronic processing unit 60 is further configured to
combine the demographic information with other vehicle information
(i.e. GPS location, duration of trip, passengers entering/exiting,
etc.) (step 116). Other vehicle information may include an intended
destination for the vehicle, the make and model of the vehicle.
Further, the clothing worn by the occupants may be utilized in
classifying the occupants and predicting what advertising to
provide thereto.
[0041] The electronic processing unit 60 is configured to improve
classifier estimates as the occupant is tracked over time (step
120). Influences such as lighting, occupant movement, and occupant
clothing may vary over time, causing the electronic processing unit
60 to make different demographic estimations over time. The
electronic processing unit 60 uses machine learning algorithms to
improve its estimates by recognizing trends or eliminating
outliers. This continuous improvement ensures the most reliable
metadata.
[0042] The electronic processing unit 60 is configured to store the
demographic information, the vehicle position information, the trip
destination information, the trip duration information and
additional information as metadata in the database 48 (step 124).
The electronic processing unit 60 classifies the data as occupant
specific data (for example, based on face, and as set forth below,
voice recognition algorithms); generic demographic data tied to a
specific vehicle at a specific time and location; or as generic
metadata that is not tied to any individual or vehicle, but
aggregated to extract demographic patterns of vehicle use and
transportation. Depending on the application, this allows for
either specific customization of the application to the user, or
broad statistics based applications that can ensure individual
privacy.
[0043] As illustrated in the flow chart 140 of FIG. 6, in some
embodiments, the microphone 46 collects occupant speech to better
ascertain occupant emotive state and reactions to external events.
The electronic processing unit 60 is configured to perform speech
recognition on audio data received from the microphone 46 (step
144). Key words or phrases are then identified (step 148).
Thereafter, the electronic processing unit 60 is further configured
to match the speaker to a face (step 152). Such matching relies on
estimating movement or speaking by a face of an occupant and
comparison with the key word. This key word and specific occupant
information is stored as metadata in the database 48 (step 160)
along with the demographic information and other metadata for each
specific user. Further, the metadata is wirelessly transmitted via
communications system 50 to the communication unit 34 of the
billboard display system 20.
[0044] Moreover, the interior occupant sensing system 40 is capable
of providing dense profile models. In one exemplary embodiment, the
electronic controller 42 is configured to fuse detected persons
with their demographic information, their emotive response and the
speech recognition of key words. Collecting occupant data, along
with vehicle dynamics, provides market researchers a real-time
response of an occupant to an advertisement displayed on the
billboard display system.
[0045] As illustrated in the flow chart 200 of FIG. 7, in one
embodiment the interior occupant sensing system 40 interacts with
the billboard display system 20 using occupant data, such as
metadata. For example, data on the number of faces in a video image
corresponding to the number of occupants, the occupant demographics
and other metrics (e.g., trip destinations and durations) are
wirelessly transmitted by the communications system 50 via V2X to
an upcoming communication unit 34 of a billboard display system
20.
[0046] More specifically, as shown in FIG. 7, a vehicle 12 enters
the zone of the billboard display system 20 (step 202). The
communications system 50 of the interior occupant sensing system 40
provides wireless transmission or wireless communication (step 206)
with the communication unit 34 of the billboard display system 20.
For communication to occur, the vehicle 12 must be within
broadcasting range of the billboard display system 20. The wireless
transmission includes the vehicle position information of the
vehicle located on a roadway relative to the billboard display 34
as determined from GPS signals.
[0047] The display controller 22 of the billboard display system 20
analyzes a population of occupants approaching the billboard
display 30 from one or more approaching vehicles (step 210).
Further, the display controller 22 is configured to analyze the
metrics (step 212) from one or more vehicles approaching the
billboard display 30. Besides the number of occupants and the
demographic information (214) of occupants in multiple vehicles
that are approaching the billboard display 30, along with trip
destinations (216) and trip durations (218) of the multiple
vehicles when the trip data is available, the metrics may include
other factors. The type of vehicle, the clothing worn by occupants,
and the time of day, are also provided as metrics in some
embodiments.
[0048] Utilizing the analyzed metrics, the display controller 22
determines and displays a targeted advertisement (step 220). The
targeted advertisement is considered most applicable to the
population consisting of the occupants of each of the vehicles in
the field of view of the billboard display 30. As set forth above,
a targeted advertisement is based on the various factors and based
on time of day. For instance, in one embodiment, advertising for
hotels or motels in mid or late evening of a long trip duration is
provided. At what is considered close to typical mealtimes,
advertising is provided for approaching restaurants. The type of
restaurants or establishments displayed in advertising is also
based on the ages of the occupants. In conclusion, the various
metrics interactively assist in the selective display of
advertising on the billboard display 30.
[0049] Upon the display of advertising for a limited time based on
the metrics, the billboard display system 20 queries the
approaching vehicles 12 for additional or updated metrics (step
230). The updated metrics include impression counts (232), emotive
responses (234), and gaze of eyes (236) of the occupants of the
approaching vehicles. The interior occupant sensing system 40 is
configured to communicate the impression counts, the emotional
states, any emotive responses and gaze of eyes of occupants to the
billboard display system (step 240), along with additional
metrics.
[0050] The display controller 22 executes a program using the above
information to target or select a future or immediate advertisement
that is more likely relevant or useful to occupants of one or more
vehicles approaching the billboard display 30. The display
controller 22 is configured to process the trip destination
information and/or the trip duration information, in combination
with the demographic information and number of occupants, to
determine advertising relevant to occupants. In one embodiment, the
emotional state of the occupants assists in selection of
advertising. For occupants in a positive mood, the advertising is
different than the advertising for occupants in an angry or bad
mood.
[0051] Further, the display controller 22 is configured to track
impression counts for current advertising. Using occupant gaze
estimations, advertisers accurately analyze and gauge the
effectiveness of various advertising schemes. Analyzing the
effectiveness from the impression counts maximizes the relevance of
the advertising to be displayed in the future to particular
occupants of one or more vehicles.
[0052] Other metrics, can be used to target advertisements. In one
operation, if the billboard display system 20 is informed that a
vehicle's fuel level is low, advertising for nearby gas stations
appears. In another scenario, child-friendly venues advertise using
the billboard display 30 to vehicles 12 carrying children as
determined from the demographic information.
[0053] In one embodiment, at least one of impression counts, gaze
estimation and emotive responses are collected and transmitted to
advertisers remotely, such as to the remote cloud server 36, to
gauge the success of advertisements provided on the billboard
display 30. The billboard display system 20 is in communication
over the internet or other communication with the cloud server 36.
Thus, the billboard display system 20 provides demographic
information to the cloud server 36 and receives advertising for
viewing on the billboard display 30.
[0054] In some implementations, the above described system is
controlled using at least one display controller 22. The display
controller 22 includes one or more processing units (e.g., a
processor, application specific integrated circuits ("ASIC"),
etc.). Likewise, the vehicle electronic controller 42 can include
one or more processing units as listed above.
[0055] The memory 26 of the display controller 22 and the memory 64
of the vehicle electronic controller 42 are non-transitory
computer-readable medium in one embodiment. Both random access
memory (RAM) and read only memory (ROM) are contemplated.
[0056] Besides the display controller 22, the electronic controller
42 uses the input/output interface 68 for sending and receiving
information from one or more sensors or systems external to the
electronic controller 42 (e.g., over a vehicle communication bus,
such as a controller area network (CAN) bus or a FlexRay bus). In
some implementations, the electronic controller 42 can also include
one or more additional internal sensors or systems.
[0057] The billboard display 30 is an electronic billboard display
that displays the electronic images from the display driver 28. The
billboard display 30 is a video screen of plasma, LCD, LED or other
digital light emitting arrangements. In some embodiments, a
loudspeaker or sound output is also provided.
[0058] While at least one video camera 44 is contemplated, a
plurality of video cameras are provided to obtain video data to
detect occupants in some embodiments.
[0059] In referencing a situation wherein two vehicles are
approaching the billboard display system 20, one vehicle can be
considered a first vehicle having a first interior occupant sensing
system with a first video camera and another vehicle can be
considered a second vehicle having a second interior occupant
sensing system with a second video camera. Of course, third,
fourth, fifth, etc. vehicles having interior occupant sensing
systems are also contemplated.
[0060] While FIG. 1 shows vehicles in one direction approaching the
billboard display 30 receiving advertising, in another embodiment
vehicles traveling in opposing directions view advertising messages
on respective sides of the billboard display 30.
[0061] With the availability of an interior video camera 44
primarily used for occupant sensing, additional demographic data of
each occupant in a vehicle 12 can be extracted from a camera image.
This demographic metadata can be stored, analyzed and fused with a
variety of other sensors and vehicle states to provide dense
demographic profiles of drivers and passengers. This data is useful
for marketers and advertisers.
[0062] Thus, the invention provides, among other things, methods
for operating a billboard display system 20 to provide advertising
directed to the occupants of vehicles 12 approaching a billboard
display 30 and for operating an interior occupant sensing system 40
to provide information or data thereto.
* * * * *