U.S. patent application number 10/973335 was filed with the patent office on 2005-04-28 for method and system for managing an interactive video display system.
Invention is credited to Belfer, Russell H., Bell, Matthew.
Application Number | 20050088407 10/973335 |
Document ID | / |
Family ID | 34520185 |
Filed Date | 2005-04-28 |
United States Patent
Application |
20050088407 |
Kind Code |
A1 |
Bell, Matthew ; et
al. |
April 28, 2005 |
Method and system for managing an interactive video display
system
Abstract
A method for managing an interactive video display system. A
plurality of video spots are displayed on the interactive video
display system. Data based on interaction with the interactive
video display system corresponding to video spots of the plurality
of video spots is gathered. The data is stored, wherein the data is
for use in managing presentation of the video spots. By analyzing
data relating to different video spots, popularity and other
metrics may be determined for the video spots, providing useful
information for managing the presentation of the video spots.
Inventors: |
Bell, Matthew; (Palo Alto,
CA) ; Belfer, Russell H.; (Burlingame, CA) |
Correspondence
Address: |
WAGNER, MURABITO & HAO LLP
Third Floor
Two North Market Street
San Jose
CA
95113
US
|
Family ID: |
34520185 |
Appl. No.: |
10/973335 |
Filed: |
October 25, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60514232 |
Oct 24, 2003 |
|
|
|
Current U.S.
Class: |
345/156 |
Current CPC
Class: |
H04N 21/812 20130101;
H04N 21/44218 20130101; H04N 21/4223 20130101; H04N 21/41415
20130101; G06Q 30/02 20130101; H04N 21/25891 20130101 |
Class at
Publication: |
345/156 |
International
Class: |
G09G 005/00; G09G
005/08 |
Claims
What is claimed is:
1. A method for managing an interactive video display system, said
method comprising: displaying a plurality of video spots on said
interactive video display system; gathering data based on
interaction with said interactive video display system
corresponding to video spots of said plurality of video spots; and
storing said data, wherein said data is for use in managing
presentation of said plurality of video spots.
2. The method as recited in claim 1 wherein said data comprises
information about what virtual objects displayed by said
interactive video display system were interacted with.
3. The method as recited in claim 1 wherein said data comprises
information about a location of said interaction relative to a
display of said interactive video display system.
4. The method as recited in claim 1 wherein said interaction is
determined using a person tracking process.
5. The method as recited in claim 1 wherein said interaction is
determined according to a foreground/background classification
image.
6. The method as recited in claim 1 wherein said storing said data
comprises storing said data to a local memory of said interactive
video display system.
7. The method as recited in claim 1 wherein said storing said data
comprises: transmitting said data to an external computer system;
and storing said data to a memory of said external computer
system.
8. The method as recited in claim 1 further comprising analyzing
said data for use in managing presentation of said plurality of
video spots.
9. The method as recited in claim 8 wherein said analyzing said
data comprises determining popularity for at least one video spot
of said plurality of video spots based on said data.
10. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered at the end of said
displaying of at least one video spot of said plurality of video
spots for determining the number of people in the viewing area of
said interactive video display system at the end of said displaying
of said video spot.
11. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered from said displaying of
at least one video spot of said plurality of video spots for
determining the average number of people in the area of said
interactive video display system during said displaying of said
video spot.
12. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered at the beginning and
end of said displaying of at least one video spot of said plurality
of video spots for determining the difference in the number of
people in the viewing area of said interactive video display system
from the beginning to the end of said displaying of said video
spot.
13. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered for said displaying of
at least one video spot of said plurality of video spots for
determining the total number of people who entered the viewing are
viewing area of said interactive video display system and the
number of people who exited the viewing area of said interactive
video display system during said displaying of said video spot.
14. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered for said displaying of
at least one video spot of said plurality of video spots for
determining the length of time a person spends interacting with
said interactive video display system.
15. The method as recited in claim 8 wherein said analyzing said
data comprises analyzing said data gathered for said displaying of
at least one video spot of said plurality of video spots for
determining the amount of movement during said displaying of said
video spot.
16. The method as recited in claim 8 wherein said analyzing said
data comprises determining a first user attraction level for at
least one video spot of said plurality of video spots based on said
data.
17. The method as recited in claim 8 wherein said analyzing said
data comprises determining a group appeal for at least one video
spot of said plurality of video spots based on said data.
18. The method as recited in claim 8 further comprising gathering
external information for use in said analyzing said data.
19. The method as recited in claim 8 further comprising adjusting a
display schedule of said plurality of video spots based on said
analyzing said data.
20. The method as recited in claim 19 wherein said display schedule
is automatically adjusted based on said analysis.
21. The method as recited in claim 19 wherein said display schedule
is manually adjusted based on said analysis.
22. The method as recited in claim 20 wherein particular said spots
are displayed on said interactive video display system when no
interaction is taking place on said interactive video display
system.
23. A system for managing an interactive video display system, said
system comprising: a display for displaying a plurality of video
spots on said interactive video display system; a data gatherer for
gathering data based on interaction with said interactive video
display system corresponding to video spots of said plurality of
video spots; and a data storage storing said data, wherein said
data is for use in managing presentation of said plurality of video
spots.
24. The system as recited in claim 23 wherein said data comprises
information about what virtual objects displayed by said
interactive video display system were interacted with.
25. The system as recited in claim 23 wherein said data comprises
information about a location of said interaction relative to a
display of said interactive video display system.
26. The system as recited in claim 23 wherein said interaction is
determined using a person tracking process.
27. The system as recited in claim 23 wherein said interaction is
determined according to a foreground/background classification
image.
28. The system as recited in claim 23 wherein said data storage is
a local data storage of said interactive video display system.
29. The system as recited in claim 23 wherein said data storage is
a data storage of an external computer system.
30. The system as recited in claim 23 further comprising a data
analyzer for analyzing said data for use in managing presentation
of said plurality of video spots.
31. The system as recited in claim 30 wherein said analyzer is
operable to determine popularity for at least one video spot of
said plurality of video spots based on said data.
32. The system as recited in claim 30 wherein said analyzer is
operable to determine a first user attraction level for at least
one video spot of said plurality of video spots based on said
data.
33. The system as recited in claim 30 wherein said analyzer is
operable to determine a group appeal for at least one video spot of
said plurality of video spots based on said data.
34. The system as recited in claim 30 wherein said analyzer
receives external data for use in said analyzing said data.
35. The system as recited in claim 30 further comprising a video
spot scheduler for adjusting a display schedule of said plurality
of video spots based on said analyzing said data.
36. The system as recited in claim 35 wherein said display schedule
is automatically adjusted based on said analysis.
37. The system as recited in claim 35 wherein said display schedule
is manually adjusted based on said analysis.
38. The system as recited in claim 36 wherein particular said spots
are displayed on said interactive video display system when no
interaction is taking place on said interactive video display
system.
39. A computer-usable medium having computer-readable program code
embodied therein for causing a computer system to perform a method
for managing an interactive video display system, said method
comprising: displaying a plurality of video spots on said
interactive video display system; gathering data based on
interaction with said interactive video display system
corresponding to video spots of said plurality of video spots; and
storing said data, wherein said data is for use in managing
presentation of said plurality of video spots.
40. The computer-usable medium as recited in claim 39 wherein said
data comprises information about what virtual objects displayed by
said interactive video display system were interacted with.
41. The computer-usable medium as recited in claim 39 wherein said
data comprises information about a location of said interaction
relative to a display of said interactive video display system.
42. The computer-usable medium as recited in claim 39 wherein said
interaction is determined using a person tracking process.
43. The computer-usable medium as recited in claim 39 wherein said
interaction is determined according to a foreground/background
classification image.
44. The computer-usable medium as recited in claim 39 wherein said
storing said data comprises storing said data to a local memory of
said interactive video display system.
45. The computer-usable medium as recited in claim 39 wherein said
storing said data comprises: transmitting said data to an external
computer system; and storing said data to a memory of said external
computer system.
46. The computer-usable medium as recited in claim 39 wherein said
method further comprises analyzing said data for use in managing
presentation of said plurality of video spots.
47. The computer-usable medium as recited in claim 46 wherein said
analyzing said data comprises determining popularity for at least
one video spot of said plurality of video spots based on said
data.
48. The computer-usable medium as recited in claim 46 wherein said
analyzing said data comprises determining a first user attraction
level for at least one video spot of said plurality of video spots
based on said data.
49. The computer-usable medium as recited in claim 46 wherein said
analyzing said data comprises determining a group appeal for at
least one video spot of said plurality of video spots based on said
data.
50. The computer-usable medium as recited in claim 46 further
comprising gathering external information for use in said analyzing
said data.
51. The computer-usable medium as recited in claim 46 wherein said
method further comprises adjusting a display schedule of said
plurality of video spots based on said analyzing said data.
52. The computer-usable medium as recited in claim 51 wherein said
display schedule is automatically adjusted based on said
analysis.
53. The computer-usable medium as recited in claim 51 wherein said
display schedule is manually adjusted based on said analysis.
54. The computer-usable medium as recited in claim 52 wherein
particular said spots are displayed on said interactive video
display system when no interaction is taking place on said
interactive video display system.
Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims priority from co-pending U.S.
Provisional Patent Application No. 60/514,232, filed on Oct. 24,
2003, entitled "METHOD AND SYSTEM FOR MANAGING CAPTURED IMAGE
INFORMATION IN AN INTERACTIVE VIDEO DISPLAY SYSTEM," by Bell et
al., and assigned to the assignee of the present application, which
is herein incorporated by reference.
FIELD OF INVENTION
[0002] The present invention relates to the field of visual
electronic displays. Specifically, embodiments of the present
invention relate to a self-contained interactive video display
system.
BACKGROUND OF THE INVENTION
[0003] Advertising is used to convey various types of information
to an audience. To fully maximize the performance of an
advertisement, it is desirable for the advertiser to be able to
gather information regarding effectiveness of the advertisement(s).
This effectiveness measure may comprise both how many people saw
the advertisement, whether people paid attention to it. In general,
for traditional offline advertising (e.g., billboards, posters,
television commercials) data gathering is typically based on
information related to where the advertisement is placed. For
example, for a billboard advertisement, effectiveness may be
measured by the amount of automobile traffic that passes the
billboard. Similarly, for a television commercial, effectiveness,
or popularity, may be based on popularity ratings for the
television show during which the television commercial was aired.
In this way, information regarding the popularity of an
advertisement may be inferred based on the advertisement placement
information. However, it is typically very difficult to directly
measure popularity of traditional advertisements. Because these
advertisements are not interactive, it is difficult to know if
people are actually paying attention to them.
[0004] Recent technological advancements have led to the creation
of a new type of advertising medium, the interactive video display
system. Interactive video display systems allow real-time
unencumbered human interactions with video displays. Natural
physical motions by human users are captured by a computer vision
system and used to drive visual effects. The computer vision system
usually uses images captured by a video camera as input and has
software processes that gather real-time information about people
and other objects in the interactive area viewed by the camera.
[0005] For interactive video display systems, it is possible to
gather information regarding displayed images based on the same
methods used for traditional offline advertising mediums. However,
in addition to providing real-time interactions, it would be
desirable to have an interactive video display system that is
capable of capturing and managing information derived from such
real-time interactions not available to traditional offline
advertising mediums.
SUMMARY OF THE INVENTION
[0006] Various embodiments of the present invention, a method and
system for managing an interactive video display system, are
described herein. In one embodiment, a plurality of video spots are
displayed on the interactive video display system.
[0007] Data based on interactions with the interactive video
display system corresponding to video spots of the plurality of
video spots is gathered. In one embodiment, the interaction is
determined according to person tracking. In another embodiment, the
interaction is determined according to a foreground/background
classification image. In one embodiment, the data includes
information about what virtual objects displayed by the interactive
video display system were interacted with. In one embodiment, the
data includes information about a location of the interaction
relative to a display of the interactive video display system.
[0008] The data may then be stored and may be used in managing
presentation of the video spots. In one embodiment, the data is
stored at a local memory of the interactive video display system.
In another embodiment, the data is transmitted to an external
computer system and is stored in a memory of the external computer
system.
[0009] In one embodiment, the data is analyzed for use in managing
presentation of the plurality of video spots. In one embodiment,
analyzing the data includes determining popularity for at least one
video spot of the plurality of video spots based on the data. In
another embodiment, analyzing the data includes determining a
"first user attraction level" for at least one video spot of the
plurality of video spots based on the data. In another embodiment,
analyzing the data includes determining a group appeal for at least
one video spot of the plurality of video spots based on the data.
In one embodiment, external data is gathered for use in analyzing
the data.
[0010] In one embodiment, the display schedule of the plurality of
video spots is adjusted based on analyzing data. In one embodiment,
the display schedule is automatically adjusted based on the
analysis. In another embodiment, the display schedule is manually
adjusted based on the analysis.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings, which are incorporated in and
form a part of this specification, illustrate embodiments of the
invention and, together with the description, serve to explain the
principles of the invention:
[0012] FIG. 1A illustrates a projection interactive video display
system in accordance with an embodiment of the present
invention.
[0013] FIG. 1B illustrates a self-contained interactive video
display system in accordance with an embodiment of the present
invention.
[0014] FIG. 2 illustrates a system for managing an interactive
video display system in accordance with an embodiment of the
present invention.
[0015] FIG. 3 illustrates a process for managing an interactive
video display system in accordance with an embodiment of the
present invention.
DETAILED DESCRIPTION
[0016] Reference will now be made in detail to various embodiments
of the invention, an electronic device for monitoring the presence
of objects around a second electronic device, examples of which are
illustrated in the accompanying drawings. While the invention will
be described in conjunction with these embodiments, it is
understood that they are not intended to limit the invention to
these embodiments. On the contrary, the invention is intended to
cover alternatives, modifications and equivalents, which may be
included within the spirit and scope of the invention as defined by
the appended claims. Furthermore, in the following detailed
description of the invention, numerous specific details are set
forth in order to provide a thorough understanding of the
invention. However, it will be recognized by one of ordinary skill
in the art that the invention may be practiced without these
specific details. In other instances, well known methods,
procedures, components, and circuits have not been described in
detail as not to unnecessarily obscure aspects of the
invention.
[0017] Some portions of the detailed descriptions, which follow,
are presented in terms of procedures, steps, logic blocks,
processing, and other symbolic representations of operations on
data bits that can be performed on computer memory. These
descriptions and representations are the means used by those
skilled in the data processing arts to most effectively convey the
substance of their work to others skilled in the art. A procedure,
computer executed step, logic block, process, etc., is here, and
generally, conceived to be a self-consistent sequence of steps or
instructions leading to a desired result. The steps are those
requiring physical manipulations of physical quantities. Usually,
though not necessarily, these quantities take the form of
electrical or magnetic signals capable of being stored,
transferred, combined, compared, and otherwise manipulated in a
computer system. It has proven convenient at times, principally for
reasons of common usage, to refer to these signals as bits, values,
elements, symbols, characters, terms, numbers, or the like.
[0018] It should be borne in mind, however, that all of these and
similar terms are to be associated with the appropriate physical
quantities and are merely convenient labels applied to these
quantities. Unless specifically stated otherwise as apparent from
the following discussions, it is appreciated that throughout the
present invention, discussions utilizing terms such as "presenting"
or "gathering" or "storing" or "transmitting" or "analyzing" or
"determining" or "adjusting" or "managing" the like, refer to the
action and processes of an electronic system (e.g., 200 of FIG. 2),
or similar electronic computing device, that manipulates and
transforms data represented as physical (electronic) quantities
within the electronic device's registers and memories into other
data similarly represented as physical quantities within the
electronic device memories or registers or other such information
storage, transmission or display devices.
[0019] Various embodiments of the present invention in the form of
one or more exemplary embodiments will now be described. In one
embodiment, the interactive video display system includes a vision
system that captures and processes information relating to a scene.
The processed information is used to generate certain visual
effects that are then displayed to viewers via an interactive
display device. People are able to interact with such visual
effects on a real-time basis.
[0020] FIG. 1A illustrates a projection interactive video display
system 100 in accordance with an embodiment of the present
invention. Projection interactive video display system 100 uses a
camera system 105, an illuminator that illuminates surface 102
being viewed by camera 105, a projector 110 that projects an image
120 onto the interactive space 115 of surface 102, and a local
computer (not shown) that takes as input the image of camera 105
and outputs a video image to projector 110.
[0021] The local computer processes the camera 105 input to discern
information about the position and movement of people (or moving
objects)in the volume in front of surface 102. In one embodiment,
the local computer processes the camera 105 input to discern on a
pixel-by-pixel basis what portions of the volume in front of
surface 102 (e.g., interactive space 115) are occupied by people
(or moving objects) and what portions of surface 102 are
background. The local computer may accomplish this by developing
several evolving models of what the background is believed to look
like, and then comparing its concepts of the background to what
camera 105 is currently imaging. Components of the local computer
that process camera 105 input are collectively known as the vision
system. Various embodiments of projection interactive video display
system 100 and the vision system are described in co-pending U.S.
patent application Ser. No. 10/160,217, filed on May 28, 2002,
entitled "INTERACTIVE VIDEO DISPLAY SYSTEM," by Bell, and assigned
to the assignee of the present application, and in co-pending U.S.
Provisional Patent Application No. 60/514,024, filed on Oct. 24,
2003, entitled "METHOD AND SYSTEM FOR PROCESSING CAPTURED IMAGE
INFORMATION IN AN INTERACTIVE VIDEO SYSTEM," by Bell, and assigned
to the assignee of the present application, both of which are
herein incorporated by reference.
[0022] In one embodiment, projection interactive video display
system 100 is coupled to a remote computer system 130. In one
embodiment, remote computer system 130 is a remote server for
collecting data gathered by projection interactive video display
system 100. In one embodiment, remote computer system 130 is
configured to collect data gathered by multiple projection
interactive video display systems 100 that are located in
physically distinct locations. It should be appreciated that remote
computer system 130 may be located in a different physical location
than projection interactive video display system 100.
[0023] FIG. 1B illustrates a self-contained interactive video
display system 150 in accordance with an embodiment of the present
invention. Self-contained interactive video display system 150
displays an image onto display screen 155, and uses a camera (not
shown) to detect people and objects in interactive space 160. A
local computer (not shown), also referred to as the image system,
takes as input the image of the camera and outputs a video image to
display screen 155.
[0024] Various embodiments of self-contained interactive video
display system 150 are described in co-pending U.S. patent
application Ser. No. 10/946,263, filed on Sep. 20, 2004, entitled
"SELF-CONTAINED INTERACTIVE VIDEO DISPLAY SYSTEM," by Bell et al.,
and assigned to the assignee of the present application, co-pending
U.S. patent application Ser. No. 10/946,084, filed on Sep. 20,
2004, entitled "SELF-CONTAINED INTERACTIVE VIDEO DISPLAY SYSTEM,"
by Bell, and assigned to the assignee of the present application,
and co-pending U.S. patent application Ser. No.______, filed on
Sep. 20, 2004, entitled "INTERACTIVE VIDEO WINDOW DISPLAY SYSTEM,"
by Bell, and assigned to the assignee of the present application,
all of which are herein incorporated by reference. Furthermore,
various embodiments of the vision system are described in
co-pending U.S. patent application Ser. No. 10/160,217, filed on
May 28, 2002, entitled "INTERACTIVE VIDEO DISPLAY SYSTEM," by Bell,
and assigned to the assignee of the present application, and in
co-pending U.S. Provisional Patent Application No. 60/514,024,
filed on Oct. 24, 2003, entitled "METHOD AND SYSTEM FOR PROCESSING
CAPTURED IMAGE INFORMATION IN AN INTERACTIVE VIDEO SYSTEM," by
Bell, and assigned to the assignee of the present application, both
of which are herein incorporated by reference.
[0025] In one embodiment, self-contained interactive video display
system 150 is coupled to a remote computer system 170. In one
embodiment, remote computer system 170 is a remote server for
collecting data gathered by self-contained interactive video
display system 150. In one embodiment, remote computer system 170
is configured to collect data gathered by multiple self-contained
interactive video display systems 150 and/or projection interactive
video display systems 100 (FIG. 1A) that are located in physically
distinct locations. It should be appreciated that remote computer
system 170 may be located in a different physical location than
self-contained interactive video display system 150. Many other
forms of interactive video display systems exist. These systems may
use different kinds of display media, and different sensing
apparatus. However, the techniques described in this patent are
fully applicable to these other systems as well.
[0026] FIG. 2 illustrates a system 200 for managing an interactive
video display system in accordance with an embodiment of the
present invention. System 200 includes interactive video display
system 210 (e.g., projection interactive video display system 100
of FIG. 1A or self-contained interactive video display system 150
of FIG. 1B) and remote computer system 230 (e.g., remote computer
system 130 of FIG. 1A or remote computer system 170 of FIG. 1B).
Interactive video display system 210 includes display 212, vision
system 214, video spot storage 216, and data gathering module 218.
In one embodiment, display 212, vision system 214, video spot
storage 216, and data gathering module 218 are components of a
computer system of interactive video display system 210. Remote
computer system 230 includes data storage 232, data analysis module
234, and video spot schedule 236. It should be appreciated that in
various embodiments of the present invention, any or all of the
components and functionality of remote computer system 230 may be
local and included in interactive video display system 210.
[0027] Data Gathering
[0028] In an exemplary aspect, interactive video display system 210
presents a rich opportunity to gather data about interactions with
and usage of the interactive displays. This data is gathered at
data gathering module 218, and is useful for many kinds of
interactive contents and applications. In one exemplary embodiment,
the data is useful for designing advertising contents 20 as it
allows advertisers to analyze where and when their advertisements
are most popular, thus, allowing them to adjust their
advertisements and/or schedules accordingly.
[0029] The data may be analyzed locally by the same computer that
handles vision system 214. Alternatively, interactive video display
system 210 may be networked, allowing the data for one or more
systems to be analyzed from remote computer system 230. The data
from multiple systems may be pooled to increase the total amount of
data or to identify differences between locations.
[0030] By using a person-tracking process, real-time information
relating to the present and past positions of people viewed by a
camera of vision system 214 or a separate camera, can be
determined. Using this information, additional information can be
derived including, for example, the number of people on or near the
display 212, the time at which each person enters and leaves the
display 212 (e.g., interactive area 115 of FIG. 1A), and whether
they are moving and interacting with the displayed image. In one
embodiment of interactive video display system 210, such as, a
floor or wall-projected interactive display, the boundaries of the
display screen are known. Thus, based on the respective positions
of the people, additional information about the people within
display 212 can be derived. For example, specific information can
be determined based on the number of people who are interacting
with Interactive video display system 210 and the number of people
who are mere spectators. In addition, with face-recognition
systems, it is also possible to gather demographic data such as
gender. It should be appreciated that the person tracking system
may take the form of a face-tracking system, in which heads are
counted and tracked instead of entire bodies. A variety of
person-tracking processes exist. These processes may use one or
more of a variety of systems and techniques including, but not
limited to, stereo cameras, time-of-flight cameras, depth
segmentation, color segmentation, feature tracking, face detection,
template matching, adaptive template matching, and blob
tracking.
[0031] Similar data can be gathered from a foreground/background
classification image, which may be produced by vision system 214.
The portion of the image that is classified as foreground is a
rough approximation of the number of people viewed by the camera.
If the width and height of the screen viewed by the camera and the
approximate cross-sectional size of a typical person from the
camera's point of view are known, then the approximate portion of
the screen that is turned to foreground by the presence of a single
person can be computed. Consequently, a conversion can be made from
foreground portion to number of people. Alternatively, by doing
some experiments with a person in front of the camera, the
approximate portion of foreground that corresponds to a single
person for a given installation can be determined. This "foreground
portion" data can be used to roughly estimate how popular the
display is at a given time, and whether people are entering or
leaving the display. Because the physical boundaries of interactive
video display system 210 may only take up a portion of the camera's
view, the foreground/background classification image can be
segmented into different regions, such as, "on the display",
"within one foot of the display's boundary" "off the display, but
within 4 feet of it" and the foreground portions for each region
can be recorded separately. Small thumbnail or full-size copies of
the vision foreground-background classification image can also be
directly logged for later analysis.
[0032] Data from the person-tracking information and/or the
foreground portions may be referred to as vision data and can be
written (e.g., stored) on a periodic basis (e.g., once per second)
to a log file. In one embodiment, the log file is stored in data
storage 232. This log file can then be analyzed, either locally or
centrally, at a later time by data analysis module 234. In
addition, the log entries may be time-stamped and may contain
information about what content is running at that time.
[0033] In addition to the vision data, data about the specific
interactions that took place on the display can be gathered. For
example, suppose that interactive video display system 210 is
showing a survey that asks users questions and takes their
responses by having them touch virtual buttons. The votes on the
virtual survey could be recorded in the logs along with the vision
data. In the case of video game content, the scores and actions of
the players could be added to the log file. In the case of
interactive informational content, such as, a virtual shopping
catalog, each instance of a product or item being viewed could be
recorded in the log file. In general, any information about an
instance of human interaction with a virtual object on the screen
can be logged for later analysis. This can be used as a valuable
feedback tool for advertisers, game designers, and content
creators.
[0034] Use of Data for Monitoring
[0035] In many situations, the display or screen of interactive
video display system 210 requires maintenance and cleaning. In
public installations, this maintenance and cleaning generally takes
place after hours, when the display is turned off. In order to
verify that maintenance and cleaning is taking place on a regular
basis, the data logs can be checked for activity after hours.
[0036] Analysis of Content Popularity
[0037] In other situations, interactive video display system 210
sequentially displays different pieces of content, which will be
referred to as "spots" or "video spots", each for a length of time.
The spots are stored in video spot storage 216. This series of
spots can be conceptually compared to a series of television
commercials. There is flexibility in the number of spots, the
length of time that they play, and the order in which they play. In
one embodiment, video spot scheduler 236 controls the scheduling of
the spots, wherein video spot scheduler 236 may be located locally
or remotely, as described above.
[0038] Data analysis module 234 is operable to perform data
analysis on the gathered data stored in data storage 232. In one
embodiment, the popularity of each spot can be determined based on
the logged data as described above. For example, a spot's
popularity can be measured by taking the average of the logged
values for the foreground portion or the number of people on the
screen during the periods when the spot was showing. However, there
are various issues that should be considered with the foregoing
approach. First, the number of people on the screen when the spot
begins playing is determined by the popularity of the previous
spot. For example, a spot that follows an unpopular spot will tend
to have lower activity levels than it otherwise would have. Second,
the number of people interacting with a spot depends on the number
of people near the display; if very few people are in the venue
where the interactive system is installed, then relatively few
people will interact with the system. Ways to handle the foregoing
problems will be further described below.
[0039] Controlling Effects of Prior Spot Popularity
[0040] There are several ways to reduce or eliminate the effects of
the previous spot's popularity on the spot that is being evaluated.
In one embodiment, the order in which spots are played is
randomized, so that each spot follows each other spot with equal
probability. There may be some deviation from true randomness to
prevent the same spot from playing twice in too short a time
period. Thus, any effect of the previous spot on the current spot's
popularity would be averaged out across all spots, giving a
somewhat better sense of the spot's true popularity.
[0041] In another embodiment, the length of time that the spot runs
can be increased so that the effects of the previous spot are
limited.
[0042] Furthermore, there are other ways of measuring various
aspects of popularity that are less sensitive to the popularity of
the previous spot. The following are some illustrative examples. In
the case of examples that describe what to do for each showing of
the spot, presume that the statistic gathered is averaged over the
number of showings. Nearly all these techniques work for both
person-tracking data and foreground portion data. Techniques
described earlier explain how to turn foreground portion data into
a rough estimate of the number of people. These techniques can be
applied equally well to data from the camera's whole image or data
from a specified region, such as the area within the interactive
display or the area near the interactive display.
[0043] In one embodiment, for each showing of the spot, only the
data for the last few seconds of the showing is analyzed when
computing the average number of people or foreground portion. This
average is referred to as "average popularity". In another
embodiment, for each showing of the spot, the difference between
the showing's average popularity and the average popularity during
the previous spot is computed. This shows whether the current spot
caused an increase in the number of people on the screen.
[0044] The above two approaches can be combined; in another
embodiment, the difference between the showing's average popularity
and the average popularity during the previous spot is computed.
However, only the last few seconds of the showing of the current
spot and the previous spot are used when computing the
averages.
[0045] In another embodiment, for each showing of the spot, the
difference between the number of people (or foreground portion) at
the beginning of the showing and the number of people (or
foreground portion) at the end of the showing is computed. The
beginning and end could either be instantaneous or refer to the
average of the first few seconds and last few seconds
respectively.
[0046] It should be appreciated that the aforementioned approaches
to determining popularity attempt to measure how many people were
interacting with the display as a result of a particular spot.
However, there are other types of statistics that can be gathered
with regard to human interactions with the interactive display. The
next few paragraphs describe other such measures.
[0047] In a further embodiment, for each showing of the spot, the
number of people who entered the display and the number of people
who left the display during the showing are counted. Popular spots,
of course, have more people enter and fewer people leave, in some
cases, though, it would be desirable to have many people enter and
leave, thus allowing the content to be seen by as many people as
possible. The information about the number of people entering and
leaving can be derived directly from person tracking data or
estimated from the foreground portion data by looking for quick
rises and drops (of a particular minimum size) in the foreground
portion data.
[0048] In another embodiment, for each showing of the spot, the
average length of time that people playing with the spot have been
on the display is recorded. Depending on the kind of message
delivered by the interactive content, the person controlling the
system may want either a long or a short length of stay.
[0049] In another embodiment, the amount of movement that takes
place during the spot's showings is recorded. The amount of
movement can be derived as the average speed of people on the
display, which can be found by examining the position information
in the person-tracking data. An alternative measure of the amount
of movement can be calculated from the foreground/background images
by computing the portion of pixels that switched from foreground to
background or vice versa between two such images a very short time
apart. In order to compute the average amount of movement, this
image difference would ideally be made several times during each
showing of the spot. This allows the person controlling the system
to distinguish between spots that promote lively behavior and spots
that create a more sedate atmosphere but are nonetheless
popular.
[0050] In another example, for each spot, the number of showings
for which no one was interacting with the display the beginning (or
no one interacted with the previous spot) but at least one person
was interacting at the end is determined. This measures the spot's
"first user attraction level," e.g., how good it is attracting a
person to interact with the display when there currently is no one
interacting with the display. In a similar way, a spot's "group
appeal" can be characterized by the number of times that one person
was playing with the spot at the beginning of the showing and
multiple people were playing with the spot at the end of the
showing.
[0051] Controlling Overall Traffic Level
[0052] Embodiments of the present invention provide a way to
account for the number of people in the venue where interactive
video display system 210 is installed. One way to do this would be
to simply obtain attendance data from the venue; many venues have
such data by day and even by hour. Alternatively, the attendance
data can be obtained by performing surveys or spot checks on the
number of people in attendance, or by use of a wide-angle camera
and analysis of the camera image to determine the number of people
in attendance. Then, the popularity of each spot relative to the
overall number of people who saw the interactive display system can
be determined. Then, a given spot's popularity could then be
compared fairly across venues with very different levels of
attendance.
[0053] Alternatively, if it is desired to compare the relative
popularity of different spots, all the spots could be run at the
same general time of day for the same days and in the same venues
so that any differences in overall traffic level affect all spots
equally.
[0054] External Factors Affecting Popularity
[0055] The popularity of a given spot may vary greatly depending on
a variety of other conditions or factors. These external factors
240 include, but are not limited to, time of day, day of the week,
season, weather, physical location of installation (geography),
type of interactive installation (wall projection, floor projection
etc.), gender, age, income demographic of people visiting the venue
of the installation, and how frequently the spot is shown, etc.
[0056] Spots that are popular under one set of conditions may be
less popular under another set of conditions. In one embodiment,
data analysis module 234 is operable to receive external data 240
in its analysis of the data. Since this data is either recorded in
the spot logs or can be matched (based on time and location) to the
spot logs, statistics on the popularity of a spot can be determined
given some or all of these conditions.
[0057] Using Popularity Information
[0058] If a goal is to schedule a set of spots for a given
interactive video display that optimizes the overall popularity of
the display, then past popularity data from one or more interactive
video displays can be analyzed to determine what spots should be
chosen.
[0059] It is useful to put the popularity log information, along
with associated external factors in a central searchable database
format such as MySQL, especially if there are multiple
installations of interactive video displays.
[0060] The people responsible for determining the schedule of spots
can make use of this database by querying for the popularity data
for these spots from the database. They have the option of limiting
this data to similar external factors (e.g., looking up the
popularity of spots running on weekdays from 10:00 PM to 12:00 AM
at mall installations in New England) so as to allow the most
accurate judgments to be made. The database could then produce a
list of the most popular spots given those conditions. Based on
what the scheduler wants, different methods of computing
popularity, such as the ones described earlier, may be
employed.
[0061] Automatic Scheduling
[0062] The spot popularity data can also be used to allow for
automatic scheduling at video spot scheduler 236, both at a micro
and at a macro level. At a micro level, an interactive video
display system could run spots that are found through the log
processing to have high "first user attraction level" (as defined
earlier) when there is no one at the display and run spots with
high "group appeal" (as defined earlier) when there is currently
one person at the display. In addition, the system could look at
its own spot popularity data over the last few minutes or hours,
and change the schedule to show popular spots more often or stop
showing unpopular spots. The system could even directly solicit
feedback from users. For example, the system could display an
interactive button that asks users to touch the button if they like
the spot and want to see more like it.
[0063] At a macro level, a variety of machine learning processes
can be employed, including but not limited to, neural networks,
hidden Markov models, a mixture of Gaussian models, and principal
component analysis, to build a model of the relationships between
the popularity of each showing of each spot and the set of
conditions under which that showing occurred. These machine
learning processes could then automatically predict what set of
spots would perform best at any given time and place, and
automatically reschedule them to optimize their popularity.
[0064] In one embodiment, as shown in FIG. 3, the present invention
provides a process 300 for managing an interactive video display
system, are described herein. In one embodiment, process 300 is
carried out by processors and electrical components (e.g., an
interactive video display system) under the control of computer
readable and computer executable instructions, such as interactive
video display system 210 of FIG. 2. Although specific steps are
disclosed in process 300, such steps are exemplary. That is, the
embodiments of the present invention are well suited to performing
various other steps or variations of the steps recited in FIG.
3.
[0065] At step 310 of process 300, a plurality of video spots are
displayed on the interactive video display system. In one
embodiment, the plurality of video spots are displayed in a pseudo
random order. In one embodiment, the length of time that a
particular video spot is displayed is adjusted.
[0066] At step 320, data based on interaction with the interactive
video display system corresponding to video spots of the plurality
of video spots is gathered. In one embodiment, the interaction is
determined according to person tracking. In another embodiment, the
interaction is determined according to a foreground/background
classification image.
[0067] At step 330, the data is stored, wherein the data is for use
in managing presentation of the video spots. In one embodiment, the
data is stored at a local memory of the interactive video display
system. In another embodiment, the data is transmitted to an
external computer system and is stored in a memory of the external
computer system.
[0068] At step 340, in accordance with one embodiment, the data is
analyzed for use in managing presentation of the plurality of video
spots. In one embodiment, analyzing the data includes determining
popularity for at least one video spot of the plurality of video
spots based on the data. In another embodiment, analyzing the data
includes determining a first user attraction level for at least one
video spot of the plurality of video spots based on the data. In
another embodiment, analyzing the data includes determining a group
appeal for at least one video spot of the plurality of video spots
based on the data. In one embodiment, external data is gathered for
use in analyzing the data.
[0069] At step 350, in one embodiment, the display schedule of the
plurality of video spots is adjusted based on analyzing data. In
one embodiment, the display schedule is automatically adjusted
based on the analysis. In another embodiment, the display schedule
is manually adjusted based on the analysis. In one embodiment, the
display schedule is fed into step 310 of process 300 for displaying
the video spots.
[0070] In an exemplary implementation, the present invention is
implemented using software in the form of control logic, in either
an integrated or a modular manner. Alternatively, hardware or a
combination of software and hardware can also be used to implement
the present invention. Based on the disclosure and teachings
provided herein, a person of ordinary skill in the art will know of
other ways and/or methods to implement the present invention.
[0071] It is understood that the examples and embodiments described
herein are for illustrative purposes only and that various
modifications or changes in light thereof will be suggested to
persons skilled in the art and are to be included within the spirit
and purview of this application and scope of the appended claims.
All publications, patents, and patent applications cited herein are
hereby incorporated by reference for all purposes in their
entirety.
[0072] Various embodiments of the invention, a method and system
for managing an interactive video display system, are thus
described. While the present invention has been described in
particular embodiments, it should be appreciated that the invention
should not be construed as limited by such embodiments, but rather
construed according to the below claims.
* * * * *