U.S. patent application number 10/821750 was filed with the patent office on 2004-09-23 for data correlation and analysis tool.
This patent application is currently assigned to DiscoverWhy Inc.. Invention is credited to Maguire, John, Potter, Terry.
Application Number | 20040187167 10/821750 |
Document ID | / |
Family ID | 21830069 |
Filed Date | 2004-09-23 |
United States Patent
Application |
20040187167 |
Kind Code |
A1 |
Maguire, John ; et
al. |
September 23, 2004 |
Data correlation and analysis tool
Abstract
A system for capturing a video image of the scene and for
correlating data relative to the scene is disclosed. The system
comprises a collection device which uses as inputs environmental
stimuli and video and audio stimuli which correlates the various
data such that statistical analysis can be performed.
Inventors: |
Maguire, John; (Concord,
MA) ; Potter, Terry; (Acton, MA) |
Correspondence
Address: |
BROMBERG & SUNSTEIN LLP
125 SUMMER STREET
BOSTON
MA
02110-1618
US
|
Assignee: |
DiscoverWhy Inc.
Bedford
MA
|
Family ID: |
21830069 |
Appl. No.: |
10/821750 |
Filed: |
April 9, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10821750 |
Apr 9, 2004 |
|
|
|
09413692 |
Oct 7, 1999 |
|
|
|
09413692 |
Oct 7, 1999 |
|
|
|
08929600 |
Sep 15, 1997 |
|
|
|
5995941 |
|
|
|
|
60026128 |
Sep 16, 1996 |
|
|
|
Current U.S.
Class: |
725/135 ;
348/E7.086; 348/E7.09; 725/132; 725/142; 725/24 |
Current CPC
Class: |
H04N 7/188 20130101;
H04N 7/181 20130101; G06Q 30/0203 20130101; G06Q 30/0201 20130101;
G06Q 30/02 20130101 |
Class at
Publication: |
725/135 ;
725/132; 725/024; 725/142 |
International
Class: |
H04N 007/08; H04N
007/16; H04N 005/91; H04N 007/173 |
Claims
What is claimed is:
1. An apparatus for determining a response to a stimulus, the
apparatus comprising: a stimulus input that receives a stimulus
signal representing the stimulus; a response input that receives at
least one response signal, each response signal being indicative of
a response to the stimulus; and a correlator coupled with the
stimulus input and the response input, the correlator correlating
the received stimulus with each received response signal as a
function of time to produce an associative mapping.
2. The apparatus as defined by claim 1 wherein the associative
mapping is stored in an associative cache.
3. The apparatus as defined by claim 1 wherein the stimulus signal
comprises a plurality of frames, the associative mapping
correlating each frame with a portion of each response signal.
4. The apparatus as defined by claim 1 wherein at least one of the
one or more input devices is a sensor that measures an
environmental condition.
5. The apparatus as defined by claim 1 wherein the stimulus
comprises a time-delayed presentation.
6. The apparatus as defined by claim 1 wherein the associative
mapping associates at least one of video, audio and response
variables with time.
7. The apparatus as defined by claim 1 further comprising: an
output that directs an output signal to a display device, the
output signal including data for displaying the associative mapping
substantially instantaneously after the creation thereof.
8. The apparatus as defined by claim 1 wherein the response input
includes at least a first group of responses having first
variables, and a second group of responses having second variables,
at least one variable being different between the first group and
the second group, the first group being disposed at a first angle
to the stimulus that differs from a second angle of the second
group to the stimulus.
9. The apparatus as defined by claim 1 further comprising: at least
one monitoring device adapted to receive the stimulus and generate
the stimulus signal, the at least one monitoring device detecting
light outside the visible spectrum.
10. An apparatus as defined by claim 1 wherein the correlator
includes a stored stimulus signal to compare against the stimulus
signal and generate a difference signal representative of the
differences between the stimulus signal and the stored signal.
11. The apparatus as defined by claim 10 wherein the processor
alarms if the difference signal exceeds a threshold value.
12. The apparatus as defined by claim 1 further comprising a video
recorder being adapted to record an image generated by the display
substantially instantaneously after the creation thereof.
13. The apparatus as defined by claim 1 further comprising: an
analyzer in electrical communication with the correlator, the
analyzer adapted to perform statistical analysis on the input
signals from the response input to find selected segments of the
stimulus signal.
14. The apparatus as defined by claim 13 wherein the analyzer
determines a point of statistical interest as measured against
predetermined criteria.
15. The apparatus as defined by claim 13 wherein the analyzer
interpolates information based upon the input signals from each of
the one or more input devices.
16. The apparatus as defined by claim 13 wherein the analyzer
extrapolates information based upon the input signals from the
response input.
17. The apparatus as defined by claim 1 further comprising: a
computer in electrical communication with the correlator such that
the correlator transmits the stimulus signal and the input signals
in raw form to the computer.
18. The apparatus as defined by claim 17 wherein the correlator
performs digital signal processing on the stimulus signal and the
input signals forwarded to the computer.
19. The apparatus as defined by claim 18 wherein the computer
comprises a graphical user interface through which a user selects
which statistical analysis is performed.
20. The apparatus as defined by claim 18 wherein the computer
comprises a graphical user interface for displaying the associative
mapping in real time.
21. The apparatus as defined by claim 1 wherein the associative
mapping is addressable by content.
22. The apparatus as defined by claim 1 wherein the response input
is coupled to a network.
23. The apparatus as defined by claim 22 wherein the stimulus input
is coupled to the network.
24. A computer program product for use on a computer system for
determining the reason that an audience generates a response to a
stimulus, the computer program product comprising a computer usable
medium having computer readable program code thereon, the computer
readable program code including: program code for recording the
response from the audience to the stimulus; program code for
performing a statistical analysis of the response to determine at
least one notable reaction; program code for replaying the at least
one notable reaction to the audience; program code for querying the
audience as to why the at least one notable reaction was received;
and program code for recording a second response to why the at
least one notable reaction was received.
25. A method of determining an audience's response to a stimulus,
the method comprising: receiving, in real time, at least one
response to the stimulus; correlating, by time, the at least one
response and the stimulus to generate an associative mapping of the
at least one response; and storing the associative mapping such
that the associative mapping is accessible by content of the at
least one response.
26. The method as defined by claim 25 further comprising: receiving
search criteria representative of content of the at least one
response; and generating a response signal having data identifying
the at least one response that meet the search criteria.
27. The method as defined by claim 26 further comprising:
forwarding the response signal to a display device to display the
at least one response that meet the search criteria.
28. The method as defined by claim 25 wherein the act of
correlating comprises: generating an analyzed variable having the
at least one response and the stimulus at a given time.
29. The method as defined by claim 25 wherein the associative
mapping correlates the stimulus over time with the at least one
response.
30. A computer program product for use on a computer system for
determining an audience's response to a stimulus, the computer
program product comprising a computer usable medium having computer
readable program code thereon, the computer readable program code
including: program code for receiving, in real time, at least one
response to the stimulus; program code for correlating, by time,
the at least one response and the stimulus to generate an
associative mapping of the at least one response; and program code
for storing the associative mapping such that the associative
mapping is accessible by content of the at least one response.
31. The computer program product as defined by claim 30 further
comprising: receiving search criteria representative of content of
the at least one response; and generating a response signal having
data identifying the at least one response that meet the search
criteria.
32. The computer program product as defined by claim 31 further
comprising: forwarding the response signal to a display device to
display the at least one response that meet the search
criteria.
33. The computer program product as defined by claim 30 wherein the
program code for correlating comprises: program code for generating
an analyzed variable having the at least one response and the
stimulus at a given time.
34. The computer program product as defined by claim 30 wherein the
associative mapping correlates the stimulus over time with the at
least one response.
35. An apparatus for determining an audience's response to a
stimulus, the apparatus comprising: an input for receiving, in real
time, at least one response to the stimulus; a correlator for
correlating, by time, the at least one response and the stimulus to
generate an associative mapping of the at least one response; and a
storage module operatively coupled with the correlator, the storage
module storing the associative mapping such that the associative
mapping is accessible by content of the at least one response.
36. The apparatus as defined by claim 35 further comprising: a
search input that receives search criteria representative of
content of the at least one response; and means for generating a
response signal having data identifying the at least one response
that meet the search criteria.
37. An apparatus for processing a response to a stimulus for
display in real time, the apparatus comprising: a response input
that receives at least one response signal, each response signal
corresponding in time to the stimulus; a processor operatively
coupled with the response input, the processor associating the at
least one response signal with at least one portion of the stimulus
over time to produce an associative mapping representative of the
at least one response to the at least one portion of the stimulus;
and an output operatively coupled with the processor, the output
forwarding a display signal having data representing the
associative mapping.
38. The apparatus as defined by claim 37 further comprising: a
display device for receiving the display signal and displaying the
at least one response and the portion of the stimulus as
represented in the associative mapping.
39. The apparatus as defined by claim 37 further comprising a
comparator for comparing the stimulus to an archive of a prior
response to a prior stimulus stored on a storage device.
40. A method for processing a response to a stimulus for display in
real time, the method comprising: receiving at least one response
signal, each response signal corresponding in time to the stimulus;
associating the at least one response signal with at least one
portion of the stimulus over time to produce an associative mapping
representative of the at least one response to the at least one
portion of the stimulus; and generating a display signal having
data representing the associative mapping.
41. The method as defined by claim 40 further comprising:
displaying the at least one response and the portion of the
stimulus as represented in the associative mapping.
42. The method as defined by claim 40 further comprising comparing
the stimulus to an archive of a prior response to a prior stimulus
stored on a storage device.
43. A computer program product for use on a computer system for
processing a response to a stimulus for display in real time, the
computer program product comprising a computer usable medium having
computer readable program code thereon, the computer readable
program code including: program code for receiving at least one
response signal, each response signal corresponding in time to the
stimulus; program code for associating the at least one response
signal with at least one portion of the stimulus over time to
produce an associative mapping representative of the at least one
response to the at least one portion of the stimulus; and program
code for generating a display signal having data representing the
associative mapping.
44. The computer program product as defined by claim 43 further
comprising: program code for displaying the at least one response
and the portion of the stimulus as represented in the associative
mapping.
45. The computer program product as defined by claim 43 further
comprising program code for comparing the stimulus to an archive of
a prior response to a prior stimulus stored on a storage
device.
46. A computer program product for use on a computer system for
allowing a user to interactively search an audience's response to
one or more stimuli, the computer program product comprising a
computer usable medium having computer readable program code
thereon, the computer readable program code including: program code
for providing a stimulus to the audience; program code for
recording one or more responses to the stimulus in real-time;
program code for correlating by time the one or more responses to
the one or more stimuli to generate an associative mapping of the
one or more responses and the stimuli; program code for storing the
associative mapping such that the associative mapping is accessible
by content of the one or more responses and the one or more
stimuli; program code for prompting the user for search criteria
representative of content of the one or more responses; and program
code for displaying the user the one or more responses and the one
or more stimuli meeting the search criteria.
47. The computer program product as defined by claim 46 wherein the
program code for correlating the response to the one or more
stimuli comprises program code for generating an analyzed variable,
the analyzed variable including the one or more responses and the
one or more stimuli at a given time.
48. The computer program product as defined by claim 46 wherein the
search criteria include a time duration of the one or more stimuli
surrounding the response.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation application from a U.S.
patent application with Ser. No. 09/413,692, filed Oct. 7, 1999,
which claims priority from U.S. Pat. No. 5,995,941, filed Sept. 15,
1997 and issued Nov. 30, 1999, which claims priority from
provisional U.S. patent application Ser. No. 60/026,128, all three
disclosures of which are incorporated herein, in their entirety, by
reference.
BACKGROUND OF THE INVENTION
[0002] The present invention related generally to apparatus and
methods for correlating and analyzing data. More particularly, the
invention relates to an apparatus and method for correlating data
with respect to video images such that analysis on the data can be
performed and video images associated with data segments can be
readily retrieved. Many industries use optical systems in
conjunction with various sensors to monitor environmental
conditions. Examples of such industries are market research,
security, and quality control. Various collection tools are
employed in collecting the data and subsequently organizing the
data. A common thread among these commercially available systems is
that the data must be manually correlated with video data from the
optical systems. For example, in the field of market research being
performed on an oral presentation may include video images of the
person making the oral presentation as well as feedback from the
sample group. The feedback is generally obtained using electronic
equipment which gauges the audience members reaction to the oral
presentation. Once the presentation is complete the data from each
of the audience members must then be correlated to that of the
video segments. That is, when strong positive or negative reactions
are attained, the relevant segment of the video must be located and
data and the audience must be correlated therewith in order for the
audience feedback to be meaningful. Often, this is a tedious, time
consuming manual task that delays feedback to the oral presenter
from two (2) to six (6) weeks.
[0003] The common practice is then to provide a written report
detailing the analysis of the individual events that the audience
found interesting in either a positive or negative manner. The
report further identifies key responses that correlated to those
events. Occasionally, along with the report a market research firm
would provide the video tape of the oral presentation for further
analysis by the presenter.
[0004] These and other objects of the invention will be obvious and
will appear hereinafter.
SUMMARY
[0005] The aforementioned and other objects are achieved by the
invention which provides a data correlation and analysis tool and a
method associated therewith. The invention comprises a system for
capturing a video image of a scene and for correlating data
relative to the scene.
[0006] In the data correlation and analysis tool, a collection
means is used which receives the data from remote electronic
sources and receives a video signal representative of the video
image from a video camera.
[0007] The remote electronic sources that produce the data can be
any of various types of sensors ranging from manually driven
dial-type controls for human input to environmental controls, such
as temperature gauges or flow controls in the case of process
control, for example.
[0008] Electrically connected to the collection means is a
correlation means which correlates the data to the video signal. In
this way the environmental responses are associated with the video
images from the video camera substantially instantaneously.
Additionally, statistical analysis can now be performed on the data
to find particular segments of the video which were associated with
highs or lows or any of various other points that may be of
interest to us inquiring or analyzing the information.
[0009] In further aspects, the invention provides methods in accord
with the apparatus described above. The aforementioned and other
aspects of the invention are evident in the drawings and in the
description that follows.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The foregoing and other objects of this invention, the
various features thereof, as well as the invention itself, may be
more fully understood from the following description, when read
together with the accompanying drawings in which:
[0011] FIG. 1 shows a block diagram of a data correlation and
analysis tool in accordance with the invention;
[0012] FIGS. 2A and 2B show block diagrams of additional
applications of the data correlation and analysis tool shown in
FIG. 1;
[0013] FIG. 3 shows the data collection correlation tool and
analysis tool of FIG. 1 in greater detail;
[0014] FIG. 4 shows a table representative of correlated data as
found in memory within the data correlation and analysis tool of
FIG. 3;
[0015] FIG. 5 shows a table representative of correlated data which
would be found in a memory associated with the data correlation and
analysis tool of FIG. 1; and
[0016] FIGS. 6A and 6B shows sample user interfaces for a computer
display presenting the correlated and analyzed data generated by
the invention.
DETAILED DESCRIPTION
[0017] While the present invention retains utility within a wide
variety of data correlation and analysis systems, and data
acquisition systems generally, and may be embodied in several
different forms, it is advantageously employed in connection with
market research systems. Though this is the form of the preferred
embodiment and will be described as such, this embodiment should be
considered illustrative and not restrictive.
[0018] FIG. 1 shows a block diagram of a data correlation and
analysis system 10 being used for market research. The data
correlation and analysis system 10 uses processing module 12 which
has various inputs from the environment. In this embodiment, a
presenter. 14 is making an oral presentation to a sample audience
24, 28. The presenter 14 could be interested in audience feedback
for any of various reason: he/she has an upcoming important oral
presentation and wants to know audience reaction; or has a new
product or a series of products and is trying to ascertain public
reaction to the product, for example. Additionally, the presenter
14 could be trying to gain judgement analysis on such topics as
political, governmental, or legal presentations, for example.
[0019] In this embodiment, the presenter 14 speaks into a
microphone 18 and has a video image captured by a video camera 20,
both of which are fed into the processing module 12. One skilled in
the art will realize that the presenter 14 need not be local for
the invention to operate. The presenter 14 can be local or offsite,
and the presentation can be in real time or time delayed. Within
the processing module 12, the audio signal from the microphone 18
and the video signal from the video camera 20 are transmitted into
a correlation unit 22.
[0020] Simultaneously therewith, an audience segment 24 inputs via
response meters 26 their impression regarding the presentation
being made by the presenter 14. One skilled in the art will realize
that the audience can have varying population and demographics
depending upon an amount of error is judged acceptable for the
subsequent statistical analysis of their opinions.
[0021] In practice, as the presenter 14 speaks the members of the
first audience segment 24 will judge positively or negatively in
real time their impression of what is being said or done by the
presenter 14. This impression is dialed into the response meters 26
by indicating on the response meter a numerical value
representative of that impression. Each member of the audience has
a response meter 26 to ensure that individual opinions are
monitored. One skilled in the art will realize that the response
meter 26 can be an analog or digital device which is commonly
available in the art. In this embodiment, an analog meter is used
which has a dial with a predetermined range, such as from one to
ten, where one equates to a poor opinion of the presentation and
ten means the presentation is excellent.
[0022] A member of the first audience segment 24 then uses the
response meter 26 to dial in his/her impression of the presentation
by selecting a value in that range. The response signal from the
response meter 26 is fed back into the processing module 12 and
into the correlation unit 22 where the response signal is
correlated by time with the video signal and the audio signal.
[0023] In a simple data correlation and analysis system, the
aforedescribed system could be complete with each member of the
audience dialing in his/her impressions via the response meter 26
and being correlated. The hereinafter described analysis can then
be performed without need for any additional information. In this
embodiment though, a second audience segment 28 is shown also
having response meters 26. This second audience segment 28 can be
used for any of numerous reasons. The market research firm may wish
to know, for example, how do the audiences impressions change with
respect to their angle to the presenter, when they are in a
separate room from the presenter 14, or in another city, inter
alia. In this embodiment, the second audience segment 28 is located
at an angle substantially different from that of the first audience
segment 24.
[0024] A second video camera 30 is then located near the second
audience segment 28. Although the second video camera 30 is
optional, it does provide information on what is the actual image
being perceived by the second audience segment 28. In this
embodiment, the information obtained from these two groups can then
be interpolated or extrapolated to many varying angles and
distances from the presenter 14 with or without additional video
feedback associated with each one.
[0025] Another form of data from the environment of the audience
segments 24, 28 is environmental sensory information itself. An
example of such environmental sensory information is that of
temperature shown by a thermometer 34. Thermometers, moisture
meters, et cetera, can be disposed around each audience segment
such that data of environmental conditions is then correlated via
the correlation unit 22 to the video image providing additional
feedback for the presenter 14. Such a layout could show that as an
audience gets warmer their opinions of the presentation go down,
for example.
[0026] As previously stated, the various video, audio and data
signals are input into the correlation unit 22. The correlated data
36 is then passed to an analysis unit 38 where the correlated data
36 is analyzed. Generally speaking, statistical analysis is used in
the analysis unit 38 to find points of interest in the presentation
by the presenter 14. For example, the analysis unit 38 can
determine all low points where the audience was in general
agreement of having a negative impression. The presenter 14 would
be interested in such information such that it can be removed from
the presentation or altered to avoid such negative reactions.
[0027] The form of the analysis can exceed simple averaging
techniques. Numerous forms of statistical analysis can be done. For
example, demographic information can be input into the analysis
unit and can be stored in the correlation unit 22 or externally in
a computer 40. The correlated data 36 can then be analyzed by
demographic information such that the reactions of males versus
females, for example, at any point in time can be analyzed.
Further, analysis can be done by age or any various other
demographic criteria that had been requested of the audience
segments 24, 28.
[0028] In the case of using an external computer 40, analyzed
correlated data can be passed to the computer or the raw
information can be passed to the computer 40. If much of the
processing is to be performed by the computer 40, then one of the
purposes of the analysis unit 38 would be digital signal processing
such that signals are smoothed and noise is reduced within the
correlated data.
[0029] The computer 40 displays to user of the computer 40 through
a graphical user interface ("GUI") 42 any statistical analysis that
is requested as well as its associated video and audio segments.
The GUI 42 can display the data in real time where the correlated
data 36 is essentially passed through the analysis unit 38 to the
computer 40 and is displayed. Or, the GUI 42 can operate on
archival information where the information is stored on a bulk
storage device, such as a computer hard disk, and analysis is
performed on the archived information.
[0030] FIGS. 2A and 2B show additional embodiments of the
invention. FIG. 2A shows a data correlation and analysis system 50
being used for quality control where a product 54 comes down a
conveyor belt 52 to a quality control station having a camera 56
and a set of sensors 58. Depending on the type of product 54 that
is being analyzed, the camera 56 can be a video camera which takes
a video image of the product 54 in which case the video image is
compared to a stored video image. Based upon statistical analysis,
the data correlation and analysis system 50 then rejects or accepts
the product 54. The camera 56 could also be a sonar or x-ray device
for examining welds in manufactured devices. Further, in the case
of welds, additional environmental information can include the
temperature at which the weld was made for better quality control
of the welds.
[0031] FIG. 2B shows another embodiment of the invention where the
data correlation and analysis system 60 is being used for security
purposes. In this embodiment, a video camera 66 is located in a
room 62 having a door 64 which is being monitored. An environmental
sensor, in this embodiment a temperature gauge 68, is also located
in the room.
[0032] The video signal from the video camera 66 and temperature
information from the thermometer 68 are passed back to the
processing module 12 where each reading of the thermometer 68 and
each frame or frames of the video are associated with time and
analyzed. In this way, a statistical correlation can be found and
statistical anomalies can be flagged to security personnel. For
example, decreasing temperature can be correlated to the door 64
being opened alerting security personnel to the door 64 having been
opened and marking for the security personnel the time of the
decreased temperature and thus the entry. One skilled in the art
will understand that numerous types such as door sensors and smoke
sensors and quantities of sensors can be used to gain more
information about the environment.
[0033] FIG. 3 shows an additional embodiment of the invention where
a VCR 76 is included in the processing module 12. In this
embodiment, the video signal from the video camera 20 is passed
into the processing module 12 and into the correlation unit 22, but
is simultaneously passed into video storage device, such as a VCR
76, a writable CD or other device without regard to whether the
device is analog or digital. Likewise, the audio signal from the
microphone 18 is passed into the correlation unit 22 and is also
passed into the VCR 76. Since both the video signal and the audio
signal in this embodiment are analog signals, the VCR 76 can create
a video tape of the presentation in a standard format, such as VHS
for example. The presenter 14 can then have a full video tape of
his/her presentation upon completion of the presentation.
[0034] Substantially simultaneously, the second audience segment 28
is using the response meter 26 to dial in their feedback to the
presentation as previously described. In this embodiment, the
response meters 26 are again analog devices wherein an analog
signal representative of the response is passed into the
correlation unit 22. One skilled in the art will realize that
digital response meters can also be used. It should also be
apparent that any of the response meters 26 can be wireless, and as
such transport the response by RF to an antenna connected to the
processing module 12. Examples of such digital response meters are
computer devices where a computer display is shown to the audience
member and responses are either input numerically via a keyboard, a
touch-screen display is used, or any of various pointing
devices.
[0035] Within the correlation unit 22, the video signal, audio
signal and the numerous response signals pass through analog to
digital converters 70, 72, 78, 80 where the signals are converted
into digital form. Digital signals can then be passed into the
analysis unit 38 where they are selectively stored on a memory
device 74. A cache 21, as more fully described hereinafter, can
also be present in the correlation unit 22 and assist with passing
the digital signals from the memory device 74.
[0036] Depending on the implementation, the memory device 74 can be
a magnetic disk or can be solid-state memory.
[0037] The analysis unit 38 subsequently performs statistical
analysis on the output of the correlation unit 22 using a
microprocessor 82. In this embodiment a summary video can be made
where segments of the video information can be parsed based on
interesting statistical correlations among the audience members
thereby creating a summary video having only segments which are of
interests to the presenter 14. For example, for basic highlights or
lowlights, the analysis can simply look for somewhat uniform highs
and lows in the responses and transmit video segments common to
those responses.
[0038] If desired; demographics can be used to further refine the
statistical analysis as previously described. The demographics can
be input via the computer 40 such that ages, the sex and various
other information relating to the audience members is known to the
analysis unit 38 and video segments can be parsed based on
demographic information. For example, the presenter 14 may be
interested in knowing what the males thought of the presentation
versus the females, or what different age groups felt about
different segments of the presentation then the summary video would
present the highlights and lowlights for the desired demographic
group.
[0039] In this embodiment, the mass storage device 74 is a
high-capacity magnetic storage device. If the video is being
archived on the mass storage device 74, the video must be
compressed. The reason for this is that video images use a
substantial amount of storage: capacity. Current video technology
uses 30 frames per second where each frame has a normal resolution
of 640 by 480 pixels. While this resolution can be altered, the.
aforementioned resolution is considered to be an acceptable video
image.
[0040] The video is then generally in color and, therefore, has 24
bits of color associated with each pixel, eight bits per red,
green, and blue ("RGB") color, creating approximately 1 megabyte of
video information per frame at 30 frames per second. One skilled in
the art will realize that the resolutions and the storage
capacities used in the preferred embodiment are dependent upon the
current state of the art. Higher resolutions may be used as the art
advances in the area of image compression, or lower cost, higher
capacity bulk storage devices become available.
[0041] For a one-hour video, this creates approximately 100
gigabytes of video information. This amount does not include audio
or any of the response information necessary to do a proper
analysis. Therefore, the video data must be compressed. In
preferred embodiment, compression is performed within the
correlation unit 22 such that the video is reduced to ten frames
per second thereby cutting the storage requirements by one third,
though one skilled in the art will realize that increasing the
number of frames per second is advantageous.
[0042] FIG. 4 shows a table of the contents of a typical memory
used in the data correlation and analysis system. In a preferred
embodiment a multi-channel associative cache is-used. The
multi-channel associative cache is an associative memory model
which associates multiple video, audio and response variables 94
and newly created variables 96 with the dimension of time. In FIG.
4 there are multiple, up to `n` video and audio stimulus streams
that are gathered and stored within correlation unit 22.
[0043] The multi-channel associative cache logs the location of the
stored and synchronized digitized multiple video and audio frames,
as well as associations in the real time process, i.e., the
response variables and their responses with these frames, other
processed variables and new variables. These associations are then
ordered as a function of time.
[0044] This associative memory then allows access to associative
process variables 94, the new variables 96 as well as video and
audio frames 92 by content as opposed to by physical address. Thus,
analysis of the associated information is easily accessible.
[0045] This is referred to as a multi-channel associative cache
because there are multiple, separate streams of input. In the
associative cache the process variable might be a set of "male"
respondents who are responding to one or more of the video-audio
stimulus. In this example there could be up to 100 male respondents
during the video frames associated with the first time slice,
t.sub.1. Any number of actual responders are possible.
[0046] FIG. 5 is another method by which the content of a typical
memory can be displayed. Again, the multi-channel associative cache
associates with time 100 multiple variables 102 (v1, v2, . . . vm,)
such as multiple video, audio or text, with process variables 104
(p1,p2, . . . pO) such as responses, and environmental variables,
and the newly created variables 106 (nv1, nv2, . . . nvy,) such as
averages, for example. The newly created variables 106 can be one
variable, such as an average, or a multitude of variables, such as
a set of values of how far each process variable is from the mean,
for example.
[0047] In this Figure, however, not only is a location designated
in the table as a "floc" stored with each variable 102, the
presence of separate objects (o1, o2 . . . on) within each of the
variables 102 is noted by the presence of a 1. In this way, the
presenter 14 can also see how objects affected the response. For
example, at a political rally the system or presenter can designate
as an object each nominee. The system would design-ate the nominees
presence by a 1 for the nominee's object in the video variable and
the nominee's voice by a 1 for the nominee's object number in the
audio variable. Thus, the campaign can determine, for example, how
the politician was received when they were on stage versus when
they were talking on stage. If a nominee's spouse was designated as
an object, the campaign can have analyzed the effect of the
spouse's presence on the nominee's acceptance, for example.
[0048] The table can also convey the frequency of the variables 102
as is noted by the presence or absence of a "floc" in the table.
Reading a column in the table allows the presenter to visually see
the intermittent nature of the audio signal. In addition to
communicating visually the frequency of events, the presence or the
absence of a variable over time, can also assist the presenter to
choose an appropriate time period it should instruct the analysis
unit 38 to wait after the audio segment before analyzing the
response. In the preferred embodiment, the analysis unit 38 will
allow for a lag time between the one or more audio-video variables
102 and the process variables 104 upon which it will analyze, but
this can be overridden by the presenter.
[0049] Further, although the table can store locations of the
audio-video variables by a pointer, a representation of the video
frame can also be stored in the table. The representation would
normally be much smaller than the frame.
[0050] Turning now to FIGS. 6A and 6B which show control screens
that allow the operator to control the automatic analysis and
automatic creation of the summary audio-video which is shown above.
In this screen, one or more processed variables are available to
analyze along with an access to one or more audio-video variables.
Analysis of a selected set of process variables are then available
to be displayed along with its associated time series. Video
segments are also available to be searched to determine associated
process variables and their resulting analysis. The video-audio
search can be done via a video search frame or clip 118 or
supplement of a video frame clip. The video-audio segments
associated with the variables or analysis of the variables can be
displayed in a video subwindow 110. One can even select a set of
criteria, highs or lows for examples, so that analysis then done
can automatically find the associated video segments and play them
back or create a new video.
[0051] A text search string 116 can also be used to locate segments
of a video so that the associated responses or their analyses can
be retrieved, further analyzed and used to create new stimulus for
the research group. Text strings are usually a result of automatic
analysis of the audio streams and are also stored in the
associative map. Alternative display forms are easily created that
allow for the display of multiple process variables 138, 140, 142,
144, multiple analyzed processed variables as well as scrolling
text windows 132 as the video is played 130.
[0052] Pop down windows allow for the control of what statistical
analysis is to be performed on the selected process variables as
well as preferences and other control parameters. These pop down
windows also allow one to tell the system how to deal with sample
data and summary video outputs and inputs.
[0053] For example, the variable list 112 shows a list of the
variables that might be used in a particular study. A user would
select a process variable and the associative cache would return
from the analysis unit 38 values for that variable. The analysis
unit 38 would manipulate the data, such as smooth it and calculate
highs and lows, inter alia, and present to the GUI 42 the audio
data over time 120 and the response over time 122. Through the GUI
42, the presenter 14 can tell the analysis unit 38 to create other
new variables or tell the associative cache to find other
associated variables.
[0054] The invention may be embodied in other specific forms
without departing from the spirit or essential characteristics
thereof. The present embodiments are, therefore, to be considered
in all respects as illustrative and not restrictive, the scope of
the invention being indicated by the appended claims rather than by
the foregoing description, and all changes which come within the
meaning and range of equivalency of the claims are therefore
intended to be embraced therein.
* * * * *