U.S. patent application number 12/170059 was filed with the patent office on 2010-01-14 for system and method for calibrating and normalizing eye data in emotional testing.
Invention is credited to Jakob DE LEMOS, Ole Baunbaek Jensen, Golam Reza Sadeghnia.
Application Number | 20100010370 12/170059 |
Document ID | / |
Family ID | 41319895 |
Filed Date | 2010-01-14 |
United States Patent
Application |
20100010370 |
Kind Code |
A1 |
DE LEMOS; Jakob ; et
al. |
January 14, 2010 |
SYSTEM AND METHOD FOR CALIBRATING AND NORMALIZING EYE DATA IN
EMOTIONAL TESTING
Abstract
A system and method is provided for calibrating and normalizing
eye data of one or more subjects prior to and/or during emotional
testing of the subjects. In particular, initially performing one or
more calibration or normalization operations prior to an emotional
test of a subject may result in accurate evaluations of emotional
responses based on measurements of eye data. Additionally, further
calibration or normalization performed during the emotional test
may be used to refine the initial calibration or normalization,
further increasing the accuracy of the evaluated emotional
responses.
Inventors: |
DE LEMOS; Jakob; (Copenhagen
V, DK) ; Jensen; Ole Baunbaek; (Copenhagen S, DK)
; Sadeghnia; Golam Reza; (Copenhagen V, DK) |
Correspondence
Address: |
PILLSBURY WINTHROP SHAW PITTMAN, LLP
P.O. BOX 10500
MCLEAN
VA
22102
US
|
Family ID: |
41319895 |
Appl. No.: |
12/170059 |
Filed: |
July 9, 2008 |
Current U.S.
Class: |
600/558 |
Current CPC
Class: |
G06K 9/00604 20130101;
A61B 5/163 20170801; A61B 3/113 20130101; A61B 3/112 20130101; A61B
5/165 20130101; A61B 5/6821 20130101; A61B 5/16 20130101; A61B
5/1104 20130101 |
Class at
Publication: |
600/558 |
International
Class: |
A61B 3/00 20060101
A61B003/00 |
Claims
1. A computer-implemented system for emotional testing of subjects
based on measurement and processing of eye data, including an
emotion response tool for determining subjects' emotional responses
to test stimuli based at least on measurement and processing of eye
data, the system comprising at least one processing device
configured to: use a calibration module to induce a desired
emotional state in a subject prior to performing emotional testing
on the subject using test stimuli; use a baseline level
determination module to determine and record an emotional baseline
level of the subject prior to performing the emotional testing; use
a light intensity calibration module to present to the subject
stimuli of different light intensity values and measure light
reflex data at each light intensity value to determine the
subject's ocular response to each light intensity, the determined
ocular response including a change in pupil dilation from one
intensity value to another intensity value and a rate of the change
in the pupil dilation from one intensity value to another intensity
value; and use an interslide calibration module to condition the
subject's eyes between test stimuli, the interslide calibration
module configured to present an interslide having a predetermined
light intensity based on a light intensity of subsequent test
stimuli, wherein the subject's eyes can be conditioned to the
predetermined light intensity so that when the subsequent test
stimuli are presented, any ocular response thereto will be based
primarily upon the subject's emotional response to the test stimuli
as opposed a change in light intensity.
2. The system of claim 1, the at least one processing device
further configured to use a gaze calibration module to collect and
analyze gaze data to determine a validity of the collected gaze
data.
3. The system of claim 2, wherein the gaze calibration module is
configured to repeat the collecting and analyzing, or take another
action, when the collected gaze data is determined to be
invalid.
4. The system of claim 1, wherein the light intensity calibration
module is configured to present a first visual stimulus having a
first light intensity value, a second visual stimulus having a
second light intensity value, a third visual stimulus having the
first light intensity value, and a fourth visual stimulus having a
third light intensity value.
5. The system of claim 4, wherein the first light intensity value
is a zero intensity value corresponding to a black display, the
second light intensity value is a half intensity value
corresponding to a grey display, and the third light intensity
value is a full intensity value corresponding to a white
display.
6. The system of claim 4, further comprising an eye tracking device
configured to measure eye data before, during, and/or after
presentation of each visual stimulus.
7. The system of claim 1, wherein the baseline level determination
module is configured to induce a desired emotional baseline level
in the subject prior to performing the emotional testing.
8. The system of claim 1, further comprising a test stimuli
presentation module configured to present the test stimuli to the
subject, an eye tracking device configured to measure eye data
during presentation of each of the test stimuli, and the emotion
response tool, which includes at least one module for processing
the measured eye data to determine an emotional response of the
subject to each of the presented test stimuli.
9. The system of claim 1, wherein the emotion response tool can use
information relating to the emotional baseline level for individual
subjects to determine the subjects' emotional responses.
10. The system of claim 1, wherein the emotion response tool can
use light reflex information for individual subjects to determine
the subjects' emotional responses.
11. The system of claim 1, wherein the emotion response tool can
use eye data measured during various calibration phases to
normalize results of the emotional testing.
12. The system of claim 1, wherein the emotion response tool
comprises a software application, operable to execute on the
processing device, comprising one or more software modules enabling
the processing device to perform functions including calibration,
stimuli presentation, data collection, and data analysis.
13. A computer-implemented method for emotional testing of subjects
based on measurement and processing of eye data, wherein an emotion
response tool is used to determine subjects' emotional responses to
test stimuli based at least on measurement and processing of eye
data, the method comprising: inducing a desired emotional state in
a subject prior to performing emotional testing on the subject
using test stimuli; determining and recording an emotional baseline
level of the subject prior to performing the emotional testing;
presenting to the subject stimuli of different light intensity
values and measure light reflex data at each light intensity value
to determine the subject's ocular response to each light intensity,
the determined ocular response including a change in pupil dilation
from one intensity value to another intensity value and a rate of
the change in the pupil dilation from one intensity value to
another intensity value; and conditioning the subject's eyes
between test stimuli, the interslide calibration module configured
to present an interslide having a predetermined light intensity
based on a light intensity of subsequent test stimuli, wherein the
subject's eyes can be conditioned to the predetermined light
intensity so that when the subsequent test stimuli are presented,
any ocular response thereto will be based primarily upon the
subject's emotional response to the test stimuli as opposed a
change in light intensity.
14. The method of claim 13, further comprising collecting and
analyzing gaze data to determine a validity of the collected gaze
data.
15. The method of claim 14, further comprising repeating the
collecting and analyzing, or taking another action, when the
collected gaze data is determined to be invalid.
16. The method of claim 13, wherein presenting to the subject
stimuli of different light intensity values includes presenting a
first visual stimulus having a first light intensity value, a
second visual stimulus having a second light intensity value, a
third visual stimulus having the first light intensity value, and a
fourth visual stimulus having a third light intensity value.
17. The method of claim 16, wherein the first light intensity value
is a zero intensity value corresponding to a black display, the
second light intensity value is a half intensity value
corresponding to a grey display, and the third light intensity
value is a full intensity value corresponding to a white
display.
18. The method of claim 16, further comprising measuring eye data
before, during, and/or after presentation of each visual
stimulus.
19. The method of claim 13, further comprising inducing a desired
emotional baseline level in the subject prior to performing the
emotional testing.
20. The method of claim 13, further comprising: presenting the test
stimuli to the subject; measuring eye data during presentation of
each of the test stimuli; and processing the measured eye data
using the emotion response tool to determine an emotional response
of the subject to each of the presented test stimuli.
21. The method of claim 13, wherein the emotion response tool can
use information relating to the emotional baseline level for
individual subjects to determine the subjects' emotional
responses.
22. The method of claim 13, wherein the emotion response tool can
use light reflex information for individual subjects to determine
the subjects' emotional responses.
23. The method of claim 13, wherein the emotion response tool can
use eye data measured during various calibration phases to
normalize results of the emotional testing.
24. The method of claim 13, wherein the emotion response tool
comprises a software application, operable to execute on a
processing device, comprising one or more software modules enabling
the processing device to perform functions including calibration,
stimuli presentation, data collection, and data analysis.
25. The system of claim 1, wherein the test stimuli includes an
aroma.
26. The method of claim 13, wherein the test stimuli includes an
aroma.
Description
FIELD OF THE INVENTION
[0001] The invention relates to calibrating and/or normalizing eye
data for one or more subjects prior to, during and/or after
emotional response testing based at least on measurements of the
eye data for the subjects.
BACKGROUND OF THE INVENTION
[0002] In many circumstances, measuring a person's emotional state
and/or emotional response to various stimuli may provide valuable
information. For example, when marketers, researchers, or other
entities desire information relating to emotional responses,
various stimuli may be presented to a subject to evaluate an
emotional response to the presented stimuli. In general, the
stimuli presented to subjects may include a visual stimulus, such
as still or moving images, slides, and/or videos. As used herein, a
"subject" may generally include, for example, an individual
respondent, person, or other test subject for which emotional
response data may be desired. In any particular data collection,
analysis, or other session testing for emotional responses,
subjects may participate actively (e.g., responding to
instructions, viewing and responding to various stimuli, etc.) or
passively (e.g., collecting data from an unaware subject). As used
herein, "emotional response testing" may generally include a
variety of activities during one or more test stimuli are presented
to a subject to determine the subject's emotional response to the
test stimuli (e.g., advertising and/or marketing studies, voter
polling, and/or other testing).
[0003] Recently, the assignee of the present application has
developed a tool referred to as the Emotion Tool.TM., which
provides objective and non-intrusive techniques for evaluating a
subject's emotional response and visual attention to stimuli such
as print ads, market research materials, brochures, or other
stimuli. Some of the techniques for evaluating a subject's
emotional response may include measuring and processing various
forms of eye data for the subject (e.g., pupil dilation, blink
rate, eye movement, etc.). For example, visual stimuli may be
presented to the subject on a computer monitor having an
eye-tracking device coupled thereto. The eye-tracking device may
therefore be used to collect raw eye data from the subject, and the
raw eye data may be processed to provide a psycho-physiological
interpretation of an emotional response to the presented stimuli.
Further details and examples relating to this tool and the
techniques used therein can be found in U.S. Patent Application
Publication No. 2007/0066916, the disclosure of which is hereby
incorporated by reference in its entirety.
[0004] Although performing certain calibration steps prior to
emotional response testing may generally be known, existing
calibration techniques typically focus primarily on gaze tracking
(e.g., tracking a location where a subject may be looking at a
given moment). Existing techniques for calibrating gaze tracking
typically involve presenting a series of indicators at different
positions on a monitor (e.g., white circles on a black background),
and determining where on the monitor the subject is looking
relative to the position of each of the indicators. However, simply
calibrating for gaze fails to appreciate that different test
subjects can often have different emotional profiles and/or
different emotional states at the time of testing (e.g., accurately
evaluating a subject's emotional response to a stimulus may depend
on whether the subject was happy, angry, confused, or in another
emotional state when the test began). Existing techniques that seek
to compensate for an initial or preexisting emotional state have
focused on attempting to induce a neutral emotional state prior to
beginning an emotional response test (e.g., presenting a
presumptively neutral slide to the subject to induce the neutral
emotional state).
[0005] However, these techniques have a limited effect. For
example, in some circumstances, merely attempting to induce a
neutral emotional state can lead to flawed test results due to
differences among test subjects relating to, among other things,
emotional profiles, responses to the presumptively emotionally
neutral stimuli, ocular physiological characteristics (e.g. pupil
size, pupil dilation range, response time, blink characteristics,
eye movement characteristics), and/or other differences. For
example, where different test stimuli have different light
intensity values, various test subjects may experience different
physiological responses to the intensity of the stimuli (e.g. pupil
dilation may vary from one subject to another). This may be thought
of as a light reflex, which is a physical reaction as opposed to an
emotional response. In another example, problems can arise because
various subjects may look at different portions of a monitor before
test stimuli are shown, or worse, subjects may look away from the
monitor altogether. These variations, among others, can often lead
to errors in the measurement of emotional response.
[0006] Existing and known techniques for calibrating and
normalizing eye data to be used in emotional response testing
suffer from these and other drawbacks.
SUMMARY OF THE INVENTION
[0007] Various aspects of the invention overcome these and other
drawbacks of prior techniques for emotional testing based on
measurements and evaluations of eye data.
[0008] According to one implementation of the invention, a
calibration phase may be performed prior to emotional testing of
one or more subjects to induce a desired emotional state (e.g. an
emotionally neutral or other desired state). The desired emotional
state may be induced, for example, by presenting certain stimuli to
the subject (e.g. a presumptively emotionally neutral stimuli).
[0009] Subsequently, an emotional baseline level may be measured
for the subject to determine actual characteristics of the
subject's emotional state. For example, values for various forms of
eye data may be measured to determine whether the subject has
reached the desired emotional state (e.g., pupil size, eye
movement, blink rate, and/or other eye data). The emotional
baseline values may be recorded, providing a set of data that can
be used to help ensure that different subjects have a comparable
emotional state prior to beginning an emotional response test,
and/or to enable use of one or more normalization techniques after
testing has begun to determine the subject's actual emotional
response to one or more test stimuli.
[0010] According to one implementation of the invention, the
calibration phase performed prior to the emotional testing may
include presenting to the subjects one or more calibration stimuli
that have different light intensities (e.g., a zero intensity
stimulus, a half intensity stimulus, and a full intensity
stimulus). The subject's ocular response to the different light
intensities may be measured to determine the subject's light
reflex. For example, as the light intensity increases from zero
intensity to an intermediate intensity, and then to a full
intensity, the range and/or rate of pupil dilation or other forms
of eye data may be measured. The measured ocular response
information can be used to calibrate and/or normalize the subjects'
responses to different test stimuli having different light
intensities during testing.
[0011] According to one implementation of the invention, one or
more conditioning stimuli may be presented to the subject during an
emotional testing phase. The conditioning stimuli may include one
or more "interslides" (or other forms of stimuli) having
predetermined characteristics. The conditioning stimuli may be
presented to the subject during various phases of the emotional
testing (e.g. between test stimuli). Assuming the emotional testing
includes presenting one or more visual test stimuli (e.g. slides),
then the conditioning stimuli may be referred to as interslides. If
other forms of stimuli are used then other forms of conditioning
stimuli may be used. If used, interslides may include one or more
slides to neutralize light reflex, for example, to condition a
subject between stimuli on which the emotional testing focuses. For
example, an interslide conditioning stimulus may have a
predetermined light intensity based on a light intensity of a
subsequent visual test stimulus. As a result, the interslide
conditioning stimulus may condition the subject's eyes to the light
intensity of the subsequent test stimulus, such that any ocular
response that occurs upon presenting the test stimulus can be based
primarily upon the emotional response to the test stimulus, rather
than a change in light intensity.
[0012] In another example, the conditioning stimuli may include a
fixation indicator designed to induce the subject to gaze at a
predetermined location of a monitor (e.g., a central point on the
monitor, or another point). As such, an eye tracking device may
determine whether the subject is looking at the fixation indicator
prior to presenting the test stimuli. This can help normalize gaze
data by having subjects look at a common point before a test
stimuli is presented. This avoids erroneous gaze data readings.
[0013] According to one implementation of the invention, various
normalization techniques may be used to account for different
emotional baseline values, different response profiles to light
intensity, and/or other variations among test subjects. These and
other techniques may be used together, individually, or in various
combinations thereof. For example, a fixation indicator may also be
used in combination with sequential variations in light intensity
in order to calibrate for both gaze and pupil size/dilation
range.
[0014] In another example, the calibration and normalization
techniques may be used to account for variations in environment
that impact emotional responses of test subjects (e.g. from one
test to another, at different test sites, etc.). For instance, an
identical emotional response test may be presented to a plurality
of subjects at a first test site and a second test site, and based
on the measurements of the baseline values during the calibration
phases, a determination can be made as to whether the first test
site or the second test site induces certain emotional states in
the test subjects.
[0015] According to one implementation of the invention, a system
for calibrating and normalizing eye data of one or more subjects
may operate in a calibration phase, a test phase, and/or other
phase. During the calibration phase, the system may lead the
subject through a gaze calibration process, which includes
collecting gaze data using an eye-tracking device. The collected
gaze data may then be analyzed to determine a validity of the
collected gaze data. For example, the collected gaze data may be
determined as valid when the subject at least generally looks at
certain locations on a monitor that correspond to fixation
indicators (e.g., a cross or other indicator to draw the subject's
attention). However, when the gaze data appears to be invalid
(e.g., because the subject was not looking at the monitor or not
looking at the fixation indicators), the gaze calibration may be
repeated or other action may be taken, such as prompting the
subject to look at the fixation indicators. Upon collecting valid
gaze data, the calibration phase may include a short pause prior to
a subsequent calibration process that calibrates for emotional
baseline, ocular physiological measurement, and/or other forms of
eye date. It will also be apparent the calibration phase may
proceed directly to the subsequent calibration processes without
having the short pause.
[0016] According to one implementation of the invention, if
desired, the system may cause the monitor to display a fixation
indicator to draw the subject's attention to a particular location
when the gaze calibration has completed. A light intensity response
calibration process may then be implemented to measure eye
dilation, pupil dilation size, or another ocular response to
changes in light intensity. Various visual stimuli having different
light intensity values may be displayed on the monitor at the
location where the fixation indicator was presented. For example, a
slide or other visual stimuli having a first light intensity may be
presented (e.g. a zero intensity or black slide), where the
fixation indicator may be shown in connection with the zero
intensity slide if desired (e.g. a white cross may be presented on
the otherwise black slide). Next, a slide having an intermediate
light intensity may be presented followed by another zero intensity
slide (e.g. a half intensity or grey slide may be presented prior
to another a zero intensity or black slide). Then, the system may
show another slide having a full light intensity (e.g. a white
slide). In one implementation, the zero intensity and intermediate
intensity stimuli may be presented multiple times to gain greater
accuracy relating to the variations in the subjects' pupil
diameter, while the full intensity stimulus need be shown only once
because the full light intensity may not evoke as great a variance
in pupil diameter response due to light reflex. The subjects'
maximum and minimum pupil diameter may be measured for the zero and
intermediate intensity stimuli, and an average pupil diameter may
be computed using the maximum pupil diameter following the zero
intensity stimuli and the maximum pupil diameter following the
intermediate intensity stimuli. When the light intensity
calibration has completed, another slide or stimulus may be
presented to induce a desired emotional state in the subject to be
tested (e.g. a half intensity or grey slide may be presented to
induce an emotionally neutral state). Various eye data may then be
measured to ensure that the subject has reached the desired
emotional state, and one or more of the calibration processes may
be repeated until the desired emotional state has been reached.
[0017] According to one implementation of the invention, during the
test phase, the system may present various test stimuli on the
monitor to measure the subject's emotional response. The eye data
measured during the calibration phase may therefore be used to
normalize the measurements taken during the test phase. For
example, where the calibration phase shows that the subject was in
a confused state prior to the test phase, a confused response to a
given test stimulus may not necessarily indicate that the test
stimulus caused the confusion. In another example, when the
calibration phase shows that the subject was in an unhappy state
prior to the test phase, a pleasurable response to a given test
stimulus may indicate that the test stimulus was particularly
effective in inducing pleasure in the subject. Additionally and/or
alternatively, various interslides or conditioning stimuli may be
used during the test phase to reduce variations from one subject to
another. For example, when a given test stimulus induces a
significant emotional response in a subject, the conditioning
stimuli may induce a more neutral state in the subject to ensure
that any subsequent response does not carry the effect of the prior
stimulus. This may be used, for example, to establish uniform (or
other desired) test conditions. Other examples and techniques for
using the calibration and conditioning stimuli will be
apparent.
[0018] According to one implementation of the invention, the system
for calibrating and normalizing eye data may include one or more
output devices for presenting calibration, test, conditioning, and
other stimuli, one or more input devices for collecting eye data,
one or more processing devices for analyzing the collected eye
data, and one or more data repositories for storing the collected
and analyzed eye data.
[0019] According to one implementation of the invention, the input
devices may include one or more of an eye-tracking device, a manual
input device, a sensor, a microphone, a touch-screen display,
and/or other input devices to receive input, including eye data,
from one or more subjects. The eye-tracking device may include a
camera and/or another known eye-tracking device that can record and
track various properties of a subject's eyes (e.g., pupil size,
blink rate, eye position or gaze, eye movement, etc.). The
eye-tracking device may be coupled to a display device, integrated
with the display device, and/or configured as a stand-alone
device.
[0020] According to one implementation of the invention, the
eye-tracking device may interface with the processing devices via
any suitable wired or wireless connection (e.g., a USB link), and
the processing devices may further interface with the output
devices that present the calibration stimuli, testing stimuli,
conditioning stimuli, and/or stimuli to the subject. The processing
devices may therefore include one or more applications to enable
the various features and functions of the invention, including one
or more modules to perform functions relating to presenting stimuli
and analyzing the subject's responses thereto. Non-limiting
examples of such modules may include one or more of a calibration
module, a stimuli presentation module, a data collection module, a
data analysis module, an output module, and/or other modules. The
calibration module may comprise one or more of a gaze calibration
module, a pupil variation calibration module, an emotional baseline
calibration module, an interslide calibration module, and/or other
calibration modules. Furthermore, it will be apparent that one or
more of the modules comprising the applications may be combined,
and that for some purposes, all of the modules may or may not be
necessary. The system may be combined with a survey module, facial
expression analysis system and modules, behavioral determination
systems, cognitive determination systems and/or other physiological
measurements.
[0021] According to one implementation of the invention, the
calibration module may perform one or more calibration steps during
a calibration phase, prior to emotional testing of subjects, as
described in greater detail above. More particularly, the
calibration module may generally include the gaze calibration
module to determine whether a subject was looking at an appropriate
output device or an appropriate location on the output device.
Further, also as described above, the calibration module may
include the pupil variation calibration module to determine a
subject's pupil diameter and pupil response to differing light
intensities. For example, the pupil variation calibration module
may sample a pupil size of one or more subjects at different light
intensities to detect variations, the absolute values, or ranges in
the subjects' pupil diameter in response to the different light
intensities.
[0022] According to one implementation of the invention, the
results of the sampling that the pupil variation calibration module
performs may be stored in the data repositories for use during an
emotional testing phase. Because pupil diameter variations due to
light intensity of various test stimuli may impact the accuracy of
eye data collected in response to presenting the test stimuli, the
pupil diameter variations may have to be removed from the collected
eye data to obtain eye data that corresponds only to the subject's
emotional response to the presented test stimuli. For example, as
described above, an average pupil diameter of the subject may be
computed and used as a scaling factor to remove pupil diameter
variations due to light intensity. In particular, pupil data
collected during the calibration phase may be used to normalize
pupil data relating to the subject's response to different stimuli
presented during emotional testing. For example, when the
calibration phase results in a determination that the subject has a
given average pupil diameter when presented with a stimulus having
a certain light intensity, that average pupil diameter may form the
scaling factor for test stimuli having comparable light intensities
(e.g., the average pupil diameter may be subtracted from the pupil
diameter measured in response to the test stimulus, yielding a
change in pupil diameter that corresponds only to the subject's
emotional response to the test stimulus).
[0023] According to one implementation of the invention, the
calibration module may include the emotional baseline calibration
module to adjust or otherwise condition a subject's emotional
level. In particular, prior to emotional testing, the emotional
baseline calibration module may attempt to induce an emotional
state in the subject that is as close as possible to a desired
emotional state (e.g., an emotionally neutral and/or other desired
state). For example, the emotional baseline calibration module may
present a series of emotionally neutral stimuli to the subject via
the output devices until a blink rate pattern, a pupil response, a
saccadic movements, and/or other eye properties reach a desired
level. Any given emotionally neutral stimulus or combination of
emotionally neutral stimuli related to any of the body's five
senses may be presented to the subject. For example, in one
implementation, a soothing voice may address the subject to place
the subject in a relaxed state of mind. Further, the soothing voice
or another emotionally neural stimulus may or may not be
accompanied one or more visually pleasant emotionally neutral
stimuli and/or other stimuli.
[0024] According to one implementation of the invention, the
calibration module may include the interslide calibration module to
calibrate and/or condition subjects prior to and/or during an
emotional test (e.g., the interslide calibration module may present
emotionally neutral interslide stimuli to a subject in between or
among test stimuli). For example, the interslide calibration module
may create the interslide stimuli to have a light intensity
identical to a subsequent stimulus that will actually be used in
the test. In another example, the interslide calibration module may
create the interslide stimuli to have a pixel representation used
in the actual test stimulus (e.g., pixel values to be used in the
actual test stimulus may be scrambled to create the interslide
calibration stimulus with a distinct image yet the same overall
light intensity). If non visual stimuli are used (e.g. aroma), the
conditioning stimuli may be aromatically neutral (e.g. pure air) or
other aroma-based conditioning.
[0025] Various other objects, features, and advantages of the
invention will be apparent through the detailed description of the
implementations and drawings attached hereto. It will also be
understood that both the foregoing general description and the
following detailed description are exemplary and not restrictive of
the scope of the invention.
BRIEF DESCRIPTION OF THE DRAWINGS
[0026] FIG. 1 illustrates an exemplary system for performing
calibration of one or more subjects prior to and during emotional
testing of the subjects, according to one aspect of the
invention.
[0027] FIG. 2 illustrates an exemplary method for performing
calibration of one or more subjects prior to emotional testing of
the subjects, according to one aspect of the invention.
[0028] FIG. 3 illustrates an exemplary method for performing
calibration of one or more subjects during emotional testing of the
subjects, according to one aspect of the invention.
DETAILED DESCRIPTION
[0029] FIG. 1 illustrates an exemplary system 100 for performing
calibration of one or more subjects prior to and during emotional
testing of the subjects according to one implementation of the
invention. As shown, the system 100 may comprise various components
to implement various aspects of the invention, and may be
configured to perform various calibration steps prior to and/or
during emotional testing of one or more subjects.
[0030] The system 100 may include at least one of a computer 110,
one or more input devices 120 for collecting eye data, one or more
output devices 140 for presenting information to the subjects, and
one or more data repositories 170 for storing collected and
analyzed eye data. The computer 110 may be operatively coupled to
the input devices 120, the output devices 140, and the data
repository 170 via one or more interfaces 105.
[0031] The one or more input devices 120 may comprise one or more
of an eye-tracking device 122, a manual input device 124, a sensor
126, a microphone 128, a touch-screen display 130, and/or other
input devices 132 to receive input, including eye data, from one or
more subjects. The eye-tracking device 122 may include a camera
and/or another known eye-tracking device that can record and track
various properties of a subject's eyes (e.g., pupil size, blink
rate, eye position or gaze, eye movement, etc.).
[0032] The eye-tracking device 122 may be coupled to a display
device 142, integrated with the display device 142, and/or
configured as a stand-alone device. The manual input device 124 may
include one or more of a keyboard, a mouse, and/or another input
device that subjects can use to manually input information. The
sensors 126 may include one or more emotion detection sensors
and/or other sensors. The emotion detection sensors may comprise,
for example, one or more physiological sensors such as galvanic
skin response sensors, facial recognition sensors, and/or other
sensors that can detect various physiological responses from
subjects. The subjects may use the microphone 128 to provide
voice-based inputs (e.g. when providing a verbal response to
various instructions, stimuli, and/or other information).
[0033] The touch-screen display 130 may be provided to accept
manual input from subjects (e.g., physical contact or pressure
applied to a screen via the subjects' finger, a stylus, and/or
another body part and/or apparatus). Additionally, in one
implementation, the display device 142 may comprise a touch-screen
display that can be used to accept manual input in addition to
presenting instructions, stimuli, and/or other information to the
subjects.
[0034] According to one implementation, the one or more output
devices 140 may include one or more of the display device 142, a
speaker 144, and/or another output devices 146. The display device
142 may comprise one or more monitors, such as a cathode ray tube
display, a digital flat panel display, a liquid crystal display, a
plasma display, and/or any other display device suitable for
presenting instructions, messages, visual stimuli, and/or other
information to subjects. The speaker 144 may comprise one or more
speakers for audibly reproducing audio instructions or messages,
audible stimuli, and/or other information to subjects.
[0035] According to one implementation, the one or more databases
170 may be operatively connected to computer 110, and may include
and/or interface with one or more databases and/or other resources
for storing various types of data. According to one implementation
of the invention, the databases 170 may include a calibration
database 172, a stimuli database 174, a collected data database
176, an analysis results database 178, and/or other databases
180.
[0036] The calibration database 172 may store information relating
to one or more calibration stimuli for presentation to subjects
prior to and/or during emotional testing of the subjects. The
calibration stimuli may comprise one or more stimuli to induce an
emotionally neutral state, vary a light intensity, fixate a gaze,
or calibrate other eye properties of a subject. The calibration
database 172 may also store one or more conditioning stimuli that
may be presented to subjects during emotional testing of the
subjects (e.g., as "interslides" in between and among test
slides).
[0037] The stimuli database 174 may store information relating to
one or more test stimuli for presentation to subjects during
emotional testing of the subjects. As previously noted, the test
stimulus or stimuli presented to subjects may comprise any stimulus
or combination of stimuli relating to one or more of the subject's
five senses (i.e., sight, sound, smell, taste, and touch). The
stimulus may comprise any real, analog, or electronic stimulus that
can be presented to the subject via known or future-developed
technology. Examples of visual stimuli can include, but are not
limited to, pictures, artwork, charts, graphs, text, movies,
multimedia or interactive content (e.g., video games), and/or other
stimuli having visual characteristics. Other types of stimuli may
be also be presented to the subjects, either together with or
separately from the visual stimulus. The stimuli may be stored on
any suitable storage media, and can include live scenarios, textual
stimuli (e.g., surveys or questionnaires), olfactory stimuli (e.g.,
aromas), audible stimuli (e.g., music, recorded voices, sound
accompanying a commercial, etc.), or any other suitable stimulus
for which an emotional response test may be desired.
[0038] The collected data database 176 may store information
relating to various forms of eye data (e.g., pupil dilation, blink
rate, eye movement, eye position, and/or other eye properties). The
computer 110 may acquire the eye data from the eye-tracking device
122 or another of input devices 120. The collected eye data may
generally relate to physiological conditions of subjects (e.g.,
acquired from emotion detection sensors), which has been collected
from the subjects during the various calibration, conditioning,
normalization, and testing phases described herein. The computer
110 may analyze the data in the collected data database 176 to
determine emotional responses of the subjects to calibration,
conditioning, test, or other stimuli. Results of analyzing that
data may then be stored in the analysis results database 178.
[0039] According to one implementation of the invention, the
computer 110 may include one or more applications 150 to enable the
various features and functions of the invention, including a
stimuli presentation module 154 to present stimuli to a subject via
the output devices 140, a data collection module 156 to collect eye
data and other information from the input devices 120, and a data
analysis module 158 to analyze the data collected from the subject
in response to the presented stimuli. In addition, a calibration
module 152 can be used to calibrate, condition, and otherwise
normalize eye data prior to and/or during emotional response
testing of the subject. The calibration module 152 may comprise one
or more of a gaze calibration module 152a, a pupil variation
calibration module 152b, an emotional baseline calibration module
152c, an interslide calibration module 152d, and/or other
calibration modules. Furthermore, it will be apparent that one or
more of the modules comprising the applications may be combined,
and that for some purposes, all of the modules may or may not be
necessary.
[0040] The calibration module 152 may perform one or more
calibration steps during a calibration phase, prior to emotional
testing of subjects. In particular, the calibration module 152 may
perform one or more calibration steps prior to emotional testing of
subjects, including one or more of a gaze fixation calibration, a
pupil variation calibration, and an emotional baseline calibration.
Additionally, the calibration module 152 may perform one or more
interslide or conditioning calibration steps during the emotional
testing of subjects.
[0041] More particularly, the calibration module 152 may include
the gaze calibration module 152a to determine whether a subject was
looking at an appropriate output device 140 or an appropriate
location on the output devices 140. The gaze calibration module
152a perform a gaze calibration process, which may include
instructing a subject to track, with his or her eyes, movement of a
visual indicator displayed on display device 142. For example, the
visual indicator may assume various shapes, sizes, and/or colors
(e.g., a small white cross displayed against a black background).
The eye tracking device 122 may then track the subject's gaze as a
location on the display device 142 where the subject is currently
looking (e.g., x, y, z co-ordinates defining a display position).
The calibration module 152 may therefore use the gaze calibration
module 152a to establish a frame of reference for the subject's
gaze.
[0042] In particular, during the gaze calibration process, the data
collection module 156 may collect gaze data via the eye-tracking
device 122 (e.g., a location on the output devices 140 where the
subject may be looking). The data analysis module 158 may then
analyze the collected gaze data to determine whether the gaze data
is valid (e.g., the gaze data may be rendered invalid upon
determining that the subject was not looking at the display device
142). When the gaze data has been determined to be invalid, the
gaze calibration module 152a may repeat the gaze calibration
process until valid gaze data can be obtained. When repeating the
gaze calibration process, the gaze calibration module 152a may
instruct the subject to reposition themselves relative to one or
more of the output tests 140, and further to track the movement of
the visual indicator on the display device 142. In one
implementation, the test for the subject may be terminated when the
gaze data (from the eye tracker) is invalid.
[0043] For example, various data processing techniques can be used
to determine if there is noise in the signal. Additionally, if the
gaze coordinate(s) is (are) outside a predetermined desired range,
the gaze data can be considered invalid. If there is no reaction
and/or no change in data over a predetermined period, this may be
determined to be invalid data. Other criteria may be used.
[0044] According to one implementation, the calibration module 152
may include the pupil variation calibration module 152b to
determine a subject's average pupil diameter, response to differing
light intensities (e.g. light reflex), or other pupil variations.
For example, the pupil variation calibration module may sample a
pupil size for one or more subjects at different light intensities
to detect variations, the absolute values, or ranges in the
subjects' pupil diameter in response to the different light
intensities. To enable compensation for these differences, the
pupil variation calibration module 152b may be used to calculate
pupil diameter variations due to light intensity of various test
stimuli. For example, pupil variation calibration module 152b may
present one or more emotionally neutral light intensity stimuli of
different light intensities (e.g., zero intensity, intermediate
intensity, full intensity) to subjects via the display device 142.
The eye-tracking device 122 may measure various eye properties of
the subjects at the different light intensities (e.g., pupil size,
pupil dilation, blink rate, and/or other properties).
[0045] Pupil variation calibration module 152b may sample the pupil
size of the subject at the different light intensities to detect
variations in the subject's pupil diameter in response to the light
intensity stimuli of different light intensities. The absolute
values of the subject's pupil diameter at different light
intensities may be measured, as well as the ranges of the subject's
pupil diameter across the different light intensities. To gain
better accuracy in determining variations in the subject's pupil
diameter, the pupil variation calibration module 152b may present
the zero intensity and the intermediate intensity stimuli in a
predetermined sequence. In one implementation, the full intensity
stimulus may only be shown once because the full intensity does not
evoke a great variance in pupil diameter response. Pupil variation
calibration module 152b may measure the subject's maximum and
minimum pupil diameter for the zero intensity light stimuli and the
half intensity light stimuli presented during the predetermined
sequence. The pupil variation calibration module 152b may then
compute the average pupil diameter as the average of the maximum
pupil diameter for the zero intensity stimuli and the maximum pupil
diameter for the intermediate intensity stimuli.
[0046] According to one implementation of the invention, the
calibration module 152 may include the emotional baseline
calibration module 152c to adjust or otherwise condition a
subject's emotional level. In particular, prior to emotional
testing, the emotional baseline calibration module 152c may attempt
to induce an emotional state in the subject that is as close as
possible to a desired emotional state (e.g., an emotionally neutral
and/or other desired state). For example, the emotional baseline
calibration module 152c may present a series of emotionally neutral
stimuli to the subject via the output devices 140 until a blink
rate pattern, a pupil response, a saccadic movement, and/or other
eye properties reach a desired level. Any given emotionally neutral
stimulus or combination of emotionally neutral stimuli related to
any of the body's five senses may be presented to the subject. For
example, in one implementation, a soothing voice may address the
subject to place the subject in a relaxed state of mind. Further,
the soothing voice or another emotionally neural stimulus may or
may not be accompanied one or more visually pleasant emotionally
neutral stimuli and/or other stimuli, which may or may not include
emotionally neutral stimuli.
[0047] According to one implementation of the invention, the
calibration module 152 may include the interslide calibration
module 152d to calibrate and/or condition subjects prior to and/or
during an emotional test (e.g., the interslide calibration module
152d may present emotionally neutral interslide stimuli to a
subject in between or among test stimuli). For example, the
interslide calibration module 152d may create the interslide
stimuli to have a light intensity identical to a subsequent
stimulus that will actually be used in the test. In another
example, the interslide calibration module 152d may create the
interslide stimuli to have a pixel representation used in the
actual test stimulus (e.g., pixel values to be used in the actual
test stimulus may be scrambled to create the interslide calibration
stimulus with a distinct image yet the same overall light
intensity).
[0048] Stimuli presentation module 154 may be used to present to a
subject one or more calibration stimuli during a calibration phase,
one or more conditioning stimuli during a conditioning calibration
phase, and one or more test stimuli during an emotional testing
phase. For example, various types of stimuli may be retrieved from
one or more of the calibration database 172 and/or the stimuli
database 174, and presented to the subject via the display device
142, the speaker 144, and/or other output devices 148. In one
implementation, the calibration database 172 and the stimuli
database 174 may be included in a common stimuli database. The
stimuli presentation module may also be or include an aroma
synthesizer to generate aromas as test stimuli. Thus the stimuli,
in various forms, may be stored or generated in real-time. In which
case, the conditioning stimuli may be aroma based (or
aroma-neutral, such as unscented or fresh air).
[0049] According to one implementation of the invention, the data
collection module 156 may collect various forms of eye data,
physiological data, and/or other data from the subject during each
of the calibration phase, the conditioning phase, and the emotional
testing phase. The data that the data collection module 156
collects may subsequently be stored in the collected data database
176.
[0050] According to one implementation of the invention, the data
analysis module 158 may analyze the collected data (e.g. eye data
and/or other data) in the collected data database 176. For example,
the data analysis module 158 may analyze the data in the collected
data database 176 to determine patterns and variations in gaze
data, eye movement, pupil diameter, pupil size, blink rate, or
otherwise for various subjects. Moreover, the data analysis module
158 may analyze the eye data in view of stimuli presented at
different light intensities to determine scaling factors or
criteria to normalize subsequent analysis that occurs during
emotional testing of the subjects. As a result, the data analysis
module 158 can determine an emotional impact of various stimuli
based on the analysis of the eye data and other information in the
collected data database 176. The results that data analysis module
158 produces may be directed for storage in the analysis results
database 178.
[0051] FIG. 2 illustrates an exemplary method 200 for performing
calibration of one or more subjects during a calibration phase,
prior to conducting emotional testing of the subjects. The
operations to be described in further detail herein may be
accomplished using one or more of the components of the system
described in greater detail above and, in some implementations,
various of the operations may be performed in different sequences,
in other orders, simultaneously, or various other operations may be
performed along with some or all of the operations illustrated in
FIG. 2. Accordingly, the description of the operations presented
herein should be regarded as exemplary only.
[0052] In an operation 202, a subject may be positioned in front of
one or more output devices, at least one of which includes an
eye-tracking device (e.g., sitting, standing, or otherwise). The
output devices may be used to present various calibration stimuli
to the subject, while the eye-tracking device may collect
eye-related information from the subject for calibration.
[0053] Gaze calibration may then be performed in an operation 204.
In particular, the gaze calibration operation 204 may include
instructing the subject to track, with his or her eyes, a moving
visual indicator displayed on a display device. The eye-tracking
device may therefore track the subject's eye movement to determine
where on the display device the subject looks. The location where
the subject looks may be defined as x, y, z and/or other
co-ordinates. As such, the gaze calibration operation 204 may
establish a frame of reference for the subject's gaze pattern
(e.g., an eye movement pattern).
[0054] In a decisional operation 206, the gaze data or other data
collected via the eye-tracking device during the gaze calibration
operation 204 may be analyzed to determine whether the gaze data is
valid. For example, the gaze data may be rendered invalid when
analysis of the gaze data indicates that the subject was not
looking at the display device or a given location or sequence of
locations on the display device (e.g., corresponding to the
location of the moving visual indicator). When decisional operation
206 determines that the gaze data is invalid, gaze calibration
operation 204 may be repeated until valid gaze data can be
obtained. In one implementation, repeating the gaze calibration
operation 204 may include instructing the subject to re-position
themselves, as in operation 202, prior to instructed the subject to
again track the movement of the visual indicator on the display
device in operation 204. In one implementation, when operation 206
determines that the gaze data is invalid, or when invalid gaze data
is collected a predetermined number of times, the calibration may
be terminated for the subject.
[0055] When decisional operation 206 does determine the gaze data
to be valid, a pupil variation calibration may be performed in an
operation 208. The pupil variation calibration may include
presenting the subject with a predetermined sequence of one or more
calibration stimuli having predetermined light intensity values or
emotional criteria (e.g., neutral and/or other criteria of a
stimulus). For example, the pupil variation calibration may present
emotionally neutral stimuli having different light intensities
(e.g., zero intensity, intermediate intensity, full intensity) via
the display device to determine the subject's pupil diameter and
pupil response to different light intensities. In one example, the
predetermined sequence may include a zero intensity or black
stimulus, followed by an intermediate intensity or gray stimulus.
Then, another zero intensity or black stimulus may be presented
followed by another intermediate intensity or gray stimulus.
Thereafter, a full intensity or white stimulus may be
presented.
[0056] The eye-tracking device may track and/or measure various eye
properties of the subjects (e.g., pupil size, pupil dilation, blink
rate, and/or other properties) at the different light intensities.
Additionally, the eye-tracking device may sample the eye properties
of the subjects at one or more rates that can enable the system to
accurately measure the values for the eye properties. For example,
the eye-tracking device may establish an average pupil diameter by
taking a maximum, minimum, or average pupil diameter when the zero
intensity stimuli were presented and averaging that pupil diameter
with a maximum, minimum, or average pupil diameter when one or more
of the intermediate or full intensity stimuli were presented. Thus,
using the tracked data, variations in the subject's pupil diameter
at different light intensities may be determined, and this data may
subsequently be used to calibrate and normalize the subject's
response to different test stimuli during emotional testing.
[0057] In an operation 210, an emotional baseline calibration may
be performed. For example, one or more stimuli having a
presumptively desired emotional impact (e.g. one or more
emotionally neutral stimuli) may be presented to the subject via
the display device or another output device. Eye properties or
other sensory characteristics may be measured to determine an
emotional state of the subject (e.g., blink rate, pupil size, eye
movement, heart rate, pulse rate, etc.). Thereafter, a decisional
operation 212 may include analyzing the eye properties or other
sensory characteristics measured in operation 210 to determine
whether the emotional state of the subject matches a desired
emotional state. For example, the desired emotional state may
generally include a neutral emotional state, although it will be
apparent that other emotional states may be used as the emotional
baseline, depending on the particular purpose of the emotional
testing to follow. When the decisional operation 212 determines
that the subject is not in the desired emotional state, the
emotional baseline calibration operation 210 may be repeated until
the blink rate pattern, pupil response, saccadic movements, heart
rate, and/or other eye properties or sensory characteristics
demonstrate that the subject has reached the desired emotional
state, whereby emotional testing of the subject may commence in an
operation 214.
[0058] FIG. 3 illustrates an exemplary method 300 for performing
calibration of one or more subjects during a conditioning phase,
which may occur while conducting emotional testing of the subjects.
The operations to be described in further detail herein may be
accomplished using one or more of the components of the system
described in greater detail above and, in some implementations,
various of the operations may be performed in different sequences,
in other orders, simultaneously, or various other operations may be
performed along with some or all of the operations illustrated in
FIG. 3. Accordingly, the description of the operations presented
herein should be regarded as exemplary only.
[0059] In one implementation, the conditioning phase may include
presenting one or more conditioning stimuli to a subject prior to,
in between, or among test stimuli presented to the subject. The
conditioning stimuli may include one or more "interslides" having
predetermined characteristics, which may be presented to the
subject during various phases of emotional testing. As such, where
emotional testing includes presenting one or more test stimuli
slides to a subject, the interslide conditioning stimuli may
include one or more slides having a primary purpose not of
emotional response testing, but of conditioning the subject prior
to those stimuli on which the emotional testing will focus.
[0060] For example, in an operation 302, the conditioning stimuli
may include a fixation indicator presented to the subject at a
predetermined location on a display device (e.g., at or near the
center of the display device). The fixation indicator may include
any suitable stimulus (e.g., a visual stimulus) to draw the
subject's attention thereto. The fixation indicator may generally
fix the subject's gaze at the predetermined location on the display
device to condition or otherwise normalize the subject for one or
more subsequent test stimuli. For example, the fixation indicator
may be presented to establish an emotionally neutral gaze condition
in the subject. In one implementation, operation 302 may include
determining whether the subject's gaze has been suitably fixed
based on a prior calibration of the subject's gaze (e.g., as
determined during the aforementioned calibration process).
[0061] In another example, in an operation 304, the conditioning
stimuli may include one or more emotionally neutral conditioning
stimuli. The emotionally neutral interslides may increase the
accuracy of eye data collected during emotional testing because the
emotionally neutral interslides serve to bring the subject's
emotional state back to a neutral state after having been exposed
to test stimuli that can include strong emotional content.
[0062] For example, the interslide stimuli may have a predetermined
light intensity, which may be based on a light intensity of a
visual test stimulus subsequently presented in an operation 306.
The interslide stimuli may have the same overall light intensity as
the actual test stimulus presented in operation 306 (e.g., by
scrambling pixels of the test pixels to create a distinct image
having the same overall light intensity). As a result, the
interslide stimuli presented in operation 304 may condition the
subject's eyes to the light intensity of the test stimulus
presented in operation 306. Therefore, when subsequently measuring
various forms of eye data in an operation 308 (e.g., blink rate,
pupil response, eye movement, etc.), any measured ocular response
can be attributed primarily to the subject's emotional response to
the test stimulus, rather than a change in light intensity.
[0063] In an operation 310, a determination may be made as to
whether to present one or more further test stimuli to the subject.
If further test stimuli are to be presented, the emotional testing
of the subject may be continued, for example, by presenting a gaze
fixation stimuli (e.g., operation 302) and/or emotionally neutral
interslide conditioning stimuli (e.g., operation 304). After again
presenting one or more of the conditioning stimuli, the further
test stimuli may be presented and the subject's emotional response
thereto measured (e.g., operations 306-308). However, it will be
apparent that, in various implementations, one or more of
operations 306 and 308 may be repeated without presenting the
interslide conditioning stimuli.
[0064] One way in which the calibration data may be used is to
determine a dynamic range of pupil dilation for each subject so
that actual pupil dilation during testing can be normalized (using
various known normalization techniques).
[0065] Aspects and implementations may be described as including a
particular feature, structure, or characteristic, but every aspect
or implementation may not necessarily include the particular
feature, structure, or characteristic. Further, when a particular
feature, structure, or characteristic has been described in
connection with an aspect or implementation, it will be understood
that such feature, structure, or characteristic may be included in
connection with other aspects or implementations, whether or not
explicitly described. Thus, various changes and modifications may
be made to the preceding description without departing from the
scope or spirit of the invention, and the specification and
drawings should therefore be regarded as exemplary only, and the
scope of the invention determined solely by the appended
claims.
* * * * *