U.S. patent application number 16/109298 was filed with the patent office on 2020-02-27 for gaming machine and method for evaluating player reactions.
The applicant listed for this patent is ARISTOCRAT TECHNOLOGIES AUSTRALIA PTY LIMITED. Invention is credited to Gregory Paul Schwartz.
Application Number | 20200066093 16/109298 |
Document ID | / |
Family ID | 69587090 |
Filed Date | 2020-02-27 |
United States Patent
Application |
20200066093 |
Kind Code |
A1 |
Schwartz; Gregory Paul |
February 27, 2020 |
GAMING MACHINE AND METHOD FOR EVALUATING PLAYER REACTIONS
Abstract
An electronic gaming machine includes a display, a digital
camera device, a credit input mechanism, and a processor programmed
to perform operations comprising: (i) receiving, from the digital
camera device, a digital image of the player; (ii) determining an
emotional state of the player by performing facial expression
analysis on the digital image; (iii) determining an emotion level
of the player by categorizing the emotional state of the player
based on the determined emotional state, the categorizing includes
a first state representing a positive emotional level and a second
state representing another emotional level; (iv) determining that
the emotional level is the other emotional level; and (v)
automatically initiating a game session action during the game play
session, the game session action is configured to cause the player
to transition to the positive emotional level.
Inventors: |
Schwartz; Gregory Paul; (Las
Vegas, NV) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
ARISTOCRAT TECHNOLOGIES AUSTRALIA PTY LIMITED |
North Ryde |
|
AU |
|
|
Family ID: |
69587090 |
Appl. No.: |
16/109298 |
Filed: |
August 22, 2018 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G07F 17/3239 20130101;
G07F 17/3206 20130101 |
International
Class: |
G07F 17/32 20060101
G07F017/32 |
Claims
1. An electronic gaming machine comprising: a cabinet; a display
device configured to display visual aspects of a wagering game; a
digital camera device integrated into the cabinet and configured to
capture digital image data of a player of the electronic gaming
machine during a game play session of a wagering game; a credit
input mechanism including at least one of a card reader, a ticket
reader, a bill acceptor, and a coin input mechanism, the credit
input mechanism configured to receive a credit wager; a storage
medium having instructions stored thereon; and a game controller
enclosed by the cabinet and configured to execute instructions
stored in a tangible, non-transitory, computer-readable storage
medium, which, when executed by the game controller, cause the game
controller to at least: transmit an instruction to the digital
camera device to initiate capturing of an initial digital image of
the player; receive, from the digital camera device, the initial
digital image of the player over a communications network coupling
the digital camera device and the game controller; submit the
initial digital image as input to a preliminary image processor
software module, the preliminary images processor is configured to
output a cropped facial image of the player by identifying a facial
position of a face of the player within the initial digital image
and cropping the initial digital image around the identified facial
position; in response to output of the cropped facial image, submit
the cropped facial image to an emotion analysis software module,
the emotion analysis software module is configured to evaluate the
cropped facial image using facial expression analysis techniques to
determine a current emotional state of the player; determine an
emotion emotional level of the player by categorizing the emotional
state of the player output by the emotion analysis software module,
the categorizing includes a first state representing a positive
emotional level and a second state representing another emotional
level; automatically initiate a game session action at the
electronic gaming machine and during the game play session when the
determined emotional level is the other emotional level, the game
session action is configured to cause the player to transition to
the positive emotional level.
2. The electronic gaming machine of claim 1, wherein the
preliminary image processor is further configured to: determine an
area within the initial digital image representing a face of the
player; and crop the initial digital image to include only the
determined area in the cropped facial image prior to evaluating the
cropped facial image.
3. The electronic gaming machine of claim 1, wherein the game
session action includes one of initiating a light show, adjusting
colors of the wagering game, and altering audio presented during
the wagering game.
4. The electronic gaming machine of claim 1, wherein the game
session action includes one of calling an attendant, providing
complimentary services, and providing complimentary credits for the
wagering game.
5. The electronic gaming machine of claim 1, wherein when executed,
the instructions further cause the game controller to at least:
capture gameplay data associated with the wagering game
contemporaneous to the capturing of the initial digital image;
timestamp the gameplay data and the emotional level to generate a
time-synchronized log of the game play session; and store the
time-synchronized log for analysis by one or more of a game
developer of the wagering game and a casino operator providing the
electronic gaming machine.
6. The electronic gaming machine of claim 1, wherein when executed,
the instructions further cause the game controller to at least
generate an engagement level of the player based at least in part
on the determined emotional state.
7. The electronic gaming machine of claim 6, wherein when executed,
the instructions further cause the game controller to at least:
determine a gaze direction of the player based on the initial
digital image; determine that the gaze direction of the player is
not directed at the electronic gaining machine, wherein generating
an engagement level of the player is further based at least in part
on the determining that the gaze direction of the player is not
directed at the electronic gaming machine.
8. The electronic gaming machine of claim 5, further comprising a
microphone configured to capture audio data near the electronic
gaming machine, wherein when executed, the instructions further
cause the game controller to at least: detect a source of noise not
related to the electronic gaming machine; and store the audio data
in the time-synchronized log.
9. The electronic gaming machine of claim 1, wherein when executed,
the instructions further cause the game controller to at least:
receive, from the camera device, digital video of an environment
behind the player; perform motion analysis on the digital video to
determine the presence of people walking near the electronic gaming
machine; and automatically identify foot traffic as a source of
distraction for the player based on the motion analysis.
10. The electronic gaming machine of claim 1, wherein the second
state represents a neutral emotional level, wherein the
categorizing includes a third state representing a negative
emotional level.
11. A method of analyzing facial expressions of a player, the
method being implemented on an electronic gaming machine, the
electronic gaming machine including at least one processor in
communication with at least one memory device, a digital camera
device configured to capture digital image data of the player
during a game play session of a wagering game, and with a display,
the method comprising: transmitting an instruction to the digital
camera device to initiate capturing of an initial digital image of
the player over a communications network coupling the digital
camera device and the game controller; receiving, from the digital
camera device, the initial digital image of the player over the
communications network; submitting the initial digital image as
input to a preliminary image processor software module, the
preliminary image processor is configured to output a cropped
facial image of the player by identifying a facial position of a
face ref the player within the initial digital image and cropping
the initial digital image around the identified facial position; in
response to the outputting of the cropped facial image, submit the
cropped facial image to an emotion analysis software module, the
emotion analysis software module is configured to evaluate the
cropped facial image using facial expression analysis techniques to
determine a current emotional state of the player; determining an
emotional level of the player by categorizing the emotional state
of the player output by the emotion analysis software module, the
categorizing includes a first state representing a positive
emotional level and a second state representing another emotional
level; automatically initiating a game session action at the
electronic gaming machine and during the game play session when the
determined emotional level is the other emotional level, the game
session action is configured to cause the player to transition to
the positive emotional level.
12. The method of claim 11, wherein the preliminary image processor
is further configured to: determine an area within the initial
digital image representing a face of the player; and crop the
initial digital image to include only the determined area in the
cropped facial image prior to evaluating the cropped facial
image.
13. The method of claim 11, wherein the game session action
includes one of initiating a light show, adjusting colors of the
wagering game, and altering audio presented during the wagering
game.
14. The method of claim 11, wherein the game session action
includes one of calling an attendant, providing complimentary
services, and providing complimentary credits for the wagering
game.
15. The method of claim 11, further comprising: capturing gameplay
data associated with the wagering game contemporaneous to the
capturing of the initial digital image; timestamping the gameplay
data and the emotional level to generate a time-synchronized log of
the game play session; and storing the time-synchronized log for
analysis by one or more of a game developer of the wagering game
and a casino operator providing the electronic gaming machine.
16. The method of claim 11, further comprising generating an
engagement level of the player based at least in part on the
determined emotional state.
17. The method of claim 16, further comprising: determining a gaze
direction of the player based on the initial digital image;
determining that the gaze direction of the player is not directed
at the electronic gaming machine, wherein generating an engagement
level of the player is further based at least in part on the
determining that the gaze direction of the player is not directed
at the electronic gaming machine.
18. The method of claim 15, further comprising: detecting, using
audio data captured by a microphone of the electronic gaining
machine, a source of noise not related to the electronic gaming
machine; and storing the audio data in the time-synchronized
log.
19. The method of claim 11, further comprising: receiving, from the
camera device, digital video of an environment behind the player;
performing motion analysis on the digital video to determine the
presence of people walking near the electronic gaming machine; and
automatically identifying foot traffic as a source of distraction
for the player based on the motion analysis.
20. The method of claim 11, wherein the second state represents a
neutral emotional level, wherein the categorizing includes a third
state representing a negative emotional level.
Description
TECHNICAL FIELD
[0001] The field of disclosure relates generally to electronic
gaming, and more particularly to an electronic gaming machine and
method that evaluate player reactions during game play.
BACKGROUND
[0002] Electronic gaming machines (EGMs), or gaming devices,
provide a variety of wagering games such as, for example, and
without limitation, slot games, video poker games, video blackjack
games, roulette games, video bingo games, keno games, and other
types of games that are frequently offered at casinos and other
locations. Play on EGMs typically involves a player establishing a
credit balance by inserting or otherwise submitting money and
placing a monetary wager (deducted from the credit balance) on one
or more outcomes of an instance, or play, of a primary game,
sometimes referred to as a base game. In many games, a player may
qualify for secondary games or bonus rounds by attaining a certain
winning combination or other triggering event in the base game.
Secondary games provide an opportunity to win additional game
instances, credits, awards, jackpots, progressives, etc. Awards
from any winning outcomes are typically added back to the credit
balance and can be provided to the player upon completion of a
gaming session or when the player wants to "cash out."
[0003] Slot games are often displayed to the player in the form of
various symbols arranged in a row-by-column grid, or "matrix."
Specific matching combinations of symbols along predetermined
paths, or paylines, drawn through the matrix indicate the outcome
of the game. The display typically highlights winning combinations
and outcomes for ready identification by the player. Matching
combinations and their corresponding awards are usually shown in a
"pay-table" that is available to the player for reference. Often,
the player may vary his/her wager to included differing numbers of
paylines and/or the amount bet on each line. By varying the wager,
the player may sometimes alter the frequency or number of winning
combinations, the frequency or number of secondary games, and/or
the amount awarded.
[0004] During game play, players may experience a variety of
emotions and levels of engagement with their game. Players that
enjoy and are highly engaged in a game tend to continue or repeat
playing that game. Accordingly, providing engaging and entertaining
games is desirable, both for the players as well as for casino
operators and game developers.
BRIEF DESCRIPTION
[0005] In one embodiment, an electronic gaming machine is provided.
An electronic gaming machine includes a display, a digital camera
device configured to capture digital image data of a player of the
electronic gaming machine during a game play session of a wagering
game, a credit input mechanism including at least one of a card
reader, a ticket reader, a bill acceptor, and a coin input
mechanism, the credit input mechanism configured to receive a
credit wager, a storage medium having instructions stored thereon,
and a processor. The processor is coupled to the display, the
credit input mechanism, and the storage medium. When executed, the
instructions cause the processor to at least: (i) receive, from the
digital camera device, a digital image of the player; (ii)
determine an emotional state of the player by performing facial
expression analysis on the digital image; (iii) determine an
emotion level of the player by categorizing the emotional state of
the player based on the determined emotional state, the
categorizing includes a first state representing a positive
emotional level and a second state representing another emotional
level; (iv) determine that the emotional level is the other
emotional level; and (v) automatically initiate a game session
action during the game play session, the game session action is
configured to cause the player to transition to the positive
emotional level.
[0006] In another embodiment, a method of analyzing facial
expressions of a player is provided. The method is implemented on
an electronic gaming machine, the electronic gaming machine
includes at least one processor in communication with at least one
memory device, a digital camera device configured to capture
digital image data of the player during a game play session of a
wagering game, and with a display. The method includes: (i)
receiving, from the digital camera device, a digital image of the
player; (ii) determining an emotional state of the player by
performing facial expression analysis on the digital image; (iii)
determining an emotion level of the player by categorizing the
emotional state of the player based on the determined emotional
state, the categorizing includes a first state representing a
positive emotional level and a second state representing another
emotional level; (iv) determining that the emotional level is the
other emotional level; and (v) automatically initiating a game
session action during the game play session, the game session
action is configured to cause the player to transition to the
positive emotional level.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] An example embodiment of the subject matter disclosed will
now be described with reference to the accompanying drawings.
[0008] FIG. 1 is a diagram of exemplary EGMs networked with various
gaming-related servers;
[0009] FIG. 2 is a block diagram of an exemplary EGM;
[0010] FIG. 3 is a diagram of an exemplary gaming device that may
be used to capture player engagement and player emotion of a player
and enhance player experience during game play;
[0011] FIG. 4 is a component diagram illustrating an example
embodiment of gaming device.
[0012] FIG. 5 illustrates example images of a person (e.g., the
player) in various emotional states as exhibited by facial
expression; and
[0013] FIG. 6 is a flowchart illustrating a process for analyzing
facial expressions of the player.
DETAILED DESCRIPTION
[0014] An electronic gaming machine (EGM) (or "gaming device") is
described herein, in which player engagement and player emotion are
captured during game play and used to enhance player experience. A
player may experience a wide array of emotions during a game play
session (e.g., from the time the player first sits down at the EGM
and begins play, and until they stand up and leave the EGM). A
particular game design or game features may provide differing
levels of interest, entertainment, or engagement levels, each of
which may be subjective to the player. Further, a gaming
environment where the player plays the EGM, such as a casino, may
present distractions or other environmental characteristics (e.g.,
based on where the EGM is positioned within the casino or on other
occurrences happening near the EGM). Such distractions may have an
impact on player engagement and enjoyment.
[0015] In an example embodiment, the EGM includes a digital camera
device that operates in conjunction with an expression evaluation
engine and a game controller to capture emotional states of the
player during game play. More specifically, the digital camera
device is integrated into the EGM and faces the player, capturing
facial expressions of the player during game play. The expression
evaluation engine receives digital video of the player and
evaluates facial expressions made by the player to determine an
emotional state of the player at various times. For example, the
expression evaluation engine may determine that the player is
joyful, angry, or sad. The game controller categorizes an emotion
level of the player (e.g., positive, neutral, negative) based on
the detected emotional state. Further, the game controller may also
determine a level of engagement (e.g., high, low) of the player
based on the emotional state (a level of arousal, e.g., calm,
bored, alert, nervous).
[0016] The game controller may treat the positive emotion level as
the most desirable emotion level of the player (e.g., where the
player's excitement level and engagement with the game are high).
Players may experience this level, for example, when there are
exciting features of the game occurring, when the player has
recently won a large payout, or when they relate to or are excited
by the game design. The neutral level represents an emotion level
in which the player is content, calm, or relaxed. Players may
experience this level, for example, when they have recently
experienced only modest wins, when there are no particularly
interesting features of the game being activated, or when they are
moderately disengaged from game play. The neutral level may
commonly occur for many players at various times during game play
and, while not necessarily undesirable, the player's excitement and
engagement are only moderate. As such, the neutral level is less
desirable than the positive level. The negative level represents an
emotion level in which the player is, for example, upset, angry,
sad, bored, or fatigued. Players may experience this level, for
example, when they have regularly lost over recent plays, when they
do not enjoy the particular game or game features, when they are
confused by the game, or when they have played too long without
positive experiences in the game. The negative level is the least
desirable emotion level for the player.
[0017] In some embodiments, the game controller may perform various
actions during a game play session based on the player's present or
recent emotion level (referred to herein as "game session
actions"). For example, when the expression evaluation engine
determines that the player is at a negative emotion level (e.g.,
presently, or consistently for a period of time), or when the
player is unengaged with the game, the game controller may perform
game session actions in an attempt to increase the player's emotion
level. Game session actions may include, for example, adjusting
colors of the game (e.g., to make the game more southing, or to
change the appearance of the game to attract the player's
attention), temporarily increasing a volume level of music or other
sounds, increasing game music tempo, initiating a light show or
entertaining video presentation, or offering free plays or other
"comps" to the player.
[0018] In some embodiments, the game controller may capture game
play data contemporaneously with emotion level data of the player
during game play (collectively, "session data"). The game
controller may transmit session data to a central server or store
session data for later retrieval. Session data synchronizes game
play data (e.g., what is occurring during a particular game) with
emotion level data (e.g., what emotion level is being experienced
by the player at that time), thereby allowing an analysis of player
reaction to particular game play events or other environmental
factors. Session data may be used (e.g., by game developers or
casino operators) to evaluate player experience during game play
sessions. For example, session data may allow game developers to
evaluate which features of the game caused player excitement, joy,
or anger, or what makes players get distracted or excited about the
game. Session data may allow casino operators, or others, to
evaluate player enjoyment of particular games in their casino or
levels of distraction or other environmental conditions that may be
present in particular areas of their casino. Such "real-world"
session data provides benefits over typical test environment data
inasmuch as the session data is generated under actual real-world
conditions (e.g., at the casino property, with typical
distractions) and by actual players (e.g., in a broad spectrum of
states, playing with their own money).
[0019] As used herein, the terms "primary game" and "base game" may
refer to games initiated in response to one of a plurality of game
initiation events, such as a wager or credit being received by or
transferred to an EGM, as described herein. A primary game may be
associated with a primary game outcome represented by a plurality
of primary game symbols or primary game reels, each of which may
include a plurality of primary game symbols, and each of which may
be selected based upon a random number generated by a random number
generator.
[0020] Further, as used herein, the terms "secondary game" and
"bonus game" may refer generally to a game or a component of a game
involving procedures in addition to the primary game. In some
embodiments, a bonus game may be triggered from a primary game and
may be associated with a bonus game outcome, which may be different
from the primary game outcome. For example, a bonus game may be
initiated after, or during, a primary game and in response to the
occurrence of a particular condition, such as a "trigger condition"
occurring during the primary game. In some embodiments, a bonus
game may be a scheduled or timed bonus (e.g., for an individual
machine or a bank of machines). A bonus game may result in a bonus
game outcome or bonus award that increases a primary game award or
adds a bonus game award to a primary game award.
[0021] FIG. 1 is a diagram of exemplary EGMs networked with various
gaming-related servers in a gaming system 100. Gaming system 100
operates in a gaming environment, including one or more servers, or
server computers, such as slot servers of a casino, that are in
communication, via a communications network, with one or more EGMs,
or gaming devices 104A-104X, such as EGMs, slot machines, video
poker machines, or bingo machines, for example. Gaming devices
104A-104X may, in the alternative, be portable and/or remote gaming
devices such as, for example, and without limitation, a smart
phone, a tablet, a laptop, or a game console.
[0022] Communication between gaming devices 104A-104X and servers
102, and among gaming devices 104A-104X, may be direct or indirect,
such as over the Internet through a web site maintained by a
computer on a remote server or over an online data network
including commercial online service providers, Internet service
providers, private networks, and the like. In other embodiments,
gaming devices 104A-104X communicate with one another and/or
servers 102 over wired or wireless RF or satellite connections and
the like.
[0023] In certain embodiments, servers 102 may not be necessary
and/or preferred. For example, the present invention may, in one or
more embodiments, be practiced on a stand-alone gaming device such
as gaming device 104A and/or gaming device 104A in communication
with only one or more other gaming devices 104B-104X (i.e., without
servers 102).
[0024] Servers 102 may include a central determination gaming
system server 106, a ticket-in-ticket-out (TITO) system server 108,
a player tracking system server 110, a progressive system server
112, and/or a casino management system server 114. Gaming devices
104A-104X may include features to enable operation of any or all
servers for use by the player and/or operator (e.g., the casino,
resort, gaming establishment, tavern, pub, etc.). For example, a
game outcome may be generated on a central determination gaming
system server 106 and then transmitted over the network to any of a
group of remote terminals or remote gaming devices 104A-104X that
utilize the game outcome and display the result to the player.
[0025] Gaming device 104A is often of a cabinet construction that
may be aligned in rows or banks of similar devices for placement
and operation on a casino floor. The gaming device 104A often
includes a main door 116 that provides access to the interior of
the cabinet. Gaming device 104A typically includes a button area or
button deck 120 accessible by a player that is configured with
input switches or buttons 122, a bill validator 124, and/or
ticket-out printer 126.
[0026] In FIG. 1, gaming device 104A is shown as a Relm XL.TM.
model gaming device manufactured by Aristocrat.RTM. Technologies,
Inc. As shown, gaming device 104A is a reel machine having a gaming
display area 118 including a plurality of mechanical reels 130,
typically 3 or 5 mechanical reels, with various symbols displayed
there on. Reels 130 are then independently spun and stopped to show
a set of symbols within the gaming display area 118 that may be
used to determine an outcome to the game.
[0027] In many configurations, gaming machine 104A may have a main
display 128 (e.g., video display monitor) mounted to, or above,
gaming display area 118. Main display 128 may be, for example, a
high-resolution LCD, plasma, LED, OLED, or microLED panel that may
be flat or curved as shown, a cathode ray tube, or other
conventional electronically controlled video monitor.
[0028] In certain embodiments, bill validator 124 may also function
as a "ticket-in" reader that enables the player to use a
casino-issued credit ticket to load credits onto gaming device 104A
(e.g., in a cashless TITO system). In such cashless embodiments,
gaming device 104A may also include a "ticket-out" printer 126 for
outputting a credit ticket when a "cash out" button is pressed.
Cashless ticket systems are well known in the art and are used to
generate and track unique bar-codes printed on tickets to allow
players to avoid the use of bills and coins by loading credits
using a ticket reader and cashing out credits using ticket-out
printer 126 on gaming device 104A.
[0029] In certain embodiments, a player tracking card reader 144, a
transceiver for wireless communication with a player's smartphone,
a keypad 146, and/or an illuminated display 148 for reading,
receiving, entering, and/or displaying player tracking information
can be provided. In such embodiments, a game controller within
gaming device 104A communicates with player tracking server system
110 to send and receive player tracking information.
[0030] Gaming device 104A may also include, in certain embodiments,
a bonus topper wheel 134. When bonus play is triggered (e.g., by a
player achieving a particular outcome or set of outcomes in the
primary game), bonus topper wheel 134 is operative to spin and stop
with indicator arrow 136 indicating the outcome of the bonus game.
Bonus topper wheel 134 is typically used to play a bonus game, but
could also be incorporated into play of the base game, or primary
game.
[0031] A candle 138 may be mounted on the top of gaming device 104A
and may be activated by a player (e.g., using a switch or one of
buttons 122) to indicate to operations staff that gaming device
104A has experienced a malfunction or the player requires service.
The candle 138 is also often used to indicate a jackpot has been
won and to alert staff that a hand payout of an award may be
needed.
[0032] In certain embodiments, there may also be one or more
information panels 152 that may be, for example, a back-lit
silkscreened glass panel with lettering to indicate general game
information including, for example, a game denomination (e.g.,
$0.25 or $1), pay lines, pay tables, and/or various game related
graphics. In some embodiments, information panels 152 may be
implemented as an additional video display.
[0033] Gaming device 104A traditionally includes a handle 132
typically mounted to the side of main cabinet 116 that may be used
to initiate game play.
[0034] Many or all of the above described components may be
controlled by circuitry (e.g., a gaming controller) housed inside
main cabinet 116 of gaming device 104A, the details of which are
shown in FIG. 2.
[0035] Not all gaming devices suitable for implementing embodiments
of the gaming systems, gaming devices, or methods described herein
necessarily include top wheels, top boxes, information panels,
cashless ticket systems, and/or player tracking systems. Further,
some suitable gaming devices have only a single game display that
includes only a mechanical set of reels and/or a video display,
while others are designed, for example, for bar tables or table
tops and have displays that face upwards.
[0036] Exemplary gaming device 104B shown in FIG. 1 is an Arc.TM.
model gaming device manufactured by Aristocrat.RTM. Technologies,
Inc. Where possible, reference numeral identifying similar features
of gaming device 104A are also identified in gaming device 104B
using the same reference numerals. Gaming device 104B, however,
does not include physical reels 130 and instead shows game play and
related game play functions on main display 128. An optional topper
screen 140 may be included as a secondary game display for bonus
play, to show game features or attraction activities while the game
is not in play, or any other information or media desired by the
game designer or operator. In some embodiments, topper screen 140
may also or alternatively be used to display progressive jackpot
prizes available to a player during play of gaming device 104B.
[0037] Gaming device 104B includes main cabinet 116 having main
door 118 that opens to provide access to the interior of gaming
device 104B. Main door 118, or service door, is typically used by
service personnel to refill ticket-out printer 126 and collect
bills and tickets inserted into bill validator 124. Main door 118
may further be accessed to reset the machine, verify and/or upgrade
the software, and for general maintenance operations.
[0038] Exemplary gaming device 104C shown in FIG. 1 is a Helix.TM.
model gaming device manufactured by Aristocrat.RTM. Technologies,
Inc. Gaming device 104C includes a main display 128A that is in a
landscape orientation. Although not illustrated by the front view
illustrated in FIG. 1, landscape display 128A has a curvature
radius from top to bottom. In certain embodiments, display 128A is
a flat panel display. Main display 128A is typically used for
primary game play while a secondary display 128B is used for bonus
game play, to show game features or attraction activities while the
game is not in play, or any other information or media desired by
the game designer or operator.
[0039] Many different types of games, including mechanical slot
games, video slot games, video poker, video black jack, video
pachinko, keno, bingo, and lottery, may be provided with or
implemented within gaming devices 104A-104C and other similar
gaming devices. Each gaming device may also be operable to provide
many different games. Games may be differentiated according to
themes, sounds, graphics, type of game (e.g., slot game vs. card
game vs. game with aspects of skill), denomination, number of
paylines, maximum jackpot, progressive or non-progressive, bonus
games, Class II, or Class III, etc.
[0040] FIG. 2 is a block diagram of an exemplary gaming device 200,
or EGM, connected to various external systems, including TITO
system server 108, player tracking system server 110, progressive
system server 112, and casino management system server 114. All or
parts of gaming device 200 may be embodied in game devices
104A-104X shown in FIG. 1. The games conducted on gaming device 200
are controlled by a game controller 202 that includes one or more
processors 204 and a memory 208 coupled thereto. Games are
represented by game software or a game program 206 stored on memory
208. Memory 208 includes one or more mass storage devices or media
housed within gaming device 200. One or more databases 210 may be
included in one or more databases 210 for use by game program 206.
A random number generator (RNG) 212 is implemented in hardware
and/or software and is used, in certain embodiments, to generate
random numbers for use in operation of gaming device 200 to conduct
game play and to ensure the game play outcomes are random and meet
regulations for a game of chance.
[0041] Alternatively, a game instance, or round of play of the
game, may be generated on a remote gaming device such as central
determination gaming system server 106, shown in FIG. 1. The game
instance is communicated to gaming device 200 via a network 214 and
is then displayed on gaming device 200. Gaming device 200 executes
game software to enable the game to be displayed on gaming device
200. In certain embodiments, game controller 202 executes video
streaming software that enables the game to be displayed on gaming
device 200. Game software may be loaded from memory 208, including,
for example, a read only memory (ROM), or from central
determination gaming system server 106 into memory 208. Memory 208
includes at least one section of ROM, random access memory (RAM),
or other form of storage media that stores instructions for
execution by processor 204.
[0042] Gaming device 200 includes a topper display 216. In an
alternative embodiment, gaming device 200 includes another form of
a top box such as, for example, a topper wheel, or other topper
display that sits on top of main cabinet 218. Main cabinet 218 or
topper display 216 may also house various other components that may
be used to add features to a game being played on gaming device
200, including speakers 220, a ticket printer 222 that prints
bar-coded tickets, a ticket reader 224 that reads bar-coded
tickets, and a player tracking interface 232. Player tracking
interface 232 may include a keypad 226 for entering player tracking
information, a player tracking display 228 for displaying player
tracking information (e.g., an illuminated or video display), a
card reader 230 for receiving data and/or communicating information
to and from media or a device such as a smart phone enabling player
tracking. Ticket printer 222 may be used to print tickets for TITO
system server 108. Gaming device 200 may further include a bill
validator 234, buttons 236 for player input, cabinet security
sensors 238 to detect unauthorized opening of main cabinet 218, a
primary game display 240, and a secondary game display 242, each
coupled to and operable under the control of game controller
202.
[0043] Gaming device 200 may be connected over network 214 to
player tracking system server 110. Player tracking system server
110 may be, for example, an OASIS.RTM. system manufactured by
Aristocrat.RTM. Technologies, Inc. Player tracking system server
110 is used to track play (e.g., amount wagered and time of play)
for individual players so that an operator may reward players in a
loyalty program. The player may use player tracking interface 232
to access his/her account information, activate free play, and/or
request various information. Player tracking or loyalty programs
seek to reward players for their play and help build brand loyalty
to the gaming establishment. The rewards typically correspond to
the player's level of patronage (e.g., to the player's playing
frequency and/or total amount of game plays at a given casino).
Player tracking rewards may be complimentary and/or discounted
meals, lodging, entertainment and/or additional play. Player
tracking information may be combined with other information that is
now readily obtainable by casino management system server 114.
[0044] Gaming devices, such as gaming devices 104A-104X and 200,
are highly regulated to ensure fairness and, in many cases, gaming
devices 104A-104X and 200 are operable to award monetary awards
(e.g., typically dispensed in the form of a redeemable voucher).
Therefore, to satisfy security and regulatory requirements in a
gaming environment, hardware and software architectures are
implemented in gaming devices 104A-104X and 200 that differ
significantly from those of general-purpose computers. Adapting
general purpose computers to function as gaming devices 200 is not
simple or straightforward because (1) regulatory requirements for
gaming devices, (2) harsh environments in which gaming devices
operate, (3) security requirements, and (4) fault tolerance
requirements. These differences require substantial engineering
effort and often additional hardware.
[0045] When a player wishes to play gaming device 200, he/she can
insert cash or a ticket voucher through a coin acceptor (not shown)
or bill validator 234 to establish a credit balance on the gaming
machine. The credit balance is used by the player to place wagers
on instances of the game and to receive credit awards based on the
outcome of winning instances of the game. The credit balance is
decreased by the amount of each wager and increased upon a win. The
player can add additional credits to the balance at any time. The
player may also optionally insert a loyalty club card into card
reader 230. During the game, the player views the game outcome on
game displays 240 and 242. Other game and prize information may
also be displayed.
[0046] For each game instance, a player may make selections that
may affect play of the game. For example, the player may vary the
total amount wagered by selecting the amount bet per line and the
number of lines played. In many games, the player is asked to
initiate or select options during course of game play (such as
spinning a wheel to begin a bonus round or select various items
during a feature game). The player may make these selections using
player-input buttons 236, primary game display 240, which may
include a touch screen, or using another suitable device that
enables a player to input information into gaming device 200.
[0047] During certain game events, gaming device 200 may display
visual and auditory effects that can be perceived by the player.
These effects add to the excitement of a game, which makes a player
more likely to continue playing. Auditory effects include various
sounds that are projected by speakers 220. Visual effects include
flashing lights, strobing lights, or other patterns displayed from
lights on gaming device 200 or from lights behind information panel
152, shown in FIG. 1.
[0048] When the player wishes to stop playing, he/she cashes out
the credit balance (typically by pressing a cash out button to
receive a ticket from ticket printer 222). The ticket may be
"cashed-in" for money or inserted into another machine to establish
a credit balance for play.
[0049] FIG. 3 is a diagram of an exemplary gaming device 300 that
may be used to capture player engagement and player emotion of a
player 302 and enhance the player experience during game play.
Gaming device 300 may be similar to gaming devices 104A-104X or
gaming device 200 (e.g., may include any of the described
components of such devices), and may be networked with other gaming
devices or computing devices as described above with reference to
FIGS. 1 and 2. In the example embodiment, gaming device 300
includes one or more digital camera devices (or just "cameras")
310A, 310B, 310C (collectively, "cameras 310") that capture digital
images or video of the player 302 and a surrounding environment
(not labelled) as the player 302 plays a wagering game. Further, in
the example embodiment, gaming device 300 includes a game
controller and an expression evaluation engine (both not depicted
in FIG. 3) internal to gaming device 300, and may include one or
more sensors 350. In other embodiments, the expression evaluation
engine may and associated methods may be performed on a server
system (e.g., central determination gaming system server 106,
player tracking system server 110, or such).
[0050] In the example embodiment, camera(s) 310A is positioned
below main display 240 and is oriented such as to capture digital
images or video of a face 320 and head 322 of the players 302 when
the players 302 are positioned during game play. Gaming device 300
may be a stand-up device (e.g., as depicted in FIG. 3) or may
include a seat or bench (not shown) at which the players 302 may
sit. As such, camera(s) 310A may be oriented to accommodate the
particular configuration of gaming device 300 and common height
ranges or positions of players 302 such as to facilitate capturing
digital video of the face 320 and head 322 of the players 302. In
some embodiments, gaming device 300 may additionally or
alternatively include cameras 310 positioned at different
locations, such as with camera devices 310B and 310C. Facial
expression processing by the expression evaluation engine may be
improved when performed with digital images or video of the face
320 of the player 302 from approximately in front of the player 302
and near a gaze direction 330 of the player 302. In this example,
the wagering game is primarily presented in a primary display area
340 of the main display 240. For example, the display area 340 may
be used to present a base game or aspects of bonus games, where
other areas of the main display 240 may be used for supplemental
features or other presentations. In other words, much of the player
302's focus is typically given to the primary display area 340. As
such, cameras positioned near or around primary display area 340
may provide video that yields improved facial expression
evaluation.
[0051] Various environmental conditions can impact the emotion
level of the player 302. For example, the position of gaming device
300 within a casino may introduce distractions or other factors of
dissatisfaction such as other nearby gaming devices 300 or gaming
tables, high foot traffic, near smoking areas, high noise areas,
hot or cold areas, or high occurrence of spectators. The player
302's reactions to such environmental conditions may manifest in
facial expressions of the player. Such environmental conditions may
be referred to herein as "environmental factors" inasmuch as the
environmental conditions impact the emotion level of the player
302. Gaming device 300 may be configured to analyse environmental
factors and detect such distraction and dissatisfaction in the
player 302.
[0052] Gaming device 300 may utilize digital video from cameras 310
or sensor data from other sensors included within gaming device 300
to detect environmental conditions around the player 302. For
example, digital images or video from camera 310C may be oriented
such as to capture a wider view around gaming device 300 and may be
analysed, for example, to capture motion of pedestrians near gaming
device 300 and detect whether gaming device 300, for example, is
positioned in a high-traffic area, or is near a tournament play
environment. Similarly, digital video from cameras 310 may be
analysed to detect spectators near gaming device 300 or cell phone
use of the player 302. In some embodiments, gaming device 300 may
use sensor data to detect player gestures. For example, gaming
device 300 may use video from cameras 310 or from a motion detector
camera (not shown in FIG. 3) to detect slumping in the player 302's
shoulders, a startled reaction by the player 302, or smartphone use
by the player 302. Such gestures may be used to evaluate emotion
level of the player (e.g., level of distraction due to smartphone
use, disappointment with slumped shoulders, and so forth).
[0053] In some embodiments, gaming device 300 may use digital video
from cameras 310 to evaluate a gaze direction or point of gaze of
the player 302 (e.g., where they player 302 is looking). For
example, gaze direction may be evaluated based on a combination of
an orientation of the head 322 of the player 302 relative to
shoulders 324 of the player 302 (e.g., as the player 302 turns
their head 322), an orientation of the shoulders 324 of the player
302 relative to the gaming device 300 (e.g., as they turn their
head 322 to look over their shoulders 324), a normalized
orientation of facial points, or eye tracking methods for measuring
eye movement. Analysis of an approximate gaze direction may be
computed and used to evaluate whether or when the player 302 is
actively watching the game (e.g., looking at primary display area
340 or elsewhere on main display 240) or otherwise looking at
gaming device 300 (e.g., looking at buttons 236). Such analysis may
be calibrated, for example, at times where focus on the gaming
device 300 is likely to be high (e.g., when the player 302 first
sits down, when a significant win occurs). If the gaze direction is
determined to be outside a configured area (e.g., relative to the
dimensions of the gaming device 300 and of the player 302), then
the level of engagement of the player 302 may be evaluated "low,"
where if the gaze direction is determined to be within the
configured area, then the level of engagement may be evaluated as
"high." In some embodiments, gaming device 300 may be configured to
track an amount of time that the player 302 gazes at the gaming
device (e.g., at primary display area 340) during a specific game
event or bonus. For example, some players may have varying levels
of interest in a particular bonus game being presented in the
primary display area 340, and may manifest that level of
disinterest by looking away from the gaming device 300 (e.g.,
looking at their smartphone, turning their head 322 toward another
nearby machine, talking to another player, and so forth). Players
having more interest in that same bonus game may gaze at the
primary display area 340 more than others. As such, the collected
gaze data may be synchronized with the game play data to evaluate
whether the player 302 is focused on (e.g., gazing at) the bonus
game as the bonus game is being presented. Such gaze data may be of
interest in later analysis (e.g., by casino operators or game
developers). In some embodiments, the gaming device 300 may use
such gaze data to attempt to reacquire the focus of the player 302.
For example, the gaming device 300 may play an additional sound or
display an additional video feature when detecting that the player
302 is not gazing at the primary display area 340.
[0054] In some embodiments, gaming device 300 may include one or
more additional sensors such as, for example, a microphone (e.g.,
for capturing audio data around gaming device 300), a thermometer
(e.g., for capturing ambient temperature experienced by the player
302 near gaming device 300), a smoke detector (e.g., for detecting
air quality near gaming device 300), a motion detector camera
(e.g., for detecting motion of the player 302 or other spectators
or other foot traffic), or a thermal camera (e.g., for capturing
temperature data of the player 302, for detecting spectators or
foot traffic near gaming device 300) (each not separately
depicted). Audio from the microphone may be used to determine
environmental conditions such as high ambient noise (e.g.,
capturing an ambient noise level, and calibrated to detect noise
not originating from gaming device 300, which may be an aggravant
to some people), the player 302 engaging in speech (e.g.,
distracted by conversation, phone), nearby noise events (e.g., a
distinct game win event at a neighbouring gaming device 300, an
alert from a cell phone of the player 302), or nearby speech (e.g.,
the speech of people other than the player 302). Gaming device 300
may include multiple microphones, some of which may be calibrated
to focus on the sitting or standing location of the player 302, and
audio analysis may use multiple audio streams to differentiate
sounds originating at or near the player 302 to other sounds. In
some embodiments, a direction of a noise event may be determined
(e.g., using multiple audio streams from directional microphones).
In some embodiments, game sound level may be increased or lowered
based on an ambient noise level around the gaming device 300.
[0055] In some embodiments, the thermometer or thermal camera may
be used to capture an ambient temperature near the player 302.
Players that are too warm or too cold may become uncomfortable,
which may lead to overall dissatisfaction during game play, thereby
affecting emotion level. Casino operators, once aware, may wish to
change heating, ventilation, and air conditioning (HVAC) properties
near gaming device 300. In some embodiments, gaming device 300 may
include an integrated air conditioner, heater, or fan, and gaming
device 300 may automatically engage such devices based on the
ambient temperature near gaming device 300 (e.g., for climate
control). In some embodiments, the smoke detector may be used to
detect the quality of air near gaming device 300, and may be used
to automatically activate the fan or otherwise move fresh air into
the area around gaming device 300.
[0056] In some embodiments, gaming device 300 may allow players 302
to capture and broadcast streaming media (e.g., live streaming in
real time). During game play, the player 302 may elect to broadcast
streaming media of themselves as they play the wagering game. Audio
of the player 302 and ambient environmental sounds (e.g., game play
audio) is captured by the microphone(s) and video of the player 302
is captured by the camera(s) 310. In some embodiments, audio and
video of game play (e.g., what is shown on displays 240, 242) may
also be captured. Such audio and video data of the player 302 may
be referred to herein as "personal streaming data." In some
embodiments, gaming device 300 may allow players 302 to share their
own personal streaming data with other players (e.g., at other
gaming devices 300 within a casino premises), for example, allowing
friends or relatives to watch each other's personal streaming data
(e.g., game play, audio, video). This allows players to share their
gaming experiences during a gaming session when not near each other
(e.g., communicating, viewing game play, sharing emotional moments,
and so forth). Gaming device 300 may present any or all of the game
play audio and video or personal audio and video of one player 302
to another player 302. Such streaming video may be viewed on one of
the displays 240, 242, for example, in a dedicated location within
the main display 240 or secondary display 242, as a
picture-in-picture, or as a movable window. Streaming audio of the
other player may be output on gaming device 300 through speakers
220. In some embodiments, gaming device 300 may allow players 302
to share their personal streaming data via social media or live
streaming sites (e.g., Facebook.RTM. Live, Periscope.RTM.,
Twitch.RTM., and such). (FACEBOOK is a registered trademark of
Facebook, Inc., a Delaware Corporation; PERISCOPE is a registered
trademark of Twitter, Inc., a Delaware Corporation; TWITCH is a
registered trademark of Twitch Interactive, Inc., a Delaware
Corporation). For example, the personal streaming data of the
player 302 may be transmitted via network 214 out to the Internet.
As such, players 302 may have enhanced excitement brought by
remotely sharing their experience with others as they experience
game play. Further, game developers and casino operators may also
experience benefits from live streaming. For example, game
popularity of a wagering game may be increased through the
additional exposure of game play to others via the streaming, thus
boosting overall interest in the wagering game. Such increased
exposure and interest may cause more players to visit casinos
offering that wagering game, or may cause streaming players to play
for longer (e.g., to continue their streams when the streaming
player has numerous viewers).
[0057] FIG. 4 is a component diagram 400 illustrating an example
embodiment of gaming device 300. It should be understood that,
while FIG. 4 illustrates additional components of gaming device 300
not shown in FIG. 3, not all components of gaming device 300 or
game controller 202 are shown for ease of illustration. In the
example embodiment, game controller 202 of gaming device 300
includes a preliminary image processor, a game presentation module
422, an expression evaluation engine 424, a game session actions
module 426, an emotion analysis module 428, an environmental
analysis module 430, a gaze analysis module 432, an HVAC controller
module 434, a session synchronization module 436, and a gameplay
data module 438. Further, gaming device 300 includes one or more
digital cameras 310, one or more microphones 410A, a thermal camera
410B, a smoke detector 410C, a thermometer 410D, and a motion
detector camera 410E (collectively referred to herein as "sensors
410"). Sensors 410 may be similar to sensors 350. Output from each
of the sensors 410 may be used by various modules of game
controller 202 to evaluate the emotion and engagement level of the
player 302.
[0058] In the example embodiment, game presentation module 422
provides the wagering game to the player 302 during game play. For
example, game presentation module 422 may display simulated reels
of a slot-style game or cause mechanical reels to spin upon a wager
being placed by the player 302 (e.g., a base game, a bonus game)
using the RNG 212 (e.g., similar to the game program 206),
generating game outcomes, and so forth. In some situations, such as
under U.S. gaming regulations, gaming device 300 may be prohibited
from altering an outcome of the wagering game. As such, the various
modules of game controller 202 may use game presentation module 422
to manipulate aspects of appearance of the wagering game, but does
not impact game outcomes (e.g., does not cause the player 302 to
win more often). In other situations, such as in social gaming
contexts (e.g., where no money wager is taken), such gaming devices
may not be prohibited by law to manipulate game outcomes. In such
contexts, game presentation module 422 may manipulate game outcomes
based on an emotion level or engagement level of the player 302
(e.g., generating higher win percentages, larger wins, activating
bonus games, and so forth).
[0059] The expression evaluation engine 424 performs facial
expression analysis on digital video of the player 302 (e.g.,
during game play) to determine an emotional state of the player
302. In some embodiments, expression evaluation engine 424 is a
third-party product configured to evaluate facial expressions, such
as those made available by iMotions, Inc. (a Delaware corporation),
Affectiva (a Delaware corporation), and Noldus (a Netherlands
company). Such expression evaluation engines 424 perform aspects of
image analysis of the facial features using, for example, eyes and
eye corners, orientation of eyebrows, mouth corners, nose tip, or
facial muscles. FIG. 5 illustrates example images 510A-D
(collectively, "images 510") of a person (e.g., the player 302) in
various emotional states as exhibited by facial expression. The
person is expressing happiness or joy in image 510A, surprise in
image 510B, anger in image 510C, and sadness in image 510D. Each of
the images 510 represent digital images (or frames of digital
video) of the player 302 captured by one or more of the cameras 310
during game play. It should be understood that the images 510 and
resulting determined emotional states shown in FIG. 4 are merely
exemplary, and that other emotional states may be determined.
[0060] Returning again to FIG. 4, in the example embodiment,
expression evaluation engine 424 generates an emotional state
output based on the input image 510. Emotional state output may
include such emotional states as, for example, tense, nervous,
stressed, upset, alert, excited, elated, happy, sad, depressed,
bored, fatigued, content, serene, relaxed, or calm. In some
embodiments, preliminary image processor 420 may perform image
processing operations prior to sending images 510 to expression
evaluation engine 424. For example, preliminary image processor 420
may capture a broader image (not shown) of the player 302 (e.g.,
including upper and lower body of the player 302 and background of
the environment behind the player 302. Preliminary image processor
420 may perform face detection of the broader image to identify the
position of the face 320 of the player 302 (e.g., via Viola Jones
Cascaded Classifier), and may crop the broader image to generate
images 410.
[0061] The emotion analysis module 428 categorizes the emotional
state determined by the expression evaluation engine 424 to
determine an emotion level. In the example embodiment, the emotion
analysis module 428 categorizes the player 302 into one of three
emotion levels: positive, neutral, and negative. The positive
emotion level represents the most desirable emotion level of the
player, where the player's excitement level and engagement with the
game are high. Players may experience this level, for example, when
there are exciting features of the game occurring, when the player
has recently won a large payout, or when they relate to or are
excited by the game design. The neutral level represents an emotion
level in which the player is content, calm, or relaxed. Players may
experience this level, for example, when they have recently
experienced only modest wins, when there are no particularly
interesting features of the game being activated, or when they are
moderately disengaged from game play. The neutral level may
commonly occur for many players at various times during game play
and, while not necessarily undesirable, the player's excitement and
engagement are only moderate. As such, the neutral level is less
desirable than the positive level. The negative level represents an
emotion level in which the player is, for example, upset, angry,
sad, bored, or fatigued. Players may experience this level, for
example, when they have regularly lost over recent plays, when they
do not enjoy the particular game or game features, when they are
confused by the game, or when they have played too long without
positive experiences in the game. The negative level is the least
desirable emotion level for the player.
[0062] In the example embodiment, the emotion analysis module 428
assigns an emotion level of positive when the determined emotional
state of the player 302 is happy, excited, satisfied, engaged,
joyful, surprised, elated, or positive. An emotion level of neutral
is assigned when the determined emotional state of the player 302
is content, neutral, or limited attention. An emotion level of
negative is assigned when the determined emotional state of the
player 302 is angry, fearful, sad, disgusted, negative, or
distracted.
[0063] In some embodiments, the emotion analysis module 428 may,
additionally or alternatively, determine an engagement level of the
player 302 based on the determined emotional state of the player
302. In one embodiment, an engagement level of high is assigned
when the determined emotional state of the player 302 is tense,
nervous, stressed, upset, alert, excited, elated, or happy. An
engagement level of low is assigned when the determined emotional
state of the player 302 is sad, depressed, bored, fatigued,
content, serene, relaxed, or calm.
[0064] Further, in the example embodiment, the emotion analysis
module 428 initiates a process of collecting images 510 and driving
expression evaluation engine 424 to determine emotional states. In
the example embodiment, the emotion analysis module 428 initiates
image collection and processing on a periodic basis, such as at a
pre-determined frequency (e.g., four times a second, once a
second). More specifically, and for example, the emotion analysis
module 428 prompts the camera 310A to capture an image 510 of the
player 302, which is transferred to expression evaluation engine
424 (optionally through preliminary image processor), thereby
receiving the emotional state of the player 302 and generating the
emotion level therefrom. In some embodiments, the emotion analysis
module 428 initiates image collection and processing based on
particular trigger events. For example, the emotion analysis module
428 may initiate image capture and processing upon the player 302
interacting with the gaming device 300 (e.g., when the player 302
cards into the gaming device 300, when the player 302 presses the
button 236 to spin the reels in a base game, when the player 302
presses a call button to summon a cocktail hostess), based upon
game-based trigger events (e.g., a particular symbol appearing
during the wagering game, a particular feature being triggered
during the wagering game, upon generating an outcome of the
wagering game), or based on environmental conditions (e.g., when
loud noises are detected nearby, when an excessive amount of foot
traffic is detected nearby, when a nearby gaming device has
produced a significant win event).
[0065] In the example embodiment, the game session actions module
426 is configured to perform various game session actions based on
the determined emotional state, emotion level, or engagement level
(e.g., in an effort to move the player 302 from less desirable
emotion levels to more desirable emotion levels). Game session
actions may include, for example, adjusting game colors or other
aspects of presentation of the wagering game (e.g., as presented by
game presentation module 422), initiating play of or altering a
volume level of game audio or music, offering complimentary items
or services ("comps," e.g., free plays, meals at casino
restaurants, free beverages, hotel room, shopping experience at
casino shops), initiating beverage services at the gaming device
300, or increasing player tracking account credit balance or status
level. In some embodiments, game session actions may include
offering special rewards for player birthday, anniversary, or
individual or group celebration for a special occasion. In
situations in which engagement level of the player 302 is
determined to be low, the game session actions module 426 may
identify one or more other games of potential interest to the
player 302 (e.g., based on comparing the current wagering game or
gaming machine 300 to past playing experience, game type,
manufacturer, and so forth). As such, the game session action may
include providing the game recommendations, determining whether and
which recommended gaming devices are presently unoccupied at the
current casino property, and providing a casino map identifying
location of the recommended gaming devices. Game session actions
may include alerting the casino operator of the player 302's low
engagement level or poor emotional level (e.g., automatically
transmitting an alert to a host), which can cause the host to visit
and interact with the player 302 to improve their mood, offer
comps, or otherwise improve the player's experience. In
non-wagering (e.g., social) gaming, game session actions may
include altering outcomes of the social game.
[0066] In some embodiments, game session actions module 426 may
initiate a game sessions action (e.g., comp'ing the player 302 with
$5 credit for free plays) upon determining that the player 302's
emotion level being negative, or being not positive, for a
pre-determined period of time (e.g., for five minutes) or for a
pre-determined number of plays (e.g., through 20 plays). In some
embodiments, the game session action may also be conditioned based
on aggregate wagering outcomes of the player 302 (e.g.,
experiencing net outcomes below a pre-determined threshold over the
same period of time or over the same number of plays). For example,
if the player 302 exhibits negative or neutral emotion level over
20 plays while experiencing a net positive wagering outcome over
those 20 plays, then the game session actions module 426 may not
perform a game session action. In contrast, if the player 302
exhibits both a negative or neutral emotion level as well as a net
wagering outcome below a pre-determined threshold (e.g., more than
$100 loss) over those 20 plays, then game session actions module
426 may initiate the game session action. In some embodiments, the
casino operator may configure the pre-determined threshold level
(of net wagering outcomes) for each particular game session action
(e.g., on an action-by-action basis), and may configure different
threshold levels for different players 302 (e.g., on a
player-by-player basis). In some embodiments, game session actions
module 426 may initiate harm minimization or self-exclusion for
problem gamblers (e.g., based on net wagering outcomes and emotion
level).
[0067] In some embodiments, the emotional state or emotion level
may be aggregated over a window of time (e.g., to develop an
average emotional state or an average emotion level over that time
period). Some emotional states may be fleeting, brief, or
temporary. For example, the player 302 may become distracted based
on the conduct of a nearby player or angry based on an incident in
one game that quickly provides a happy emotional state (e.g., an
unanticipated winning outcome). As such, the game controller 202
may utilize an average emotional state or average emotional level
when initiating game session actions so as to avoid reacting to
what is only a temporary state from which the player 302 may
naturally recover. In some embodiments, game session actions may be
initiated only when a non-positive or a negative emotion level is
detected a pre-determined number of times within a window of time.
In some embodiments, a subsequent game session action may be
initiated only after a pre-determined amount of time since the
previous game session action was initiated (e.g., using a timeout
or countdown timer).
[0068] In some embodiments, environmental analysis module 430
collects environmental data about environmental conditions (e.g.,
near gaming device 300) that may impact player experience or
emotion level of the player 302 during game play. Environmental
analysis module 430 may initiate sensor data collection from
sensors 410, such as capturing audio via microphones 410A,
capturing thermal images or thermal video via thermal camera 410B,
capturing air quality data via smoke detector 410C, capturing
temperature data via thermometer 410D, or capturing motion data via
motion detector camera 410E. In some embodiments, environmental
data may be used (e.g., as a factor) to evaluate engagement level
or emotion level or to initiate game session actions. In some
embodiments, environmental data may be collected (e.g., as a
component of game play session data) and used in offline analytics
(e.g., to correlate changes in emotional state or emotion level
with possible environmental conditions occurring near the player
302 or gaming device 300 during game play). Collection of
environmental data may be initiated as described above with regard
to emotional states, or may be initiated based on a change in
emotion level (e.g., when the player 302 changes from positive to
neutral or negative).
[0069] In some embodiments, gaze analysis module 432 determines the
gaze direction 330 of the player 302 during game play. The gaze
direction 330 may be used (e.g., exclusively, or as a factor along
with emotional state) to determine the engagement level of the
player 302. For example, if the gaze direction 330 of the player
302 is determined to be away from the primary display area 340 or
elsewhere on main display 240 for a pre-determined period of time
(e.g., 5 seconds), then gaze analysis module 432 may determine that
the engagement level of the player is low. Conversely, if the gaze
direction 330 is determined to be within the primary display area
340 or elsewhere on main display 240 for a pre-determined period of
time (e.g., 5 seconds), then the engagement level of the player may
be set to high. Gaze analysis module 432 may acquire digital images
or video from cameras 310 or motion detector camera 410E for use in
determining the gaze direction 330 (e.g., via shoulder orientation,
head orientation, eye orientation, and so forth). Further, gaze
analysis module 432 may include gaze direction data as a part of
game session data. In some embodiments, gaze analysis module 432
may perform gaze analysis on people other than the player 302. For
example, when the gaming machine 300 is not currently in use, gaze
analysis module 432 may detect when a passer-by gazes at the gaming
machine 300 and may initiate an "attract mode" (e.g., additional
lights and sounds, demo presentations, and so forth) that is
configured to automatically entice the passer-by to play the gaming
machine 300.
[0070] In some embodiments, the HVAC controller module 434
initiates game session actions to alter environmental conditions
around gaming device 300, and may be based on environmental
conditions (e.g., as captured by thermometer 410D, thermal camera
device 410B, or smoke detector 410C, or as generated by
environmental analysis module 430). Such game session actions may
include, for example, engaging a fan, heater, or air conditioner
(none of which are shown) of gaming device 300. In some
embodiments, HVAC controller module 434 may collect environmental
conditions data as a part of game session data.
[0071] In the example embodiment, gameplay data module 438 collects
data regarding game play. Gameplay data may include, for example,
game-specific data for each play of the wagering game (e.g., amount
wagered, net wagering outcome, base spin results, features
initiated, bonus game results, jackpots won), session data (e.g.,
player profile information, credits established, comps given, time
in and time out of gaming device 300, play rate (e.g., rate or
speed of game play), aggregate outcome, drink service requests),
and proximity gameplay data (e.g., activity level of nearby EGMs,
jackpots or other large wins occurring during the gaming
session).
[0072] Session synchronization module 436, in the example
embodiment, captures and synchronizes the data generated by the
various modules of game controller 202 in a time-synchronized log.
For example, session synchronization module 436 collects and
synchronizes emotional state data generated by expression
evaluation engine 424, emotion level data generated by emotion
analysis module 428, game session actions initiated by game session
actions module 426 or HVAC controller module 434, environmental
data captured by environmental analysis module 430 or HVAC
controller module 434, gaze direction data generated by gaze
analysis module 432, and gameplay data captured by gameplay data
module 438. Such data may be referred to collectively herein as
"session data." In the example embodiment, components of session
data may be timestamped by session synchronization module 436
(e.g., when received), or timestamped using a shared clock (not
shown) at a time of occurrence or capture by the associated module
(e.g., when generated). As such, each of the disparate types of
session data may be examined together, with the timestamps acting
as chronological synchronization to depict what was happening or
what data was being generated at any given time during a game play
session.
[0073] In some embodiments, session synchronization module 436 may
store session data for later retrieval (e.g., download by a
technician). In some embodiments, session synchronization module
436 may transmit any or all of the session data to a
network-connected server (e.g., casino management system server
114, player tracking system server 110, an EGM manufacturer server
(not shown), or such). Session synchronization module 436 may
remove or otherwise sanitize personally identifiable information
(PII) from the session data prior to removal (e.g., to protect
privacy). In the example embodiment, no raw video or images of the
player 302 are stored with session data, other than for transient
use as described above (e.g., in generating emotional state). In
some embodiments, the gaming device 300 may allow the player to opt
into or out of any of the sensor collection or analysis data
described herein.
[0074] In some embodiments, session data may be used by game
developers or manufacturers of gaming device 300. For example, game
developers may analyse aspects of game design and impacts on
players' emotional states or emotion levels (e.g., how different
players react to game aesthetics, game features, or other aspects
of game play, what causes players to lose focus or interest). Such
data may help game developers design better games or improve
existing games. Developers may examine situations in which a
particular player exhibits a low emotion level for one particular
game but then exhibits a higher emotion level for another game and
may then attempt to determine whether the differing emotion level
was due to gaming outcomes or whether the difference was based on
some aspect of game design (e.g., based on music, theme, colors).
In some embodiments, developer feedback may be shared with third
parties, casinos, or manufacturers.
[0075] In some embodiments, session data may be used by the casino
operator of gaming device 300. For example, casino operators may
analyse emotional states or emotion levels of the same type of
gaming device 300 at two different locations within a property, or
different types of gaming devices 300 at a common location. Such
data may help operators detect sources of distraction or discomfort
for players and take remedial actions to improve emotion levels for
future players. Casino operators may store session data as a part
of player profiles, an overall emotion level of the player 302
during a given session or period of time, and may provide other
comps to players if they exhibit poor emotion levels. The gaming
device 300 may allow the casino operator to change the game
presented to the player 302. For example, if the player 302 has
become disinterested in (e.g., low engagement, non-positive
emotional level) the current wagering game, the gaming device 300
may provide a menu of games from which the player 302 may
choose).
[0076] In some embodiments, gaming device 300 may include haptic
sensors (not shown) that allow gaming device 300 to detect certain
types of actions taken by the player 302. For example, game
controller 202 may be configured to use haptic sensors to detect
when the player 302 stands up from, sits down in, or fidgets while
seated in a chair during the gaming session. Game controller 303
may be configured to use haptic sensors (e.g., a piezoelectric
sensor) to detect when the player 302 slams the gaming device 300.
Such player actions may be used to evaluate the emotion level of
the player 302. For example, a player slamming their hands on the
button deck 120 or displays 240, 242 may indicate a frustrated
emotion level, and the player fidgeting in their seat may be
indicative of a level of distraction or discomfort.
[0077] FIG. 6 is a flowchart illustrating a process 600 for
analyzing facial expressions of the player 302, as described above.
In the exemplary embodiment, process 600 is performed by game
controller 300 (shown in FIG. 3).
[0078] In the exemplary embodiment, game controller 300 receives
610 a digital image of the player 302 (e.g., from the camera device
510A). Game controller 300 also determines 620 an emotional state
of the player by performing facial expression analysis on the
digital image. Game controller 300 also determines 630 an emotion
level 632 of the player by categorizing the emotional state of the
player based on the determined emotional state, the categorizing
includes a first state representing a positive emotional level
(e.g., "Category 1" 640, "positive") and a second state
representing another emotional level (e.g., "Category 2" 642,
"neutral", or "Category 3" 644, "negative"). Game controller 300
further determines that the emotional level is the other emotional
level (e.g., "Category 2" 642 or "Category 3" 644). Game controller
300 automatically initiate a game session action during the game
play session, the game session action is configured to cause the
player to transition to the positive emotional level. If emotion
level 632 is "positive", or within "Category 1" 640, then no game
session action is taken 650. If emotion level 632 is "neutral," or
within "Category 2" 642, then game controller 300 may test 660
whether to implement a local or system game session action, such as
initiating 662 a drink offer or summoning an attendant, or
initiating 664 a light show or music. If emotion level 632 is
"negative," or within "Category 3" 644, then game controller may
test 660 whether to implement a local or system game session
action, such as providing 674 comps, free plays, or summoning an
attendant.
[0079] In some embodiments, the method 600 includes determining an
area within the digital image representing a face of the player,
and cropping the digital image to include only the determined area
prior to performing the facial expression analysis. In some
embodiments, the game session action includes one of initiating a
light show, adjusting colors of the wagering game, and altering
audio presented during the wagering game. In some embodiments, the
game session action includes one of calling an attendant, providing
complimentary services, and providing complimentary credits for the
wagering game.
[0080] In some embodiments, the method 600 further includes
capturing gameplay data associated with the wagering game,
timestamping the gameplay data and the emotion level to generate a
time-synchronized log of the game play session, and storing the
time-synchronized log for analysis by one or more of a game
developer of the wagering game and a casino operator providing the
electronic gaming machine. In some embodiments, the method 600
includes generating an engagement level of the player based at
least in part on the determined emotional state. In some
embodiments, the method 600 includes determining a gaze direction
of the player based on the digital image, and determining that the
gaze direction of the player is not directed at the electronic
gaming machine, and generating an engagement level of the player is
further based at least in part on the determining that the gaze
direction of the player is not directed at the electronic gaming
machine.
[0081] In some embodiments, the method 600 also includes detecting,
using audio data captured by a microphone of the electronic gaming
machine, a source of noise not related to the electronic gaming
machine, and storing the audio data in the time-synchronized log.
In some embodiments, the method 600 also includes receiving, from
the camera device, digital video of an environment behind the
player, performing motion analysis on the digital video to
determine the presence of people walking near the electronic gaming
machine, and automatically identifying foot traffic as a source of
distraction for the player based on the motion analysis. In some
embodiments, the second state represents a neutral emotional level,
wherein the categorizing includes a third state representing a
negative emotional level.
[0082] A computer, controller, or server, such as those described
herein, includes at least one processor or processing unit and a
system memory. The computer, controller, or server typically has at
least some form of computer readable non-transitory media. As used
herein, the terms "processor" and "computer" and related terms,
e.g., "processing device", "computing device", and "controller" are
not limited to just those integrated circuits referred to in the
art as a computer, but broadly refers to a microcontroller, a
microcomputer, a programmable logic controller (PLC), an
application specific integrated circuit, and other programmable
circuits "configured to" carry out programmable instructions, and
these terms are used interchangeably herein. In the embodiments
described herein, memory may include, but is not limited to, a
computer-readable medium or computer storage media, volatile and
nonvolatile media, removable and non-removable media implemented in
any method or technology for storage of information such as
computer readable instructions, data structures, program modules,
or other data. Such memory includes a random access memory (RAM),
computer storage media, communication media, and a
computer-readable non-volatile medium, such as flash memory.
Alternatively, a floppy disk, a compact disc-read only memory
(CD-ROM), a magneto-optical disk (MOD), and/or a digital versatile
disc (DVD) may also be used. Also, in the embodiments described
herein, additional input channels may be, but are not limited to,
computer peripherals associated with an operator interface such as
a mouse and a keyboard. Alternatively, other computer peripherals
may also be used that may include, for example, but not be limited
to, a scanner. Furthermore, in the exemplary embodiment, additional
output channels may include, but not be limited to, an operator
interface monitor.
[0083] As indicated above, the process may be embodied in computer
software. The computer software could be supplied in a number of
ways, for example on a tangible, non-transitory, computer readable
storage medium, such as on any nonvolatile memory device (e.g. an
EEPROM). Further, different parts of the computer software can be
executed by different devices, such as, for example, in a
client-server relationship. Persons skilled in the art will
appreciate that computer software provides a series of instructions
executable by the processor.
[0084] While the invention has been described with respect to the
figures, it will be appreciated that many modifications and changes
may be made by those skilled in the art without departing from the
spirit of the invention. Any variation and derivation from the
above description and figures are included in the scope of the
present invention as defined by the claims.
* * * * *