U.S. patent application number 11/519229 was filed with the patent office on 2007-06-28 for interactive gaming method and apparatus with emotion perception ability.
This patent application is currently assigned to Industrial Technology Research Institute. Invention is credited to Cheng-Hung Chang, Shun-Nan Liou, Ying-Ko Lu, Ming-Jye Tsai.
Application Number | 20070149282 11/519229 |
Document ID | / |
Family ID | 38194575 |
Filed Date | 2007-06-28 |
United States Patent
Application |
20070149282 |
Kind Code |
A1 |
Lu; Ying-Ko ; et
al. |
June 28, 2007 |
Interactive gaming method and apparatus with emotion perception
ability
Abstract
The present invention relates to an interactive gaming method
and apparatus with emotion perception ability, by which not only
gestures of a user can be detected and used as inputs for
controlling a game, but also physiological attributes of the user
such as heart beats, galvanic skin response (GSR), etc., can be
sensed and used as emotional feedbacks of the game affecting the
user. According to the disclosed method, the present invention
further provides an interactive gaming apparatus that will
interpret the signals detected by the inertial sensing module and
the bio sensing module and use the interpretation as a basis for
evaluating the movements and emotions of a user immediately, and
then the evaluation obtained by the interactive gaming apparatus is
sent to the gaming platform to be used as feedbacks for controlling
the game to interact with the user accordingly. Therefore, by the
method and apparatus according to the present invention, not only
the harmonics of human motion can be trained to improve, but also
the self-control of a user can be enhanced.
Inventors: |
Lu; Ying-Ko; (Guishan
Shiang, TW) ; Tsai; Ming-Jye; (Xinpu Town, TW)
; Liou; Shun-Nan; (Kaohsiung City, TW) ; Chang;
Cheng-Hung; (Taichung City, TW) |
Correspondence
Address: |
BRUCE H. TROXELL;SUITE 1404
5205 LEESBURG PIKE
FALLS CHURCH
VA
22041
US
|
Assignee: |
Industrial Technology Research
Institute
|
Family ID: |
38194575 |
Appl. No.: |
11/519229 |
Filed: |
September 12, 2006 |
Current U.S.
Class: |
463/36 |
Current CPC
Class: |
A63F 13/212 20140902;
A63F 13/235 20140902; A63F 2009/243 20130101; A63F 2300/204
20130101; A63F 2009/2402 20130101; A63F 2300/1012 20130101; A63F
13/211 20140902; A63F 2009/2408 20130101; A63F 13/06 20130101; A63F
2300/105 20130101; A63F 13/428 20140902; A63F 2009/2436
20130101 |
Class at
Publication: |
463/36 |
International
Class: |
A63F 9/24 20060101
A63F009/24 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 27, 2005 |
TW |
094146650 |
Claims
1. An interactive gaming method with emotion perception ability,
comprising steps of: sensing a movement for generating a gesture
signal accordingly while detecting at least a bio signal caused by
emotional variation; receiving and processing the gesture signal
and the least one bio signal so as to generate a processed signal
accordingly; and interpreting the movement and the emotion
variation with reference to the processed signal while using the
interpretation as basis to issue a digital signal accordingly.
2. The interactive gaming method of claim 1, further comprising a
step of: enabling an interactive apparatus to display an
interaction with respect to the digital signal.
3. The interactive gaming method of claim 2, wherein the
interactive apparatus is a device selected from the group
consisting of a display and a gaming platform.
4. The interactive gaming method of claim 1, wherein the bio signal
is a signal selected form the group consisting of: a heart rate
signal, a blood oxygen level signal, a blood perfusion rate signal,
a skin impedance signal, a temperature signal and a moisture
signal.
5. The interactive gaming method of claim 1, wherein the gesture
signal comprises at least an acceleration signal.
6. An interactive gaming apparatus with emotion perception ability,
comprising: a sensing unit, for detecting and measuring a movement
to generate a gesture signal accordingly and for detecting and
measuring at least a physiological attribute to generate at least a
bio signal accordingly; and a signal processing unit, electrically
connected to the sensing unit, for receiving and interpreting the
gesture signal and the at least one bio signal while using the
interpretation as basis to issue a digital signal accordingly.
7. The interactive gaming apparatus of claim 6, wherein the sensing
unit further comprises: an inertial sensing module, electrically
connected to the signal processing unit, for detecting and
measuring the movement so as to generate the gesture signal
accordingly; and a bio sensing module, electrically connected to
the signal processing unit, for detecting and measuring at least a
physiological attribute to generate the at least one bio signal
accordingly.
8. The interactive gaming apparatus of claim 6, further comprising:
a displaying unit, coupled to the signal processing unit for
enabling the same to received the digital signal.
9. The interactive gaming apparatus of claim 6, further comprising:
a gaming platform, capable receiving the digital signal while using
the received digital signal to interact.
10. The interactive gaming apparatus of claim 6, further
comprising: a displaying unit, coupled to the signal processing
unit for enabling the same to received the digital signal; and a
gaming platform, capable receiving the digital signal from the
signal processing unit while using the received digital signal to
interact.
11. The interactive gaming apparatus of claim 6, wherein the bio
sensing module is a module comprising devices selected form the
group consisting of an optical heart rate sensor, an electronic
heart rate sensor, an optical blood oxygen level sensor, an optical
blood perfusion rate meter, an electronic skin impedance sensor, a
temperature sensor and a moisture sensor.
12. The interactive gaming apparatus of claim 6, wherein the
interactive gaming apparatus is substantially a device selected
from the group consisting of a game wrist sleeve, a game groove, a
joystick, and a gamepad.
13. The interactive gaming apparatus of claim 6, wherein the
interactive gaming apparatus is substantially a device selected
from the group consisting of a mobile phone, a personal digital
assistant, and a portable game console.
14. The interactive gaming apparatus of claim 6, wherein the signal
processing unit further comprises: a first calculation unit, for
receiving and processing the gesture signal to form a first
electronic signal accordingly; and a second calculation unit, for
receiving and processing each bio signal to form a second
electronic signal accordingly.
15. The interactive gaming apparatus of claim 14, wherein the
signal processing unit further comprises: a signal
processing/emotion interpretation module, for receiving the first
and the second electronic signals while performing an operation
upon the received signals so as to issue a digital signal
accordingly.
16. The interactive gaming apparatus of claim 6, wherein inertial
sensing module is substantially a first inertial part, capable of
detecting a displacement of a movement moving along a first axis
and a rotation of the movement rotating about a second axis.
17. The interactive gaming apparatus of claim 6, wherein inertial
sensing module is substantially a second inertial part, capable of
detecting the displacement moving along a second axis and the
rotations rotating about a first axis and a third axis.
18. The interactive gaming apparatus of claim 17, wherein the
second inertial sensing part further comprising: a primary
accelerometer, for detecting the displacement moving along the
second axis; and an auxiliary accelerometer, being arranged apart
from the primary accelerometer by a specific distance, for enabling
the auxiliary accelerometer to detect an acceleration of one
degree-of-freedom; wherein the accelerations measured by the
primary accelerometer and the auxiliary accelerometer are subjected
to a differential operation so as to detect the rotation rotating
about the third axis.
19. The interactive gaming apparatus of claim 17, wherein the
second inertial sensing part further comprising: an accelerometer,
for detecting the displacement moving along the second axis and the
rotation rotating about the first axis; and a gyroscope, for
detecting the rotation rotating about the third axis.
20. The interactive gaming apparatus of claim 6, wherein inertial
sensing module is substantially a third inertial part, capable of
detecting the rotation rotating about the a third axis.
21. The interactive gaming apparatus of claim 6, wherein inertial
sensing module is a module composed of a first inertial part, a
second inertial part and a third inertial part.
22. The interactive gaming apparatus of claim 6, wherein the
sensing unit further comprises a wireless transmission module while
the gaming platform further comprises a wireless reception
module.
23. The interactive gaming apparatus of claim 6, further comprising
a networking interface, for enabling the interactive gaming
apparatus to connect to a network so as to download and update
softwares resided in the interactive gaming apparatus.
24. The interactive gaming apparatus of claim 6, capable of
executing a game software.
25. The interactive gaming apparatus of claim 6, further comprising
a storage media, for providing memory ability to the interactive
gaming apparatus.
26. An interactive gaming apparatus with emotion perception
ability, comprising: an inertial sensing module, for detecting and
measuring a movement to generate a gesture signal accordingly; a
bio sensing module, for detecting and measuring at least a
physiological attribute to generate at least a bio signal
accordingly; a signal processing unit, electrically connected to
the inertial sensing module and the bio sensing module, for
receiving and interpreting the gesture signal and the at least one
bio signal while using the interpretation as basis to issue a
digital signal accordingly; and a displaying unit, coupled to the
signal processing unit for enabling the same to received the
digital signal.
27. An interactive gaming apparatus with emotion perception
ability, comprising: a sensing unit, for detecting and measuring a
movement to generate a gesture signal accordingly and for detecting
and measuring at least a physiological attribute to generate at
least a bio signal accordingly; and a signal processing unit,
electrically connected to the sensing unit, for receiving and
interpreting the gesture signal and the at least one bio signal
sensed while using the interpretation as basis to issue a digital
signal according to the evaluation; and a gaming platform, capable
receiving the digital signal.
28. The interactive gaming apparatus of claim 27, wherein the
sensing unit further comprises: an inertial sensing module,
electrically connected to the signal processing unit, for detecting
and measuring the movement so as to generate the gesture signal
accordingly; and a bio sensing module, electrically connected to
the signal processing unit, for detecting and measuring at least a
physiological attribute to generate the at least one bio signal
accordingly.
29. The interactive gaming apparatus of claim 27, wherein the
signal processing unit is electrically connected to a signal
processing/emotion interpretation module, and the signal
processing/emotion interpretation module is arranged at a location
selected from the group consisting a position on the sensing unit
and a position on the gaming platform.
Description
FIELD OF THE INVENTION
[0001] The present invention relates to a gaming method and
apparatus, and more particularly, to an interactive gaming method
and apparatus with emotion perception ability, which is capable of
interpreting a user's gestures and bio statuses detected
respectively by the inertial sensing module and the bio sensing
module while using the interpretation as feedbacks for controlling
a game to interact with the user accordingly so as to train and
improve the motion harmonics and self-control of the user.
BACKGROUND OF THE INVENTION
[0002] With the evolution of video games, the current trend is to
enable the game characters of a video game to interact with players
through certain input devices of the game platform. The
conventional input devices, such as mice, keyboards, or joysticks,
etc., simply rely on a cross key and push button arranged thereon
for enabling game character movement and command selection.
[0003] However, as the popular of so-called "virtual reality" (VR)
game systems, players are enabled to experience computer-generated
environments instead of merely interacting with them over a display
screen. Such systems typically require the user to don goggles,
through which he or she reacts naturally to the changing virtual
environment, generating signals that the computer interprets to
determine the state and progress of the presented environment. In
these virtual reality game systems, the realistic feeling can be
enforced if signals perceivable to the computer can be generated in
accordance to the gestures and emotional responses of the user in
real time. Moreover, as the signals generated by the combinations
of user's gestures and emotional responses have far more varieties
than that of button-pressing input devices, the control of the user
to video game is greatly enhanced.
[0004] With this in mind, an improved input device, which is
capable of detecting and interpreting a user's external gestures
and internal physiological/emotional responses while using the
interpretation as feedbacks for controlling a game to interact with
the user, are in great need.
[0005] Many attempts have been made to develop such input devices,
one of which is disclosed in U.S. Pat. No. 5,534,917, entitled
"VIDEO IMAGE BASED CONTROL SYSTEM". The control system of the
aforesaid patent uses a video camera to capture images of a user,
such that the captured imaged are processed and analyzed to be
applied in a three-dimensional VR game. Although the abovementioned
system can allow a device to be controlled through movement or
positioning of a user within the field of view of a camera, which
is preferred than those conventional mice, keyboards and joysticks,
the type of control offered by the abovementioned system requires a
plurality of video cameras to operate simultaneously that the whole
system is bulky, difficult to set up and costly. In addition, the
abovementioned system has no usefulness in detecting user's
physiological/emotional responses while he/she is playing the
three-dimensional VR video game.
[0006] Another attempt is disclosed in U.S. Pat. No. 6,909,420,
entitled "DEVICE INDICATING MOVEMENTS FOR SOFTWARE". The system of
the aforesaid patent comprises at least a sleeve for putting on
over the joints of a users and at least a movement sensor fixed to
each sleeve, by which a movement of the user wearing the sleeves on
the limbs thereof can be detected and used as input of a game
platform for enabling the user to interact with a game operating on
the game platform. However, since the movement sensor used in the
abovementioned system is an on/off sensor, the sleeves can not
detect sophisticated movements and displacement since the sensor
with on two settings, i.e. ON and OFF, is not sufficiently
sensitive. In addition, the abovementioned system also has no
usefulness in detecting user's physiological/emotional
responses.
[0007] Yet, another attempt is disclosed in U.S. Pat. No.
6,190,314, entitled "COMPUTER INPUT DEVICE WITH BIIOSENSORS FOR
SENSING USER EMOTIONS", as shown in FIG. 1. In FIG. 1, the
aforesaid input device 10 is comprised of: a biosensor 101, for
detecting and measuring pulses of a user by a means of infrared; a
temperature sensor, for measuring temperature variation of the
user; and a contact electrode, for measuring galvanic skin response
(GSR) of the user; such that the physiological attributes relating
to the user's emotional state, including anger, fear, sadness,
disgust, joy and surprise, etc., can be recognized and measured.
However, the input device 10 is an optical mouse or a mechanical
mouse that can only detect user's hand movements on a plane and is
connected to a computer 102, it is not operable in space and thus
can not be applied in a three-dimensional VR video game
[0008] From the above description, it is noted that the prior-art
interactive control systems are lack of means of using
physiological attributes relating to a user's emotional state and
the user's gestures as inputs to a video game for enabling the user
to interact with game characters of the video game. Therefore, a
novel interactive gaming method and apparatus with emotion
perception ability is needed.
SUMMARY OF THE INVENTION
[0009] The primary object of the present invention is to provide an
interactive gaming method and apparatus with emotion perception
ability, by which not only gestures of a user can be detected and
used as inputs for controlling a game, but also physiological
attributes of the user such as heart beats, pulses, blood oxygen
level, general somatic activities, galvanic skin response (GSR),
etc., can be sensed and used as emotional feedbacks of the game
affecting the user, that is, the interactive gaming apparatus will
interpret the detected physiological attributes and gestures of a
user of the gaming apparatus, and employ the interpretation as a
basis for evaluating the movements and emotions of the user
immediately, and then the evaluation obtained by the interactive
gaming apparatus is sent to the gaming platform to be used as
feedbacks for controlling the game to interact with the user
accordingly.
[0010] Another object of the invention is to provide an interactive
gaming method and apparatus with emotion perception ability, which
is compact and commercially feasible by the utilization of micro
inertial sensing modules, non-invasive bio sensing modules and
wireless communication means.
[0011] Yet, another object of the invention is to provide an
interactive gaming method and apparatus with emotion perception
ability, capable of using an micro inertial sensing module to
detect the displacement/rotation movements of hands holding the
gaming apparatus and thus generating electronic signals to be
received and processed by a signal processor while enabling the
processed signals to be used as input signals of a gaming platform,
and therefore, freeing the hands from those button-pressing
operations for generating input signals and thus releasing the
fingers from pain and fatigue.
[0012] One further object of the invention is to provide an
interactive gaming method and apparatus with emotion perception
ability, capable of using an micro inertial sensing module to
detect the displacement/rotation movements of hands holding the
gaming apparatus and thus generating electronic signals to be
received and processed by a signal processor while enabling the
processed signals to be used as input signals of a gaming platform,
and therefore, increasing the information displaying area of the
gaming apparatus since the conventional press buttons can be
eliminated from the interactive gaming apparatus of the
invention.
[0013] To achieve the above object, the present invention provides
an interactive gaming method with emotion perception ability, which
comprises steps of: sensing a movement for generating a gesture
signal accordingly while detecting at least a bio signal caused by
emotional variation; receiving and processing the gesture signal
and the least one bio signal so as to generate a processed signal
accordingly; and interpreting the movement and the emotion
variation of the user with reference to the processed signal while
using the interpretation as basis for evaluating the movements and
emotions of the user so as to issue a digital signal
accordingly.
[0014] Preferably, the interactive gaming method with emotion
perception ability further comprises a step of: enabling an
interactive apparatus to display an interaction with respect to the
digital signal. In a preferred aspect, the interactive apparatus
can be a device selected from the group consisting of a display and
a gaming platform.
[0015] Preferably, the bio signal can be a signal selected form the
group consisting of: a heart rate signal, a blood oxygen level
signal, a blood perfusion rate signal, a skin impedance signal, a
temperature signal and a moisture signal.
[0016] Preferably, the gesture signal comprises at least an
acceleration signal.
[0017] Moreover, to achieve the above object, the present invention
provides an interactive gaming apparatus with emotion perception
ability, which comprises: an inertial sensing module, for sensing a
movement and generating a gesture signal accordingly; a bio sensing
module, for sensing at least a physiological attribute and
generating at least a corresponding bio signal accordingly; and a
signal processing unit, electrically connect to the inertial
sensing module and the bio sensing module, for receiving and
interpreting the gesture signal and the at least one bio signal
while using the interpretation as basis for issuing a digital
signal accordingly.
[0018] Preferably, the bio sensing module is a module comprising
devices selected form the group consisting of an optical heart rate
sensor, an electronic heart rate sensor, an optical blood oxygen
level sensor, an optical blood perfusion rate meter, an electronic
skin impedance sensor, a temperature sensor and a moisture
sensor.
[0019] Preferably, the interactive gaming apparatus with emotion
perception ability of the invention is substantially a device
selected from the group consisting of a game wrist sleeve, a game
groove, a joystick, and a gamepad.
[0020] Preferably, the signal processing unit further comprises: a
first calculation unit, for receiving and processing the gesture
signal to form a first electronic signal accordingly; a second
calculation unit, for receiving and processing each bio signal to
form a second electronic signal accordingly; and a signal
processing/emotion interpretation module, for receiving the first
and the second electronic signals while using the received signals
as basis for evaluating the movement and emotions of the user so as
to issue a digital signal according to the evaluation.
[0021] Preferably, the inertial sensing module further comprises a
first inertial sensing part, for detecting a displacement of a
movement moving along a first axis and a rotation of the movement
rotating about a second axis.
[0022] Preferably, the inertial sensing module further comprises a
second inertial sensing part, for detecting the displacement moving
along the second axis and the rotations rotating about the first
axis and a third axis. Moreover, the second inertial sensing part
further is comprised of: a primary accelerometer, for detecting the
displacement moving along the second axis; and an auxiliary
accelerometer, being arranged apart from the primary accelerometer
by a specific distance, for enabling the auxiliary accelerometer to
detect an acceleration of one degree-of-freedom; wherein the
accelerations measured by the primary accelerometer and the
auxiliary accelerometer are subjected to a differential operation
so as to detect the rotation rotating about the third axis.
[0023] In a preferred embodiment of the invention, the inertial
sensing module further comprises a second inertial sensing part,
for detecting the displacement moving along the second axis and the
rotation rotating about the first axis and a third axis. Moreover,
the second inertial sensing part further is comprised of: an
accelerometer, for detecting the displacement moving along the
second axis and the rotation rotating about the first axis; and a
gyroscope, for detecting the rotation rotating about the third
axis.
[0024] Preferably, the inertial sensing module further comprises a
third inertial sensing part, for detecting the rotation rotating
about the third axis.
[0025] In addition, to achieve the above object, the present
invention provides an interactive gaming apparatus with emotion
perception ability, which comprises: [0026] a sensing unit, further
comprising: [0027] an inertial sensing module for detecting and
measuring a movement to generate a gesture signal accordingly; and
[0028] a bio sensing module, for detecting and measuring at least a
physiological attribute to generate at least a bio signal
accordingly; [0029] a signal processing unit, electrically connect
to the inertial sensing module and the bio sensing module, for
receiving and interpreting the gesture signal and the at least one
bio signal while using the interpretation as basis for issuing a
digital signal accordingly; and [0030] a gaming platform, capable
receiving the digital signal from the signal processing unit while
using the received digital signal to control a game played
thereon.
[0031] Preferably, the sensing unit further comprises a wireless
transmission module, and the gaming platform further comprises a
wireless reception module.
[0032] Preferably, the sensing unit is substantially a device
selected from the group consisting of a game wrist sleeve, a game
groove, a joystick, and a gamepad.
[0033] Preferably, the signal processing unit further comprises: a
first calculation unit, for receiving and processing the gesture
signal to form a first electronic signal accordingly; and a second
calculation unit, for receiving and processing each bio signal to
form a second electronic signal accordingly; wherein the first
calculation unit can be arranged at a location selected from the
group consisting a position on the sensing unit and a position on
the gaming platform; and that is the same to the second calculation
unit.
[0034] Preferably, the signal processing unit is electrically
connected to a signal processing/emotion interpretation module,
whereas the signal processing/emotion interpretation module is
arranged at a location selected from the group consisting a
position on the sensing unit and a position on the gaming
platform.
[0035] In a preferred aspect of the invention, the signal
processing unit further comprises: a first calculation unit, for
receiving and processing the gesture signal to form a first
electronic signal accordingly; a second calculation unit, for
receiving and processing each bio signal to form a second
electronic signal accordingly; and a signal processing/emotion
interpretation module, for receiving the first and the second
electronic signals while using the received signals as basis to
issue a digital signal accordingly; wherein the signal
processing/emotion interpretation module can be arranged at a
location selected from the group consisting a position on the
sensing unit and a position on the gaming platform.
[0036] In addition, the first calculation unit can be arranged at a
location selected from the group consisting a position on the
sensing unit and a position on the gaming platform, and that is the
same to the second calculation unit.
[0037] Furthermore, to achieve the above object, the present
invention provides an interactive gaming apparatus with emotion
perception ability, which comprises: [0038] an inertial sensing
module for detecting and measuring a movement to generate a gesture
signal accordingly; [0039] a bio sensing module, for detecting and
measuring at least a physiological attribute to generate at least a
bio signal accordingly; [0040] a signal processing unit,
electrically connect to the inertial sensing module and the bio
sensing module, f or receiving and interpreting the gesture signal
and the at least one bio signal while using the interpretation as
basis for issuing a digital signal accordingly; and [0041] a
displaying unit, coupled to the signal processing unit for
receiving the digital signal.
[0042] Preferably, the interactive gaming apparatus of the
invention further comprises a networking interface, for enabling
the interactive gaming apparatus to connect to a network for
downloading and updating softwares resided in the interactive
gaming apparatus.
[0043] Preferably, the interactive gaming apparatus with emotion
perception ability of the invention is substantially a mobile
phone, a personal digital assistant, or a portable gaming
console.
[0044] Preferably, the interactive gaming apparatus with emotion
perception ability of the invention is capable of executing a game
software, whereas the game software can be a software selected from
the group consisting of a multimedia game, a 2D/3D animation game
and a music playing software with tempo/volume control.
[0045] Preferably, the interactive gaming apparatus of the
invention further comprises a memory, for storing the video game
and icons of emotional expression.
[0046] Other aspects and advantages of the present invention will
become apparent from the following detailed description, taken in
conjunction with the accompanying drawings, illustrating by way of
example the principles of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
[0047] FIG. 1 is a schematic view of a conventional computer input
device disclosed in U.S. Pat. No. 6,190,314.
[0048] FIG. 2 is a flowchart depicting steps of an interactive
gaming method with emotion perception ability according to a
preferred embodiment of the invention.
[0049] FIG. 3A is a schematic view of an interactive gaming
apparatus with emotion perception ability according to a first
preferred embodiment of the invention.
[0050] FIG. 3B is a schematic view of an interactive gaming
apparatus with emotion perception ability according to a second
preferred embodiment of the invention.
[0051] FIG. 4 is a schematic diagram showing the application of the
interactive gaming apparatus with emotion perception ability of the
first preferred embodiment of the invention.
[0052] FIG. 5A is a schematic view of an interactive gaming
apparatus with emotion perception ability according to a third
preferred embodiment of the invention.
[0053] FIG. 5B is a schematic diagram depicting the operation of
the inertial sensing module and the bio sensing module of the third
preferred embodiment of the invention.
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0054] For your esteemed members of reviewing committee to further
understand and recognize the fulfilled functions and structural
characteristics of the invention, several preferable embodiments
cooperating with detailed description are presented as the
follows.
[0055] Please refer to FIG. 2, which is a flowchart depicting steps
of an interactive gaming method with emotion perception ability
according to a preferred embodiment of the invention. The flow of
the interactive gaming method 4 of FIG. 2 starts at step 40. At
step 40, a movement is sensed for generating a gesture signal
accordingly while detecting at least a bio signal caused by
emotional variation, and then the flow proceeds to step 41. At step
41, the gesture signal and the least one bio signal are received
and processed so as to generate a processed signal accordingly, and
then the flow proceeds to step 42. It is noted that the bio signal
can be a signal selected form the group consisting of: a heart rate
signal, a blood oxygen level signal, a blood perfusion rate signal,
a skin impedance signal, a temperature signal and a moisture
signal, and the gesture signal is an acceleration signal. At step
42, the movement and the emotion variation are interpreted with
reference to the processed signal while using the interpretation as
basis to issue a digital signal accordingly, and then the flow
proceeds to step 43. At step 43, an interactive apparatus is
enabled to display an interaction with respect to the digital
signal. In a preferred aspect, the interactive apparatus can be a
display, or a gaming platform.
[0056] By the aforesaid method, an interactive gaming apparatus
with emotion perception ability can be provided. Please refer to
FIG. 3A, which is a schematic view of an interactive gaming
apparatus with emotion perception ability according to a first
preferred embodiment of the invention. In FIG. 3A, the interactive
gaming apparatus with emotion perception ability 2 is comprised of
a sensing unit 21 and a gaming platform 23. The sensing unit 21 can
be arranged on a game wrist sleeve, a game groove, a joystick, and
a gamepad, which is used for detecting a movement of a user and
physiological attributes of the same while playing a video game.
Moreover, the sensing unit 21 includes an inertial sensing module
211, a bio sensing module 211 and a signal processing unit 213. The
inertial sensing module 211 is used for detecting and measuring
accelerations or variations of angular velocity resulting from the
movement of the user while generating a gesture signal accordingly.
In this preferred embodiment, the inertial sensing module 211 is
either being device capable of sensing a movement measured along a
specific axis of a plane or along a specific direction define on
the plane, or being a device capable of sensing a movement in a
space defined by a 3-D Cartesian coordinates of X, Y, Z axes. In
addition, the bio sensing module 212 is used for sensing at least a
physiological attribute of the user and generating at least a bio
signal accordingly; wherein the bio sensing module 212 can be an
optical module comprising devices selected form the group
consisting of a heart rate sensor, a blood oxygen level sensor, a
blood perfusion rate meter, a skin impedance sensor, a temperature
sensor and a moisture sensor.
[0057] The signal processing unit 213 is electrically connected to
the inertial sensing module 211 and the bio sensing module 212 for
receiving the gesture signal from the inertial sensing module 211
and the bio signal from the bio sensing module 212. In a preferred
embodiment of the invention, the signal processing unit 213 further
includes a first calculation unit 2131, a second calculation unit
2132 and a signal processing/emotion interpretation module 2133;
wherein the first calculation unit 2131 is used for receiving and
processing the gesture signal to form a first electronic signal
accordingly; and the second calculation unit 2132 is used for
receiving and processing each bio signal to form a second
electronic signal accordingly. In addition, the sensing unit 21
further comprises a wireless transmission module 214, which is used
for encoding the first and the second electronic signals while
transmitting the encoded signals to the gaming platform 23.
[0058] In the first preferred embodiment shown in FIG. 3A, the
signal processing/emotion interpretation module 2133 is arranged on
the gaming platform 23 such that, as the wireless reception module
231 of the gaming platform 23 receives encoded signals transmitted
from the wireless transmission module 24, the received encoded
signals are decoded and sent to the signal processing/emotion
interpretation module 2133 to be interpreted and used as a basis
for evaluating the movements and emotions of a user immediately,
and then the obtained evaluation is sent to the gaming platform 23
to be used as feedbacks for controlling a game or a content 232 to
interact with the user accordingly, such as controlling the tempo,
the speed, the batting average, shooting rate, or size/color of
game characters, etc. of the game playing on the gaming platform
23. Moreover, the gaming platform 23 can be a computer, a game
console or a multimedia player.
[0059] Please refer to FIG. 3B, which is a schematic view of an
interactive gaming apparatus with emotion perception ability
according to a second preferred embodiment of the invention. The
configuration of the interactive gaming apparatus with emotion
perception ability 2 of FIG. 3B is generally the same as that shown
in FIG. 3A, and the only difference is that the signal
processing/emotion interpretation module 2133 is arranged on the
sensing unit 21 instead of the gaming platform 23. In addition, for
simplified the structure of the sensing unit 21 as well as saving
the power consumption of the same, it is also feasible to move and
arrange the whole signal processing unit 213 on the gaming platform
so that the wireless transmission module 214 simply transmits
signals detected by the inertial sensing module 211 and the bio
sensing module 212 while leaving the entire posterior signal
processing to be performed on the gaming platform 23.
[0060] Please refer to FIG. 4, which is a schematic diagram showing
the application of the interactive gaming apparatus with emotion
perception ability of the first preferred embodiment of the
invention. As shown in FIG. 4, the sensing unit 21 is a wrist
sleeve having an inertial sensing module 211, a bio sensing module
212, a signal processing unit 213 and a wireless transmission
module 214; wherein the bio sensing module 213 is further composed
of a optical bio sensor 2121 and a impedance electrode 2122. The
optical bio sensor 2121 is used for measuring physiological
attributes, such as blood oxygen level, pulse, heart beat, and
blood perfusion rate, etc. When a video game like a shooting game,
a game of aero exercising, a ball game, a boxing game or a fighting
game of martial art, etc., is played by a user wearing the wrist
sleeve, the emotional responses and the gestures of the user will
be detected by the bio sensing module 212 and the inertial sensing
module 211 and enables the two modules to issue signals,
corresponding to the detection, to be received and processed by the
signal processing unit 213. Thereafter, the processed signals are
sent to the wireless reception module 231 of the gaming platform 23
by the wireless transmission module 214 and further to the signal
processing/emotion interpretation module 2133 to be interpreted and
used as a basis for evaluating the movements and emotions of a user
immediately, and then the obtained evaluation is sent to the gaming
platform 23 to be used as feedbacks for controlling a game or a
content 232 to interact with the user accordingly.
[0061] Please refer to FIG. 5A, which is a schematic view of an
interactive gaming apparatus with emotion perception ability
according to a third preferred embodiment of the invention. As seen
in FIG. 4, the housing of the interactive gaming apparatus with
emotion perception ability 3 is configured with a displaying unit
33, a networking interface 37 and a bio interface 38. The
displaying unit can be a flat panel display of liquid crystal
display, or organic light emitting diode panel, but is not limited
thereby. The networking interface 37 is used for enabling the
interactive gaming apparatus 3 to connect to a network either
through a wired manner or a wireless manner so as to download and
update softwares resided in the interactive gaming apparatus 3.
Moreover, the interactive gaming apparatus 3 further comprises a
memory, which is used for storing the video game and icons of
emotional expression. It is noted that the interactive gaming
apparatus 3 can be applied in a variety of devices, such as a
mobile phone, a personal digital assistant (PDA), and a portable
game console, and so on.
[0062] The internal sensing/processing mechanism of the interactive
gaming apparatus 3 is shown in FIG. 5B, which shows the operation
of the inertial sensing module and the bio sensing module of the
third preferred embodiment of the invention. As seen in FIG. 5B,
the internal sensing/processing mechanism of the interactive gaming
apparatus 3 is arranged on a substrate 30, which is comprised of an
inertial sensing module 31, a signal processing unit 31 and a bio
sensing module 34. It is noted that the bio sensing module 33 is
connected to the bio interface 38 of FIG. 5A, such that
physiological attributes of a user can be detected by the bio
sensing module 34 though the bio interface 38 as the thumb of the
user is engaged with the bio interface 38 while the interactive
gaming apparatus 3 is held in the hand of the user.
[0063] Moreover, the inertial sensing module 31 is an inertial
sensor of sensing a least one degree-of-freedom sensibility. In
this preferred embodiment, the inertial sensing module 31 is an
inertial sensor capable of sensing a movement of six
degree-of-freedom, which is comprised of an X-axis inertial sensing
part 311, a Y-axis inertial sensing part 312 and a Z-axis inertial
sensing part 313. That is, as inertial sensing module 31 is place
in a space defined by a 3-D Cartesian coordinates of X, Y, Z axes,
a movement of six degree-of-freedom can be sensed by the inertial
sensing module 31 and generate a sensing signal accordingly, that
the X-axis inertial sensing part 311 is enabled to sense a
displacement measured along the X axis and the rotation
.omega..sub.y about the Y axis, and the Y-axis inertial sensing
part 312 is enabled to sense a displacement measured along the Y
axis and the rotations .omega..sub.y, .omega..sub.z respectively
about the Y and Z axis, and the Z-axis inertial sensing part 313 is
enabled to sense a displacement measured along the Z axis. In
addition, the signal processing unit 32 is coupled to the inertial
sensing module 31 and the displaying unit (not shown in FIG. 5B),
so that the signal processing unit 32 can receive and process the
sensing signal and then transmit the processed signal to the
displaying unit for enabling the displaying unit to display game
characters with respect to the processed signal.
[0064] In the preferred embodiment of FIG. 5B, the X-axis inertial
sensing part 311 and the Z-axis inertial sensing part 313 are
respectively an accelerometer, and the Y-axis inertial sensing part
312 is further comprised of: a primary accelerometer 3121, for
detecting the displacement moving along the Y axis; and an
auxiliary accelerometer 3122, being arranged apart from the primary
accelerometer 3121 by a specific distance D, for enabling the
auxiliary accelerometer to detect an acceleration of one
degree-of-freedom; wherein the accelerations measured by the
primary accelerometer 3121 and the auxiliary accelerometer 3122 are
subjected to a differential operation so as to detect the rotation
rotating about the Z axis. As for the rotation about the X axis can
be selected to be sensed by the primary accelerometer 3121 or the
auxiliary accelerometer 3122.
[0065] As the inertial sensing module 31 is subjecting to a
movement of a user holding the interactive gaming apparatus 3, the
sensing signal is sent to the signal processing unit 32 to be
interpreted and used as a basis for evaluating the movement of the
user, and then the evaluation is used as feedbacks for controlling
the game character 35 of FIG. 5A to interact with the user
accordingly. Moreover, the inertial sensing module 31 also can be
used for controlling the command selection of the menu 36 shown in
FIG. 5A, so that number of conventional press buttons required in
the interactive gaming apparatus 3 can be minimized and thus the
displaying area are enlarged. Furthermore, the inertial sensing
module 31 can enable the interactive gaming apparatus 3 to have
more ways of controlling the game character 35. Preferably, the
software resided in the interactive gaming apparatus 3 can be a
multimedia game, a 2D/3D animation game, or a music playing
software with tempo/volume control.
[0066] Referring to the emotion sensing capability of the
interactive gaming apparatus of the invention, as the interactive
gaming apparatus 3 of FIG. 5A is held in the both hands of a user,
the fingers of the user will engage with the bio interface 38 so
that the sensing signal of the bio sensing module 34 is sent to the
signal processing unit 32 to be interpreted and used as a basis for
evaluating the emotions of the user, and then the evaluation is
used as feedbacks for controlling the game character 35 to interact
with the user accordingly. In a preferred aspect, the bio sensing
module 34 can be employed for measuring emotion indexes or indexes
of physiological attribute while the result of the measuring can be
displayed on the displaying unit in a graphic manner.
[0067] In the first and the second embodiments of the invention,
the inertial sensing module can be a stand-along X-axis inertial
sensing part, a stand-along Y-axis inertial sensing part, or a
stand-along Z-axis inertial sensing part. However, also in the
first and the second embodiments of the invention, the inertial
sensing module can be a module comprising a X-axis inertial sensing
part, a Y-axis inertial sensing part, and a Z-axis inertial sensing
part, so as to enable the sensing module to sensing a movement of
six degree-of-freedom. As the principle of the aforesaid module is
known in prior art that it is not described further herein.
[0068] Although the interactive gaming apparatus 3 of FIG. 5A is
applied in a portable game console, it is not limited thereby. That
is, the interactive gaming apparatus can be applied in a video game
station or control pad thereof, such as joystick, and hand-held
controller, etc. Moreover, although the sensing unit of the
invention is fitted on a wrist sleeve as shown in FIG. 4, it is not
limited thereby and can be fitted to any means worn on elbow, knee,
waist, etc., such that all limb and body movements of the user can
be detected.
[0069] To sum up, the present invention provides a compact and
commercially feasible interactive gaming apparatus with emotion
perception ability, by which not only gestures of a user can be
detected and used as inputs for controlling a game, but also
physiological attributes of the user can be sensed and used as
emotional feedbacks of the game affecting the user, that is, the
interactive gaming apparatus will interpret the detected
physiological attributes and gestures of a user of the gaming
apparatus, and employ the interpretation as a basis for evaluating
the movements and emotions of the user immediately, and then the
evaluation obtained by the interactive gaming apparatus is sent to
the gaming platform to be used as feedbacks for controlling the
game to interact with the user accordingly.
[0070] While the preferred embodiment of the invention has been set
forth for the purpose of disclosure, modifications of the disclosed
embodiment of the invention as well as other embodiments thereof
may occur to those skilled in the art. Accordingly, the appended
claims are intended to cover all embodiments which do not depart
from the spirit and scope of the invention.
* * * * *