U.S. patent application number 11/580302 was filed with the patent office on 2007-08-16 for online interactive multimedia system and the transmission method thereof.
This patent application is currently assigned to Industrial Technology Research Institute. Invention is credited to Sheng-Wen Jeng, Zhi-Wei Lian, Shun-Nan Liou, Ming-Jye Tsai.
Application Number | 20070190506 11/580302 |
Document ID | / |
Family ID | 38369007 |
Filed Date | 2007-08-16 |
United States Patent
Application |
20070190506 |
Kind Code |
A1 |
Jeng; Sheng-Wen ; et
al. |
August 16, 2007 |
Online interactive multimedia system and the transmission method
thereof
Abstract
The present invention relates to an online interactive
multimedia system and the transmission method, that the interactive
system is adapted to be applied by users of a plurality of remote
ends for enabling users at his/her remote end to interact with each
other in an online fashion, and each remote end comprises: a at
least a micro inertial sensing module, a micro processing device,
an audio/video device, a network device, and a displaying device;
wherein each micro inertial sensing module is capable of detecting
and measuring motions of a user operating the remote end so as to
generate gesture signals accordingly; the micro processing device
is used for receiving/transmitting and processing audio/video as
well as the gesture signals from/to other remote ends in an online
fashion, while using the processed signals to control a software
arranged in the online interactive multimedia system for enabling
the displaying device to display images accordingly; the
audio/video device is used for providing real-time audio/video
service between the remote end and another remote end connected
thereto through a network; and the network device, for connecting
the remote end to a network system for enabling the remote end to
transmit/receive audio/video and the gesture signals to/from
another. remote ends.
Inventors: |
Jeng; Sheng-Wen; (Tainan
City, TW) ; Lian; Zhi-Wei; (Xindian City, TW)
; Liou; Shun-Nan; (Kaohsiung City, TW) ; Tsai;
Ming-Jye; (Xinpu Town, TW) |
Correspondence
Address: |
BRUCE H. TROXELL
SUITE 1404
5205 LEESBURG PIKE
FALLS CHURCH
VA
22041
US
|
Assignee: |
Industrial Technology Research
Institute
|
Family ID: |
38369007 |
Appl. No.: |
11/580302 |
Filed: |
October 13, 2006 |
Current U.S.
Class: |
434/247 |
Current CPC
Class: |
A63B 2225/50 20130101;
A63F 13/211 20140902; A63B 2102/02 20151001; A63B 2102/16 20151001;
A63F 13/212 20140902; A63B 2220/40 20130101; A63F 13/10 20130101;
A63F 13/335 20140902; A63B 24/0087 20130101; A63F 2300/1087
20130101; A63B 2024/0096 20130101; A63B 2102/32 20151001; A63F
13/235 20140902; A63F 2300/1012 20130101; A63F 13/428 20140902;
A63F 13/12 20130101; A63F 2300/6045 20130101 |
Class at
Publication: |
434/247 |
International
Class: |
A63B 69/00 20060101
A63B069/00 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 26, 2005 |
TW |
094146489 |
Aug 11, 2006 |
TW |
095129463 |
Claims
1. An online interactive multimedia system, adapted to be applied
by users of a plurality of remote ends for enabling users at
his/her remote end to interact with each other, each remote end
comprising: at least a micro inertial sensing module, each capable
of detecting and measuring motions of a user operating a remote end
so as to generate a first gesture signals accordingly; a micro
processing device, for receiving/transmitting the first gesture
signal from/to other remote ends for forming a second gesture
signal thereon, while processing the first and the second gesture
signals thereof to generate a processing signal accordingly; and a
displaying device, connected to the micro processing device for
enabling the same to receive the processing signal.
2. The system of claim 1, wherein the micro inertial sensing module
is primarily comprised of at least a micro inertial sensor, a
wireless transceiver, and peripheral circuits.
3. The system of claim 1, wherein the micro inertial sensing module
is comprised of at least an accelerometer.
4. The system of claim 1, wherein the micro inertial sensing module
is comprised of at least a gyroscope.
5. The system of claim 1, wherein the micro inertial sensing module
is enabled to be fitted to any part of a user's body.
6. The system of claim 1, wherein the micro inertial sensing module
can be arranged inside a package.
7. The system of claim 6, wherein the package is formed into a
shape of a batting means selected from the group consisting of a
bat, a table tennis racket, a golf club, and the like.
8. The system of claim 1, wherein the micro processing device is
able to control the software according to the online fashion.
9. The system of claim 1, wherein the first and the second gesture
signals are transmitted by a means of network transmission, whereas
the means of network transmission is a means selected from the
group consisting of an ADSL network, a cable local area network, a
Bluetooth communication network, a 3G mobile network and a GPRS
mobile network.
10. An online interactive multimedia system, adapted to be applied
by users of a plurality of remote ends for enabling users at
his/her remote end to interact with each other, each remote end
comprising: at least a micro inertial sensing module, each capable
of detecting and measuring motions of a user operating a remote end
so as to generate a first gesture signals accordingly; an
audio/video device, for fetching a first audio/video data from the
remote end; a micro processing device, for receiving/transmitting
the first gesture signal and the first audio/video data from/to
other remote ends for forming a second gesture signal and a second
audio/video data thereon, while the processing the first and the
second gesture signals as well as the first and the second
audio/video data to generate a processing signal accordingly; and a
displaying device, connected to the micro processing device for
enabling the same to receive the processing signal.
11. The system of claim 10, wherein the micro inertial sensing
module is primarily comprised of at least a micro inertial sensor,
a wireless transceiver, and peripheral circuits.
12. The system of claim 10, wherein the micro inertial sensing
module is comprised of at least an accelerometer.
13. The system of claim 10, wherein the micro inertial sensing
module is comprised of at least a gyroscope.
14. The system of claim 10, wherein the micro inertial sensing
module is enabled to be fitted to any part of a user's body.
15. The system of claim 10, wherein the micro inertial sensing
module can be arranged inside a package.
16. The system of claim 15, wherein the package is formed into a
shape of a batting means selected from the group consisting of a
bat, a table tennis racket, a golf club, and the like.
17. The system of claim 10, wherein the audio/video device includes
audio/video receivers selected form the group consisting of
camcorders, microphones, speakers, and the like.
18. The system of claim 10, wherein the first and the second
audio/video data received by the micro processing device are
outputted from the audio/video device.
19. The system of claim 10, wherein the micro processing device is
able to control the software according to the online fashion.
20. The system of claim 10, wherein the first and the second
gesture signals are transmitted by a means of network transmission,
whereas the means of network transmission is a means selected from
the group consisting of an ADSL network, a cable local area
network, a Bluetooth communication network, a 3G mobile network and
a GPRS mobile network.
21. A transmission method of an online interactive multimedia,
adapted to be applied by users of a plurality of remote ends of the
online interactive multimedia system for enabling users at his/her
remote end to interact with each other, comprising steps of:
detecting a gesture of a user operating a remote end for generating
a first gesture signal accordingly; receiving/transmitting the
first gesture signal from/to other remote ends for forming a second
gesture signal thereon; processing the first and the second gesture
signals to generate a processing signal accordingly; and receiving
the processing signal while displaying corresponding images and
motions with respect to the received processing signal.
22. The method of claim 21, further comprising steps of: fetching a
first audio/video data form a remote end; receiving/transmitting
the first audio/video data from/to other remote ends for forming a
second gesture signal thereon; processing the first and the second
audio/video data to generate an audio/video processing signal
accordingly; and receiving the audio/video processing signal while
displaying corresponding audios and videos with respect to the
received audio/video processing signal.
23. The method of claim 21, wherein the processing signal is
comprised of a first signal, acquired from the processing of the
first gesture signal, and a second signal, acquired from the
processing of the second gesture signal.
24. The method of claim 23, wherein the displaying of the
corresponding images and motions with respect to the received
processing signal further comprise steps of: displaying images and
motions corresponding to the remote end with respect to the first
signal; and displaying images and motions corresponding to the
other remote ends with respect to the second signal.
Description
FIELD OF THE INVENTION
[0001] The present invention relates to an online interactive
multirmedia system and the transmission method thereof, and more
particularly, to an online video conferencing means capable of
enabling remote users to interact with each other, not only by
video and audio, but also by motions and gestures; that is, a user
of the interactive system not only can perform video conference
with other remote users in an online fashion, but also can
further-have gesture interaction with them, so that the interactive
system can be employed as a platform for interactive sport game, an
online fitness training program, or a remote rehabilitation
program.
BACKGROUND OF THE INVENTION
[0002] As the lifestyle and diet of modern people tends to change
in the footsteps of time, more and more high-calorie foods are
consumed while just activities of daily living are performed. This
has resulted in an increasing number of persons who are obese. This
is a problem even amongst children. It is known that a good
exercise can help you burn off calories and increase your metabolic
rate. Therefore, there are all kinds of fitness center and aerobic
exercise clubs available for obese people. However, most of the
exercise programs provided by such facilities either are performed
at a specific location where trainees can receive instructions in
group from a few trainers or instructors, or are capable of
providing exercise videos to be purchased by trainees for enabling
each trainee to perform the exercise by oneself at home while
following the instructions of the videos. Although the exercise
video is capable of enabling a person to perform exercise at any
time and place, it can not motivate the trainee to develop an
exercise routine that can become part of one's everyday life since
it is usually boring and robotic, and can not enable the trainee to
obtain instructions from trainer instantly for enabling the same to
perform the exercise correctly.
[0003] One of such system is the electronic health monitoring
system disclosed in U.S. Pat. No. 6,144,837, entitled "Method and
apparatus for interactively monitoring a physiological condition
and for interactively providing health-related information", as
shown in FIG. 1. In the operating of the health monitoring system
10, a multimedia device 15 is used to play an optic disc 17
containing programs of relating training/testing while interacting
with a user therewith, and the same time that the physiological
conditions of the user can be detected and converted into relating
physiological signals by the electrodes 11, 12, 13 whereas the
generated physiological signals are transmitted to be displayed on
the monitor 16 by the interface device 14, and consequently, the
physiological condition of the user is revealed. Although the
aforesaid system can assist a user to exercise a training or
testing by oneself at home, it is still short of interaction as it
can not enable any two users, each at one's home, to interact with
each other, and as the programs contained in the optic disc 17 for
providing training/test that can be used in the system 10 are all
designed to be practice following a specific schedule and can not
be changed. Therefore, it is not practical. In addition, it is not
economical since once the programs contained in the optic disc are
done by the user, additional spending will be necessary to purchase
disc containing new exercise programs.
[0004] Other examples of such systems are the two interactive
teaching systems disclosed in TW Pat. No. 92119274 and the TW Pat.
No. 91111827, which are online systems capable of utilizing
multimedia devices and Internet to enable interaction between
teachers and students, and vice versa. The operation modes of the
aforesaid teaching systems are similar to that of the popular MSN
messenger. Although the interaction of the aforesaid teaching
system is enhanced, the interaction is restricted to audio and
video only, and the users of the systems still can not interact
with each other by gesture movements.
SUMMARY OF THE INVENTION
[0005] In view of the disadvantages of prior art, the primary
object of the present invention is to provide an online interactive
multimedia system and the transmission method thereof, capable of
enabling remote users to interact with each other, not only by
video and audio, but also by motions and gestures; that is, a user
of the interactive system not only can perform video conference
with other remote users in an online fashion, but also can further
have gesture interaction with them, so that the interactive system
can be employed as a platform for interactive sport game, an online
fitness training program, or a remote rehabilitation program.
[0006] To achieve the above object, the present invention provides
an online interactive multimedia system, adapted to be applied by
users of a plurality of remote ends for enabling users at his/her
remote end to interact with each other, each remote end comprising:
[0007] at least a micro inertial sensing module, each capable of
detecting and measuring motions of a user operating a remote end so
as to generate a first gesture signals accordingly; [0008] a micro
processing device, for receiving/transmitting the first gesture
signal from/to other remote ends for forming a second gesture
signal thereon, while processing the first and the second gesture
signals thereof to generate a processing signal accordingly; and
[0009] a displaying device, connected to the micro processing
device for enabling the same to receive the processing signal.
[0010] Preferably, the micro inertial sensing module is primarily
comprised of at least a micro inertial sensor, a wireless
transceiver, and peripheral circuits.
[0011] Preferably, the micro inertial sensor can be a device
selected from the group consisting of an accelerometer and a
gyroscope.
[0012] Preferably, the micro inertial sensing module can be fitted
to any part of a user's body.
[0013] Preferably, the micro inertial sensing module can be
arranged inside a package.
[0014] Preferably, the package can be formed into a bat, a table
tennis racket, and a golf club.
[0015] Preferably, the first and the second gesture signals are
transmitted by a means of network transmission, whereas the means
of network transmission is a means selected from the group
consisting of an ADSL network, a cable local area network, a
Bluetooth communication network, a 3G mobile network and a GPRS
mobile network.
[0016] To achieve the above object, the present invention further
provides an online interactive multimedia system, adapted to be
applied by users of a plurality of remote ends for enabling users
at his/her remote end to interact with each other, each remote end
comprising: [0017] at least a micro inertial sensing module, each
capable of detecting and measuring motions of a user operating a
remote end so as to generate a first gesture signals accordingly;
[0018] an audio/video device, for fetching a first audio/video data
from the remote end; [0019] a micro processing device, for
receiving/transmitting the first gesture signal and the first
audio/video data from/to other remote ends for forming a second
gesture signal and a second audio/video data thereon, while the
processing the first and the second gesture signals as well as the
first and the second audio/video data to generate a processing
signal accordingly; and [0020] a displaying device, connected to
the micro processing device for enabling the same to receive the
processing signal.
[0021] Preferably, the audio/video device includes camcorders,
microphones, speakers, and so on.
[0022] Preferably, the audio signal received by the micro
processing device is outputted from the audio/video device.
[0023] Moreover, to achieve, the above object, the present
invention provides a transmission method of an online interactive
multimedia, adapted to be applied by users, of a plurality of
remote ends of an online interactive multimedia system for enabling
users at his/her remote end to interact with each other, the method
comprising steps of: [0024] detecting a gesture of a user operating
a remote end for generating a first gesture signal accordingly;
[0025] receiving/transmitting the first gesture signal from/to
other remote ends for forming a second gesture signal thereon;
[0026] processing the first and the second gesture signals to
generate a processing signal accordingly; and [0027] receiving the
processing signal while displaying corresponding images and motions
with respect to the received processing signal.
[0028] In a preferred aspect, the transmission method further
comprises steps of: [0029] fetching a first audio/video data form a
remote end; [0030] receiving/transmitting the first audio/video
data from/to other remote ends for forming a second gesture signal
thereon; [0031] processing the first and the second audio/video
data to generate an audio/video processing signal accordingly; and
[0032] receiving the, audio/video processing signal while
displaying corresponding audios and videos with respect to the
received audio/video processing signal.
[0033] Preferably, as the processing signal is comprised of a first
signal, acquired from the processing of the first gesture signal,
and a second signal, acquired from the processing of the second
gesture signal, the displaying of the corresponding images and
motions with respect to the received processing signal further
comprise steps of: [0034] displaying images and motions
corresponding to the remote end with respect to the first signal;
and [0035] displaying images and motions corresponding to the other
remote ends with respect to the second signal.
[0036] Other aspects and advantages of the present invention will
become apparent from the following detailed description, taken in
conjunction with the accompanying drawings, illustrating by way of
example the principles of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
[0037] FIG. 1 shows an electronic health monitoring system
disclosed in U.S. Pat. No. 6,144,837.
[0038] FIG. 2 is a flow chart illustrating a preferred embodiment
of the invention.
[0039] FIG. 3 shows an online interactive multimedia system
according to a preferred embodiment of the invention.
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0040] For your esteemed members of reviewing committee to further
understand and recognize the fulfilled functions and structural
characteristics of the invention, several preferable embodiments
cooperating with detailed description are presented as the
follows.
[0041] Please refer to FIG. 2, which is a flow chart illustrating a
preferred embodiment, of the invention. In FIG. 2, the transmission
method of an online interactive multimedia, adapted to be applied
by users of a plurality of remote ends of an online interactive
multimedia system for enabling users at his/her remote end to
interact with each other, starts at step 30. At step 30,
information required for the transmission method is acquired before
the flow proceeds to, step 31, which include two sub-steps 301,
302. At sub-step 301, motions of a user operating a remote end is
detected and measured so as to generate a first gesture signals
accordingly; and at sub-step 302, a first audio/video data of the
aforesaid remote end is fetched.
[0042] At step 31, an operation is performed at the remote end for
receiving the first gesture signal and the first audio/video data
and transmitting the same to another remote end for causing a
second gesture signal and a second audio/video data to be formed on
the another remote end; and then the flow proceeds to step 32. At
step 32, the first gesture signal, the first audio/video data, the
second gesture signal and the second audio/video data are
processed, whereas the process of step 32 further comprises steps
of: processing the first and the second gesture signal to form a
motion processing signal; and processing the first and the second
audio/video data to form an audio/video processing signal; and then
the flow proceeds to step 33. Moreover, in a preferred aspect, the
signal transmission of the aforesaid flow chart is enabled by a
means of network transmission, whereas the means of network
transmission is a means selected from the group consisting of an
ADSL network, a cable local area network, a Bluetooth communication
network, a 3G mobile network and a GPRS mobile network.
[0043] At step 33, the motion processing signal and the audio/video
processing signal are received and used for generating
corresponding images, sounds and motions with respect to the above
processing, which further comprises steps of: displaying images and
motions corresponding to the remote end with respect to a first
signal, acquired from the processing of the first gesture signal
and the first audio/video data; and displaying images and motions
corresponding to the other remote ends with respect to a second
signal, acquired from the processing of the second gesture signal
and the second audio/video data. Thus, as users at the plural
remote ends of the invention are enabled to communicate with each
other through a means of network transmission, the display at a
remote end not only is enabled to display images corresponding to
the motions of the user operating the same, but also it is enabled
to display images corresponding to the motions of the other users
operating other remote end. For instance, a plurality of sub-frames
can be presented on a display for enabling the same to display
images relating to users at other remote ends.
[0044] Please refer to FIG. 3, which shows an online interactive
multimedia system according to a preferred embodiment of the
invention. In the FIG. 3, the two users A, B respectively operating
two remote ends 20, 30 of the online interactive multimedia system
are connected through a network system 40 for enabling signals to
be transmitted to and from between the two remote ends 20, 30. In
this preferred embodiment, more than one micro inertial sensing
modules 21 are fitted to the user A operating the remote end 20
while more than one micro inertial sensing modules 31 are fitted to
the, user B operating the remote end 30, whereas each of the micro
inertial sensing modules 21, 31 can be enabled to be fitted to any
part of the users A, B in respective, such as hand, waist, hip,
leg, or even head, or neck. That is, it is preferred to attach to
the movable parts of a human body. Each of the micro inertial
sensing modules 21, 31 is capable of detecting and measuring
motions of the users A, B in respective so as to generate gesture
signals accordingly while transmitting the gesture signals by
wireless transceiver arranged respectively in each of the micro
inertial sensing modules 21, 31. Preferably, each of the micro
inertial sensing modules 31, 31 is primarily comprised of at least
a micro inertial sensor, whereas each micro inertial sensor can be
an individual accelerometer, and an individual gyroscope, an
assembly of a plural accelerometers, an assembly of a plural
gyroscopes, or the combinations thereof. The aforesaid
accelerometers or gyroscopes should function with corresponding
peripheral circuits so as to perform an operation of acceleration
calculation and signal processing, that is known to those skilled
in the art and is voided to be described further herein. As the
gesture signals are generated by the detection of each of the micro
inertial sensing modules 21, 31 upon the movement of the users A, B
in respective, the gesture signals are respectively sent to the
micro processing devices 23, 33 to be evaluated and interpreted.
Any one of the two micro processing devices 22, 33 is capable of
receiving/transmitting and processing audio/video signals as well
as the gesture signals from/to other remote ends in an online
fashion, while using the processed signals to control an
interactive software of the online interactive multimedia system
for enabling the displaying devices 24, 34 to display images
accordingly. It is noted that the means of network transmission is
a means selected from the group consisting of an ADSL network, a
cable local area network, a Bluetooth communication network, a 3G
mobile network and a GPRS mobile network whereas the means of
network transmission is a means selected from the group consisting
of an ADSL network, a cable local area network, a Bluetooth
communication network, a 3G mobile network and a GPRS mobile
network. As seen in FIG. 3, each displaying devices 24, 34 has a
sub-displaying frame, which can be used to display a second image,
such as a close-up of the head of the user A, or a still image,
etc. Moreover, as each of the micro inertial sensing modules 21, 31
is detecting respectively the motions of the two users A, B, two
audio/video devices are used for providing a real-time audio/video
service between the two remote ends 20, 30 according to the online
fashion, that each of the two audio/video devices 20, 30 includes
audio/video receivers selected form the group consisting of
camcorders, microphones, speakers, and the like.
[0045] From the above description, it is concluded that a
transmission method of an online interactive multimedia, adapted to
be applied by users of a plurality of remote ends of the online
interactive multimedia system is provided, which comprises steps
of: [0046] (a) detecting gestures of users A, B, respectively
operating remote ends 20, 30 for generating-gesture signals
accordingly by the use of micro inertial sensing modules 21, 31;
[0047] (b) receiving/transmitting and processing audio/video
signals as well as the gesture signal from/to other remote end in
an online fashion, while using the processed signals to control a
software for respectively enabling displaying devices 24, 34 to
display images accordingly by the use of micro processing devices
23, 33; [0048] (c) providing a real-time audio/video service
between the two remote ends 20, 30 according to the online fashion
by the use of audio/video devices 22, 32 in respective; [0049] (d)
connecting the two remote ends 20, 30 to a network system 40 for
enabling the two remote ends 20, 30 to transmit/receive audio/video
and the gesture signals to/from each other; and [0050] (e)
displaying the audio/video signals and the gesture signal received
from the network system 40 on the displaying devices 24, 34 in
respective.
[0051] Furthermore, instead of arranging each of the micro inertial
sensing modules 21, 31 at any location of the two user A, B, each
micro inertial sensing modules 21, 31 can be arranged inside a
package, whereas the package is formed into a shape of a batting
means selected from the group consisting of a bat, a table tennis
racket, a golf club, and the like. Such that, by cooperating with
matching software and displaying device, a batting movement can be
simulated using the package. For instance, If the user A operating
the remote end 20 is holding a gold-club-shaped package containing
a micro inertial sensing module 21, the swing of the user A can be
detected by the micro inertial sensing module 21 and then the
result of the detection is transmitted to the micro processing
device 23 while the audio/video device 22 is also transmitting the
sounds and images of the swing to the micro processing device 23;
and then the micro processing device 23 is enabled to transmit all
the signals received thereby to another remote end 30 through the
network system 40 where the micro processing device 33 will process
all those signals to be displayed on the display device 34.
Similarly, the above description is suitable to the user B
operating the remote end 30 for display the motions of the user B
on the displaying device 24 of the remote end 20 operated by the
user A. Assuming that user A operating the remote end 20 is a
student and user B operating the remote end 30 is a coach, the
coach is able to see the swing of the student on his/her displaying
device 34 in real time so that he/she can specifically correct the
gesture, strength and trajectory of the awing instantly. Assuming
that user A operating the remote end 20 is a patient and user B
operating the remote end 30 is a doctor, the doctor is able to see
the rehabilitation exercise performed by the patient on his/her
displaying device 34 in real time so that he/she can specifically
direct the exercise instantly. In addition, online game can be
enabled by the user of the network server of the network system 40,
that is, the two users A, B, are able to select an on line ball
game on the network system so as to play the game against each
other. It is noted that the amount of user is not limited by the
two users A, B, as described in the preferred embodiment shown in
FIG. 3, that it can be as many as specified by the online software
containing in the online interactive multimedia system.
[0052] To sum up, the present invention provides an online
interactive multimedia system and the transmission method thereof,
and more particularly, to an online video conferencing means
capable of enabling remote users to interact with each other, not
only by video and audio, but also by motions and gestures; that is,
a user of the interactive system not only can perform video
conference with other remote users in an online fashion, but also
can further have gesture interaction with them.
[0053] While the preferred embodiment of the invention has been set
forth for the purpose of disclosure, modifications of the disclosed
embodiment of the invention as well as other embodiments thereof
may occur to those-skilled in the art. Accordingly, the appended
claims are intended to cover all embodiments which do not depart
from the spirit and scope of the invention.
* * * * *