U.S. patent application number 12/609438 was filed with the patent office on 2010-11-04 for systems and methods for optimizing one or more audio tracks to a video stream.
Invention is credited to John Hajime FORD.
Application Number | 20100279822 12/609438 |
Document ID | / |
Family ID | 43030817 |
Filed Date | 2010-11-04 |
United States Patent
Application |
20100279822 |
Kind Code |
A1 |
FORD; John Hajime |
November 4, 2010 |
SYSTEMS AND METHODS FOR OPTIMIZING ONE OR MORE AUDIO TRACKS TO A
VIDEO STREAM
Abstract
Disclosed are systems and methods for detecting the movements in
an exerciser's repetitive movement pattern that, when pre-recorded
audio footfall sound files are played, most closely support the
illusion that the user is generating the sounds with their own
"virtual" footfalls, thus creating an enhanced virtual exercise
experience. The system generates a signal to communicate the
virtual footfall timing with a media playback system, and plays
back media files that support the virtual exercise experience.
Inventors: |
FORD; John Hajime; (San
Francisco, CA) |
Correspondence
Address: |
GREENBERG TRAURIG, LLP (SV);IP DOCKETING
2450 COLORADO AVENUE, SUITE 400E
SANTA MONICA
CA
90404
US
|
Family ID: |
43030817 |
Appl. No.: |
12/609438 |
Filed: |
October 30, 2009 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61110559 |
Nov 1, 2008 |
|
|
|
61248151 |
Oct 2, 2009 |
|
|
|
Current U.S.
Class: |
482/8 |
Current CPC
Class: |
A63B 2230/06 20130101;
A63B 2225/50 20130101; A63B 2225/20 20130101; A63B 71/0686
20130101; A63B 2071/0638 20130101; A63B 71/0622 20130101; A63B
2071/0625 20130101 |
Class at
Publication: |
482/8 |
International
Class: |
A63B 71/00 20060101
A63B071/00 |
Claims
1. An exercise system comprising: a video playing device in
communication with a central processing unit and a user input
interface connected thereto adapted and configured to receive one
or more inputs from a user and transmit the one or more inputs to
the central processing unit; a video frame sequence encoded onto a
storage medium in communication with and playable on the video
playing device; a video monitor in communication with the video
playing device for displaying the video frame sequence reproduced
by the video playing device; a video frame rate controller
configured to vary a rate of display of sequential frames in the
video frame sequence reproduced by the video playing device on the
video monitor in response to the one or more inputs from the user;
one or more audio rate controllers configured to vary a rate of
delivery of two or more audio files on an audio playing device in
response to the one or more inputs from the user; and an exercise
device.
2. The system of claim 1 wherein the video frame rate controller
comprises a computer program stored on a storage medium with the
prerecorded video frame sequence, the program being operable with
the central processing unit in the video playing device when the
video sequence is played to modify the frame display rate in
response to the signal from a user interface rate detector.
3. The system of claim 2 wherein the signal from the user interface
rate detector is one or more of a biological parameter measurer,
and an exercise device operation detector.
4. The system of claim 3 wherein the user interface rate detector
is a heart rate signal receiver for receiving a signal from a
transmitter worn by the user controlling the rate at which the
video is displayed on the video monitor.
5. The system of claim 1 wherein the audio rate controller
comprises a computer program stored on a storage medium with an
audio sequence, the program being operable with the central
processing unit in the audio playing device when the audio sequence
is played to modify the audio delivery rate in response to the
signal from the user interface rate detector.
6. The system of claim 5 wherein the signal from the user interface
rate detector is one or more of a biological parameter measurer,
and an exercise device operation detector.
7. The system of claim 6 wherein the interface rate detector is a
heart rate signal receiver for receiving a signal from a
transmitter worn by the user and generating a signal usable by the
video player for displaying a user's heart rate on the display.
8. The system of claim 1 wherein the user input is one or more of
each of manual input and automatic input.
9. The system of claim 8 wherein the manual input is one or more of
height, weight, age, difficulty level, target training rate, target
heart rate, and target activity level.
10. The system of claim 8 wherein the automatic input is a signal
from one or more of a biological parameter measurer, and an
exercise device operation detector.
11. The system of claim 1 wherein the exercise device is selected
from the group comprising a bicycle, a stair stepper, and an
elliptical trainer.
12. The system of claim 1 wherein the rate of delivery of the two
or more audio files are delivered asynchronous to each other.
13. The system of claim 1 wherein each of the recorded video frame
sequences includes a frame time stamp and the video frame rate
controller is adapted to generate a variable time adjustment factor
in proportion to the one or more inputs from the user and a
adjustor adapted to apply a factor to the frame time stamp to
determine a modified time at which the next video frame sequence is
to be displayed.
13. An apparatus for engaging a user of an exercise device
interactively in viewing a video frame sequence of an activity on a
video monitor comprising: a video player in communication with the
video monitor adapted and configured to play a video frame sequence
on the video monitor; a detector adapted and configured to detect a
rate of exercise by the user on the exercise apparatus and transmit
a signal proportional to the rate of exercise; a video controller
in communication with the video player adapted and configured to
receive the signal proportional to the rate of exercise and
transmit a signal generated in response to the signal proportional
to the rate of exercise to the video player wherein the video
player adjusts the rate at which the video frame sequence is
displayed; two or more audio controllers in communication with an
audio player adapted and configured to receive the signal
proportional to the rate of exercise and transmit one or more
signals generated in response to the signal proportional to the
rate of exercise to the audio player wherein the audio player
adjusts the rate at which the audio is played.
14. The apparatus according to claim 13 wherein the exercise device
is selected from the group comprising a bicycle, a stair stepper
and an elliptical trainer.
15. The apparatus according to claim 13 wherein the signal from the
user interface rate detector is one or more of a biological
parameter measurer, and an exercise device detector.
16. The apparatus according to claim 15 wherein the interface rate
detector is a heart rate signal receiver for receiving a signal
from a transmitter worn by the user and generating a signal usable
by the video player for displaying a user's heart rate on the
display.
17. A method of controlling a video frame sequence display rate of
a video playback sequence in a device in response to an external
signal comprising the steps of: (a) accessing a duration time stamp
for a current video frame; (b) determining from the external signal
an adjustment value for changing the display rate (c) obtaining a
modified duration time to a next frame by adding the adjustment
value to the duration time stamp; (d) displaying the next frame
when the modified duration time has passed; (e) accessing a
duration time stamp for a current audio data packet; (f)
determining from the external signal an adjustment value for
changing the rate of delivery of the audio data packet; (g)
obtaining a modified duration time to a next audio frame by adding
the adjustment value to the duration time stamp; and (h) playing
the next audio data packet when the modified duration time has
passed.
18. The method of claim 17 further comprising the step of repeating
steps (a) through (d) for each subsequent video frame and steps (e)
through (h) for each subsequent audio data packet.
19. The method of claim 17 wherein the accessing step (a) further
comprises the steps of: i) setting a time offset to current clock
time; ii) displaying a current video frame; and iii) accessing a
frame time stamp and a duration time stamp for the current video
frame.
20. The method of claim 19 wherein the step of determining further
comprises the step of comparing the external signal to
predetermined criteria to determine the adjustment value.
21. The method of claim 20 wherein the external signal is a user
variable exercise rate signal.
22. The method of claim 20 wherein the step of displaying (d)
further comprises the steps of: i) adding the modified duration
value to the time offset to obtain a next frame time; and ii)
displaying the next frame when clock time exceeds the next frame
time.
23. The method of claim 17 wherein the accessing step (e) further
comprises the steps of: i) setting a time offset to current clock
time; ii) playing a current audio data packet; and iii) accessing a
frame time stamp and a duration time stamp for the current audio
data packet.
24. The method of claim 23 wherein the step of determining (f)
further comprises the step of comparing the external signal to
predetermined criteria to determine the adjustment value.
25. The method of claim 24 wherein the external signal is a user
variable exercise rate signal.
26. The method of claim 23 wherein the step of playing (h) further
comprises the steps of: i) adding the modified duration value to
the time offset to obtain a next audio data packet time; and ii)
playing the next audio data packet when clock time exceeds the next
frame time.
Description
CROSS-REFERENCE
[0001] This application claims the benefit of U.S. Provisional
Patent Application No. 61/110,559, filed Nov. 1, 2008, entitled
"System and Method for Optimizing One or More Audio Tracks to a
Video Stream," by John Hajime Ford, and U.S. Provisional Patent
Application No. 61/248,151 filed Oct. 2, 2009, entitled "System and
Method for Optimizing One or More Audio Tracks to a Video Stream,"
by John Hajime Ford, which applications are incorporated herein by
reference.
BACKGROUND OF THE INVENTION
[0002] The results from the 1999-2002 National Health and Nutrition
Examination Survey (NHANES), using measured heights and weights,
indicate that an estimated 65 percent of U.S. adults are either
overweight or obese. The breakdown indicates that an estimated 30%
are obese (which is defined as a Body Mass Index of 30 or higher),
while 35% are overweight. Body Mass Index ("BMI") is calculated by
dividing a person's body weight in kilograms by their height in
meters squared. In addition to these statistics for adults, an
estimated 15.5 percent of adolescents (ages 12 to 19) and 15.3
percent of children (ages 6 to 11) are estimated to be obese in the
United States.
[0003] Although it seems counterintuitive, perhaps as a result of
these alarming statistics, the United States is currently
experiencing a boom in the health club industry. The number of
health clubs in the US has climbed to 26,830 from just 15,372 in
the year 2000--a 75% increase. An estimated 41 million Americans
belong to health clubs and that number is expected to increase to
over 50 million by the year 2010. The 2004 US health club industry
comprised a 14.8 billion dollar market, more than double the 7.3
billion dollar market seen just one decade prior. These market
changes can be correlated to the American obesity crisis, an
appearance focused culture, and the increasing health awareness of
the population, and are not expected to slow or stop in the
foreseeable future.
[0004] As reported by the Centers for Disease Control and
Prevention (www.cdc.gov), physical activity can bring many health
benefits. In fact, it has been found that people who enjoy
participating in moderate intensity or vigorous intensity physical
activity on a regular basis lower their risk of developing coronary
heart disease, stroke, non-insulin dependent (type 2) diabetes
mellitus, high blood pressure, and colon cancer by 30-60% (US DHHS,
1996). Additionally, active people have lower premature death rates
than people who are the least active. Even among frail and older
people, mobility can be improved through physical activity. Butler,
R N, et al., "Physical Fitness: Benefits of Exercising for the
Older Patient" Geriatrics 53(10): 46-52 (1998). Researchers have
even found a correlation between walking and a lower incidence of
Alzheimers.
[0005] One trend has been the development of dozens of completely
new types of group fitness that emerge each year. These fitness
classes are typically designed to combine two popular forms of
fitness or activities into one class. For example, YogaSpin, a
combination of Yoga and group stationary cycling, Extreme Boot
Camp, Cheerleading School, Cycling Karaoke, and countless
variations based on the popular Cardio Kickboxing and Pilates
exercise platforms.
[0006] Another trend that has been noted in the industry is that
when people work out with another person, their effort level tends
to increase as they try to match the effort of their work-out
partner. That benefit is not realized by persons working out alone,
for example, on a treadmill.
SUMMARY OF THE INVENTION
[0007] A solution is needed to provide a mechanism for people
working out alone to attain a training benefit of working out with
a work-out partner.
[0008] An aspect of the invention provides a system that detects
the movements in an exerciser's repetitive movement pattern that,
when pre-recorded audio footfall sound files are played, most
closely support the illusion that the user is generating the sounds
with their own "virtual" footfalls, thus creating an enhanced
virtual exercise experience. The system generates a signal to
communicate the virtual footfall timing with a media playback
system, and plays back media files that support the virtual
exercise experience.
[0009] An aspect of the invention is directed to an exercise
system. The exercise system comprises: a video playing device in
communication with a central processing unit and a user input
interface connected thereto adapted and configured to receive one
or more inputs from a user and transmitting the input to the
central processing unit; a prerecorded video frame sequence encoded
onto a storage medium in communication with and playable on the
video playing device; a video monitor in communication with the
video playing device for displaying the prerecorded video frame
sequence reproduced by the video playing device; a video frame rate
controller configured to vary a rate of display of sequential
frames in the video frame sequence reproduced by the video playing
device on the monitor in response to the one or more inputs from
the user wherein each of the recorded video frame sequences
includes a frame time stamp and the video frame rate controller is
adapted to generate a variable time adjustment factor in proportion
to the one or more inputs from the user and a adjustor adapted to
apply a factor to the frame time stamp to determine a modified time
at which the next video frame sequence is to be displayed; two or
more audio rate controllers configured to vary a rate of delivery
of two or more audio files in response to a signal from user
interface rate detector wherein the rate of delivery of the two or
more audio files are delivered asynchronous to each other; and an
exercise device. In some aspects, the video frame rate controller
is adapted and configured to comprise a computer program stored on
a storage medium with the prerecorded video frame sequence. In such
configurations, the program is operable, for example, with the
central processing unit in the video playing device when the
prerecorded video sequence is played to modify the frame display
rate in response to the signal from the user interface rate
detector. Additionally, the signal from the user interface rate
detector is one or more of a biological parameter measurer, and an
exercise device detector. Suitable exercise devices include, for
example, a bicycle, a stair stepper, and an elliptical trainer.
[0010] Another aspect of the invention is directed to an apparatus
for engaging a user of an exercise device interactively in viewing
a video frame sequence of an activity on a video monitor. The
apparatus comprises a video player in communication with the video
monitor adapted and configured to play a video frame sequence on
the video monitor; a detector adapted and configured to detect a
rate of exercise by the user on the exercise apparatus and
transmitting a signal proportional to the rate of exercise; a video
controller in communication with the video player adapted and
configured to receive the signal proportional to the rate of
exercise and transmit a signal generated in response to the signal
proportional to the rate of exercise to the video player wherein
the video player adjusts the rate at which the video frame sequence
is displayed; and one or more audio controllers in communication
with an audio player adapted and configured to receive the signal
proportional to the rate of exercise and transmit one or more
signals generated in response to the signal proportional to the
rate of exercise to the audio player wherein the audio player
adjusts the rate at which the audio is played. Suitable exercise
devices include, for example, a bicycle, a stair stepper and an
elliptical trainer. Moreover, in some configurations, the signal
from the user interface rate detector is one or more of a
biological parameter measurer, and an exercise device detector.
Additionally, the interface rate detector is a heart rate signal
receiver for receiving a signal from a transmitter worn by the user
and generating a signal usable by the video player for displaying a
user's heart rate on the display.
[0011] Still other aspects of the invention are directed to a
method of controlling a video frame sequence display rate of a
prerecorded video playback sequence in a device in response to an
external signal, in which each video frame has a unique frame time
stamp and a duration time stamp indicating the time between
successive frames. The method typically comprises the steps of: a)
accessing the duration time stamp for a current video frame; b)
determining from the external signal an adjustment value for
changing the display rate; c) obtaining a modified duration time to
a next frame by adding the adjustment value to the duration time
stamp; d) displaying the next frame when the modified duration time
has passed; e) accessing the duration time stamp for a current
audio data packet; f) determining from the external signal an
adjustment value for changing the rate of delivery of the audio
data packet; g) obtaining a modified duration time to a next audio
frame by adding the adjustment value to the duration time stamp; h)
playing the next audio data packet when the modified duration time
has passed; and i) repeating steps a through d for each subsequent
video frame and steps e through h for each subsequent audio data
packet. The accessing step can further comprise the steps of: i)
setting a time offset to current clock time; ii) displaying a
current video frame; and iii) accessing a frame time stamp and a
duration time stamp for the current video frame. Moreover, the step
of determining can further comprise the step of comparing the
external signal to predetermined criteria to determine the
adjustment value. Additionally, as will be appreciated by those
skilled in the art, the external signal is a user variable exercise
rate signal. In other embodiments, the step of displaying further
comprises the steps of: i) adding the modified duration value to
the time offset to obtain a next frame time; and ii) displaying the
next frame when clock time exceeds the next frame time.
INCORPORATION BY REFERENCE
[0012] All publications, patents and patent applications mentioned
in this specification are herein incorporated by reference to the
same extent as if each individual publication, patent or patent
application was specifically and individually indicated to be
incorporated by reference.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The novel features of the invention are set forth with
particularity in the appended claims. A better understanding of the
features and advantages of the present invention will be obtained
by reference to the following detailed description that sets forth
illustrative embodiments, in which the principles of the invention
are utilized, and the accompanying drawings of which:
[0014] FIG. 1A-C are overviews of a system having a server, a CPU,
a monitor, storage media, input devices, etc. which can be used by
a user;
[0015] FIG. 2 is block diagram of the apparatus in accordance with
the invention shown FIG. 1b;
[0016] FIG. 3 is a flow diagram of the video frame rate control
program encoded on the video disk;
[0017] FIG. 4 is a flow diagram of a adjustment factor
subroutine;
[0018] FIGS. 5A-B are is flow diagrams of adjustment factor
subroutines; and
[0019] FIGS. 6A-B are logic flow diagrams of an embedded processor
in an interface unit.
DETAILED DESCRIPTION OF THE INVENTION
[0020] An aspect of the invention provides a video stream that has
one or more audio tracks associated with the video stream, one of
which correlates to the sounds of, for example footfalls and which
is variable in response to input and which can be used, for
example, as a training guide to encourage people to increase their
exertion rate as they perceive competing against another person. A
second audio track is provided that can correspond to normal
ambient-type noise that might be heard by a user in the environment
represented by the video stream, music or whatever background noise
a user selects. Changes to the video rate correlated audio can be
in response to user input or in response to a sensing of a user
parameter. In order to achieve the objectives of the invention a
variety of components are required: computer systems, components
and networks; exercise devices; and software which transforms input
from the exercise device and/or computer system into a video and
audio output which is delivered to a user.
I. Computer Systems, Components and Networks
[0021] The systems and methods disclosed herein can be achieved
using a variety of computer systems, components and networks which
are either incorporated into the exercise devices described below
or are in communication with the devices.
[0022] FIG. 1A is a block diagram showing a representative example
logic device through which reviewing, manipulating, or analyzing
information relating to the present invention can be achieved. Such
data can be in relation to exercise, one or more physiological
parameters, exertion, or any other suitable parameter desired to be
measured of a subject.
[0023] A computer system (or digital device) 100 that may be
understood as a logical apparatus that can read instructions from
media 111 and/or network port 105, which can optionally be
connected to server 109 having fixed media 112. The computer system
100 can also be connected to the Internet or an intranet. The
system includes CPU 101, disk drives 103, optional input devices,
illustrated as keyboard 115 and/or mouse 116 and optional monitor
107. Data communication can be achieved through the indicated
communication medium to a server 109 at a local or a remote
location. The communication medium can include any means of
transmitting and/or receiving data. For example, the communication
medium can be a network connection, a wireless connection or an
internet connection. It is envisioned that data relating to the
present invention can be transmitted over such networks or
connections. The computer system can be adapted to communicate with
an participant parameter monitor and/or an apparatus on which a
participant is engaged in exercise. Additionally, the components of
these logic devices can be incorporated, wholly or partially, into
an exercise device, as will be appreciated by those skilled in the
art.
[0024] A user or participant 122 can optionally also be connected
to a variety of monitoring devices, as described below. The
monitoring devices can further be adapted and configured to
interact with the system. As will be appreciated by those skilled
in the art, the computer system, or digital device, 100 can be any
suitable device. Other suitable devices capable of being adapted
for use in the system described herein include, for example, video
iPod (Apple Corp., Cupertino Calif.), or a portable DVD player or
DVD Walkman.RTM. (Sony Corporation). Such devices would be suitable
in a variety of situations.
[0025] In an embodiment, a computer-readable medium includes a
medium suitable for storing, analyzing and transmitting target
information of interest, such as that information discussed in more
detail below. The medium can include a result regarding a disease
condition or state of a subject, wherein such a result is derived
using the methods described herein.
[0026] FIG. 1B depicts another exemplary computing system 100. The
computing system 100 is capable of executing a variety of computing
applications 138, including computing applications, a computing
applet, a computing program, or other instructions for operating on
computing system 100 to perform at least one function, operation,
and/or procedure. Computing system 100 may be controlled by
computer readable instructions, which may be in the form of
software. The computer readable instructions can contain
instructions for computing system 100 for storing and accessing the
computer readable instructions themselves. Such software may be
executed within CPU 102 to cause the computing system 100 to
perform desired functions. In many known computer servers,
workstations and personal computers CPU 102 is implemented by
micro-electronic chips CPUs called microprocessors. Optionally, a
co-processor, distinct from the main CPU 102, can be provided that
performs additional functions or assists the CPU 102. The CPU 102
may be connected to co-processor through an interconnect. One
common type of coprocessor is the floating-point coprocessor, also
called a numeric or math coprocessor, which is designed to perform
numeric calculations faster and better than the general-purpose CPU
102.
[0027] In operation, the CPU 102 fetches, decodes, and executes
instructions, and transfers information to and from other resources
via the computer's main data-transfer path, system bus 140. Such a
system bus connects the components in the computing system 100 and
defines the medium for data exchange. Memory devices coupled to the
system bus 140 include random access memory (RAM) 124 and read only
memory (ROM) 126. Such memories include circuitry that allows
information to be stored and retrieved. The ROMs 126 generally
contain stored data that cannot be modified. Data stored in the RAM
124 can be read or changed by CPU 102 or other hardware devices.
Access to the RAM 124 and/or ROM 126 may be controlled by memory
controller 120. The memory controller 120 may provide an address
translation function that translates virtual addresses into
physical addresses as instructions are executed.
[0028] In addition, the computing system 100 can contain
peripherals controller 128 responsible for communicating
instructions from the CPU 102 to peripherals, such as, printer 142,
keyboard 118, mouse 116, and data storage drive 143. Display 108,
which is controlled by a display controller 134, is used to display
visual output generated by the computing system 100. Such visual
output may include text, graphics, animated graphics, and video.
The display controller 134 includes electronic components required
to generate a video signal that is sent to display 108. Further,
the computing system 100 can contain network adaptor 136 which may
be used to connect the computing system 100 to an external
communications network 132.
[0029] Computing system 100, described above, can be deployed as
part of a computer network 150 or can be wholly or partially
incorporated into a suitable exercise device, such as those
described below. In general, the above description for computing
environments applies to both server computers and client computers
deployed in a network environment. FIG. 1C illustrates an exemplary
illustrative networked computing environment 100, with a server in
communication with client computers via a communications network
150. As shown in FIG. 1B, server 110 may be interconnected via a
communications network 150 (which may be either of, or a
combination of a fixed-wire or wireless LAN, WAN, intranet,
extranet, peer-to-peer network, virtual private network, the
Internet, or other communications network) with a number of client
computing environments such as tablet personal computer 102, mobile
telephone 104, telephone 106, personal digital assistant 108, and a
personal computer 112. Additionally, an exercise input device 12 is
provided in communication with the network environment. In a
network environment in which the communications network 150 is the
Internet, for example, server 110 can be dedicated computing
environment servers operable to process and communicate data to and
from client computing environments via any of a number of known
protocols, such as, hypertext transfer protocol (HTTP), file
transfer protocol (FTP), simple object access protocol (SOAP), or
wireless application protocol (WAP). Additionally, networked
computing environment 100 can utilize various data security
protocols such as secured socket layer (SSL) or pretty good privacy
(PGP). Each client computing environment can be equipped with
operating system 138 operable to support one or more computing
applications, such as a web browser (not shown), or other graphical
user interface (not shown), or a mobile desktop environment (not
shown) to gain access to server computing environment 100. An
exercise device 12 is in communication with one or more devices
shown in the network 150. The exercise device 12 can be a treadmill
(as illustrated), or any other suitable device.
[0030] In operation, a user (not shown) may interact with a
computing application running on a client computing environment to
obtain desired data and/or computing applications as the user is
exercising using the exercise device 12 adapted and configured to
communicate with the computing application and/or interact with a
biological parameter measurer 144 adapted and configured to
communicate with the computing application via, for example, an
interface unit 142. The data and/or computing applications may be
stored on server computing environment 100 and communicated to
cooperating users through client computing environments over
exemplary communications network 150. A participating user may
request access to specific data and applications housed in whole or
in part on server computing environment 100. These data may be
communicated between client computing environments and server
computing environments for processing and storage. Server computing
environment 100 may host computing applications, processes and
applets for the generation, authentication, encryption, and
communication data and applications and may cooperate with other
server computing environments (not shown), third party service
providers (not shown), network attached storage (NAS) and storage
area networks (SAN) to realize application/data transactions.
[0031] Biological parameter measurement devices 144 include, for
example, devices capable of measuring a biologic function, such as
heart rate, blood pressure, blood sugar or glucose levels, body
temperature, tissue oxygenation, volume of O.sub.2 (VO2), pulse
oximeter measurement (Sp O.sub.2), electroencephalogram measurement
(EEG), O.sub.2 saturation, or any other measurement that can be
obtained from the participant as would be appreciated by those
skilled in the art. See, for example, U.S. Patent Publications U.S.
2005/0166373 entitled Case Structure for Sensor Structure
Attachable to and Detachable From a Shoe (Saasko et al.), U.S.
2005/0135039 entitled Electric Circuit and Transmission Method for
Telemetric Transmission (Klemetti), U.S. 2005/0130802 entitled
Arrangement, Method and Computer Program for Determining Physical
Activity Level of Human Beings (Kinnunen), U.S. 2005/0111307
entitled Electronic Wrist Device (Saaski et al.), U.S. 2005/0111306
entitled Portable Wrist-Worn Personal Electronic Device (Saaski et
al.), U.S. 2005/0017850 entitled Mechanical Measuring Device and a
Measuring Method (Nissala), U.S. 2005/0004436 entitled Method and
Device for Weight Management of Humans (Nissala), U.S. 2004/0220738
entitled Portable Personal Data Processing Device (Nissala), and
U.S. 2004/0220485 entitled Method and Device for Measuring Heart
Rate, and for Manufacturing the Device (Rytky). U.S. Pat. Nos.
6,832,109 entitled Wrist-Worn Device for Displaying and Setting
Heart Rate Parameters (Nissala); 6,754,517 entitled Apparatus for
Measuring Electrocardiograph Signal (Nissila); 6,714,812 entitled
Method of Performing Operating Settings in Heart Rate Measurement
Arrangement, and Heart Rate Measurement Arrangement (Karjalainen);
6,687,535 entitled Controlling of Fitness Exercise (Hautala et
al.); 6,605,044 entitled Caloric Exercise Monitor (Bimbaum);
6,584,344 entitled Method and Apparatus for Measuring Heart Rate
(Hannula); 6,554,773 entitled Method and Arrangement for Blood
Pressure Measurement (Nissila); 6,553,247 entitled Electrode Belt
of Heart Rate Monitor (Rytky); 6,540,686 entitled Measurement
Relating to Human Body (Heikkila et al.); 6,443,904 entitled
Determination of Stress Level of Fitness Exercise (Nissila);
6,954,661 entitled Blood Sugar Measuring Apparatus (Cho et al.);
6,746,415 entitled Method of Blood Constituent Monitoring Using
Improved Disposable Eletrocorporeal Conduit (Steuer et al.);
5,251,632 entitled Tissue Oxygen Measuring System (Delpy);
4,368,740 entitled Physiologic Analyzer (Binder); 6,912,413
entitled Pulse Oximeter (Rantala et al.); 6,879,850 entitled Pulse
Oximeter with Motion Detector (Kimball); 6,829,496 entitled Blood
Component Measurement Apparatus (Nagai et al.); 6,950,697 entitled
Electroencephalagram Acquisition Method and System (Jordan);
6,829,502 entitled Brain Response Monitoring Apparatus and Method
(Hong et al.); 6,510,340 entitled Method and Apparatus for
Encephalography (Jordan); 6,909,912 entitled Non-Invasive Perfusion
Monitor and System, Specially Configured Oximeter Probes, Methods
of Using Same, and Covers for Probes (Melker); and 6,850,789
entitled Combination SPO2/temperature Measuring Apparatus
(Scheitzer Jr., et al.). Parameter measurement devices also
include, for example, a participant monitoring their pulse manually
by placing a finger, for example, on the carotid artery or near the
wrist to determine the number of beats per minute. See also, U.S.
Pat. No. 5,170,780 to Rosenfeld for Method of Credibility
Assessment Based on Feedback-Elicited P3 Responses of Brain and
European Patent Publication EP 1510175 A1 to Kostucki for Exercise
Manager Program.
II. Exercise Devices
[0032] The systems and methods disclosed herein can be achieved
using a variety of exercise devices which are modified to provide
information to one or more computer systems, components and
networks on which software is provided which is adapted and
configured to transform one or more inputs, e.g., from an exercise
device, parameter sensor, and/or computer system into a video and
audio output which is delivered to a user.
[0033] As will be appreciated by those skilled in the art,
footfalls or movement of a user can be sensed in a variety of ways.
The mechanism for sensing footfalls or movement of a user will vary
by machine type. For the three major types of fitness machines,
treadmills, elliptical machines, and stair steppers, there may not
necessarily be an actual footfall (as in the case with elliptical
machines). Additionally, there are also various hybrid machines
which are a mix of these three main types of machines. The
elliptical machines and stair steppers do not have a real footfall
to sense, as the user's motion in reality is a fluid elliptical
pattern in the case of elliptical machines, and a repetitive
vertical (but fluid with no actual foot impact) motion on a
stepper.
[0034] Treadmills: One mechanism that can be used would be to sense
the user via, for example, the extra force exerted downward on the
treadmill, or certain treadmill components, by the footfall of the
user. A force sensor may be a piezoelectric device that converts
force directly into a voltage differential, which in turn can be
correlated to a controller to provide a mechanism for providing
sensory feedback for the user. Other force sensors that produce
electric signals as an output would be possible, including for
example capacitative force sensors, optical-electric strain gauges,
and others. Another mechanism for sensing footfalls with a
treadmill includes, for example, one or more of: providing ammeters
to detect increased load on the treadmill motor, tension sensors
(electromagnetic or even piezoelectric) to detect sudden changes in
belt tension, providing acoustic pickup to sense the sound of the
footfall (a simple condenser microphone, electret microphone,
dynamic or inductance type microphone), providing user-worn sensors
such as accelerometers to detect the motion of the users feet or
body (and which can communicate--wired or wirelessly--with the
system), providing optical sensors to detect the position of the
users' body--either with an imaging system or through
beam-interruption detection (for example, with infrared lasers and
photoelectric sensors, or with a passive imaging system that can
recognize the user's body position), providing a sensor worn by the
user that is a foot/leg position sensor such as the that used in
gaming systems such as the Nintendo Wii.RTM., providing a charge
sensor that detects contact by a user (similar to touchscreen
technology), providing a sensor that detects variances in the speed
of the treadmill belt (which will slow slightly on foot impact).
See, for example, U.S. Pat. No. 7,507,187 to Dyer et al. for
Parameter Sensing System for an Exercise Device; and U.S. Pat. No.
6,336,891 to Fedrigon et al. for Interactive Exercise Pad and
System. Treadmills are described more fully in, for example, U.S.
Pat. Nos. 7,575,537 entitled Dual Direction Exercise Treadmill for
Simulating a Dragging or Pulling Action with a User Adjustable
Constant Static Weight Resistance; 7,563,203 entitled Treadmill
with Cushion Assembly; and 7,367,926 entitled Treadmill with
Moveable Console.
[0035] Elliptical Trainers: In one aspect, the elliptical could be
configured to enable a "virtual footfall" to be sensed on the
elliptical, or the moment at which a footfall sound would create
the most effective representation of running or walking, is a
position sensor that triggers when, for example, an elliptical pad
is at its forward-most point in its elliptical movement pattern, on
its down stroke, or reaches its lowest point. Alternately, speed
sensors could determine when the forward motion of a stride reaches
zero, which would also correspond to the furthest forward part of
the stride but would accommodate inconsistent stride lengths.
Elliptical trainers are described more fully in, for example, U.S.
Pat. Nos. 6,436,007 entitled Elliptical Exercise Machine with
Adjustment; 6,361,476 entitled Variable Stride Elliptical Exercise
Apparatus; and 6,077,196 entitled Adjustable Elliptical Exercise
Apparatus.
[0036] Variants could include a differential position sensor,
sensing the point at which the pads are furthest apart, which is
the same as the point at which either pad is furthest forward.
Other ways to determine footfall on an elliptical include, for
example, one or more of providing a speed sensor that senses when
the elliptical pad's horizontal speed is zero (it is exactly at the
moment in the elliptical movement pattern in which it is moving
neither forward nor back). Speed sensor could also detect when the
foot pad's vertical acceleration is highest; providing a
weight/force/pressure sensor that detects the moment at which the
greatest weight/force/pressure is being applied to the elliptical
pad, which will correspond with the beginning of downward motion on
the front most pad and thus the virtual footfall; providing a
wireless/wired worn-sensor methods as mentioned above; providing
optical/audio/infrared/etc sensors as mentioned above in treadmill
section; or using the above systems to detect speed as it
approaches zero.
[0037] Stair Stepper Exercise Devices: One mechanism to detect
footfall or an equivalent of footfall on a stepper would be, for
example, a position sensor that detects the point at which one of
the step pads/levers is at or begins descending from its highest
point. Steppers frequently have traditional step counters built-in
and these could also be piggybacked upon. All of the methods
mentioned above for ellipticals could be adjusted to work with
steppers with a vertical fluid motion rather than an elliptical
motion, including methods that detect the pad's speed approaching
zero. The trigger point for the footfall would be the point at
which either footpad begins to descend from its highest point. This
is also the point at which the most force/pressure/weight is on the
footpad and when the footpad begins downward acceleration. Stair
stepping devices are described more fully in, for example, U.S.
Pat. Nos. 7,153,238 entitled Stairclimber Apparatus Pedal
Mechanism; and 5,399,134 entitled Stair Climbing Exercise
Apparatus.
[0038] Hybrid Exercise Devices: Combinations or variants of the
methods above could be used on hybrid machines to equal effect.
III. Apparatus Configured to Correlate Audio and Visual Signals
[0039] In this system, playback of one or more audio media files is
triggered by one or more of the sensors described above. A signal
generated by the sensor is detected by the media playing device,
which then plays an appropriate media file. In addition to audio
media files, visual signals and/or media files might be generated
to correspond to the audio playback (for example a graphical shoe
that blinks on the bottom portion of the screen each time the
footfall is sensed).
[0040] Latency, or lag between the time at which a footfall is
sensed and the time at which the audio file playback is initiated,
can greatly diminish the effectiveness/enjoyment/motivation of the
experience. Latency, even measured in amounts as small as tens of
milliseconds, can alter the illusion created by the product of a
user's actual footfalls. One method of addressing latency is to use
software/system elements to make the playback of audio files
predictive, rather than timed to correspond with specific
occurrences of actual footfalls. One variant of this method would
be to: characterize the system latency through product testing,
determine how much latency exists for each machine through testing
and then, during the exercise session, determine the cadence of the
user, such as by using the sensors sensing the time between a
plurality of footfalls and using this information to create an
average footfall timing pattern. Thereafter begin playing the
footfall audio files in accordance with the rolling average time
between steps generated by the above.
[0041] Apply the known system latency determined through step 1 of
the method described above to advance the timing of the audio file
playback by an amount equal to the system latency. Then allow the
user to use any number of possible input devices, such as buttons,
levers, touch screen elements, etc, to adjust the timing of the
predictive audio playback up or down by small increments to adjust
the latency correction as needed.
[0042] Another variant of this method would be to not predict the
moment associated with the actual footfall, but to use any number
of the sensor methods above to sense a moment that precedes the
moment of actual footfall by a fixed or predictable amount of time.
Thus the signal to play the footfall audio file would be generated
in advance, and taking known system latency into account, as in the
first variant, the playback could be timed to match the actual
moment of footfall.
[0043] Another variant of this method might be to "cover up" the
latency to some extent by playing an audio file that "fades up" or
increases in volume throughout playback. This could lead the user
to believe that the audio file is timed correctly, but just happens
to be initially inaudible because of the nature of the sound.
[0044] To create an effective virtual experience, the audio files
played back by the footfall triggers could be configured to match
the sounds of real footfalls pre-recorded on terrain similar to
that shown in a video media presentation on the machine's display.
The footfall audio files could thus vary throughout the course of a
virtual exercise experience based on the various types of terrain
being displayed. The characteristics of the audio file playback
could be altered to enhance the experience in several ways:
[0045] The volume of the footfall audio file playback, or some
portion of it, could be adjusted interactively based on movement
speed or other exercise parameters, such as to represent louder
footsteps when running faster.
[0046] The length or amount of the total footfall audio actually
played back could be interactively adjusted based on movement speed
or other exercise parameters, such as to represent quicker, lighter
steps when running faster. This could occur by either cutting off
playback (or combining cut-off with a fade-out of volume) or by
speeding up playback beyond the files original speed to achieve
shorter playback of the entire audio file.
[0047] Standard stereo audio playback functionality could be used
to pan footfall audio playback to the right or left speakers in the
audio system alternately to more accurately represent the virtual
experience by representing the right or left footstep.
[0048] This raises issues of sensing the left/right footstep,
rather than footsteps in general. In the case of steppers and
ellipticals, this is a simple task, as the footpads are independent
of one another and any of the discussed sensing methods can easily
be adjusted to sense the footfall of either the left or right pad.
In the case of the treadmill, the belt is a solid single element,
and sensing either the left or right is more challenging. There are
however methods that could do so, such as via multiple
pressure/weight/force sensors that are positioned on different
sides of the belt, differential tension/drag sensor that is capable
of detecting greater tension on one side of the belt, other
possible methods.
[0049] The specific audio file played could be varied based on an
exercise parameter, such as speed, incline, or resistance. The
sound of an actual footfall will be different depending on whether
the person generating that footfall is moving fast or slow, or over
flat or steep terrain, for example. Thus footfalls pre-recorded on
the same type of terrain might be played with different
characteristics to adjust for this as noted above, and/OR different
pre-recorded footfalls might be played that more accurately
represent the exercisers parameters over the same terrain. For
example, there could be a sound for fast movement over gravel,
another file for slow movement over gravel, and yet another for
movement over steep gravel.
[0050] Alternatively, when a series of audio footfalls are played
back to exercisers, many exercisers experience an instinctual
motivation to match the sounds of these footfalls with their actual
steps. Thus a motivational/coaching system using the audio
techniques described herein could be greatly enhanced through the
use of programmed footfall patterns.
[0051] In one implementation of this system, pre-recorded footfalls
with varying characteristics to represent varying terrain or
exercise parameters as described above, could be played
independently of the activity of the exerciser, in a pre-determined
pattern which is known to support the motivation of the
exerciser.
[0052] The timing of these footfalls could be adjusted by the user
themselves, or by the program, based on the program itself or the
program in conjunction with information input by or detected from
the user (such as desired difficulty level, height, age, heart
rate, or other parameters).
[0053] The timing of the footfalls could be manually adjustable by
the user through standard input devices such as buttons, touch
screen elements, etc.
[0054] FIG. 2 is a block diagram of an apparatus in accordance with
the invention shown in FIG. 1. The apparatus 210 includes the
exercise machine 12 connected to an interface unit 236 which is in
turn connects to the CPU 202 of a player. In one configuration a
prerecorded data storage medium 248, such as a compact laser disc
(CD), is inserted into a player 220. This CD contains media data
including a set of digitized video frames and audio data packets
and an encoded program which modifies the player control program
250. In other configurations, the pre-recorded data is contained on
a storage medium which is streamed to a local device via the
Internet or an intranet.
[0055] For purposes of illustrates, it will be appreciated that the
CPU 202 includes a control program 250, a content buffer 252 which
decodes and decompresses the data stream read from, for example,
the CD 248, a dashboard display overlay generator 254, and a set of
two or more display buffers 256 and 258. The player 220 also
optionally includes inputs for signals receivable from, for
example, a remote system CPUs 260.
[0056] Each CD can be configured to include a program which
manipulates the control program 250 to manipulate the rates at
which the audio visual are displayed.
[0057] Additional displays, such as real time, or near real time,
exercise inputs from an interface unit 232 such as speed, cadence,
total distance traveled, lap distance, calories burned, time
elapsed, time remaining, and the like can be provided. Moreover,
these parameters can be calculated and displayed on, for example, a
`dashboard` overlay on the visual monitor 218. Such parameters can
be generated in overlay block 254 and fed to the display buffers
256 and 258. The overall control of the video sequences, and the
start/stop of the exercise overlay program can be manipulated via a
suitable user input, such as keypad 236 mounted on an interface
unit 232.
[0058] The rate at which the video is displayed and one or more
audio sequences are played in accordance with the present invention
is controllable by a program shown in block diagram form in FIG. 3.
As will be appreciated by those skilled in the art, the program can
be encoded on a CD and operated on the operating control program
250 in the CPU of the player 220. Alternatively, the program can be
stored on a remote network location that is accessible locally.
[0059] The present invention is adapted and configured to utilize
one or more of each video and audio recorded sequences. Suitable
sequences include those which can be recorded using a POV (Point of
View) videotape camera mounted on an athlete's head or vehicle such
as a bicycle. Each video frame recorded is assigned and has
recorded with it a unique frame time stamp value and a duration
stamp value for accessing the next frame packet of data upon
playback. Typically, in real time recording, the duration stamp is
a constant value, for example, about 8 milliseconds. These frame
stamp and duration values function to allow a playback apparatus to
sequence and time the reproduced display. The audio may be
interlaced with the video or may be a separate track recording. The
recorded sequence of video and audio are then digitized and
recorded on a video CD along with the control program described
below and shown in the Figures herein.
[0060] The basic effect of the control program of the present
invention is to control the rate at which the video frame is played
and to correlate the playback to the exercise rate of the user on
the exercise device 12 that is in communication with the system.
Therefore as the exercise rate increases, the frame rate of display
increases. This creates the experience for a user that the scenery
changes at a rate equivalent, or near equivalent, to the rate at
which the scenery would change if the user were, for example,
cycling or running at that rate where the video display was
captured. Similarly, as the exercise rate of a user decreases, the
video displays scenery at a slower and slower rate, until the user
stops and the scene displayed stops also. Thus the user achieves a
visual sensation similar to that of actually traveling through the
scenery shown in the video.
[0061] The process flow begins operation 300, for example, when the
user begins pedaling the bicycle or walking on the treadmill.
Thereafter, the program can set, in operation 302, a cumulative
time offset (T.sub.os) to the current clock time of the CPU. This
time offset tracks the total difference in time that the program
modifies the video sequence due to the exercise rate of the
user.
[0062] The program next queries, in operation 304, whether a stop
video stream flag is set. However, if the Stop Video Flag is set,
as when the user stops pedaling to rest for a period of time or
stops running on the treadmill, a control passes to a wait
operation 306. Wait operation 306 can, for example, be a process
delay on the order of 10 milliseconds, after which the query in
operation 304 would then be repeated. If the Stop Video Stream flag
is not set, control passes to operation 308.
[0063] In operation 308, the current frame time stamp value
(T.sub.f) is retrieved and the current frame duration value
(T.sub.d) is retrieved from the decompressed stream data in content
buffer 452. Then the Adjustment value (T.sub.adj) is obtained, in
operation 110, from the program sequence shown in FIG. 4. In
operation 312, the variable T.sub.next is set equal to
T.sub.f+T.sub.d+T.sub.adj+T.sub.os. This is the clock time at which
the next frame should be decompressed into the decompression buffer
452.
[0064] In operation 314, the query is made as to whether the
current CPU clock time is equal to or greater than T.sub.next. If
the clock time is not greater to T.sub.next, then the system
continues to wait 306. If the clock time is greater than
T.sub.next, then the next video frame is decompressed and added to
the decompression buffer 316 in the content buffer block 252 of the
player CPU 202. At this point, the content of the decompression
buffer is copied to a frame buffer, which in this case is either
frame buffer A or frame buffer B 320, whichever is pointed to by a
pointer. The pointer is adapted and configured to alternate between
the two frame buffers.
[0065] As soon as the decompression buffer is copied to the pointed
to frame buffer 318, control is passed to operation 320, where
display objects such as the optional dashboard indicating the
current heart rate, pulse icon and exercise status parameters, are
overlaid into the buffer indicated by the pointer. Once the frame
buffer contents are overlaid, control shifts to operation 322 and
the contents of the frame buffer pointed to is sent to the video
display or television set.
[0066] In operation 324, the frame buffer pointer is switched to
the other buffer. In operation 326, the contents of T.sub.adj are
added to the T.sub.os register so as to keep track of total
adjustments to the sequence. Control then passes to operation 328
where the program queries whether there are any more video frames
in the sequence. If there are none, the program ends in operation
330. If there are additional frames, control passes again to the
wait operation 306 and the above steps are repeated as many times
as necessary to achieve the desired output.
[0067] Video reproduction rate thus is modifiable by adding time or
subtracting time from a prerecorded video frame duration stamp
value. In other words, the value of T.sub.adj changes and thus
modifies the effect of T.sub.d. FIG. 4 describes an example of how
T.sub.adj can be modified. For example, the sequence of operations
in FIG. 4 occurs continuously to provide a value of T.sub.adj
corresponding to a user's exercise rate. As will be appreciated
from the examples below, the manner of corresponding the video
stream to a user's exercise rate can be accomplished in a variety
of ways and from a plurality of inputs, including a biological
parameter measurement device 144 (as shown in FIG. 1B), or from the
rate at which the exercise device is operated by the user, or from
a rate at which a user is moving as detected by the device.
[0068] By way of an illustrative example, a sequence begins in
operation 432 where, for example, the wheel speed input signal from
the wheel pickup is fed through the interface unit 142 into the CPU
102 through an input such as keypad 118. The current speed,
corresponding to miles per hour or kilometers per hour, can be
obtained from the raw signal in operation 434. So, for example, the
current speed can be continually updated as long as there is a
wheel speed thread from the interface unit 142. In addition,
validity checks are performed in this operation to ensure that the
signal is, in fact, a correct wheel speed signal.
[0069] Control then shifts to operation 436 where a query is made
whether current speed equals the last speed. If so, control returns
to operation 432 for another input from wheel speed. If not, the
query is made in operation 438 whether current speed is equal to
zero. If not, control transfers to operation 440 where the query is
made whether the Stop Video Flag is set. If this flag is not set,
then control passes to operation 442 where the video adjust factor,
T.sub.adj is set to a table value corresponding to the current
speed. Values can be empirically determined to give the appearance
to the user of smooth transitions between frames and may be
different for different operating systems and different video
player machine speeds.
[0070] Control then shifts to operation 444 where Last Speed is set
equal to New Speed and control again transfer to operation 432
where another signal from the wheel sensor is awaited. If the
Current Speed is equal to zero in operation 438, control transfer
to operation 446 where the Stop Video Stream flag is set and
current clock time is saved. The Stop Video Stream flag may also be
required in operation 304 to cover the situation where the user
stops to rest after beginning a sequence. In that instance, the
current clock time when this flag is set must be saved because,
after the start, in operation 300, clock time is continuously
running. Therefore, if the user stops momentarily, requiring the
video sequence to freeze, the duration of the stopped period must
be added to the time offset in order to keep the sequence operating
properly based on current clock time.
[0071] When the user again using the device by, for example,
pedaling, a signal will be produced in operation 432. Control then
sequences through operations 434 and 436 with "no" answers. In
operations 440, the answer is "yes" to the query whether the Stop
Video Flag is set Control then transfer to operation 448 resets the
Stop Video Flag and adds to T.sub.os the elapsed time between the
current clock time at flag reset and the clock time saved when the
Stop Video Stream flag was previously set. This addition to
T.sub.os accounts for the lapse while the user was idle. Control
then transfers again to operation 442 where the video adjust factor
T.sub.adj is appropriately set as described above.
[0072] The sequence illustrated in FIG. 4 is continuous and
proceeds whenever there is a signal from the wheel speed monitor
input thread from the interface unit 142. Therefore this program
sequence is constantly updating during the exercise activity. In
contrast, the sequence illustrated in FIG. 3 operates during the
time that a video frame is available to be displayed.
[0073] Turning now to FIGS. 5a and 5b, flow diagrams are provided
illustrating two versions of an audio portion of a prerecorded
sequence. There are basically two ways in which the audio data is
encoded on the video compact disc. The audio data chunks may be in
a separate file from the video file or the audio data chunks may be
interleaved with the video frame data chunks in the same file. The
FIG. 5b flow diagram is for processing audio data chunks which are
stored in a separate audio data file on the CD. FIG. 5a provides a
flow diagram for play of audio data chunks which are interleaved
with the video frame data in the same file. Either case may be used
dependent upon variables such as buffer space and storage medium
drive speed. The audio program code is encoded on the CD as is the
video program code described above with reference to FIGS. 3 and
4.
[0074] Referring now to FIG. 5a, when the video stream starts, the
audio stream starts in operation 500. A digital pointer is set to
one of at least two audio buffers in operation 502 and control
passes to operation 504 where an audio data chunk is retrieved from
the decompression content buffer 252. The audio data chunk is then
loaded into the buffer pointed to by the audio pointer in operation
506. As soon as the audio chunk is loaded into this buffer,
contents of the buffer begin to play in operation 508. Meanwhile,
operation control is passed to operation 510 where the query is
made whether the audio stream is complete. If so, control passes to
operation 512 and the audio play stream stops. If the audio stream
is not complete, control passes to operation 514 where the pointer
is shifted to the next buffer and control is passed back to
operation 504. This process repeats until there are no more audio
data chunks signifying that the audio stream is complete. In the
case just described, the audio continues to play, even when the
video slows in response to the actions of the user on the exercise
device 12.
[0075] Referring now to the interleaved audio flow diagram in FIG.
5b, when the first video frame is decompressed and loaded into the
display buffer as described above with reference to FIGS. 3 and 4,
the audio stream control begins in operation 520. First, a pointer
is set to one of at least two audio buffers in operation 522.
Control then passes to operation 524 where an audio data chunk
associated with the current video frame is retrieved from the
decompression content buffer 252. This data chunk is then loaded
into the audio buffer pointed to in operation 526 and play of this
audio chunk immediately begins in operation 528. At the same time,
control shifts to operation 530 where the stream is queried to
determine if the audio stream is complete. If not, the query is
made, in operation 532, whether the next audio data chunk is
available.
[0076] Thus, the query in operation 532 is whether the next video
frame has been called for display. This becomes important when the
user is slowing down the video display by reducing his or her
exercise rate, e.g. pedaling slower. If the next video frame has
not been called, the next audio data chunk will not be available.
In this case, operation 532 transfers control back to operation 528
and the current audio data chunk is replayed. Where the audio is
wind noise, sounds of the road, or natural background noise in the
country, the user will not likely be able to distinguish that the
audio is being "looped back". Once the next video frame is called,
the query in operation 532 will transfer control to operation 534
where the pointer is set to another buffer. Control then transfers
back to operation 524 to get another audio data chunk and
operations 526, 528, 530, and 532 are repeated until the last of
the audio stream is processed. In this case, control is transferred
to operation 536 where the audio play is stopped.
[0077] The audio buffers provide a smooth sequencing of audio chunk
play. In the present invention, a plurality of audio can be played
with the video. For example, a first audio can be played at a
constant rate independent of the video frame, while a second audio
is played at a rate which corresponds to the rate at which the
video is played. This facilitates a first audio pitch and tempo
remaining a pleasing audio signal, as would be perceived at the
location while a second audio pitch changes to correspond to the
pace at which footfalls, for example, would occur at the rate of
exercise perceived by the user, while a third audio can occur at a
rate intended to mimic a training pace or quickening pace of a
training partner in response to a differential input by a user or
calculated based on user parameters such as age, health and
training objectives. Also, as the user slows down, the on-location
sounds would not change appreciably. However, the audio volume
could be made to increase or decrease as the user increases or
decreases speed respectively. Therefore looping these audio chunks
of wind and road noise back during slow exercise periods as
described in FIG. 5b maintains the perceptive effect of the audio
being independent of the video rate of display, yet tying the audio
content to the video frames being displayed.
[0078] Additionally, the audio of, for example, the sounds of
footfalls can further be divided into individual sound packets that
are played in bursts at a tempo or pace corresponding to the
desired pace without altering the rate or speed of audio.
Alternatively, an audio of a series of, for example, footfalls
which includes a footfall followed by an interval of silence before
a subsequent footfall (and so on) can be alternately speeded up
during the intervals of silence only.
[0079] Another variation, shown in FIGS. 1a-c, is the output
connection of a control signal to the exercise device 12. The
prerecorded video may include coded data for setting the exercise
device to predetermined resistance values. In this case, the
program would include a control thread which accesses the control
data associated with the video frame which sets the resistance.
This data would then be translated into setting values which would
be transmitted to the servomotor or stepper motor connected to the
resistance control such as resistance wheel on the exercise
stand.
[0080] The interface unit 132 can be adapted and configured to
include a battery, a programmable interrupt controller (PIC) or
processor, a biological parameter measurer 144, such as a heart
rate receiver tuned to the transmission frequency of the heart rate
transmitter, a series of shift registers, a 10 MegaHertz clock,
wheel speed input terminals which mate with the contacts on the
exercise machine 12, and a nine pin output connector which connects
the interface unit 132 to a controller keypad 118. The PIC
processor can, for example, be an 8 bit device, which processes the
incoming tics, or beats from the wheel speed sensor and the ticks
from the output of the heart rate receiver. The shift registers in
the interface unit convert the data from the PIC Processor to 32
bit words for compatibility of communication with a player. The
interface unit 32 is adapted and configured to collect data from
the heart rate and wheel speed inputs and transmits the heart rate
and wheel speed data to the player when queried by the main
program. Provision may also be included in the interface unit for
providing a signal to a stepper motor or servomotor on the exercise
device to vary the resistance provided by the resistance
roller.
[0081] A logic flow diagram of the embedded program in the PIC
processor is shown in FIG. 6a. When the user turns on the player
activates the playing of a video, the interface unit processor
starts in operation 600. A run time crystal clock (RTCC) is used to
run the processor and to measure time durations between input
pulses in the PIC processor. This RTCC runs at 10 MHz and counts
between zero and 255. The output rolls over at 255. This equates to
a rollover time of about a maximum of 2 milliseconds. Operation 602
initializes several flags and registers to zero. Specifically, a
load pulse flag, a heart beat flag, a heart tic flag, a wheel
rotation flag and a wheel tic flag are set to zero. Also, a heart
count register, a heart store register, a wheel count register, and
a wheel store register are set to zero. The heart count registers
and wheel count registers are used to accumulate time tics between
input pulses from the heart rate receiver and the wheel rotation
reed switch as will become more apparent below.
[0082] Control then transfers to operation 604 where the run time
crystal clock is set to 99. This clock then counts up to 255 and
then rolls over to zero and continues counting to 255, rolling
over, and repeating. This works out to about 2 milliseconds between
rollovers. Control then shifts to operation 606 where the query is
made whether the RTCC has rolled over. The actual query is whether
the RTCC is less than 90. If yes, the RTCC is reset to 99 in
operation 608 and the heart tic and wheel tic flags are set in
operation 610. Control then proceeds to operation 612. If the RTCC
has not yet rolled over, control passes directly to operation 612
without passing through operations 608 and 610. Operation 612
queries whether a load pulse flag has been set by the control
program. This flag will be set if the program is ready to receive
input from the interface unit 32. In this case, control transfers
to operation 614 where the interface unit 32 output registers are
loaded with the contents of the heart store and wheel store
registers. These output registers are then immediately read by the
operating system through the connection through a user input
interface, such as keypad 118.
[0083] Whether or not the load pulse flag is set by the player,
control then passes to operation 616 where the query is made
whether a heartbeat flag has been set, i.e. a beat has been
received by the heart rate receiver. If a beat has been received,
control passes to operation 618 where the heart count register
contents are transferred to the heart store register. Then, in
operation 620, the heart count register is reset to zero and the
heart beat flag is reset in order to sense another heartbeat.
Control then proceeds from operation 620 to operation 622. If no
heartbeat has been received in the heart rate receiver, and thus
the heart beat flag is not set, operation 616 transfers directly to
operation 622.
[0084] In operation 622 using a bicycle, for example, as exercise
device 12, a query is made whether the wheel rotate flag has been
set by passage of a wheel magnet past a reed switch. If not,
control passes to operation 624. If the wheel rotate flag is set,
control passes to operation 626 where the contents of the wheel
count register are moved to the wheel store register. Control then
transfers to operation 628, where the wheel count register is set
to zero and the wheel rotate flag is reset in order to sense
receipt of another wheel rotation.
[0085] Operation 624 queries whether the heart tic flag is set. If
so, control transfers to operation 632 where the heart count
register is incremented and the heart tic flag is reset. Control
then transfers to operation 630. If the heart tic flag is not set,
operation 624 transfers control directly to operation 630.
[0086] Operation 630 queries whether the wheel tic flag is set. If
so, control transfers to operation 634 where the wheel count
register is incremented and the wheel tic flag is reset. Control
then passes back to operation 606. If the wheel tic flag is not
set, operation 630 passes directly back to operation 606.
[0087] The sequence of operations described in FIG. 6b is
continuous. The net effect of the sequence is to constantly update
the heart count, wheel count, heart store, and wheel store
registers as wheel rotations and heart beats are received. This
information is passed into the output registers for transmission to
the control system whenever the control program requests input
which is about 60 times per second. It is to be understood that the
above description is exemplary of one embodiment only.
[0088] As will be appreciated by this disclosure similar logic can
be applied to other exercise devices to achieve similar
results.
[0089] Other programmed method variations and equivalents for
providing the above exercise device and user input to the playing
device will become readily apparent to those skilled in the art. In
addition, other playing devices may be utilized in place of the
system. For example, an MPEG-2 compatible player coupled to a
personal computer may be used, or another CD player using a
different operating system such as a Sony video game CD player. In
these cases, the hardware in the interface unit may have to be
modified to achieve bit compatibility with the particular player
input devices. However, the basic logic flow of the example
described above could still apply.
[0090] Many of the above methods related to type and characteristic
of the audio footfall file played back also apply in this type of
implementation.
IV. Examples
[0091] Example 1
[0092] In one example, the pace of the video stream changes with
the pace of the footsteps which corresponds to the rate at which a
user is exercising as described above. The rate of the correlated
audio can be set-up to increase as a user increases the speed at
which the user performs. Thus, the pace of the video, i.e., the
rate at which the video is streamed to the screen, correlates to
the pace of, for example, the footsteps or the rate at which a user
is cycling on a stationary bike. It has been observed that, for
example, runners may find the urge to synchronize their steps to a
strong musical beat or to a repetitive sound, such as a footfall.
Similarly, cyclists may be urged to press down on the pedal at a
rate corresponding to a repetitive sound. This urge may motivate
the exerciser to perform at the video correlated pace as the speed
of the video and the correlated audio increases to correspond to
the rate at which the viewing would change if, for example, a
runner were running at a particular pace and his or her footfalls
were at a particular pace. The rate of change of each of the audio
tracks is accomplished such that the sounds continue to sound
natural but achieves the effect of a Foley artist in making the
various audio tracks to correspond to a video.
[0093] Example 2
[0094] In another example, the pace of exercise interactively
changes with the pace of the footsteps. The pace of the video,
i.e., the rate at which the video is streamed, can change in
response to a measured biological parameter. A measured parameter
may be used to provide feedback that a user should increase his or
her speed or decrease his or her speed (e.g., if heart rates are
above or below desired levels, the correlated audio can be
increased, along with the speed of the machine to increase the
heart rate to a desired or target level).
[0095] Example 3
[0096] In another example, a user could select a difficulty level
and/or or other parameters that would result in a certain speed of
footsteps being played. This could be a combination of several
parameters, such as in the case where a person might enter their
height and a difficulty level of "5". The program might know that
"5" for a tall person is a different foot pace then "5" for a short
person.
[0097] Example 4
[0098] In another example, to the extent that there was ambient
noise (e.g., water running, leaves falling, wind blowing) or music
that audio would remain at a natural rate (e.g., the rate that it
would be heard by the user in nature at) regardless of speed of
video or the correlated audio, but the sound for footfalls or any
other video correlated sound that corresponds to a user's actual
activity would change with the pace of video scenery.
[0099] Example 5
[0100] In another example, a user can select a competition rate
wherein, for example, the user performs at a base rate to establish
his or her normal training rate. Then, the user can select a
pre-defined competitor profile, or a rate of performance
improvement, e.g., 10%. In this scenario, the correlated sound will
occur at the rate of the competitor profile or the rate of
performance improvement while the video can stream at the rate at
which the user is actually performing or at the rate of the chosen
competitor profile or performance rate improvement.
[0101] Example 6
[0102] In another example of a deployment system for the methods of
the invention, a data processing system for administering course
material is provided comprising a computer or server as shown in
FIG. 1B. The computer can, if desired, be connected to a network of
remote stations as shown in FIG. 1C. In operation, the remote
stations could then serve as the data entry points for information
to be gathered, for example at a remote class site or by a
participant registering for a class on-line. Information gathered
includes, for example, information about the materials to be
presented, the participant, the environment, etc. The remote
stations can also serve as access points through which people
transmit inquiries concerning the materials presented or exercise
programs and for responding to those inquiries. Additionally, each
station can store a record of each participant's details, including
performance information for each class, and periodically send this
information to the central computer for processing.
[0103] Additionally, it will be appreciated that the exercise can
be administered in the form of interval training As will be
appreciated by those skilled in the art, the ordering of these
steps, or any steps described herein, can occur as illustrated in
the various figures, or can occur in any other order that achieves
the objectives of the invention without departing from the scope of
the invention.
[0104] In another embodiment of the invention, the system and
method can be adapted to work in conjunction with the invention
disclosed in U.S. Pat. No. 6,142,913 for "Dynamic Real Time
Exercise Video Apparatus and Method" (Ewert). Thus the system can
be adapted to incorporate varying video playback rate adjusts the
frame rate of motion video content being replayed on a display
device based on the intensity at which a participant exerts
themselves. This system supports the illusion that the participant
is actually traveling through the terrain being displayed. A result
of the use of this system is that a participant who exercises at a
higher intensity will view more footage than will a participant who
exercises at a lower intensity, as the frame rate speed will be
higher and more frames of the footage will be viewed.
[0105] An additional benefit of the systems and methods disclosed
herein is the modularity of the systems and methods.
[0106] While preferred embodiments of the present invention are
shown and described herein, it will be obvious to those skilled in
the art that such embodiments are provided by way of example only.
Numerous variations, changes, and substitutions will now occur to
those skilled in the art without departing from the invention. It
should be understood that various alternatives to the embodiments
of the invention described herein may be employed in practicing the
invention. It is intended that the claims following the description
define the scope of the invention and that methods and structures
within the scope of these claims and equivalents are covered
thereby.
* * * * *