U.S. patent application number 16/526638 was filed with the patent office on 2020-04-09 for system and methods for providing performance feedback.
The applicant listed for this patent is ISOLYNX, LLC. Invention is credited to Douglas J. DeAngelis, Edward G. Evansen.
Application Number | 20200108292 16/526638 |
Document ID | / |
Family ID | 42728851 |
Filed Date | 2020-04-09 |
![](/patent/app/20200108292/US20200108292A2-20200409-D00000.png)
![](/patent/app/20200108292/US20200108292A2-20200409-D00001.png)
![](/patent/app/20200108292/US20200108292A2-20200409-D00002.png)
![](/patent/app/20200108292/US20200108292A2-20200409-D00003.png)
![](/patent/app/20200108292/US20200108292A2-20200409-D00004.png)
United States Patent
Application |
20200108292 |
Kind Code |
A2 |
DeAngelis; Douglas J. ; et
al. |
April 9, 2020 |
SYSTEM AND METHODS FOR PROVIDING PERFORMANCE FEEDBACK
Abstract
Systems and methods provide feedback to at least one participant
in a field of play. A performance analysis device determines
performance information of each participant in the field of play,
where the performance information is based upon at least one of
determined location, speed, path, acceleration and biometrics of
said each participant. At least one output device provides
real-time feedback to the at least one participant based upon the
performance information. The real-time feedback comprises
performance information of the at least one participant and/or
performance information of one or more other participants in the
field of play.
Inventors: |
DeAngelis; Douglas J.;
(Ipswich, MA) ; Evansen; Edward G.; (West Newbury,
MA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
ISOLYNX, LLC |
HAVERHILL |
MA |
US |
|
|
Prior
Publication: |
|
Document Identifier |
Publication Date |
|
US 20190351288 A1 |
November 21, 2019 |
|
|
Family ID: |
42728851 |
Appl. No.: |
16/526638 |
Filed: |
July 30, 2019 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15386265 |
Dec 21, 2016 |
|
|
|
16526638 |
|
|
|
|
13231802 |
Feb 14, 2017 |
9566471 |
|
|
15386265 |
|
|
|
|
PCT/US2010/027349 |
Mar 15, 2010 |
|
|
|
13231802 |
|
|
|
|
61160141 |
Mar 13, 2009 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A63B 24/0062 20130101;
A63B 2230/50 20130101; A63B 24/0003 20130101; A63B 2243/007
20130101; A63B 2220/20 20130101; A63B 2220/17 20130101; A63B
2230/202 20130101; A63B 2024/0012 20130101; A63B 2220/30 20130101;
G06F 19/3481 20130101; A63B 2230/00 20130101; A63B 2230/42
20130101; A63B 2024/0025 20130101; A63B 2220/806 20130101; A63B
2071/0625 20130101; A42B 3/30 20130101; A63B 2024/0065 20130101;
A63B 2230/207 20130101; A63B 2225/50 20130101; A63B 2024/0056
20130101; A63B 2225/15 20130101; A63B 2220/12 20130101; A63B
71/0619 20130101; A63B 2230/06 20130101; A63B 2220/40 20130101;
G09B 19/0038 20130101; A63B 24/0021 20130101; G16H 20/30 20180101;
A63B 2243/0025 20130101 |
International
Class: |
A63B 24/00 20060101
A63B024/00; G09B 19/00 20060101 G09B019/00; A63B 71/06 20060101
A63B071/06; A42B 3/30 20060101 A42B003/30 |
Claims
1. A system for providing real-time audio feedback to an athlete,
comprising: one or more audio speakers configured to be co-located
with the athlete and play the real-time audio feedback to the
athlete; and a performance analysis device configured to:
continuously determine a location of the athlete from a location
device worn by the athlete, determine an audio signal, based on the
continuously-determined location of the athlete, that indicates how
well, or not, the athlete conforms to one or both of a path and a
speed, and direct the one or more audio speakers to play the audio
signal as the real-time audio feedback.
2. The audio feedback system of claim 1, the one or more audio
speakers including left and right speakers for left and right ears,
respectively, of the athlete.
3. The audio feedback system of claim 2, the left and right
speakers being mounted to a helmet worn by the athlete.
4. The audio feedback system of claim 2, the audio signal
indicating the real-time audio feedback to the athlete via a
beeping frequency of a beeping sound, a tone, a volume of one or
both of the left and right speakers, and a verbal message.
5. The audio feedback system of claim 2, the audio signal being
configured to indicate how well the athlete conforms to the speed
via a beeping frequency of a beeping sound.
6. The audio feedback system of claim 2, the audio signal being
configured to indicate how well the athlete conforms to the path
via a volume of one or both of the left and right speakers.
7. A method for providing real-time audio feedback to an athlete,
comprising: continuously determining a location of the athlete from
a location device worn by the athlete; determining an audio signal,
based on the continuously-determined location of the athlete, that
indicates how well, or not, the athlete conforms to one or both of
a path and a speed; and directing one or more audio speakers,
co-located with the athlete, to play the audio signal as the
real-time audio feedback.
8. The method of claim 7, the one or more speakers including left
and right speakers for left and right ears, respectively, of the
athlete.
9. The method of claim 8, the left and right speakers being mounted
to a helmet worn by the athlete.
10. The method of claim 8, wherein said determining the audio
signal includes determining the audio signal such that the audio
signal, when played on the left and right speakers, indicates the
real-time audio feedback to the athlete via a beeping frequency of
a beeping sound, a tone, a volume of one or both of the left and
right speakers, and a verbal message.
11. The method of claim 8, wherein said determining the audio
signal includes determining the audio signal such that the audio
signal, when played on the left and right speakers, indicates to
the athlete how well the athlete conforms to the speed via a
beeping frequency of a beeping sound.
12. The method of claim 8, wherein said determining the audio
signal includes determining the audio signal such that the audio
signal, when played on the left and right speakers, indicates to
the athlete how well the athlete conforms to the path via a volume
of one or both of the left and right speakers.
13. A system for providing real-time audio feedback to an athlete
in a field of play, comprising: one or more audio speakers
configured to be co-located with the athlete and play the real-time
audio feedback to the athlete; and a performance analysis device
configured to: continuously determine a location of the athlete
from a location device worn by the athlete, continuously determine,
from a location device worn by each of one or more other athletes
in the field of play, a location of said each of the one or more
other athletes, determine an audio signal based on the
continuously-determined location of the athlete and the
continuously-determined location of said each of the one or more
other athletes, and direct the one or more audio speakers to play
the audio signal as the real-time audio feedback.
14. The audio feedback system of claim 13, the one or more audio
speakers including left and right speakers for left and right ears,
respectively, of the athlete.
15. The audio feedback system of claim 14, the left and right
speakers being mounted to a helmet worn by the athlete.
16. The system of claim 14, the performance analysis device being
further configured to: determine a direction from which situational
pressure is coming toward the athlete, the situational pressure
being based on proximity of the athlete to each of the one or more
other athletes; and indicate the direction to the athlete via the
real-time audio feedback by controlling a volume of one or both of
the left and right speakers.
17. The audio feedback system of claim 16, the performance analysis
device being further configured to determine the proximity of the
athlete to said each of the one or more other athletes based on the
continuously-determined location of the athlete and the
continuously-determined location of said each of the one or more
other athletes.
18. The audio feedback system of claim 17, wherein the athlete is a
quarterback in an American football game, and the one or more other
athletes are defenders of the quarterback.
Description
RELATED APPLICATIONS
[0001] This application is a continuation of U.S. patent
application Ser. No. 15/386,265, filed Dec. 21, 2016, which is a
continuation of U.S. patent application Ser. No. 13/231,802, filed
Sep. 13, 2011, now U.S. Pat. No. 9,566,471, which is a continuation
of International Patent Application No. PCT/US2010/027349, filed
Mar. 15, 2010, which claims priority to U.S. Patent Application No.
61/160,141, filed Mar. 13, 2009. Each of these applications is
incorporated herein by reference in its entirety.
BACKGROUND
[0002] Traditionally, video and still images of a live event (i.e.,
video content and still image content) are created by a team of
professionals. In the case of video content, highly trained camera
persons operate one or more cameras and highly trained production
staff operate production equipment (e.g., within a production van
at a sporting event) to select camera shots and combine graphics
into a production feed. In the case of still image content, highly
skilled camera persons operate still cameras to capture still
images of an event and submit these still images to one or more
editors who select shots for use in magazines, for example.
[0003] The cost of producing video and still image content defines
the market size required to cover this cost. Thus, only events
having a sufficient market justify the cost of producing video and
still image content. Although technology has reduced the cost of
production, the cost of skilled human operators remains high.
[0004] Images from a camera may be used to visually track an object
(e.g., a golf ball) within the camera's field of view. The camera
may be motorized to allow it to move so as to maintain the moving
object within its field of view. However, such systems fail when
the camera `loses sight` of the object; for example, the camera may
lose sight of the object if the object becomes visually obscured by
another object.
[0005] For certain sporting events, cameras may be motorized to
facilitate tracking of competitors and are operated by remote
camera operator. These cameras still require the skill of a
person.
[0006] Many systems have been developed to track objects by
attaching a sensor to the object and then using the sensor to
determine the location of the object. Such object tracking provides
data (e.g., speed) to computer systems but is not known to
facilitate real image production.
SUMMARY OF THE EMBODIMENTS
[0007] In an embodiment, a system provides feedback to one or more
participants in a field of play. The system includes a performance
analysis device for determining performance information of each
participant in the field of play. The performance information is
based upon at least one of determined location, speed, path,
acceleration and biometrics of the participant. The system also
includes at least one output device for providing real-time
feedback to the participants based upon the performance
information.
[0008] In another embodiment, a system provides feedback to a
leader of one or more participants in a field of play. The system
includes a performance analysis device for determining performance
information of each of the participants, where the performance
information is based upon at least one of determined location,
speed, path, acceleration and biometrics of said each participant.
The system also includes an output device for providing real-time
feedback to the leader based upon the performance information.
[0009] In another embodiment, a method provides feedback to at
least one participant in a field of play. Successive locations of
the participant within an operational field are determined.
Performance of the participant is determined based upon the
successive locations, real-time feedback is provided to the
participant based upon the determined performance.
[0010] In another embodiment, a system provides feedback to at
least one participant in a field of play. The system includes a
performance analysis device for determining performance information
of each participant in the field of play, where the performance
information is based upon at least one of determined location,
speed, path, acceleration and biometrics of said each participant.
The system also includes at least one output device for providing
real-time feedback to the at least one participant based upon the
performance information.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG. 1 shows one exemplary system for capturing performance
data of tracked objects moving within an operational field, in an
embodiment.
[0012] FIG. 2 illustrates use of the system of FIG. 1 for measuring
skill and performance of training athletes performing a drill.
[0013] FIG. 3 shows one exemplary Perfect Execution template.
[0014] FIG. 4 shows one exemplary plot of data captured during a
play.
[0015] FIG. 5 shows one exemplary overlay of the plot of FIG. 4 and
the template of FIG. 3.
[0016] FIG. 6 shows one exemplary system for providing performance
feedback, in an embodiment.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0017] FIG. 1 shows one exemplary use of a system 100 for capturing
performance data of tracked objects 106 moving within an
operational field 108. Objects 106 for example represent runners,
football players, speed skaters, race cars or horses and/or
jockeys, and field 108 represents a running track, a football
field, an ice rink, a race track or a horse track,
respectively.
[0018] System 100 is shown with an object tracking device 102, a
recording device 120, a display device 130, an optional camera
control device 104 and five optional motorized cameras 110. If
included, cameras 110 may be situated within or around operational
field 108. Four exemplary objects 106 are shown within field 108,
each object having a location unit 112.
[0019] Object tracking device 102 utilizes location units 112 to
determine location information (e.g., coordinate data) for objects
106 within operational field 108. Object tracking device 102 sends
this location information to recording device 120 and optionally to
camera control device 104. Camera control device 104 may include a
model of operational field 108, with coordinates of each camera
110. Camera control device 104 receives coordinate data of objects
106 from object tracking device 102, determines a possible field of
view from each camera 110 to each object 106, and assigns cameras
110 to the objects based upon optimum field of view selection. For
example, for each camera 110 and for each object 106, camera
control device 106 determines a possible field of view from the
camera to the object. Then, by selecting an optimum field of view
for each object (e.g., based upon the distance from the camera to
the object, the objects position within field 108 and whether all
objects are assigned to a camera), control device 104 determines
which camera 110 to assign to each object 106. Where the number of
objects is less that the number of cameras, camera control device
104 may assign more than one camera 110 to an object 106. Where the
number of objects is more than the number of cameras, camera
control device 104 may assign one or more select cameras 110 to a
more important object 106 (e.g., the leader in a race). Each object
106 is for example prioritized such that cameras assignment is also
prioritized for that object--in a race, the leader is assigned a
higher priority to ensure best camera assignment.
[0020] As objects 106 move within field 108, object tracking device
102 may provide, for each object 106, velocity and position
information 126 to recording device 120 and to camera control
device 104. Camera control device 104 uses this velocity and
position information for optimal camera assignment to each object
106. Thus, camera control device 104 may be made aware of movement
characteristics (e.g., direction of movement) of objects 106, and
accordingly assigns or re-assigns cameras based upon camera fields
of view that include the front of object 106. Further, camera
control device 104 for example assumes that the front of object 106
faces forward as it moves, or it may instead be programmed to
identify the front of object 106 as the aspect facing the general
direction of movement of an event. Camera control device 104
accordingly assigns and controls cameras 110 to capture frontal and
side image data of object 106, in preference to rear images.
[0021] Recording device 120 records velocity and positional
information 126 received from object tracking device 102 and may
record and/or convert image data 119 from each camera 110. Camera
control device 104 may also provide annotation data 127 to
recording device 120 such that recording device 120 may record
relevant camera information (e.g., zoom, direction and azimuth)
with velocity and positional information 126.
[0022] In one embodiment, recording device 120 simultaneously
records image data 119 from each camera 110, velocity and position
information 126 and annotation data 127. Image data 119 is a signal
or signals representing data bits captured by each camera 110.
Recording device 120 includes processing software for converting
the received signal into a data stream and interpreting the data
stream as a series of images, which are then recorded as video, for
example. System 100 is thus suitable for use in autonomous still or
moving picture production.
[0023] Camera control device 104 sends annotation data 127 to
recording device 120 for recording with image data 119. Annotation
data 127 includes identification of tracked objects of image data
119. For example, if camera 110(1) is selected to maintain object
106(1) within its field of view, as the image data from camera
110(1) is recorded by recording device 120, camera control device
104 may include identification of object 106(1) within annotation
data 127 that is recorded with the image data.
[0024] Recording device 120 may generate a live feed 105
(optionally including annotation data and performance
characteristics of displayed objects) and allow interaction with a
user through display device 130.
[0025] Recording device 120 may display images from selected
cameras 110 on display device 130 together with measured
performance statistics.
[0026] Recording device 120 may include additional functionality
for determining performance statistics and movements of objects
106. In one example, recording device 120 shows a plan view of at
least part of field 108 on display 130 and plots movement of
objects 106 over select periods. Recording device 120 may also
determine performance data for objects 106 and include this
performance data, e.g., as a video overlay, when recording image
streams of object 106 and/or when displaying select image streams
on device 130 and/or live feed 105. This video overlay is for
example formatted as tabulated figures that include the performance
statistics and/or a graphical representation of the performance of
object 106.
[0027] In another embodiment, recording device 120 replays recorded
image data 119, velocity and position information 126 and
annotation data 127 as feed 105 featuring one or more objects 106.
Where system 100 is utilized as a training device by a sports team,
recording device 120 may be operated to generate output for image
feed 105 and/or display device 130 by overlaying velocity and
position information 126 and annotation data 127 onto image data
119 for one or more selected athletes. Thus, recording device 120
automatically displays recorded image streams and performance
information of the selected athlete. The video overlay and
performance information included therein is variable according to
sport and/or preference or selection of a user of recording device
120. In one example, live feed 105 may drive a large stadium
display during practice to highlight training performance.
[0028] In one embodiment, recording device 120 delivers instant
replay images or video streams that include overlaid performance
information determined from velocity and position information 126
and/or annotation data 127 for the object(s) 106 associated with
the selected image data 119.
[0029] In another embodiment, recording device 120 generates live
image feed 105 by combining a video overlay of performance
information selected from velocity and position information 126
and/or annotation data 127 and image data 119. In particular,
recording device 120 of this embodiment matches performance
information from annotation data 127 to image data 119 for each
object 106 to which a camera 110 is assigned.
[0030] Recording device 120 may also connect to (or include) a
wireless transceiver 140 that communicates with a hand held display
and control device 142 to allow a coach to utilize system 100 while
on a practice field. Further, transceiver 140 may communicate
wirelessly with one or more head-up-display devices, such as
glasses 144, worn by athletes to provide real-time display of
performance related information and other training aids. Glasses
144 may utilize, for example, Light-guide Optical Element (LOE)
technology, such as that employed in Video Eyeglasses manufactured
by Lumus Ltd.
[0031] Skill Evaluation and Improvement
[0032] Real time position data may be used to measure, evaluate and
compare athlete skill parameters such as velocity, acceleration and
change of direction agility on predefined drill paths. In addition,
using velocity and position information 126 to control cameras 110
to track objects 106 within field 108, skill and performance of
athletes represented by objects 106 may be evaluated when
performing these predefined drills, such that mechanics of the
athlete may be evaluated and corrections made to the athlete's
techniques to improve future performance.
[0033] Data Only: Performance Evaluation
[0034] FIG. 2 shows use of system 100, FIG. 1, for measuring skill
and performance of training athletes performing an arrowhead drill
200 within field 108. Arrowhead drill 200 involves two athletes
(represented as objects 106(1) and 106(2)), five cones
202(1)-202(5) placed five yards apart as depicted in FIG. 2, and
two discrete paths 212 and 214.
[0035] The first athlete (object 206(1)) starts out from a
start/finish line 216 on path 212 and the second athlete (object
206(2)) starts on path 214 once the first athlete has reached cone
202(2). By continually or periodically monitoring velocity and
position of objects 206 while performing arrowhead drill 200,
system 100 may determine agility of each athlete. For example,
determining speed, acceleration and change of direction for each
athlete may be measured as a continuous function over one or more
predefined drill paths to provide significantly more detailed
information on the athletes' agility and skill level than by
evaluating speed and acceleration as average values between
discrete points using fixed measurement devices (such as stop
watches, touch pads and photocells), as previously done. The
collection of velocity and positional information (i.e., velocity
and positional information 126) eliminates the need for mechanical
measurement devices along the drill path and allows velocity and
acceleration data to be determined at any point along the
path(s).
[0036] Performance drills are intended to predict the eventual
success level of an athlete in real world situations. In the most
primitive form, a stop watch collects time from start of the drill
to completion to provide a gross measure of overall performance and
may be useful as a first pass selection of athletes. However, this
simple timing measurement gives no indication of how the athlete
performed each element of the drill. Discrete measurement points
along the path may be added to provide additional information, but
since they are limited to discrete points along the path, this
additional information is still limited to average performance
measurements of the drill elements.
[0037] The collection of continuous data by system 100 provides
complete performance measurements, such as velocity and
acceleration, at all points along each element of the drill. In
football drills, such as arrowhead drill 200 of FIG. 2, these
parameters are particularly critical at turning (cutting) points.
Continuous data collection allows fine distinction between the
performances of multiple athletes. This distinction may be critical
in determining how well an athlete should perform in a real world
situation. For instance, a first and second athlete may have the
same overall time when performing a drill; however, the first
athlete may be achieving this time by an ability to cut right and
accelerate exceptionally well, while being slow in cutting and
accelerating left. In contrast, the second athlete may cut and
accelerate equally well to both the right and the left. In real
world situations of a football game, the second athlete with
balanced ability should consistently outperform the first athlete
who is unbalanced, since defenders facing the second athlete will
not be able to predict which way the second athlete will choose to
cut as the second athlete has equal ability in both directions,
whereas the first athlete would typically favor his fastest cut
direction.
[0038] Thus, by monitoring velocity and position information
continually (or periodically) for each athlete, weaknesses and
strengths of each athlete may be easily measured.
[0039] In the simplest form, continuous real time position data
collection may be employed on a 40 yard dash. Although existing
systems provide total time and split times at discrete points along
the path, from which the average velocity and acceleration between
the split points can be calculated, these average values are
limited by the positions of the discrete points. Since system 100
may collect position information in real time, velocity and
acceleration values may be determined as a continuous function,
allowing plots of instantaneous velocity and acceleration at any
point along the 40 yard path.
[0040] The 40 yard dash and arrowhead drill 200 are used as
examples and should not be used to limit the scope hereof. The
advantages of real time data collection over existing discrete
measurement systems hold true for all practice drills in all
sports.
[0041] Data and Camera Integration: Performance Evaluation and
Improvement
[0042] Camera control device 104, FIG. 1, may use velocity and
position information 126 from object tracking device 102 to
automatically aim one or more cameras on one or more athletes
performing a drill path such that recording device 120 may record
video of the athlete's motion and performance data. For example,
recording device 130 may generate one or more displays (e.g., on
display 130, video feed 105, handheld device 142 and/or
heads-up-display device/glasses 144) showing direction vectors
and/or acceleration plots overlaid onto video of the athlete
performing the drill to allow a coach to evaluate the athlete's
mechanics and performance at specific points on the drill path. The
coach may replay this video and data overlay to make determinations
between athletes and in coaching methods for athletes to improve
their individual performance.
[0043] Practice Field: Evaluation and Productivity
[0044] Data Only and Camera Integration: Keep Practice
Productive
[0045] On a practice field it is often difficult, if not
impossible, to know where all players are at all times. This is
especially true when players have assignments on multiple squads
such as offense, defense and any one of the special teams. If the
coach cannot find an athlete, that athlete may miss important
coaching advice and training. Further, one athlete missing from a
training session may prevent training of ten other players on a
squad until the missing athlete is found. Much valuable practice
time is often lost due to players not being in the right
location.
[0046] Since system 100 collects real time position information for
all objects 106 continually, a coach may determine the location of
any athlete by identifying the player (e.g., entering the player's
number) to system 100 such that recording device 120 may display
the current location of the player to the coach. In one example of
operation, a coach enters a player's number into handheld device
142 and recording device 120 displays a plan of field 108 showing
the location of the selected player. Optionally, system 100 may be
configured and/or operated to display live video imagery of the
player's activity to the coach on handheld device 142 by activating
one or more cameras 110 to image the current location of the
selected player. Thus, through the use of system 100, the coach
learns the location of the selected player and can see the activity
of that player, thereby allowing an instantaneous decision as to
whether to call on him, wait for him or select a substitute.
[0047] Data Only: Assignment Verification
[0048] Collected, real time position data of an athlete(s)
continuous position during a play can be used to determine whether
the athlete correctly performed his specific assignment. As an
example, consider an offense practicing plays where the receiver
routes are of particular interest.
[0049] Assignment Verification Operation:
[0050] "Perfect Execution" Capture:
[0051] Eleven players may be outfitted with positioning tags so
that their exact positions can be recorded as a play is executed.
After diagramming the play, the players may walk through the play
exactly as it is supposed to be executed. The data from all eleven
players is plotted and saved in a play library as a template, such
as Perfect Execution template 300, FIG. 3, for later comparison
against subsequent executions. For simplicity, only the receiver
paths are shown in template 300; however, in most cases the paths
of all eleven players would be captured, plotted and stored. This
can be done for all plays in a playbook and the "Perfect Execution"
need only be captured once for each play.
[0052] "Practice Execution" Capture:
[0053] The players are told to run a specific play. This play may
be executed at walk through speed, half speed or full speed. The
real time continuous position of each player is captured as the
play unfolds Immediately following the play, the data for each
player is plotted. An example of such a plot is shown in FIG. 4.
For ease of illustration, Practice Execution plot 400 shows only
the receiver paths; however, in most cases the paths of all eleven
players would be captured and plotted.
[0054] Assignment Verification:
[0055] The Practice Execution plot 400 is then overlaid on the
Perfect Execution template 300 for immediate evaluation by coaches
on the field. An example of such an overlay is shown by overlay
500, FIG. 5. The solid lines represent perfect execution template
300 and the dashed lines represent practice execution plot 400.
From this example, it is immediately obvious that the tight end and
wide receivers ran their assigned routes reasonably close to design
while the tailback clearly deviated from his assigned route.
[0056] Overlay Automation:
[0057] For practical implementation, Perfect Execution template 300
and Practice Execution plot 400 overlay automatically. Furthermore,
the overlay may be independent from where the data for each was
collected. For example, Perfect Execution template 300 may be
collected in a gym while Practice Execution plot 400 may be
collected on a practice field days or even years later. In
addition, successive plays may originate from different points on
the practice field, and they could be moving in either
direction.
[0058] Automatic overlay is accomplished by selecting two stable
reference points whose relative starting positions to each other
are known. For instance, in both Perfect Execution template 300 and
Practice Execution plot 400, the position of the right tackle is
known and stable relative to the position of the left tackle.
Therefore, this relationship can be used in establishing the
overlay starting point.
[0059] Data and Camera Integration: Mechanics and Performance
Evaluation:
[0060] Real time position data is used to aim a camera(s)
automatically at specific athletes of interest during a practice
play execution to record video of the execution. The number of
athletes videoed is limited only by the number of cameras. The
video can be viewed in real time, while the play is being executed,
or saved and reviewed later.
[0061] Real-Time Video and Performance Data:
[0062] The video stream from the camera is transmitted to a
handheld or tablet computer being held by a coach on the field. The
coach selects the player of interest on the screen before the play
is executed. While the play is being executed, the camera isolates
on this player and the video is streamed to the handheld for the
coach to view in real time. As the player moves on the field a
yellow line indicating the path traveled is laid down on the image
of the field. A data window on the screen also indicates the
current velocity and acceleration of the player.
[0063] Replay of Video and Performance Data:
[0064] In addition to streaming the video to the handheld, the
video is also streamed to disk. The stored video can be used on the
field by the coach for instant replay, which can be used in
coaching the player. The replay video can be stopped at any point
to freeze the action and highlight specific situations. The coach
can also access the velocity and acceleration of the player at any
point by touching the screen on the yellow path indication
line.
[0065] Post Analysis of Video and Performance Data:
[0066] All continuous position, velocity, acceleration and path
data, as well as video, from a practice session can be stored for
post analysis. Successive plays can be compared to isolate what
performance elements contributed to the success or failure of a
particular play execution. For instance, in the case of a wide
receiver running a route and being covered by a defender.
[0067] Post Analysis Use of Video:
[0068] Video can be used to analyze the receiver's body motion
while running his route and distinguish between moves that are
"fakes" (i.e., moves which are successful in confusing a defender),
resulting in a successful play, from moves which did not confuse
the defender, resulting in an unsuccessful play.
[0069] Post Analysis Use of Data Parameters:
[0070] Data such as velocity and acceleration at any point along
the path traveled can be compared to identify the precise point at
which burst of acceleration is particularly successful in getting
distance between the receiver and the defender.
[0071] While the examples above involve an offensive play,
identical situations exist for defensive plays as well as all
special teams plays.
[0072] Integration of Real Time Data with Audio Feedback: Training
and Task Execution
[0073] The collection of continuous real-time position data can be
used in conjunction with an "in helmet" audio system to assist
players in learning and perfecting new tasks. The audio system is
mounted in the player's helmet in stereo fashion with a miniature
speaker at the left and right ears. The speakers operate
independently allowing specific information to be communicated to
the player, in real time, through variations in tone and volume.
The following are examples of how this might be used to train
various player positions but the same principles could be applied
to all positions in learning and perfecting new tasks.
[0074] Audio Feedback: Quarterback Example
[0075] To be an effective passer, a quarterback must know how much
time he has in "the pocket" before he must get rid of the ball or
run. The most basic parameter is time, which is measured in seconds
(typically 3.5 or less). Failure to wait as long as possible could
mean making a bad decision or throwing before a receiver has had
time to get open. However, waiting 1 ms too long could result in a
sack or career-ending injury.
[0076] Continuous real time data in conjunction with audio feedback
can be used in helping a quarterback to learn the optimal time in
the pocket. Real time position data is used to determine when the
quarterback is located in the pocket. As soon as he is in position
a timer start and a beeping sound is sent to the speakers in his
helmet. This beeping increases in frequency as time expires. The
frequency continues to increase until the maximum time has elapsed
and the beeping becomes a steady tone.
[0077] In the real world, the maximum amount of time a quarterback
has in the pocket may be cut short by defenders reaching him more
quickly than anticipated. To be effective in this situation a
quarterback must learn to "feel" the pressure of defenders.
Defensive pressure usually comes from the left or right side and in
this case the quarterback must sense the pressure through
peripheral vision. Since he is focused down field looking for
potential receivers this can be a difficult skill to master. By
tracking the real time position of the defenders their proximity to
the quarterback is easily determined. To aid the quarterback in
learning to "feel" this pressure the beeping frequency can be
increased more rapidly to reflect the decreasing available time in
the pocket and the volume of the right or left speaker can be
increased to indicate the specific direction the pressure is coming
from.
[0078] Audio Feedback: Receiver Example
[0079] For a receiver to be effective, he must be where the
quarterback expects him to be. The most basic element of this is
running the proper route. Continuous real time position data in
conjunction with audio feedback can be used to help the receiver
learn his routes quickly and accurately so that he runs them the
same every time.
[0080] When the receiver first begins his route, there is tone of
equal volume in both speakers. As he moves along the path, the
system senses his position and slowly increases the volume in the
ear on the side to which he will eventually cut. Upon arriving at
the cut point, the volume in the speaker on the side of the cut
hits a peak and the speaker on the opposing side shuts down
completely.
[0081] The second element of being in the right place, and eluding
defenders, involves running the various legs of the route at
different speeds. In this instance, the tone in the example above
would be replaced by a beeping sound. The frequency of the beeping
indicates the speed at which the receiver should be running. In a
simple case, a low frequency would represent a jogging speed, a
medium frequency would represent a medium speed run and a high
frequency beeping would present full speed. The system controls the
beeping frequency by sensing what leg of a route the receiver was
on and delivering the preprogrammed speed message. In an alternate
embodiment, the beeping may have a frequency based upon a desired
cadence for the athlete, thereby indicating the desired running
speed.
[0082] Both messages (speed and direction) can be delivered to the
receiver by employing the volume and beeping frequency described
above simultaneously. The beeping frequency indicates the speed to
run and the volume of the beeping in the right or left ear
indicates the direction to turn.
[0083] Integration of Real Time Data with Visual "Heads-up"
Feedback: Training and Task Execution
[0084] The collection of continuous real-time position data can be
used in conjunction with "heads up" technology to assist players in
learning and perfecting new tasks. The heads-up technology consists
of a pair of transparent glasses 144 (FIG. 1), worn by the player,
which have the capability to accept wireless data and project this
data in the form of images, which appear to the player to be at
some distance in front of the glasses.
[0085] PATENT Attorney Docket No. 614755
[0086] Visual Heads-up Feedback: Quarterback Example
[0087] Time in the Pocket:
[0088] To be an effective passer, a quarterback must know how much
time he has in `the pocket` before he must get rid of the ball or
run. The most basic parameter is time, which is measured in seconds
(typically 3.5 or less). Failure to wait as long as possible could
mean making a bad decision or throwing before a receiver has had
time to get open. However, waiting 1 ms too long could result in a
sack or career ending injury.
[0089] Continuous real time data in conjunction with heads-up
technology can be used in helping a quarterback to learn the
optimal time in the pocket. Real time position data is used to
determine when the quarterback is located in the pocket. As soon as
he is in position, a timer starts and bar graphs located in both
the far right and left corners of his vision begin growing. When
there is 0.5 seconds remaining the graphs begin blinking and when
time has expired, they turn solid red.
[0090] In the real world, the maximum amount of time a quarterback
has in the pocket may be cut short by defenders reaching him more
quickly than anticipated. To be effective in this situation a
quarterback must learn to "feel" the pressure of defenders.
Defensive pressure usually comes from the left or right side and in
this case, the quarterback must sense the pressure through
peripheral vision. Since he is focused down field looking for
potential receivers this can be a difficult skill to master. By
tracking the real time position of the defenders, their proximity
to the quarterback is easily determined. To aid the quarterback in
learning to "feel" this pressure the bar graphs can be increased
more rapidly to reflect the decreasing available time in the
pocket. In addition, the system can sense the specific position of
the defenders and they can be represented by dots that start out at
the far most sides of the glasses and move toward the center as the
defenders get closer. When the defender dot reaches the center of
the vision area, the defender has reached the quarterback.
[0091] Receiver Looks:
[0092] In any passing play, a quarterback has multiple receivers to
select from and a limited time to make the best selection. The
process of deciding which receiver to throw to is referred to as
"receiver looks". The most effective quarterbacks are those who
methodically move through these looks to find the best receiver to
throw to. Typically, there will be a primary look, the receiver the
ball is intended to go to, a secondary look, the receiver the
quarterback will consider if the primary is tightly covered and a
safety valve. The safety valve is a receiver positioned such that
the quarterback can dump the ball off when he has reached his
maximum time in the pocket (described above). Note that a
quarterback typically only has 1.2 to 1.5 seconds to evaluate each
of his receivers so this is not an easy skill to perfect,
especially under defensive pressure.
[0093] Continuous real time data in conjunction with heads-up
technology can be used in helping a quarterback learn to rotate
methodically through his available receivers. As the quarterback
drops back into the pocket, the dynamic position of the primary
receiver appears as a dot in his vision. Since the system is
sensing the actual position of the primary receiver this dot
represents the actual, relative position of the receiver. If the
receiver starts out on the left side and runs straight up the
field, then the dot will start in the bottom left hand side of the
vision and move up the left side of the vision. If the receiver
runs across the middle then the dot will start out in the bottom
left and move up and left to right through the vision. As an aide
to help the quarterback rotate at the right time, the receiver dot
begins to blink at programmed time (say 1 second). At a
preprogrammed time (say 1.5 seconds) the primary receiver dot turns
off and the secondary receiver dot turns on and functions in the
same manner as the primary receiver dot. Following the second
interval (say 1.5 seconds or total of 3 seconds), the safety
receiver dot turns on. Optionally, dots corresponding to multiple
potential receivers are shown simultaneously in the quarterback's
vision, allowing the quarterback to select the best receiver (e.g.,
the receiver in a most desired position) to throw to, from multiple
options.
[0094] Team Position Reads:
[0095] Real time position of all or select members of a team may
also be represented with heads-up technology, enabling a player
(e.g., the quarterback) to determine when other players are in or
approaching a desired position. For example, position of players
involved in a predetermined play may appear as dots in the
quarterback's vision. When the dots achieve or approach a desired
formation, indicating that players are in position for or moving
into position for the play, the quarterback may initiate the
play.
[0096] Combining Time in the Pocket and Receiver Reads:
[0097] As a quarterback's training progresses the time in pocket
feedback and receiver look elements can be integrated to provide
the quarterback with maximum real-time feedback to become a more
effective and efficient passer.
[0098] Simulated Use of Heads-up Feedback:
[0099] Once data has been captured while using this system in
practice with real receivers and real defenders, it can be replayed
at any time. This allows the quarterback to practice and experience
the live situation without all of the other players being present
thus allowing them to work on perfecting their own individual
roles. This is essentially a video game where the quarterback is an
active participant and the actions of the receivers and defenders
are derived from data of his actual teammates. This allows players
in skill positions, such as quarterbacks, to maximize the amount of
time they spend perfecting their role, under life like conditions,
without requiring the other 21 players to be present. This video
game like ability yields an unprecedented advance in the training
of skill position players.
[0100] Visual Heads-up Feedback: Receiver Example
[0101] Route Definition:
[0102] In this situation the heads-up glasses display is used much
like the display on a GPS. The receiver is able to look right
through the transparent glasses but a line indicating the specified
route would be projected in front of him. The system continuously
senses the receiver's position and as he traveled down each leg of
the route, the path would get shorter and shorter until he reached
a turning, or cutting, point. As soon as he made his cut, the next
leg of the route would turn upward indicating forward motion in
that direction.
[0103] Route Speed:
[0104] Each leg of a route may have a different ideal speed and in
some cases, there may even be multiple speeds per leg. As the
receiver moves along the path, specified by the route definition
method described above, the ideal speed in feet/second is projected
in his field of view. The system continuously monitors his current
speed and displays this next to the ideal speed. The system
monitors the athlete's performance in real time and may provide
visual indications to the athlete when the athlete meets certain
desired criteria, or when those criteria are not being met by the
athlete. For example, if the athlete needs to speed up then his
actual speed may be projected in green and if he needs to slow down
then it may be projected in red.
[0105] Game Time: Coaching
[0106] Data Only: Assignment Verification
[0107] Assignment verification can be employed in a game situation
in exactly the same way it is employed during practice sessions.
This provides coaches with real time feedback as to which players
are performing their assignments accurately and consistently.
[0108] Data Only: Fatigue Analysis
[0109] Various parameters such as time on the field and distance
traveled may be tabulated and used to determine when a player's
effectiveness may be diminishing.
[0110] Time on the Field
[0111] The total amount of time a player is on the field during a
quarter, half or game can be determined by accumulating time when
the player is located in the region of play defined by the playing
field and both end zones. This parameter can be made even more
relevant by associating the time accumulation with the playing
clock such that time on the field is only accumulated when the game
clock is running.
[0112] Distance Traveled
[0113] In many skill positions, such as running backs, defensive
backs and receivers, total distance traveled by the athlete may be
a more accurate determinant of fatigue than time on the field. The
total distance traveled by an athlete during a quarter, half or
game can be determined by accumulating distance traveled when the
player is located in the region of play defined by the playing
field and both end zones.
[0114] Data and Camera Integration: Opposition Evaluation
[0115] Real time position data is used to aim a camera(s)
automatically at the area "around" a specific athlete of interest.
By aiming a camera at the area around a specific athlete it is
possible to determine how the opposition is responding to specific
actions of the athlete of interest. For instance, by aiming a
camera at the area around a wide receiver the defensive backfield
rotations and/or assignments of the opposition can easily be
determined.
[0116] FIG. 6 shows one exemplary system 600 for providing
performance feedback. For purposes of example, system 600 is
described below with respect to athletes. However, it will be
appreciated that system 600 is not limited to use by athletes or an
athletic team (including a coach), but may provide beneficial
performance feedback to any participant in a field of play.
[0117] System 600 includes a performance analysis device 602 that
determines a location of each of a plurality of athletes 606 within
an operational field 608. In the example of FIG. 6, athletes 606
are formed into two pursuit teams that include athletes 606(1-4)
and athletes 606(5-8), respectively, that race around circuit 609
in a direction indicated by arrows. Each athlete 606 wears a
location device 612 that is used by performance analysis device 602
to determine the location of the athlete, a biometric sensing unit
646 that senses biometric information of the athlete, and a
feedback device 644 that provides feedback from performance
analysis device 602 to the athlete. The biometric information
sensed by biometric sensing unit 646 may include one or more of:
body temperature at one or more locations, heart rate, blood oxygen
levels, hydration level, respiratory rate, perspiration salinity
level, blood sugar levels, and other blood analysis results. For
example, Cygnus Inc.'s GlucoWatch checks glucose levels every 20
minutes by sending tiny electric currents through the skin.
Biometric sensing unit 646 may sense other biometric information
without departing from the scope hereof.
[0118] Feedback device 644 may represent one or more of a visual
display, such as a liquid crystal display (e.g., a watch and a
bicycle handlebar display unit) and a HUD incorporated within
glasses (e.g., glasses 144, FIG. 1) and/or a helmet, an audio
output unit that generates audio signals such as beeps, tones and
verbal information, and an actuator that provides movement
(tactile) feedback. Where the athlete is taking part in a sport
that utilizes equipment that is ridden, such as a bicycle, feedback
device 644 may be attached to the equipment or the athlete.
Performance analysis device 602 utilizes a wireless transceiver 640
to send feedback information to each feedback device 644, to
receive biometric information from biometric sensing units 646, and
to receive location information from location units 612.
[0119] Performance analysis device 602 includes a performance
monitor 622 that utilizes real time location information from
location units 612 and biometric feedback information from
biometric sensing units 646 to determine, for each athlete 606,
performance that includes one or more of location, velocity, stress
level, and fatigue level. Performance monitor 622 may also generate
and maintain profile data for each athlete, shown as profile data
621 stored within recording device 620 of performance analysis
device 602. Profile data 621 may include, for each athlete,
historical performance such that current and future performance of
the athlete may be predicted. For example, if a particular athlete
finished a four-thousand meter training race in the morning, a
lighter level of afternoon training may be expected for that
athlete and performance analysis device 602 may not push the
athlete as hard as another athlete who was resting that
morning.
[0120] Recording device 620 may generate a live feed and allow
interaction with a user through a display device 630.
[0121] In the example of FIG. 6, each athlete 606 receives feedback
for his team. For example, each athlete 606(1-4) receives feedback
including location, stress and fatigue levels, for themselves and
each other. Similarly, each athlete 606(5-8) receives feedback
including location, stress and fatigue levels, for themselves and
each other. Feedback generator 624 may also provide feedback
defining each team performance with respect to the other. For
example, in the pursuit illustrated in FIG. 6, timing between each
last athlete of each team (i.e., athletes 606(4) and 606(8)) may be
provided to each athlete 606. As the order of each team changes
through rotation, performance analysis device 602 automatically
identifies the `last` man of the team. Performance monitor 622 and
feedback generator 624 may also provide each team with relative
position information of each team member, such that a current
leader of the team becomes aware when the last member begins to
fall too far behind. For example, feedback device 644 displays four
green dots or blips to indicate team members in position, and a red
dot or blip to indicate a team member falling behind. In one
aspect, the leader of a track cycling team becomes aware (via
performance monitor 622 and feedback generator 624) that the last
man of the team is struggling to keep pace and may thus modify the
team's pace to prevent exhaustion of the last man, in hopes of
bettering the team's collective race result.
[0122] In another example, a quarterback receives feedback,
indicating a projected fatigue level for potential receivers,
allowing an informed choice as to which receiver to select for a
particular play. Similarly, the quarterback may receive fatigue
information for defenders such that potential weaknesses may be
reduced or at least identified at the last second prior to the snap
of the football.
[0123] Each athlete 606 may receive feedback indicating the
location of other athletes relative to himself, particularly
athletes not directly in his current field of view. Further, each
athlete may receive feedback relating to their entire team, based
upon processing of data by performance analysis device 602. For
example, where a player in a football team is recovering from high
impact in a previous play, other team members will be aware of this
through feedback from performance analysis device 602.
[0124] Biometric sensing units 646 may include one or more
accelerometers that measure impact to athlete 606 and movement of
the athlete. In one example, these biometric sensors measure
cadence of the athlete running such that performance monitor 622
may determine stride length (e.g., based upon measured cadence and
speed determined from location information).
[0125] Performance analysis device 602 may also monitor athletes
606 while resting between training exercises, such that profile
data 621 for each athlete includes recovery rates.
[0126] Performance analysis device 602 may include a strategy
manager 626 that creates and/or utilizes strategy data 627 stored
within recording device 620. In one example, strategy data 627
represents plays (e.g., Perfect Execution Template 300, FIG. 3) in
an American football game, or practice session. Strategy manager
626 may define criteria (e.g., paths and speeds) for one or more
players in the play and feedback generator 624 sends these criteria
to feedback devices 644 of relevant players. Performance monitor
622 may monitor movement of each player (e.g., using location units
612) and provide feedback, via feedback generator 624, indicating
how well, or not, each player conforms to the suggested path and
speed. Feedback generator 624 may also provide, in real-time,
corrective moves and instructions to players that do not maintain
the desired criteria.
[0127] FIG. 6 also shows a coach 652 of athletes 606 utilizing a
hand held device 642 that wirelessly interacts, via transceiver
640, with performance analysis device 602. Hand held device 642 may
include a processor, a memory, an input device and a display, such
as found in an Apple.TM. iPad. Device 642 allows coach 652 to view
live performance information determined by performance analysis
device 602 for athletes 606, and feedback that is sent
automatically to the athletes from feedback generator 624. Coach
652 may provide input to object tracking device, via hand held
device 642 and transceiver 640, to adjust the level of feedback
provided by performance analysis device 602.
[0128] In an embodiment, coach 652 receives `alarm` messages from
performance analysis device 602, via wireless transceiver 640 and
hand held device 642, indicating athletes that have one or more of:
high stress levels, high fatigue levels, high temperatures, and
other abnormal biometric information. Coach 652 may then make
informed decisions to rest athletes prior to potential injuries or
life threatening conditions.
[0129] Changes may be made in the above methods and systems without
departing from the scope hereof. For example, the athletes
referenced above may represent any type of participant within an
operational field or field of play. Performance analysis device 602
for example provides biometric information to and of any "team". By
tracking and reporting biometric data of team members, performance
analysis device 602 allows team members to monitor one another,
and/or allows an instructor, coach or supervisor to monitor each
team member. In one aspect, performance analysis device 602 allows
commercial divers, astronauts or fighter pilots to monitor one
another for signs of panic (e.g., increased respiration and heart
rate) or distress even when out of sight. It should thus be noted
that the matter contained in the above description or shown in the
accompanying drawings should be interpreted as illustrative and not
in a limiting sense. The following claims are intended to cover all
generic and specific features described herein, as well as all
statements of the scope of the present method and system, which, as
a matter of language, might be said to fall therebetween.
* * * * *