U.S. patent application number 14/158423 was filed with the patent office on 2014-07-17 for real time feedback swim training system and method based on instantaneous speed.
This patent application is currently assigned to Alex Aquatics. The applicant listed for this patent is Alex Aquatics. Invention is credited to Alex Boutov, Andrei Krylov.
Application Number | 20140200116 14/158423 |
Document ID | / |
Family ID | 51165575 |
Filed Date | 2014-07-17 |
United States Patent
Application |
20140200116 |
Kind Code |
A1 |
Boutov; Alex ; et
al. |
July 17, 2014 |
Real Time Feedback Swim Training System and Method Based on
Instantaneous Speed
Abstract
A swim training system and method includes a signal delivery
sub-system configured to deliver signals to the swimmer and a
machine vision sub-system with a detector and a target marker to be
worn by a swimmer. The detector is configured to detect the target
marker. A controller is configured to operably control the machine
vision sub-system. A data processing sub-system is configured to
process data collected by the machine vision sub-system and to
generate signals for the signal delivery system. During a swim
training session, the detector tracks a location of the target
marker and transmits speed data to the data processing sub-system.
The data processing sub-system processes the speed data and
generates signals that are transmitted to the swimmer via the
signal delivery sub-system in real time during the swim session.
The signals are representative of a real time speed characteristic
of the swimmer.
Inventors: |
Boutov; Alex; (LaGrange,
IL) ; Krylov; Andrei; (St. Petersburg, RU) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Alex Aquatics |
LeGrange |
IL |
US |
|
|
Assignee: |
Alex Aquatics
LaGrange
IL
|
Family ID: |
51165575 |
Appl. No.: |
14/158423 |
Filed: |
January 17, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61753759 |
Jan 17, 2013 |
|
|
|
Current U.S.
Class: |
482/55 |
Current CPC
Class: |
A63B 2244/20 20130101;
G06T 7/292 20170101; G06T 2207/30196 20130101; G01P 3/50 20130101;
A63B 71/0686 20130101; G01S 5/16 20130101; G06T 2207/30204
20130101; G09B 19/0038 20130101 |
Class at
Publication: |
482/55 |
International
Class: |
A63B 69/12 20060101
A63B069/12 |
Claims
1. A swim training system comprising: a signal delivery sub-system
configured to deliver signals to a swimmer; a machine vision
sub-system having a detector and a target marker to be worn by the
swimmer, the detector configured to detect the target marker; a
controller configured to operably control the machine vision
sub-system; and a data processing sub-system configured to process
data collected by the machine vision sub-system and to generate
signals for the signal delivery system, wherein, during a swim
training session of the swimmer, the detector tracks a location of
the target marker and transmits speed data to the data processing
sub-system, wherein the data processing sub-system processes the
speed data and generates signals that are transmitted to the
swimmer via the signal delivery sub-system in real time during the
swim session, the signals representative of a real time speed
characteristic of the swimmer.
2. The swim training system according to claim 1, wherein the
target marker is a light belt, which carries one or more lights
visible to the machine vision sub-system.
3. The swim training system according to claim 2, wherein the one
or more lights include a plurality of omni-directional LED
lights.
4. The swim training system according to claim 1, wherein the
signal delivery sub-system includes an in-ear audio transmitter
worn by the swimmer.
5. The swim training system according to claim 1, wherein the
signal delivery sub-system includes a remote FM transmitter and an
FM receiver worn by the swimmer and tunable to a desired FM
channel.
6. The swim training system according to claim 1, wherein the data
processing sub-system includes an electronic device with a data
processor.
7. The swim training system according to claim 1, wherein the
detector of the machine vision sub-system includes one or more
video cameras.
8. The swim training system according to claim 7, wherein the one
or more video cameras are configured for underwater use and to
detect the target marker under water.
9. The swim training system according to claim 1, wherein the real
time speed characteristic includes a desired instantaneous swim
speed range between a low threshold speed and a high threshold
speed.
10. The swim training system according to claim 9, wherein the
signal delivery sub-system can deliver a first signal when an
instantaneous swim speed of the swimmer is slower than the low
threshold speed, a second signal when the instantaneous swim speed
of the swimmer is faster than the high threshold speed, and a third
signal when the instantaneous swim speed of the swimmer is within
the desired instantaneous swim speed range.
11. The swim training system according to claim 10, wherein the
first signal is a first auditory signal, the second signal is a
second auditory signal different from the first auditory signal,
and the third signal is the absence of an auditory signal.
12. A swim training method comprising: providing a signal delivery
sub-system, a data processing sub-system, and a machine vision
sub-system having a detector and a target marker; attaching the
target marker to a swimmer; placing the detector of the machine
vision sub-system so as to detect the target marker on the swimmer;
operating a controller to control the machine vision sub-system;
tracking the target marker via the detector as the swimmer moves
during a swim session; sending data from the machine vision
sub-system to the data processing sub-system during the swim
session; processing the data at the data processing sub-system; and
transmitting signals to the swimmer via the signal delivery
sub-system in real time during the swim session, the signals
representative of a real time speed characteristic of the
swimmer.
13. The swim training method according to claim 12, wherein the
step of providing includes providing one or more cameras that can
detect the target marker.
14. The swim training method according to claim 12, wherein the
step of providing includes providing a light belt that carries one
or more lights detectable by the detector of the machine vision
sub-system.
15. The swim training method according to claim 12, wherein the
step of placing includes placing the detector under water.
16. The swim training method according to claim 12, wherein the
step of operating a controller includes configuring a programmable
electronic device to operate the machine vision sub-system.
17. The swim training method according to claim 12, wherein the
step of transmitting includes transmitting an audible signal to the
swimmer.
18. The swim training method according to claim 12, wherein the
step of transmitting includes transmitting two or more different
audible signals to the swimmer via an ear piece of the signal
delivery sub-system.
19. The swim training method according to claim 12, wherein the
step of processing includes comparing a desired instantaneous swim
speed range between a low threshold speed and a high threshold
speed to an instantaneous swim speed of the swimmer during the swim
session.
20. The swim training method according to claim 19, wherein the
step of transmitting includes transmitting a first signal when an
instantaneous swim speed of the swimmer is slower than the low
threshold speed, a second signal when the instantaneous swim speed
of the swimmer is faster than the high threshold speed, and a third
signal when the instantaneous swim speed of the swimmer is within
the desired instantaneous swim speed range.
Description
RELATED APPLICATION DATA
[0001] This patent is related to and claims priority benefit of
U.S. provisional patent application Ser. No. 61/753,759, which was
filed on Jan. 17, 2013 and also entitled "Real Time Feedback Swim
Training System and Method Based on Instantaneous Speed." The
entire content of this prior filed provisional application is
hereby incorporated herein by reference.
BACKGROUND
[0002] 1. Field of the Disclosure
[0003] The present disclosure is generally directed to swim
training devices, systems, and methods, and more particularly to a
swim training system that provides real time feedback based on an
instantaneous swimming speed during swimmer training.
[0004] 2. Description of Related Art and Problems with Same
[0005] It is well known in many athletic training endeavors to
employ devices and systems to assist the athletes with objective
feedback about their performance. This is particularly true for
competitive swimming because speed is the critical indicator of
each individual swimmer's performance. Objective feedback regarding
swimming speed is critical for both swimming skill development and
competitive swim training.
[0006] Objective feedback devices, methods, and systems that assist
swim training related to swimming speed can be classified into two
categories: 1) feedback delayed until the end of a swim and 2)
real-time feedback during a swim. There are many widely known and
used methods and systems of delayed feedback, which include the
following.
[0007] Stop watch. A stop watch allows coach to measure time and
speed of a swim. The coach can communicate the measured time to the
swimmer at the end of the swim. All fluctuations of the swimming
speed within one swim are necessarily averaged. The stop watch does
not allow a coach or the swimmer to measure swimming speed
fluctuations occurring during any one swim.
[0008] Pace clock. A pace clock allows a swimmer to measure the
time and speed of any one swim. A swimmer can view and determine
the time of a previous split at the end of the swim. All
fluctuations of the swimming speed within one swim are averaged. A
pace clock also does not allow a swimmer or coach to measure
swimming speed fluctuations occurring during any one swim.
[0009] AVIDA Sports. The AVIDA Sports system provides a swimmer
with a message to the swimmer about the split time of the
immediately previous lap. The AVIDA Sports system uses the term
"Real-Time" to indicate that the split time information is provided
to the swimmer on every lap about the split time of the previous
lap. All swimming speed fluctuations occurring within one length or
lap of the pool are not averaged and not available to the swimmer
or coach.
[0010] Methods using a cord attached to swimmer's waste belt. The
following four methods employ a cord with one end that attaches to
the swimmer and another end that attaches to a deck mounted piece.
The cord stretches during the swim between the deck-mounted piece
and the swimmer.
[0011] 1) "Swim Power Test"--see http://www.globsport.org
[0012] 2) "Torrent E-Rack Swim Speed Analyzer"--see
http://www.teamtermin.com/solutions.shtml
[0013] 3) "Speedometer Swimming Measuring System"--see
http://www.sportthieme.co.uk/Swimming/Training_Equipment/Swimming_Benches-
/art=22 36041
[0014] 4) "Velocity Meter/Video Telemetry"--see
http://www.teamtermin.com/
[0015] These four methods and systems that use a cord attached to
swimmer can provide feedback about speed fluctuations, but only
after the swim is completed, not during the swim where the cord is
employed. The attached cord methods and systems also only allow the
swimmer to perform measurements when the swimmer is traveling in
one direction, away from the deck mounted piece. A regular training
swim typically involves two-way lap swimming.
[0016] Methods based on ultrasound Doppler effect. Koigerov, 1979.
Some have tried to develop improved swim training devices as well,
to varying degrees of success. For example, Koigerov/Kolmogorov
utilized Doppler effects and ultrasound to measure the speed of a
swimmer in a pool. However, accuracy is poor at lower speeds and
ultrasound reflects off the walls of the pool, further reducing
accuracy. The device is positioned at the bottom of the pool. The
wavy surface of the water can also limit the capability of the
device to locate the swimmer on the surface. The device can also be
used only with a single swimmer in the pool. Otherwise the data
would be compromised by obtaining data from the wrong swimmer. This
device is best used for obtaining data for a swimmer's starts. This
device is also not designed for real time feedback. Analysis of the
data points collected is done only after the swimmer's training
session.
[0017] Video recording analysis. Analysis of swimming speed
fluctuations based on video recording is also widely used. For
example, Karpovich, 1930, Maglischo 2003, Barbosa, et al 2012; etc.
have used video recording techniques. These methods provide
in-depth analysis of swimming speed fluctuations. However, feedback
is available to the swimmers only after the swim is completed, not
real time.
[0018] Methods, devices, and systems that can provide "real time"
feedback to the swimmer or coach regarding instantaneous swimming
speed during swim training include the following.
[0019] Real-Time Swimming Monitor. US Published Application No.
2010/0030482 to Xipu Li discloses a complex
micro-electro-mechanical system (MEMS) device that uses
accelerometers and gyroscopes. Accuracy of the results produced by
the device is questionable due to the cumulative nature of
measurement errors, inherent to accelerometers. This error
accumulation is known as measurement drift. Measurement error of a
current measurement cycle is added to the measurement errors of all
previous measurement cycles after the last calibration of the
system. Calibrations are required frequently to correct measurement
drift in order to provide more accurate speed measurements. Any
measurement drift correction or sub-system of the disclosed
Real-Time Swimming Monitor can make the system and components
complex and quite expensive.
[0020] Swim Meter. U.S. Pat. No. 5,767,417 discloses a device used
to measure swimming speed in open water or non-standard width
swimming pools. The measurements are based on flow of the water
around the swimmer's body. The disclosed Swim Meter is not intended
for measuring swimming speed fluctuations that occur within time
intervals of 100 milliseconds or less. The accuracy of the swimming
speed measurements of the Swim Meter, which is based on either of a
rotating element, a pitot tube and pressure sensor, a current
generator responsive to salt ions in the water, or an electrical
generator, is not sufficient for speed measurements within such
short time intervals.
[0021] The foregoing and other known devices, systems, and methods
for providing feedback based on swimmer's speed are capable of
determining average speed per swim. However, these devices,
systems, and methods are not capable of providing sufficiently
accurate measurement rates required for analysis of speed
fluctuations within a swimming stroke cycle. Video analysis systems
are used mostly for research into swimming stroke technique.
However, the results of video analysis are not available to the
swimmer in real time while they swim. The time delay, required for
video data processing, causes delay in feedback delivery to the
coach or swimmer. Delayed feedback is less effective than real time
feedback for stroke correction and training.
SUMMARY
[0022] The disclosed NATATOMETRY.TM. system automates analysis of
video data in order to provide speed fluctuation feedback delivered
in real time to the swimmer. The real time feedback during a swim
training session increases the effectiveness of stroke corrections
and training regimens in real time while a swimmer is training.
[0023] In one example according to the teachings of the present
disclosure, a swim training system has a signal delivery sub-system
configured to deliver signals to a swimmer and a machine vision
sub-system having a detector and a target marker to be worn by the
swimmer. The detector is configured to detect the target marker. A
controller is configured to operably control the machine vision
sub-system. A data processing sub-system is configured to process
data collected by the machine vision sub-system and to generate
signals for the signal delivery system. During a swim training
session of the swimmer, the detector tracks a location of the
target marker and transmits speed data to the data processing
sub-system. The data processing sub-system processes the speed data
and generates signals that are transmitted to the swimmer via the
signal delivery sub-system in real time during the swim session.
The signals are representative of a real time speed characteristic
of the swimmer.
[0024] In one example, the target marker can be a light belt, which
can carry one or more lights visible to the machine vision
sub-system.
[0025] In one example, the target marker can be a light belt with a
plurality of omni-directional LED lights visible to the machine
vision sub-system.
[0026] In one example, the signal delivery sub-system can include
an in-ear audio transmitter worn by the swimmer.
[0027] In one example, the signal delivery sub-system can include a
remote FM transmitter and can include an FM receiver worn by the
swimmer. The FM transmitter and/or receiver can be tunable to a
desired FM channel.
[0028] In one example, the data processing sub-system can include
an electronic device with a data processor.
[0029] In one example, the controller can be a computer, a laptop
computer, a smart phone, a tablet, or the like.
[0030] In one example, the detector of the machine vision
sub-system can include one or more video cameras.
[0031] In one example, the detector of the machine vision
sub-system can include one or more video cameras that can be
configured for underwater use and to detect the target marker under
water.
[0032] In one example, the real time speed characteristic can
include a desired instantaneous swim speed range between a low
threshold speed and a high threshold speed.
[0033] In one example, the signal delivery sub-system can deliver a
first signal when an instantaneous swim speed of the swimmer is
slower than a low threshold speed, a second signal when the
instantaneous swim speed of the swimmer is faster than a high
threshold speed, and a third signal when the instantaneous swim
speed of the swimmer is within a desired instantaneous swim speed
range between the low and high threshold speeds.
[0034] In one example, a first signal delivered to the swimmer can
be a first auditory signal, a second signal delivered to the
swimmer can be a second auditory signal different from the first
auditory signal, and a third signal delivered to the swimmer can be
the absence of an auditory signal.
[0035] In one example according to the teachings of the present
disclosure, a swim training method includes providing a signal
delivery sub-system, a data processing sub-system, and providing a
machine vision sub-system that has a detector and a target marker.
The target marker is attached to a swimmer and the detector of the
machine vision sub-system is placed so that it can detect the
target marker on the swimmer. A controller is operated to control
the machine vision sub-system. The detector tracks the target
marker as the swimmer moves during a swim session. Data is sent
from the machine vision sub-system to the data processing
sub-system during the swim session. The data processing sub-system
processes the data. Signals are transmitted to the swimmer via the
signal delivery sub-system in real time during the swim session.
The signals are representative of a real time speed characteristic
of the swimmer.
[0036] In one example, the method can include providing one or more
cameras as the detector that can detect the target marker.
[0037] In one example, the method can include providing a light
belt that carries one or more lights detectable by the detector of
the machine vision sub-system.
[0038] In one example, the method can include placing the detector
under water.
[0039] In one example, the method can include providing a
programmable electronic device as the controller and configuring
the electronic device to operate the machine vision sub-system.
[0040] In one example, the method can include transmitting an
audible signal to the swimmer.
[0041] In one example, the method can include transmitting two or
more different audible signals to the swimmer via an ear piece of
the signal delivery sub-system.
[0042] In one example, the method can include the controller
sub-system comparing a desired instantaneous swim speed range
between a low threshold speed and a high threshold speed to an
instantaneous swim speed of the swimmer during the swim
session.
[0043] In one example, the method can include transmitting a first
signal when an instantaneous swim speed of the swimmer is slower
than a low threshold speed, a second signal when the instantaneous
swim speed of the swimmer is faster than a high threshold speed,
and a third signal when the instantaneous swim speed of the swimmer
is within a desired instantaneous swim speed range between the low
and high threshold speeds.
BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Objects, features, and advantages of the present invention
will become apparent upon reading the following description in
conjunction with the drawing figures, in which:
[0045] FIG. 1 shows a simplified schematic of one example of a
NATATOMETRY system constructed in accordance with the teachings of
the present invention.
[0046] FIG. 2 shows a simplified plot of one example of real time
swim speed data over time obtained, processed, and delivered by the
NATATOMETRY system FIG. 1.
DETAILED DESCRIPTION OF THE DISCLOSURE
[0047] The disclosed real time feedback swim training system
(hereinafter the "NATATOMETRY system") solves or improves upon one
or more of the above-noted and/or other problems and disadvantages
with prior known swim training systems, methods, and devices. As
used herein, the phrase "real time" is defined as a time interval
of 100 milliseconds or less between the moment of instantaneous
speed and the beginning of a signal or indicator, such as an audio
alarm or a visual signal, being delivered to the swimmer, such as
via an ear piece or swim goggles worn by the swimmer.
[0048] In one application, real time feedback, based on
instantaneous speed, can be used by swimmers who are correcting
swimming technique to do so while swimming. Using the disclosed
NATATOMETRY system and method, a swimmer can achieve high
instantaneous speed within a stroke cycle, or avoid a low instant
speed within a stroke cycle, or stay within the range of desired
instant speed fluctuations within a stroke cycle.
[0049] In one application, real time feedback, based on
instantaneous speed, can be utilized by swimmers who are performing
drills that involve or require intentional change of speed between
stroke cycles during, for example, build up and build down drills.
During such drills, the swimmer intentionally accelerates or
decelerates through one swim. Real time feedback to the swimmer
during the swim can help them achieve the desired speed
changes.
[0050] In one example, the NATATOMETRY system can be programmed not
to deliver any signal to notify the swimmer that his or her instant
speed has fallen below any low speed threshold but can deliver a
signal a signal to notify the swimmer that their instantaneous
speed is above a high speed threshold. In one example, the
NATATOMETRY system can be programmed not to deliver any signal to
notify the swimmer that his or her instant speed is above a high
speed threshold but can deliver a signal that their speed has
fallen below a low speed threshold. In one example, the NATATOMETRY
system can be programmed to deliver different signals to notify the
swimmer when their instantaneous speed falls below or increases
above such low and high speed thresholds, respectively. The
NATATOMETRY system can be programmed to alter the thresholds during
a swim to achieve a specific training regimen that requires speed
variation during the training swim.
[0051] In one example, the disclosed NATATOMETRY system can provide
a swimmer with real time feedback regarding their instantaneous
speed in the form of audio alarm signals. In one example, the
disclosed NATATOMETRY system can deliver an audible signal such as
a high-pitch (900 Hz audio signal for example) to an ear piece worn
by the swimmer that the swimmer can hear while swimming. In one
example, the disclosed NATATOMETRY system can deliver a visual
signal that is visible to the swimmer such as a green light
illuminated through the goggles, for example, while swimming. In
one example, the disclosed NATATOMETRY system can deliver both
audio signals such as the high-pitch audio signal and visual
signals such as the green light to the swimmer when their
instantaneous speed is above a high speed threshold. These and/or
other signal types can also be used to notify the swimmer that he
or she has successfully reached a pre-set speed threshold, such as
a high threshold for instant speed in real time.
[0052] In other examples, the NATATOMETRY system can also deliver
any type of signal to notify the swimmer that his or her instant
speed has fallen below a predetermined low speed threshold. In one
example, the disclosed NATATOMETRY system can deliver a low-pitch
(400 Hz) audio signal and/or a visual signal (red light through the
goggle) to notify the swimmer that he or she has failed to avoid a
pre-set low threshold of instant speed during the previous
measurement cycle. In this particular example the NATATOMETRY
system does not deliver any signal to notify the swimmer that his
or her instant speed raised above any high speed threshold.
[0053] In one example, the disclosed NATATOMETRY system can be
configured to deliver in real time:
[0054] 1) a high-pitch (900 Hz) audio signal to notify the swimmer
that he or she has reached a pre-set high threshold of
instantaneous speed during the previous measurement cycle, or
[0055] 2) a low-pitch (400 Hz) audio signal to notify the swimmer
that he or she has failed to avoid a pre-set low threshold of
instantaneous speed during the previous measurement cycle, or
[0056] 3) the absence of a high-pitch or a low-pitch signal can
indicate to the swimmer that his or her intra-cycle speed stays
within the range of the pre-set high and low speed thresholds
during the previous measurement cycle.
[0057] FIG. 1 shows a simplified schematic of one example of a
NATATOMETRY system 10 according to the invention. The disclosed
NATATOMETRY system 10 generally can be defined as having three
sub-systems including a Machine Vision sub-system 12 for data
collection, a Data Processing sub-system 14 for signal generation,
and a Signal Delivery sub-system 16.
[0058] The Machine Vision sub-system 12 is used to measure the
location of the swimmer in a current measurement cycle and to
communicate this location to the Data Processing sub-system 14. The
Machine Vision sub-system 12 in one example includes a target
marker 18 such as a light emitting marker belt, worn by a swimmer
S, around the body near the hip joint level. The Machine Vision
sub-system 12 can also include a detector to detect the target
marker 18. In one example, the detector is in the form of one or
more video cameras 20. The NATATOMETRY system 10 can also include a
controller 22, which can be a part of the Machine Vision sub-system
12 or part of the Data Processing sub-system 14. The controller 22
can be configured to provide camera synchronization, receive video
signals from the cameras, and perform image processing in real
time, to determine the swimmer's position, based on (in one
example) Binary Large Object (BLOB) detection. A special
controller, a desktop computer, a laptop computer, a tablet
computer, a smart phone device, or the like can serve as the
controller for controlling the Machine Vision sub-system 12 and can
include controller hardware for the purpose of the disclosed
NATATOMETRY system.
[0059] The Data Processing sub-system 14 is used to calculate the
instantaneous speed values, based on a comparison of the locations
of the swimmer in a current and a previous measurement cycle. The
Data Processing sub-system 14 can be implemented on the controller
22 as in the example of FIG. 1, at least in part on the detector 18
of the of Machine Vision sub-system 12 itself, or on a separate
computer (not shown). The instantaneous speed value for a giving
measurement cycle is compared, in one example, to the pre-set
thresholds. An appropriate signal, such as an audio alarm signal,
can then be generated if the high or low speed the thresholds are
reached. In one example, a high-pitch signal can be generated if
the instantaneous speed is greater than the pre-set high speed
threshold. A low-pitch alarm signal can be generated if the
instantaneous speed is less than the pre-set low speed threshold.
No signal can be generated if the calculated current instantaneous
speed is within the range between high and low speed thresholds. If
a signal is generated during the current inspection cycle, the Data
Processing sub-system can then pass a high-pitch or low-pitch
signal on to the Signal Delivery sub-system 16.
[0060] The Signal Delivery sub-system 16 can include an out of
water, on-deck transmitter and a waterproof receiver worn by a
swimmer. The sub-system 16 can also include a receiver worn by a
coach or an on-deck speaker configured for the coach to hear. The
Signal Delivery sub-system 16 can, in one example, convert the
received high-pitch or low-pitch signal into an FM radio signal and
transmit the radio signal through the airwaves. An FM radio
receiver can be worn by a swimmer and be configured to receive the
FM signal and deliver it to a bone conduction ear-piece, which can
be mounted on or near the swimmer's ear. The coach can use, for
example, a wearable receiver, which can be identical to the
swimmer's receiver, or can carry or stand near an on-deck speaker.
The Signal Delivery sub-system 16 can utilize a separate FM radio
frequency for each of multiple swimmers to provide real time
feedback for more than one swimmer training in the water at the
same time. The controller 22 and Data Processing sub-system 14 can
be configured to accommodate multiple swimmers by detecting
multiple target markers 18 and/or receiving data collected by the
detector or detectors. Each swimmer can have their own dedicated
detector or the controller 22 can be configured to distinguish
among the different target markers 18 of multiple swimmers detected
by the same detector.
[0061] Machine Vision Sub-System Details
[0062] Target marker--In the disclosed example, the swimmer wears a
target marker 18 device near their hip joints. The target marker
includes a waist belt with Visual Identification Projectors Beacons
(see http://www.nitevis.com/VIPIR_Legacy_detail.htm) that are
attached to the waist belt. Multiple beacons (nine in one example)
are used to compensate for a swimmer's body rotation along the
lengthwise axis in direction of swimming. In another example, a low
voltage waterproof Omni Directional LED light Bulb can be used as a
target marker. The light from the target marker 18 forms a group of
adjacent pixels in the image created by the cameras 20. This group
of pixels is identified by the controller as a BLOB. The location
of the swimmer in each image corresponds to the location of the
center of the BLOB. Displacement of the BLOB from the edge of the
inspection region corresponds to the location of the swimmer at the
moment when the camera's electronic shutter is opened.
[0063] Video Camera--In this example, the Machine Vision system
includes a video camera 20 (in one example an IR camera by SONY,
Model XCEI50), or multiple such cameras, placed along the bottom of
a pool or along the underwater windows located along the bottom of
the pool. Optical characteristics of the water in the swimming pool
and pool lighting conditions can play an important role in camera
selection for the NATATOMETRY system. Water serves as an optical
filter, which filters out most of the light energy emitted in the
red and infra-red segments of the optical spectrum present in
typical pool lighting and deck heating conditions. Infra-red energy
emitted by deck heaters is effectively filtered out by water. For
above-the-water surface camera solutions, green and blue optical
filters are likely required.
[0064] Cable--In this example, a cable (not shown) or cables can
electrically connect the video cameras 20 and the controller 22. In
one example, the cable or cables can be a CA-CN17 cable or a
CV-5000/XG cable Series. The cables are used to communicate and
exchange control and video signals between the controller 22 and
the one or more video cameras 20. A separate cable can be used to
connect the controller with each of the cameras.
[0065] Machine Vision Controller--In this example, the functions of
the controller 22 can be implemented as an algorithm on a
programmable processor such as a portable computer, a smart phone,
a tablet computer, a desktop or laptop personal computer, for
storing and sending electronic signals and data. In one example,
the controller is a XG-7000 series Vision System
(XG-7002A--Flexible Image Processing System Analog camera
controller by Keyence Corporation www.keyence.com). In this
example, image collection, BLOB detection with pre-processing
filters, and data transmission can take from 35 to 45 milliseconds
depending on the complexities of the real images. The controller 22
can be configured to trigger a new measurement cycle once every 50
milliseconds.
[0066] The controller 22 can use an internal clock to synchronize
and control multiple cameras 20. The controller 22 can adjust the
camera's level of sensitivity, control opening and closing of the
electronic shutters of the cameras 20, and the like. The controller
22 receives the video signals from the cameras 20 and can process
the video signals.
[0067] The controller 22 can use, for example, two pre-processing
filters to minimize false alarms in BLOB detection. The first of
the two pre-processing filters can convert brightness of the pixels
from grey scale into binary (black or white) category, using an
adjustable threshold. The black/white threshold can be raised when
the external background light is high. The binary threshold can be
lowered when the level of external light is low. This first filter
may not be needed if there is no external light in the pool. The
second of the two pre-processing filters can convert a group of
adjacent pixels of the same brightness into one object. This can
help to minimize random fluctuations caused by the shape of the
light signal projected onto the camera 20 by the target marker 18.
The pre-processing filters may be needed to lower the rate of false
alarms caused by small random particles in the water and by light
reflections from the wavy surface of the water.
[0068] The controller 22 detects the BLOB in the inspection area.
The inspection area is configured to correspond to the area between
lane line dividers. In this manner, each of the cameras 20 can
independently process the location of the targets 18 in multiple
swimming lanes. The location of the target marker 18 is initially
determined in pixels from the edge of the camera's Field of View
(FOV).
[0069] At the end of a measurement cycle the controller transmits
strings of values to the Data Processing sub-system 14. In one
instance, the transmission can be configured to use a conventional
Ethernet E5-I cable, which connect the controller 22 with a laptop
computer used to host the Data Processing sub-system 14. One string
can be transmitted per BLOB.
[0070] The string of values can consist of comma separated fields
including: the number of the current inspection or measurement
cycle, the number of the current camera 20, the number of the
inspection area, the location of the BLOB on the X-axis, and the
like. The location measurement is converted from pixels into linear
values (yards for example) by using an arc tangent function within
the Data Processing sub-system 14, which can function as
follows:
L=D*tg(.alpha.),
where [0071] L--is the linear position of the swimmer in meters,
[0072] D--is the distance from camera to the swimmer, [0073]
.alpha.--is the angular displacement of the swimmer from the center
of the FOV, and [0074] tg (.alpha.)--is the tangent of the angular
displacement of the swimmer in the FOV of the camera.
[0075] Data Processing Sub-System Details
[0076] The Data Processing sub-system 14 generates the alarm
signals of appropriate tone and delivers the signals to the Signal
Delivery sub-system 16. Details of the processing algorithm in one
example are found in the attached listing of the script written in
Perl programming language.
TABLE-US-00001 #!/usr/bin/perl -w use Audio::Beep; use
Math::Interpolate qw(linear_interpolate); my $FALSE = 0; my $TRUE =
1; # my $DEBUG = $FALSE; my $DEBUG = $TRUE; my $BEEPED = 0; # FALSE
my $pevious_x = undef; my $Delay_OK = 0; #counter of the delayed OK
lines my $Delay_NG = 0; #counter of the delayed NG lines my $T_OK =
10; # threshold of the delayed OK lines before starting a new swim
my $T_NG = 10; # threshold of the delayed NG lines before starting
a new idle my $SwimInProgress = $FALSE; my $IdleInProgress = $TRUE;
my $Swim_number = 0; my @Delay_OK = ( ); # stack for the delayed OK
lines. my @Delay_NG = ( ); # stack for the delayed NG lines. my $OK
= 0; #couner of the "OK" lines in the current idle series. my $NG =
0; #couner of the "NG" lines in the current idle series. my
$last_measured_x = undef; # last non-interpolated, but actully
measured x my $last_OK_counter = undef; # last counter with
non-interpolated, but actully measured x while (<>){ my $line
= $_; chomp $line; $DEBUG && print "$line\n"; my
@inspection_data = split (/,/, $line); # split data by comma from
the input line my $x = $inspection_data[0]; my $counter =
$inspection_data[2]; my $judgement = $inspection_data[5]; if
($judgement =~ /\+0$/){# judgement is "OK": There is a BLOB if Swim
is in Progress - process this $line right away if
($SwimInProgress){# Swim In Progress IS TRUE foreach my $ng
(@Delay_NG){ my @inspections = ($last_OK_counter, $counter); my
@positions = ($last_measured_x, $x); $DEBUG && print
"INTERPOLATE line $ng from \@Delay_NG with non-measured_x\n";
$DEBUG && print "LIPL x1 = $last_OK_counter y1 =
$last_measured_x, x2 = $counter y2 = $x\n"; @delayed_ng = split
(/,/, $ng); # split data by comma from the input line
$delayed_counter = $delayed_ng[2]; my $delayed_x =
linear_interpolate($delayed_counter, \@inspections, \@positions);
$DEBUG && print "LIPL $delayed_x for $delayed_counter, from
@inspections, @positions\n"; if (defined ($previous_x)){ $delta =
$delayed_x - $previous_x; printf SWIMLOG "%d\t%6.1f\t%6.2f\t*\n",
$delayed_counter, $delayed_x, $delta; printf
"%d\t%6.1f\t%6.2f\t*\n", $delayed_counter, $delayed_x, $delta;
}else{ printf SWIMLOG "%d\t%6.1f\t*\n", $delayed_counter,
$delayed_x; printf "%d\t%6.1f\t*\n", $delayed_counter, $delayed_x;
} $previous_x = $delayed_x; } @Delay_NG = ( ); $DEBUG &&
print "Swim Is in Progress processing $line, increment \$OK\n";
$delta = $x - $previous_x; # printf SWIMLOG "%d\t%6.1f\t%6.2f\n",
$counter, $x, $delta; # printf "%d\t%6.1f\t%6.2f\n", $counter, $x,
$delta; printf SWIMLOG "%d\t%6.1f\t%6.2f\t", $counter, $x, $delta;
printf "%d\t%6.1f\t%6.2f\t", $counter, $x, $delta; if ($BEEPED ==
0){ print "beep was not sent yet \$BEEPED == $BEEPED\n"; if
(abs($delta) >= 5){ # beep (559.255, 100); beep (859.255, 250);
print "BEEP SLOW"; print SWIMLOG "BEEP SLOW"; # print "vvvvvvvv
$swimlog SLOW vvvvvvvv\n"; $BEEPED = 5; } }else{ print "beep should
be quiet for $BEEPED inspections\n"; $BEEPED--; } printf SWIMLOG
"\n"; printf "\n"; $previous_x = $x; $OK++; $last_measured_x = $x;
$last_OK_counter = $counter; }else{# Swim In Progress IS NOT TRUE
$DEBUG && print "Swim Is NOT in Progress yet\n"; $DEBUG
&& print "push $line\nfor later processing\n"; push
@Delay_OK, $line; if (@Delay_OK < $T_OK){ # Delay_OK is small we
have to wait before starting a new swim $DEBUG && print
"waiting to start a swim\n"; $Delay_OK++; }else{# $Delay_OK >=
$T_OK we have enough OKs we need to start a new swim
$Swim_number++; print "*************\nstarting a new swim
$Swim_number\n"; $swimlog = "swim_" . $Swim_number . ".log"; print
"NEW swim log\t$swimlog\n"; $BEEPED = $FALSE; # same as 0 open
(SWIMLOG, ">$swimlog") or die "can't open $swimlog for
appending: $!"; printf SWIMLOG "Number\tPixels\tDelta \tINTPLD\n";
printf "Number\tPixels\tDelta \tINTPLD\n"; $DEBUG && print
"process lines from @Delay_OK stack\n"; foreach my $ok (@Delay_OK){
$DEBUG && print "$ok\n"; my @inspection_data = split (/,/,
$ok); # split data by comma from the input line my $x =
$inspection_data[0]; my $counter = $inspection_data[2]; if (defined
($previous_x)){ $delta = $x - $previous_x; printf SWIMLOG
"%d\t%6.1f\t%6.2f\n", $counter, $x, $delta; printf
"%d\t%6.1f\t%6.2f\n", $counter, $x, $delta; }else{ printf SWIMLOG
"%d\t%6.1f\n", $counter, $x; printf "%d\t%6.1f\n", $counter, $x; }
$previous_x = $x; } $last_measured_x = $x; $last_OK_counter =
$counter; @Delay_OK = ( ); $SwimInProgress = $TRUE; $IdleInProgress
= $FALSE; } } }elsif(($judgement =~ /\+1$/)){# NG No BLOB if
($IdleInProgress){# Idle series is in progress # print "since No
Swim is in Progress and this new line is NG, clear accidental OKs
from Delay_OK list @Delay_OK\n"; @Delay_OK = ( ); # print "Idle Is
in Progress skipping this line, increment \$NG\n"; $NG++; }else{
$DEBUG && print "Idle Is NOT in Progress yet\n"; $DEBUG
&& print "push $line\nfor later processing\n"; push
@Delay_NG, $line; if (@Delay_NG < $T_NG){# Delay_NG is small We
have to wait before starting new idle print "Wait for before
terminating the swim\n"; $DEBUG && print "waiting to start
a idle\n"; $Delay_NG++; }else{# $Delay_NG >= $T_NG. We have to
terminate this swim and start a new idle $Idle_number++; $DEBUG
&& print "*************\nstarting a new idle
$Idle_number\n"; print "closing log $swimlog\n"; close SWIMLOG;
$previous_x = undef; $DEBUG && print "process lines from
@Delay_NG stack\n"; @Delay_NG = ( ); $IdleInProgress = $TRUE;
$SwimInProgress = $FALSE; } } }else{ $DEBUG && print
"$line\n$judgement is neither +0 nor +1\n"; }
[0077] The foregoing Data Processing sub-system 14, and the
processing program or software, can be implemented as part of the
controller, such as if the controller 22 is a computer or a device
with a data processor. The Data Processing sub-system 14 can also
be implemented separately on a different device or within a
component of the Machine Vision 12 sub-system that has a data
processor.
[0078] A computer program product can be devised for operating the
NATATOMETRY system 10 and for processing the date from the Machine
Vision sub-system 12. The product can include one or more
computer-readable storage media having stored thereon
computer-executable instructions that, when executed by one or more
processors of a computing system, cause the computing system to
perform the methods and functions of the NATATOMETRY system 10
disclosed and described herein
[0079] Signal Delivery Sub-System Details
[0080] The Signal Delivery sub-system 16 receives the signals from
the Data Processing sub-system 14 and uses the on-deck FM
transmitter and wearable waterproof FM receiver to deliver alarm
signals to the swimmer and, optionally, the coach. A separate FM
channel can be used for each swimming lane. One FM channel can be
allocated to a swimmer in each swimming lane. In one example, the
Signal Delivery sub-system 16 can be implemented as a Sennheiser EW
300 IEM G3, in-ear monitoring system with transmitter and receiver
tuned onto one of the FM channels available in the Ultra High
Frequency UF spectrum. In one example, the FM receiver can be
placed in a water proof case (Sporti Water proof MP3 Player Case
for example, see
https://www.swimoutlet.com/ProductDetails.asp?ProductCode=32813&CartID).
In one example, the FM receiver in the water proof case can be
attached to the waist belt or the target marker 18 belt worn by the
swimmer. In one example, bone conduction head phones (Audio Bone
1.0 for example, see http://www.audioboneheadphones.com/) can be
worn by the swimmer as the ear piece to provide comfortable volume
of audio alarm.
[0081] Novelty of the Disclosed NATATOMETRY System
[0082] Novelty of the disclosed NATATOMETRY system as compared to
U.S. Pat. No. 5,767,417 lies at least in the use of a Machine
Vision sub-system 12 to collect real time data about a swimmer's
location. Machine vision is utilized in the disclosed NATATOMETRY
system as a new and useful improvement over that disclosed in the
'417 patent due to its higher spatial and temporal resolution.
[0083] Novelty of the disclosed NATATOMETRY system as compared to
US Published Application 2010/0030482 lies at least in the use of a
Machine Vision sub-system 12 to collect real time data about a
swimmer's location. A machine vision system does not have the
measurement drift of an accelerometer-based system as disclosed in
the '482 publication.
[0084] Novelty of the disclosed NATATOMETRY system as compared to
the real time heart rate monitors (stroke counters, distance per
stroke monitors) lies at least in that the feedback is given to the
swimmer based on an analysis of instantaneous swimming speed, not
heart rate (stroke counters, distance per stroke monitors).
[0085] Novelty of the disclosed NATATOMETRY system as compared to
the AVIDA Swim Training System lies at least in the duration of the
real time measurement cycle. The duration of AVIDA Swim Training
System measurement cycle is one lap of swimming, which takes 9
seconds or more at a world record pace. That prior art system
averages speed per lap and does not show fluctuations of swimming
speed occurring during or within any particular lap. The disclosed
NATATOMETRY system is capable of providing duration of speed
measurement cycles of 100 milliseconds or less and is capable of
providing feedback instantaneously to the swimmer during individual
stroke cycles.
[0086] Novelty of the disclosed NATATOMETRY system as compared to
known methods of video analysis in swimming lies at least in that
the feedback signals are delivered to the swimmer and/or coach in
real time as defined herein.
[0087] In one example, the disclosed NATATOMETRY system can provide
a swimmer with real time inter-cycle or inter-stroke feedback
regarding their swimming speed. In one example, the disclosed
NATATOMETRY system can provide an audio signal to the swimmer
during a swim that is relevant to their speed. In one example, the
disclosed NATATOMETRY system utilizes a machine vision device that
can detect the location of a swimmer over a length of a pool. In
one example, the disclosed NATATOMETRY system includes a plurality
of visible lights worn by the swimmer that are detected by a
machine vision device while in the pool. In one example, the
disclosed NATATOMETRY system can detect and measure a swimmer's
speed many times during a single stroke cycle and provide real time
feedback of same to the swimmer. These and other objects, features,
and advantages of the present invention will become apparent to
those having ordinary skill in the art upon reading this
disclosure.
[0088] The Machine Vision sub-system can be programmable to
accommodate for different water depths and can be designed to be
configured on site. The Machine Vision sub-system can also be
configured to accommodate for the number of pixels in a frame. One
pixel at the center of the unit will represent a different distance
than one pixel at the edge of the field of view. Thus, as the
swimmer swims across the field of view of the unit, the unit must
be configured to compensate. A swimmer may move 70 pixels for one
data point and 75 pixels for the next date point, but be swimming
at the same speed. The unit can be configurable to accommodate.
[0089] The NATATOMETRY system can be configured to automatically
and instantaneously download all data points to the Machine Vision
sub-system controller and to the Data Processing sub-system.
Alternatively or in addition, the controller and/or the Data
Processing sub-system can store the data for later analysis. The
goal of the disclosed NATATOMETRY system is to produce real time,
accurate speed data and deliver useful information to the swimmer
in the midst of a training session relating to such real time
data.
[0090] FIG. 2 illustrates one representative example of a plot of
average speed vs. actual real time speed is shown. As can be seen,
an audible signal is delivered to the swimmer each time their
velocity or speed is either too high or too low compared to the
high and low threshold speeds.
[0091] The natatometrical data collected by the disclosed system
are sequential and obtained in constant time intervals. Depending
on the training objectives, various signals can be derived from raw
natatometrical data. Any number of results and outcomes can be
derived from the disclosed NATATOMETRY system and delivered to the
swimmer and/or coach. These can be monitored for accuracy using any
known quality control methods, such as, for example, checking a) a
threshold of minimal instant speed per period of time, b) a
threshold of maximum instant speed per period of time, c) a running
range of instant speed per period of time, or d) a running standard
deviation of instant speed per period of time, or performing a
regression analysis, i.e., slope of the linear regression and/or
performing an autocorrelation analysis to determine stroke cycle
periods.
[0092] In another example, the Machine Vision sub-system FM signals
disclosed above can be replaced using radio frequency
identification (RFID). RFID receivers can be placed along the lane
ropes. The swimmer can wear or carry an RFID tag or mark. As they
swim by the RFID receivers, the receivers can provide information
about their current location for the particular moment in time.
Then the system can derive the speed and communicate it to the
swimmer like in the earlier described NATATOMETRY system.
[0093] Although certain real time swim training systems and methods
have been described herein in accordance with the teachings of the
present disclosure, the scope of coverage of this patent is not
limited thereto. On the contrary, this patent covers all
embodiments of the teachings of the disclosure that fairly fall
within the scope of permissible equivalents.
* * * * *
References