U.S. patent application number 14/465530 was filed with the patent office on 2015-02-26 for swing with imu feedback of back swing, contact point, and follow through.
The applicant listed for this patent is Sergey Feingold, Lavie Sak. Invention is credited to Sergey Feingold, Lavie Sak.
Application Number | 20150057112 14/465530 |
Document ID | / |
Family ID | 52480881 |
Filed Date | 2015-02-26 |
United States Patent
Application |
20150057112 |
Kind Code |
A1 |
Sak; Lavie ; et al. |
February 26, 2015 |
Swing with IMU Feedback of Back Swing, Contact Point, and Follow
Through
Abstract
An apparatus, method, and system are herein disclosed for
improving an athlete's swing by detecting three phases of a
swing--the backswing, the lead up, and the follow through--and
determining whether a ball impact or highest velocity occurred
between the lead up and follow through using an inertial
measurement unit device affixed to a piece of sports equipment
without encumbering the athlete's equipment by substantially
changing the weight, balance, or structure of the equipment.
Inventors: |
Sak; Lavie; (Sterling,
VA) ; Feingold; Sergey; (Atlanta, GA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Sak; Lavie
Feingold; Sergey |
Sterling
Atlanta |
VA
GA |
US
US |
|
|
Family ID: |
52480881 |
Appl. No.: |
14/465530 |
Filed: |
August 21, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61959320 |
Aug 21, 2013 |
|
|
|
Current U.S.
Class: |
473/461 |
Current CPC
Class: |
A63B 2220/40 20130101;
A63B 69/38 20130101; A63B 2220/30 20130101; A63B 2071/0625
20130101; A63B 60/54 20151001; A63B 2024/0068 20130101; A63B 49/00
20130101; A63B 2024/0015 20130101; A63B 24/0003 20130101; A63B
2220/89 20130101; A63B 2220/36 20130101; A63B 2220/833 20130101;
A63B 2220/20 20130101; A63B 2220/24 20130101; A63B 2220/18
20130101; A63B 2225/50 20130101; A63B 24/0006 20130101; A63B
2220/801 20130101; A63B 71/0622 20130101; A63B 2024/0071 20130101;
A63B 2220/53 20130101; A63B 2220/34 20130101; G01P 15/00
20130101 |
Class at
Publication: |
473/461 |
International
Class: |
A63B 24/00 20060101
A63B024/00; A63B 71/06 20060101 A63B071/06; A63B 69/38 20060101
A63B069/38 |
Claims
1. An apparatus comprising, at least one inertial measurement unit,
wherein the inertial measurement unit comprises an accelerometer, a
gyroscope, and a magnetometer; a microcontroller operably coupled
to the inertial measurement unit; memory operably coupled to the
microcontroller; wherein the inertial measurement unit,
microcontroller, and memory are operably coupled to identify a
backswing, lead up, and follow through to validate a complete
stroke.
2. The apparatus of claim 1, further comprising a housing adapted
for attachment on a tennis racket with a string bed with a proximal
end and a distal end with mains strings oriented from the proximal
end to the distal end and cross strings oriented transverse to the
mains strings, the housing adapted to attach to the proximal end of
said string bed between the proximal end of the string bed and a
first cross string.
3. The apparatus of claim 2, wherein the housing attaches to at
least two mains strings to dampen vibrations of the tennis
racket.
4. The apparatus of claim 1, further comprising a battery and a
battery charging and protection component operably coupled to the
microcontroller.
5. The apparatus of claim 4, further comprising a piezoelectric
module operably connected to the battery charging and protection
component.
6. The apparatus of claim 1, further comprising an output display
operably coupled to the microcontroller to indicate a ball impact
occurred within the lead up, follow through, or in between the lead
up and follow through.
7. The apparatus of claim 6, wherein the output display comprises
an electronic paper display.
8. The apparatus of claim 6, wherein the output display comprises a
touch screen for user input.
9. The apparatus of claim 1, further comprising an output speaker
operably coupled to the microcontroller to make sounds regarding a
threshold metric relating to backswing, lead up, or follow
through.
10. The apparatus of claim 1, further comprising an interface
operably coupled to the microcontroller for coupling to an external
portable electronic device.
11. The apparatus of claim 10 wherein the interface comprises a
wireless interface.
12. The apparatus of claim 1, further comprising an input operably
coupled to the microcontroller for indicating that a previous set
of raw data comprised a calibrating swing.
13. The apparatus of claim 12, wherein the microcontroller compares
data associated with backswing, lead up, and follow through with
data of the calibrating swing.
14. The apparatus of claim 1 further comprising a movement module
operably coupled to the microcontroller in order to compute player
movement and estimate body position.
15. A method of improving an athlete's stroke, comprising steps of:
using an inertial memory unit, collecting raw data of distance,
angle, acceleration, and speed; using a microcontroller,
identifying backswing, lead up, ball impact, and follow through by
processing the raw data; using an output, indicating the ball
impact occurred within the lead up, follow through, or in between
the lead up and follow through; wherein the method is performed by
one or more computing devices.
16. The method of claim 15, comprising an additional step of
comparing backswing, lead up, and follow through with a calibrated
stroke.
17. The method of claim 16, further comprising validating ball
impact by removing acceleration raw data below a threshold value
associated with taps.
18. The method of claim 15, comprising an additional step of
transmitting values to an external device.
19. The method of claim 15, wherein identifying a backswing
comprises identifying a change in direction before a threshold
increase in acceleration.
20. The method of claim 19, wherein the threshold increase in
acceleration varies with a change in angle from the change in
direction.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS; BENEFIT CLAIM
[0001] This application claims the benefit of Provisional Appln.
61/959,320, filed Aug. 21, 2013, the entire contents of which is
hereby incorporated by reference as if fully set forth herein,
under 35 U.S.C. .sctn.119(e).
FIELD OF THE INVENTION
[0002] The present invention relates to inertial measurement
feedback techniques for sports equipment and, more specifically, to
improving a swing with IMU feedback of back swing, lead up, and
follow through.
BACKGROUND
[0003] Given that MEMS devices are becoming more precise and
accurate, new devices are needed to help sports equipment users
increase the quality of their game. Existing devices are designed
to examine one quantitative metric associated with a swing, and
these devices are extremely over inclusive as to what motions count
as a valid shot. For example, a MEMS device may be attached to the
shaft of a golf club to record a speed associated with a golf
swing. After a person moves the club, the device may record a
speed, but there is no indication of whether that speed was
actually associated with a swing for hitting a golf ball or simply
a practice swing. A swing may be recorded even when a user taps
their sports equipment on their shoe or bounces a ball on the
sports equipment with no intention of hitting the ball as if they
were in a game.
[0004] Determining whether a set of recordings of the MEMS device
can be associated with a swing is called stroke validation. Stroke
validation is usually confined to the quantitative scope of the
metric being measured. Thus, a threshold value of speed may define
a valid stroke for the golf club, but the speed is actually the
speed of the club at the shaft location, and has no connection to
direction or ball impact. The metric defines the max speed of a
location on the shaft of the golf club, not whether the speed was
used to effect a quality stroke.
[0005] To add some qualitative analysis, the user may set up video
equipment to record each swing. The video is later analyzed by
choosing specific swings in the video, slowing down those parts,
and overlaying the video with videos of other players or
professional models. These features allow both a player and coach
to see visual information that can facilitate learning.
[0006] Unfortunately, much of the benefit of practice involves
muscle memory. Connecting the after practice analysis of whether a
swing was a quality swing with the muscle memory associated with
making the swing is very difficult. Developing muscle memory often
requires repetition immediately after a quality swing has been
made.
[0007] Additional setbacks with the above described approaches
include setup time and bulkiness of having multiple apparatus
necessary to complete the task. A user trying to use these devices
may actually be thrown off their game because the equipment adds
weight and offsets balance of the sports equipment. A user is less
likely to swing naturally if the user has to set up the apparatus
in a particular manner, start a recording, swing, transfer the data
to a computer, and then analyze the data on a computer. The time
associated with setup makes the user less relaxed and ready to
perform a natural, effective swing.
[0008] Other setbacks associated with the above described
approaches include hardware implementation. Due to the nature of a
MEMs device being used with sports equipment, the device must be
housed mechanically in a housing that won't fall off the sports
equipment, slide around the equipment, or rattle around while using
the sports equipment. To combat these problems, the devices often
require connection to a computer to gain feedback of a stroke or
swing after a significant amount of time has passed.
[0009] The approaches described in this section are approaches that
could be pursued, but not necessarily approaches that have been
previously conceived or pursued. Therefore, unless otherwise
indicated, it should not be assumed that any of the approaches
described in this section qualify as prior art merely by virtue of
their inclusion in this section.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In the drawings:
[0011] FIG. 1 is a block diagram illustrating a system architecture
of an IMU device.
[0012] FIG. 2 is a schematic diagram illustrating an IMU device
performing as a dampener on a tennis racket.
[0013] FIG. 3 is a flowchart illustrating steps for transforming
raw data of a swing into valid, qualitative feedback.
[0014] FIG. 4A is schematic template illustrating a user display on
an IMU device.
[0015] FIG. 4B comprises a display interface as an example for
explaining embodiments of the present invention.
DETAILED DESCRIPTION
[0016] In the following description, for the purposes of
explanation, numerous specific details are set forth in order to
provide a thorough understanding of the present invention. It will
be apparent, however, that the present invention may be practiced
without these specific details. In other instances, well-known
structures and devices are shown in block diagram form in order to
avoid unnecessarily obscuring the present invention.
[0017] Embodiments are described herein according to the following
outline:
[0018] 1.0 General Overview
[0019] 2.0 Mechanical Hardware
[0020] 3.0 Electrical Hardware
[0021] 4.0 Stroke Analysis
[0022] 5.0 User Interface
[0023] 6.0 External Networking
[0024] 1.0 General Overview
[0025] In some embodiments, an IMU is used to detect a quality
swing by detecting three phases of a swing: the backswing, the lead
up, and the follow through. By associating expected values for
metrics associated with each phase of the swing, false positives
are significantly reduced. A swing is identified by recognition of
a sequence of quantitative metrics associated with the three
phases.
[0026] For example, after a threshold velocity is reached, the
preceding accelerometer data is reviewed to determine if the
threshold velocity was preceded by a slower velocity in the
opposite direction to detect a back swing. The angle of the
backswing at its farthest point back (a zero velocity in one
direction) is then compared with an acceleration that varies with
the change in angle to an expected ball contact point to detect a
lead up. The angle at this point is then compared with a
deceleration that varies with the change in angle to detect a
follow through.
[0027] In some embodiments, the vertical centerline of the body is
associated with the vertical orientation of the sports equipment
mounted with an IMU, and a horizontal plane through the shoulder is
associated with the horizontal orientation of the sports equipment
mounted with an IMU. Thus, a lead up can be further validated by
detecting whether the swing passes through the vertical centerline
of the body, and a follow through can be further validated by
detecting whether the swing ends above the shoulder.
[0028] In some embodiments, instant feedback is provided as to
whether the actual point of impact coincides with the expected
point of impact. The actual point of impact may be behind the
expected point of impact in the lead up phase. The actual point of
impact may be after the expected point of impact in the follow
through phase. Comparison of this expected point of impact with the
actual point of impact provides quality analysis to a swing or
stroke.
[0029] 2.0 Mechanical Hardware
[0030] In some embodiments, mechanical hardware for the IMU device
includes housing and a hinge mechanism. In some embodiments, the
housing comprises two pieces surrounding the IMU device. The
housing is preferably constructed of aluminum because of its high
strength to weight ratio. Thus, the walls of the housing may be
thin and still provide protection for the inner electronic
components. Other embodiments of the two housing pieces include,
but are not limited to, plastics, rubbers, composites, and other
metals. The two pieces may be attached around a piece of sports
equipment, such that some of the electrical components are in one
piece and other electrical components are in the other piece. As an
example, the two pieces may be attached to each other around the
strings of a tennis racket.
[0031] In some embodiments, the two pieces are attached with a
hinge mechanism. The hinge mechanism provides for a secure
connection on one side of the two pieces, so a mechanical fastener
on the other side of the two pieces provides the only separation
point. The mechanical faster may include a snap, slide, bolt,
rivet, clip, clamp, Velcro, insert or other mechanical fastener
type known in the art. There is a clearance around the hinge for a
printed circuit board (PCB) cable to pass from one piece to the
piece. In this manner, the electrical components of the IMU device
may separate onto two or more PCBs.
[0032] In some embodiments, the IMU latches onto sport equipment
through either a twist latch or slide latch. The twist latch
involves the front piece inserted into the back pieces or vice
versa. The front piece rotates until it locks into place, creating
a secure physical and electrical connection. The slide latch
involves the front piece sliding into the back piece, similar to a
camera tripod latching system. The front piece may also have
additional inserts or bends to lock into place, creating a secure
physical and electrical connection.
[0033] In still other embodiments, the housing comprises a one
piece housing with a clamp or grip adapted for attaching to a piece
of sports equipment. For example, the housing may have a grip fro
stings of a tennis racket.
[0034] In some embodiments, the housing functions as a part of the
sports equipment. FIG. 2 is a schematic diagram illustrating an IMU
affixed to a tennis racket depicting the location of a combined
enclosure apparatus 203 that is latched onto a tennis racket string
bed 201. The apparatus 203 resides within the middle six vertical
strings, also known as the "mains" and below (not touching) the
bottom most horizontal string, also known as the "cross". This is
the same area a vibration dampener is typically affixed. The front
unit 103 and the back unit 123, when operably attached, form the
apparatus 203.
[0035] In some embodiments, the apparatus 203 latches onto the
tennis racket string bed 201 through either a twist latch or slide
latch. The twist latch involves the front unit 103 inserted into
the back unit. The front unit 103 rotates until it locks into
place, creating a secure physical and electrical connection. The
slide latch involves the front unit 103 sliding into the back unit
123, similar to a camera tripod latching system. The front unit 103
locks into place, creating a secure physical and electrical
connection. The device only powers on when the two components of
the IMU housing are locked and connected.
[0036] The dampener may be made of metal, impact-resistant plastic,
and a rubber or rubber-like material (i.e. silicone). The dampener
may consist of a mixture of these materials to achieve the desired
feel, weight, appearance, and durability. The dampener may be a
single monolithic piece, or may consist of multiple pieces that
attach to one another.
[0037] 3.0 Electrical Hardware
[0038] In some embodiments, electrical systems are provided for
computing stroke quality, speed, stroke count, stroke type, ball
spin, ball impact, player movement, effective play time with an
inertial measurement unit (IMU) device affixed to a piece of sports
equipment without encumbering the athlete's equipment by
substantially changing the weight, balance, or structure of the
equipment.
[0039] In some embodiments, inertial measurement unit (IMU) device
comprises a MEMS (microelectromechanical systems) device preferably
containing a 3-axis accelerometer, 3-axis gyroscope, and 3-axis
magnetometer operably connected to a microcontroller responsible
for processing the raw data of the device, onboard memory for
storing data, and a battery.
[0040] An inertial measurement unit (IMU) is an electronic
component that measures and reports on velocity, orientation, and
gravitational forces, using a combination of accelerometers,
gyroscopes or angular rate sensors, and magnetometers. These
systems are more accurate than single micro-electromechanical
sensors (MEMS) devices because the sensors may be used in
conjunction to correct for bias.
[0041] In some embodiments, the IMU measures vibrations from the
string bed and sends this additional data to the microcontroller.
Impact data from the IMU provides another level of abstraction that
can be used in conjunction with racket head speed at the time of
impact, stroke quality metrics, stroke count, stroke type, and ball
spin to provide feedback to a user.
[0042] FIG. 1 is an IMU device 101 comprising a front unit 103, a
back unit 123, an external movement module 125, and an external
device 137. The front unit 103 houses direct user inputs 105, a
display 107, and a speaker 109, an inertial measurement unit 111, a
piezoelectric sensor module 113, an internal memory unit 115, a
wireless connectivity unit 117, and a microcontroller 119. The back
unit 123 houses a charge and data transfer node 133, a battery 127,
an external memory module 129, and a power harvesting unit 131. The
charge and data transfer node permits a USB connection 135 to an
external device 137, such as a smartphone, computer, or tablet,
while also controller the voltage entering into the battery for
charging.
[0043] In some embodiments, the two-piece apparatus contains two
mechanically and electronically linked components, a front unit 103
and a back unit 123. The apparatus powers on and operates when both
components are operably attached, that is, the two components are
securely attached via mechanical means. Both the front unit 103 and
back unit 123 contain electronic connection contacts. When the
front unit 103 and the back unit 123 are securely attached via
mechanical means, there is an electrical connection 121 between the
two components. This connection 121 supplies power from the
rechargeable battery 127, within the back unit 123.
[0044] The rechargeable battery 127 may have a lithium-based
chemistry, such as lithium-polymer or lithium iron phosphate.
Preferably, it is of a single cell construction with a nominal
voltage of approximately 3.7V, and a capacity in the range of
50-800 mAh. It is integrated into the back unit 123 but may be
designed to facilitate simple user replacement at end of life. In
order to charge the battery 127, there is charging and protection
circuitry within the charge and data transfer node 133. The
function of this circuitry is to provide the correct voltages and
currents to the battery 127 during charging to ensure safe
operation. This charging and protection circuitry also monitors the
battery 127 voltage level and state-of-charge of the battery 127 to
prevent an over-charge, over-voltage, or under-voltage condition.
This is critical to the safe integration of lithium-based battery
technology. The circuitry may report battery voltage and
state-of-charge to the microcontroller 119. This component may
accept a charge via a USB connection 135 inserted into the device,
via a proprietary or industry-standard power port, and/or via an
inductive charging interface. Any of these methodologies may be
used in any combination and are not mutually exclusive.
[0045] Embodiments of this component accept charge via a female USB
port, a female micro USB port, proprietary or industry-standard
power ports, or an inductive charging interface. Embodiments of
this component may also be set up to accept charge from multiple
sources. Any of these methodologies may be used in any combination
and are not mutually exclusive. The rechargeable battery 127 is
also charged via the power harvesting unit 131. The power
harvesting unit 131 may use piezoelectric actuators, solar cells,
mechanical generators, or any combination of these to generate
energy from the motion of the device and/or ambient conditions.
This energy is sent to the charge and data transfer node 133 where
it is conditioned so that it may be used to charge the battery 127.
This extends the battery life of the device to enable less frequent
charging by the user. Another aspect of the current invention
resides in a piezoelectric energy harvesting module. To extend the
life of the battery before a re-charge is required, the apparatus
may contain an energy harvesting module that utilizes the
piezoelectric effect. The module leverages an onboard piezoelectric
component to harvest energy captured from the vibration of the
strings upon any external contact with the racket string bed.
[0046] When piezoelectric actuators are used, the piezoelectric
effect is leveraged. The Piezoelectric effect is the ability of
certain crystalline materials to generate electricity in response
to being physically stressed, such as from deformation or
vibration. Multiple layers of piezoelectric materials can be
stacked to amplify the voltage and current generated by a given
stress. Energy is harvested by using a multi-layered piezoelectric
material mounted in a cantilever style (one end is free to move).
Upon encountering a shock, as when a player strikes a tennis ball
with the racket, part of the energy of the impact is absorbed by
the piezoelectric material which undergoes an initial deformation
in response to the movement, and afterwards vibrates at its
resonant frequency for a short period of time. The material may
include a small weight on the end opposite the mounting point. The
purpose of this weight is to both amplify the movement of the
piezoelectric material and to prolong the period for which it
vibrates. This serves to increase the amount of energy that can be
harvested after each impact.
[0047] Such vibration of the piezoelectric material generates an AC
voltage. This voltage is the input to a specialized component that
rectifies and conditions the voltage. The output of this component
is a low (approximately 0-5 V) DC voltage that is fed to a
capacitor. The purpose of the capacitor is to accumulate energy
after each impact until it is of a sufficient amount to be
transferred to the battery efficiently. Once the capacitor is full,
it is connected to the battery charging circuitry, which transfers
this energy from the capacitor to the battery. The capacitor then
resumes accumulating energy until the next cycle. Piezoelectric
elements used for power harvesting may be the same unit(s) used for
data capture purposes.
[0048] In some embodiments, the USB connection 135 allows the
external memory 129 to transfer data to and from an external device
137. Calculated data is sent to an external device 137, such as a
smartphone, tablet, or computer for aggregation, organization, and
sharing. The external device 137 can send data in the form of
system configuration and firmware updates to the external memory
129, within the back unit 123.
[0049] The connection 121 between the front unit 103 and the back
unit 123 also allows a two-way data transfer between the external
memory 129, within the back unit 123, to the microcontroller 119
within the front unit 103. The data transferred from the
microcontroller 119 to the external memory 129 are calculated
metrics ready for output to the user. The external memory 129 is
distinct from the internal memory 115 in that either the user or
the microcontroller 119 can access it, whereas the internal memory
115 is only used for intra-device storage and cannot be directly
accessed by the user.
[0050] The external memory 129 may be of an EEPROM, Flash, or other
non-volatile memory architecture. It may be used to store data that
must be preserved during power cycles or no power conditions, such
as user configuration data. It may further be used to store output
data from the apparatus that can be accessed by the user, either
directly or through an external device 137. The data may be
accessed via any standard wired or wireless communications
protocol, including but not limited to USB, Bluetooth, WiFi,
Zigbee, ANT, or other protocol. This data access may be performed
via the charge and data transfer node 133 or the wireless
connectivity unit 117.
[0051] The internal memory 115 may be EEPROM, Flash, or another
volatile or non-volatile memory. The internal memory 115 may
consist of more than one discrete component, and may use any
combination of memory architectures. It is used to store output
data from the device, as well as any initialization code that is
required by the device upon startup. It may be accessed directly by
the microcontroller 119, or by an external device 137 via a wired
or wireless communications protocol such as USB, Bluetooth, WiFi,
Zigbee, ANT, or other protocol. The raw data may also be
transmitted to an external device 137 via the wireless connectivity
unit 117, the USB connection 135, or both in order to be processed,
displayed, and/or stored on the external device 137.
[0052] In some embodiment, the wireless connectivity unit 117
provides an interface to transmit and receive data from an external
device 137. The wireless connectivity unit 117 may use any industry
standard wireless communication protocol and/or a proprietary
wireless protocol. It may support multiple wireless protocols. This
device may contain one or multiple components used for transferring
data over wireless protocols. These may include Wireless USB,
Bluetooth, WiFi, Zigbee, ANT, RFID, NFC, or a short-range radio
link of another type. The device may utilize one or multiple such
protocols.
[0053] The data is delivered directly to the user via an onboard
display 107 or to an external device 137. The data is captured from
the IMU 111 and a piezoelectric vibration sensor 113. There is also
an optional movement module 125 that can provide a different set of
data. The IMU 111, piezoelectric vibration sensor 113, and movement
module 125 deliver raw data to the microcontroller 119.
[0054] The IMU 111 is responsible for measuring the movement of the
device. It is a MEMS (microelectromechanical systems) device
containing a 3-axis accelerometer, 3-axis gyroscope, and 3-axis
magnetometer, in any combination. It measures different parameters:
acceleration, rotation, and magnetic field strength. It measures
each of these parameters on three orthogonal axes. The measurement
functions may be done by a single component or by multiple
components. Accelerations are measured by the accelerometer and
integrated over a period of time to compute velocity, and this may
also be integrated over a period of time to compute the path of the
device through space. Rotations are measured by the gyroscope and
are used to measure the attitude--or orientation--of the racket in
3D space. Magnetic field strength is measured by the magnetometer
and is used to supplement gyroscope readings to improve accuracy.
All 3 sensors--accelerometer, gyroscope, and magnetometer, may be
used in conjunction to correct inaccuracies and noise and provide a
more accurate measurement of the movement and orientation of the
racket.
[0055] The piezoelectric vibration sensor 113 contains a single
piezoelectric element or multiple piezoelectric elements that may
be used as a sensor to detect vibration patterns present in the
racket or strings for a period of time after an impact. This
information may be used to analyze vibration patterns to determine
if a valid shot has occurred and/or approximately where on the
racket string bed the impact occurred. This sensor may be in the
form of a MEMs device package, wherein such a device contains one
or multiple piezoelectric elements, or it may consist of one or
more discrete piezoelectric elements that are bare (i.e. not
contained in a package or integrated with any interface circuitry).
Multiple piezoelectric elements may be used to detect vibrations on
up to 3 orthogonal axes. Multiple piezoelectric elements with
distinct resonance frequencies may also be used to detect different
vibration frequencies. Piezoelectric elements used for data capture
purposes may be the same unit(s) used for power harvesting 131. In
some embodiments, the IMU replaces the peizoelectric elements in
functionality.
[0056] The movement module 125 is an optional accessory that
provides a different set of raw data to the microcontroller 119.
This data set, after calculation, creates metrics detailing the
movement of a tennis player as opposed to metrics dealing with a
tennis player's tennis stroke. The module may contain an IMU and/or
a GPS receiver. The GPS receiver may be integrated into the device
to allow reception and processing of standard civilian GPS signals.
The IMU may be used for "dead reckoning," a method by which motion
is tracked over a period of time using no external references. This
module is to be used for tracking the movement of the device and
player during a session. This data may be merged with any other
sensor data or calculated data to provide additional metrics to the
end user. The GPS receiver module may contain an integrated
antenna, or may be connected to an antenna located elsewhere within
the device. All data output by the IMU/GPS module transmits to the
microcontroller 119. GPS module data output may consist of raw GPS
signal data or processed GPS data in a standard format. In the
former case all processing of raw GPS signal streams is done
onboard the microcontroller; in the latter case all such processing
is done on the GPS receiver module itself. The transmitted IMU data
may be raw unprocessed data, or data processed by an onboard
microcontroller inside the movement module 125 in order to reduce
the burden on the microcontroller 119.
[0057] The microcontroller 119 handles the main data processing
tasks, intra-device communication, and all user input/output
functions. It handles most or all processing and
communications-related tasks between components of the device.
Specific functions may be handled by specialized components. The
microcontroller 119 is optimized for low-power operation. The IMU
111 and the piezoelectric vibration sensor 113 may also perform
some data processing functions in order to reduce the burden on the
microcontroller. In some embodiments there are a plurality of
processors that process data in parallel.
[0058] Another aspect of the current invention resides in user
input. Embodiments of the invention comprise buttons, knobs,
wheels, slides, joysticks, and switches that the user can use to
access various measurements and outputs. These inputs may be
external or virtual through a capacitive or resistive touch screen
to enable touch-based and/or gesture based input. An inertial
measurement unit also detects and recognizes user input such as a
"taps" or a motion based gestures. Any of these input methodologies
can be used in any combination, and are not mutually exclusive.
[0059] Button inputs 105 may include tactile buttons that the user
can push to access various functions, capacitive or resistive touch
screen to enable touch-based and/or gesture based input, or a touch
screen that is placed directly over the display or directly over
the protective display cover. An external device 137, such as a
connected smartphone, tablet, or computer allows for system
configuration and firmware updates. The device may also use the IMU
to detect and recognize user input such as a tap or a motion based
gesture. Any of these input methodologies may be used in any
combination and are not mutually exclusive.
[0060] In some embodiments, a display on the IMU device provides
instant feedback of quality metrics of a swing. As the device is
used mainly outdoors and in the sunlight, the screen needs high
contrast and visibility, without glare. The display may comprise
and electronic paper display that is low power, durable, and
robust, such that it can stand up to the shock and vibration
produce by impacts to the string bed.
[0061] In other embodiments, the display comprises an organic light
emitting diode (OLED) display, LCD (liquid-crystal display), or
PNLC (polymer networked liquid crystal) display. In these
embodiments, the miniaturization and manufacturing of this
technology provides for ease of replacement and customization.
[0062] Visual output may be in the form of on an onboard display
107 and audible output may be in the form of an onboard speaker
109. A transparent cover built into the housing protects the
display 107. This cover may be composed of high-strength glass or
an impact-resistant plastic, such as polycarbonate. The display 107
may also be encapsulated, or "potted," in a plastic or rubber-like
epoxy to protect it from shock and impact. This encapsulation may
be complete or partial. The display 107 may also have a backlight
or a direct illumination source for viewing under low-light
conditions. The speaker 109 may use an electromagnetic actuator or
a piezoelectric actuator. The speaker 109 may be voice coil or
piezo actuated. It may produce a constant or variable tone upon
application of a constant DC voltage, or it may accept an arbitrary
waveform to reproduce a full or partial range of audible
frequencies.
[0063] In some embodiments, one or more modules or components may
be used in different combinations in order to conserve space. For
example, the power harvesting unit 131 may not be included in an
embodiment to increase room for a battery 127.
[0064] 4.0 Stroke Analysis
[0065] In some embodiments, analysis of stroke quality requires
taking measurements of distance, angle, acceleration, path, and
speed of the stroke and identifying the backswing phase, lead up
phase, and follow through phases, and then validating each phase.
Validation of athlete's backswing requires comparing the associated
measurements with the appropriate length, height, and racket angle.
Validation of a swing's contact point requires comparing whether
the contact point is between 0.5 ft to 2.5 ft (depending on arm
length) in front of the athlete's body. Validation of an athlete's
follow through requires comparing whether the stroke is continuous
and naturally decelerates across the athlete's body.
[0066] Using the above quantitative and qualitative metrics, ball
path data may be calculated. For example, identifying top spin shot
helps predict a ball with top spin and/or side spin. Top spin
causes the ball to follow a trajectory with a smaller radius than a
ball with an equivalent velocity, but no spin. It is often utilized
to keep a ball in the court, hit sharper angles, and apply pressure
to the opponent to hit a higher bouncing ball. Side spin causes the
ball trajectory to curve to the left or to the right following
impact with the racket. A shot with side spin is used to force the
opponent out of position. Identifying a slice shot helps predict a
ball with back spin and/or side spin. Back spin causes the ball to
follow a flatter trajectory over the net and forces the opponent to
hit a lower bouncing ball.
[0067] In addition to the input captured by a user's tennis actions
through various sensors, the user may provide direct input to the
device via the button inputs, an external device, or via gestures
(i.e. tapping, shaking, rotating) that are detected by the IMU. The
direct input can notify the IMU device of that a quality shot has
been achieved, and the user would like alerts or comparisons of
similar data, so the user can try to recreate the shot.
[0068] FIG. 3 is a flow chart of operation steps between nodes in a
system, defining the calculations and transmission of data from
within the local device 301 and an external device 323. The raw
data 303 is captured, buffered 305, and compared to determine
whether the apparatus movement qualifies as a valid stroke or as
"taps" or a motion based gestures, then stored as a valid stroke
307. If a valid stroke is not detected, raw data capture is
repeated 309. Upon detection of a valid stroke 311, the raw data
303 is computed into temporary metrics 313, which are then averaged
or totaled 315, forming session metrics 319. Additionally, upon
valid stroke detection 307, the raw data 303 can be directly
transferred 317 into session metrics 319. The session metrics 319
are then transmitted 321 to the external device 323 via a USB or
wireless connection. The session metrics 319 are stored and
aggregated within the external device 323, forming progressive
metrics 325, i.e. they can be used to interpret a user's progress.
The subset of the progressive metrics 325 are then selected 327 by
a user for sharing information to other users, forming shared
metrics 329.
[0069] Raw data 303 is captured from the various sensory inputs,
that is, the IMU 111, the piezo sensor 113, and the movement module
125. Raw data is acquired continuously, but the data may be
retained on the device only for a particular interval of time with
a first-in-first-out (FIFO) hierarchy. The data comprising this
time interval is known as the buffer 305. Once the buffer 305 is
full, every new data acquisition causes the oldest acquisition in
the buffer 305 to be deleted or overwritten. In this manner the
buffer 305 always contains the most recent data for a given
duration. Physically, the buffer 305 is a block of reserved memory
that can be located on any component with sufficient free memory to
contain it. This may include the microcontroller 119, the internal
memory 115, the external memory 129, or any combination of these
components.
[0070] Once a valid stroke is detected 307, that is when raw data
303 is within an expected threshold velocity, measurements of
distance, acceleration, path, and speed associated with the
movement of a tennis racket during a typical tennis stroke, and
within an expected vibration range after contact between a tennis
ball and tennis racket, it may be computed into metrics based on
measurements from a piezoelectric sensor. An additional redundancy
can be included to eliminate counts made too closely in time by
comparing time since the last valid stroke with an expected
threshold value. All together, restricting acceptable strokes with
in an excepted threshold value allows the device to prevent false
positives and produce more accurate metrics. An example of a
prevented stroke false positive is a tennis player hitting the
racket strings on their feet and hand.
[0071] Upon a valid stroke, raw data 303 is fed into one or a
series of software algorithms that are executed on the
microcontroller 119. Different algorithms are used depending on the
desired output metric. The outputs of these algorithms are the
temporary metrics 313. Temporary metrics 313 are metrics solely on
the most recent valid stroke and are displayed to the user until
the next valid stroke metrics are to be displayed. Temporary
metrics include, but are not limited to, racket head speed, ball
spin, and stroke quality.
[0072] Session metrics 319 are total durations, totals, and
averages of temporary metrics. Session metrics include, but are not
limited to, effective play time, stroke count, average ball spin,
and average racket head speed. Average racket head speed and
average ball spin are respective averages of all valid strokes
within a session.
[0073] Racket head speed defines the quality of a tennis stroke and
its definition is addressed in the background. Racket head speed is
computed and outputted by measuring the velocity with the IMU and
then adjusting with angular velocities to compute racket head
speed. In some embodiments, a racket head speed is calculated for
each phase of a stroke: back swing, lead up, and follow through.
The comparisons of these values can help with quality analysis of a
stroke.
[0074] Ball spin defines the quality of a tennis stroke and its
definition is addressed in the background. Ball spin is computed by
isolating vertical acceleration component and the angle of the
racket head from measurements of distance, angle, acceleration,
path, and speed to calculate the expected rotations per minute on
the ball struck. This is an absolute metric of the number of
rotations per minute of the tennis ball, regardless of the type of
spin (top, back, side).
[0075] Effective play time is computed by identifying a time subset
of the total active duration of playing session when the player is
making dynamic movements to approach an oncoming ball, striking the
ball, and recovering (the reaction and player's body's reset after
finishing a stroke) from a stroke.
[0076] Stroke count is computed by storing the number of tennis
strokes completed. The counter only tracks strokes that are valid
tennis strokes, meaning that they have to meet the criteria of a
backswing, lead up, and follow through. Additionally, they may be
invalidated for shank shots or lack of ball contact. Users have the
option to pause and reset the device. Pausing the device triggers
the device to not calculate towards effective play time, shot
count, average ball spin, and average racket head speed. Resetting
the device sets effective play time to 00:00 (hr:min), stroke count
to 0 shots, average ball spin to 0 rpm, and average racket head
speed to 0 m/s.
[0077] Before the device is turned off or reset the user is given
the option to save the current set of session metrics 319.
Additionally, the user can also continually save the session at any
time. These saved metrics are used saved onto the external memory
129. These metrics can be transmitted 321 to an external device 137
via the wireless connectivity unit 117 or a USB connection 135.
Those metrics, when housed and aggregated on an external device,
become progressive metrics 325. Progressive metrics 325
quantitatively present the development of a tennis player's tennis
stroke and subsequent performance over time. Progressive metrics
325 include, but are not limited to, average racket head speed,
average ball spin, average stroke quality, total and average (per
session) effective play time, total and average (per session)
stroke count, average ball impact, average stroke type, and player
movement.
[0078] Ball impact is computed by comparing expected measurements
for on center, off center, and frame/shank hits with actual
measurements of distance, angle, acceleration, path, and speed. In
particular, the IMU 111 and Piezo sensor 113 detects data on the
vibration of the ball on the strings. The frequency components and
waveform shape of the vibration can be help to differentiate
various types of ball impacts.
[0079] Stroke type is computed by the comparison of expected
measurements for types of swings with the IMU 111 data used to
calculate the general stroke path. For example, a low backswing to
a high follow through or a high backswing to a low follow through
results in two different types of swings. Calculation of path with
measurements of distance, angle, acceleration, path, and speed
along with comparison of expected values defines types of swings,
including but not limited to a slice, topspin, a flat shot, a drop
shot, and a lob. The angle and direction of the racket head during
the back swing and prior to the contact phase of the swing
differentiates from a forehand swing and backhand swing.
[0080] Player movement defined as the distance a tennis player has
moved during a session whether that player is making dynamic
movements to meet, approach, and prepare for an oncoming ball, or
simply moving around a court. Movement is calculated through any or
all of the module inputs and processed by the microcontroller
119.
[0081] An aspect of the invention also includes a movement module
that tracks the location of the player on the court. The movement
module may contain a GPS receiver and an IMU. Player movement may
be tracked using GPS position data in conjunction with IMU data.
The IMU measures the location of the player with "dead reckoning",
a process where in the IMU is used to track absolute motion for a
period of time without any external reference point(s). The motion
that is tracked includes the distance a tennis player moves during
a session whether or not that player is making dynamic movements to
meet, approach, or prepare for an oncoming ball.
[0082] Stroke quality is computed and outputted by comparing
measurements of distance, angle, acceleration, path, and speed, to
identify the phases of the swing or stroke as backswing, contact
point, or follow through and validating measurements associated
with the identified phase. Validation simply requires comparing
actual values with stored expected values for a quality swing. For
example, validation of the player's backswing requires comparison
of expected measurements to the actual length, height, and racket
angle of the athlete's backswing. Validation of the athlete's
contact point with the tennis ball requires comparison of expected
measurements such that the contact point would be 0.5 ft to 1.5 ft
in front of the player's body. Validation of the athlete's follow
through requires comparison of expected measurements where the path
and deceleration is continuous across the athlete's body.
[0083] Consistent improvement of a swing or stroke requires
focusing both on quantitative aspects and qualitative aspects.
Overlooked in many systems is an analysis of stroke quality as well
as a comprehensive analysis of stroke quantities.
[0084] For example, tennis athletes tend to focus on one
quantitative measurement such as racket head speed, stroke count,
or ball spin. Racket head speed is the velocity of the racket head
upon contact with a tennis ball. The average tennis player's racket
head speed is 20-25 m/s. However, optimal racket head speed is
around 25-35 m/s. Stroke count is the number of valid tennis
strokes completed. A high number of repetitions increase muscle
memory, and thus, accuracy and precision of a swing. Ball spin is
the rotational speed of the tennis ball in revolutions per minute
(RPM) after impacting the racket. The more spin a player imparts to
the tennis ball, the more likely the player is to hit the ball into
the court, within bounds. The average tennis player's ball spin is
about 800 RPM-1,200 RPM, while optimal ball spin is around 1,500
RPM-2,500 RPM. While all three of these measurements are important,
many systems only focus on one quantitative measurement. This is an
issue because these metrics are inter-related. For example, when
the racket head speed is not high enough, a struck ball will not
have proper velocity or spin to have an appropriate ball path.
[0085] In some embodiments, the system provides feedback for
improving stroke quality by obtaining an optimal starting position,
path, and ending position of a swing in each of the three distinct
phases of a stroke or swing: the back swing, the lead up, and the
follow through. The backswing is the take back of the racket before
contact with the ball is made. The lead up is the motion until, a
racket connects with a tennis ball at the contact point or expected
contact point, and the follow through is the path the racket takes
after the point of contact with the ball. The quality of the stroke
is affected based on each phases starting and ending position in
relation to the athlete's body.
[0086] Effective qualitative analysis illuminates many stroke
quality issues. Examples of stroke quality issues include, a
backswing being too far past the player's opposite side, or
non-playing side; a backswing being not far enough past the
player's playing side; a contact point with the tennis ball being
too close to the player's body and the arm is either not extended
far enough out or too close to the player's body (player side); and
a path of a tennis stroke that does not elevate up through a shot
after making contact with tennis ball because the player lacks the
upward component of a stroke.
[0087] In some embodiments, quality analysis feedback is tailored
towards a serve. The serve is the most important tennis stroke and
is used to begin each point in a tennis match. It is also the most
easily manipulated stroke, where the tennis player is in full
control of the stroke, as there is no incoming ball struck by an
opponent, but rather the tennis player tosses the tennis ball. The
stroke quality components (backswing, contact point, and follow
through) of a serve are more complex and have more variations than
other tennis strokes. However, because the serve is consistent, an
athlete may give feedback to the device, when an optimal server is
performed. Then the device can give indicators about whether the
phases of other serves match the phases of the optimally defined
server.
[0088] Another qualitative aspect of swing analysis is keeping
track of each ball impact. The contact point between a ball and
racket can be classified as on center, off center, or on the frame
of the racket. An on center ball impact is a shot that is struck
within the "sweet spot" of the racket. The sweet spot is defined as
the area of the string bed where a ball is struck with relative
assurance that the ball responds in a controlled and deliberate
manner, while producing the least strain on an athlete's body. This
area is slightly below the very center of the string bed. An off
center impact occurs when the ball is struck outside the "sweet
spot" of the racket, but without contact to the tennis racket
frame. Finally, a frame or shank shot, is where the ball impact
occurs between the frame of the racket and a tennis ball. To a
tennis player, it is often obvious when the player has any one of
these impact types on an individual stroke. However, improvement
would follow from understanding the number of balls, in total, that
fall within each impact classification after an entire session.
[0089] Although tennis is highly illustrative, IMU device may be
adapted for many specific types of sports equipment, such as the
core of a bowling ball or baseball bat, the head or shaft of a golf
club or hockey stick, or the frame of other pieces of sports
equipment including rackets. Additionally, the unit can be adapted
for a helmet or vehicle attachment in various motorsports such as
Formula One, NASCAR, Rally Racing, Motocross, and Water Surface
Racing. The unit may also be adapted for board sports such as skate
boarding, surfing, snowboarding, wind surfing, skiing, water
skiing, paddle boarding, or kayaking. The unit may also be adapted
for various roller sports such as roller skating, mountain biking,
cycling. The unit may also be adapted for various Olympic sports
such as archery, athletics, badminton, basketball, beach
volleyball, boxing, canoe slalom, canoe sprint, cycling BMX,
cycling mountain bike, cycling road, cycling track, diving,
equestrian/dressage, equestrian/eventing, equestrian/jumping,
fencing, football, golf, gymnastics artistic, gymnastics rhythmic,
handball, hockey, judo, modern pentathlon, rowing, rugby, sailing,
shooting, swimming, synchronized swimming, table tennis, taekwondo,
tennis, trampoline, triathlon, volleyball, water polo,
weightlifting, wrestling freestyle, wrestling Greco-Roman, Alpine
skiing, biathlon, bobsleigh, cross country skiing, curling, figure
skating, freestyle skiing, ice hockey, luge, Nordic combined, short
track speed skating, skeleton, ski jumping, snowboard, and speed
skating. This is not meant to be an exhaustive list, merely an
attempt to give the versatility of the unit.
[0090] 5.0 User Interface
[0091] FIG. 4A is sample digital display partitioning 4A and a
sample digital display 4B, defining the total screen area 401, the
temporary metric display partition 403, the session metric display
partition 405, and the system dashboard display partition 407. The
temporary metric display partition 403 houses the racket head speed
409 (m/s), the ball spin 411 (rpm), and the stroke quality, in the
form whether ball impact occurred in the lead up 413, at the
expected contact point 415, or during the follow through 417. The
session metric display partition 405 houses the effective play time
419 (ept, hours: minutes), the stroke count 421 (shots), the
average ball spin 423 (rpm), and the average racket head speed 425
(m/s). Other metrics, such as, but not limited to ball impact,
stroke type, and player movement, may be later placed within this
display 401 or displayed solely on the display of an external
device 137.
[0092] The system dashboard display partition houses the pause
state 4427, the battery level 429, the connectivity state 431, and
the audible feedback state 433. The pause state 427 (on/off), when
on, triggers the device to not calculate and save towards effective
play time, shot count, average ball spin, and average racket head
speed. When off, the device calculates these metrics as normal. The
battery level 429 icon displays the amount of batter life
remaining. It has three states: 1/3, 2/3 and full capacity. The
connectivity state 432 (on/off), when on, toggles an active
connection between the wireless connectivity unit 117 and the
external device 137. When off, no wireless connection is made. The
volume (on/off), when on, enables the onboard speaker 109. When
off, the onboard speaker 109 is disabled.
[0093] 6.0 External Networking
[0094] In some embodiments, the analysis determined using one or
more of the above described techniques may be uploaded to a
computer and then imported to an application. The data may then be
analyzed through isolating specific swings and comparing/overlaying
the data with other swings of other players or professional models.
These features allow both a player and coach to see undisputable
visual information that can facilitate learning.
[0095] Another aspect of some embodiments is the capability to sync
and transfer data with an external device via Bluetooth or a
standard wireless protocol connection. This connection can be
configured to specifically connect to a smartphone, tablet, or
computer.
[0096] An embodiment resides in a system of collecting data through
an apparatus and aggregating said data for analyses by an
application on a smartphone, tablet, or a computer. The analyzed
data can be stored across a network with multiple nodes for storage
and access. Nodes can be configured for an integrated display
showing multiple metrics, that is, racket head speed, stroke
quality, stroke count, stroke type, ball spin, ball impact, player
movement, effective play time.
[0097] In some embodiments, the system comprises the ability to log
and share metrics and their analyses to parents, other players, and
other coaches, via the Internet (email, cloud applications, and
social media). The data analysis, including annotations and
comments, can be saved to provide a progression over time.
[0098] The user, via various social media avenues or other online
distribution means can share progressive metrics 325. Additionally,
the user has the ability to specify certain subsets of the
progressive metrics 325 to share. These shared metrics 329 include,
but are not limited to, average racket head speed, average ball
spin, average stroke quality, total and average (per session)
effective play time, total and average (per session) stroke count,
average ball impact, average stroke type, and player movement.
* * * * *