U.S. patent application number 14/465194 was filed with the patent office on 2015-02-26 for systems, articles, and methods for human-electronics interfaces.
The applicant listed for this patent is Thaimic Labs Inc.. Invention is credited to Matthew Bailey, Aaron Grant, Stephen Lake.
Application Number | 20150057770 14/465194 |
Document ID | / |
Family ID | 52481060 |
Filed Date | 2015-02-26 |
United States Patent
Application |
20150057770 |
Kind Code |
A1 |
Bailey; Matthew ; et
al. |
February 26, 2015 |
SYSTEMS, ARTICLES, AND METHODS FOR HUMAN-ELECTRONICS INTERFACES
Abstract
Human-electronics interfaces in which a wearable
electromyography ("EMG") device is operated to control virtually
any electronic device are described. In response to detected muscle
activity and/or motions of a user, the wearable EMG device
transmits generic gesture identification flags that are not
specific to the particular electronic device(s) being controlled.
An electronic device being controlled is programmed with
user-definable instructions for how to interpret and respond to the
gesture identification flags.
Inventors: |
Bailey; Matthew; (Kitchener,
CA) ; Lake; Stephen; (Kitchener, CA) ; Grant;
Aaron; (Kitchener, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Thaimic Labs Inc. |
Kitchener |
|
CA |
|
|
Family ID: |
52481060 |
Appl. No.: |
14/465194 |
Filed: |
August 21, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61869526 |
Aug 23, 2013 |
|
|
|
Current U.S.
Class: |
700/83 |
Current CPC
Class: |
G06F 3/017 20130101;
G06F 3/015 20130101; A61B 5/04888 20130101; A61B 5/1122 20130101;
A61B 5/6824 20130101; A61B 5/0488 20130101; A61B 5/6829
20130101 |
Class at
Publication: |
700/83 |
International
Class: |
G06F 3/01 20060101
G06F003/01; A61B 5/0488 20060101 A61B005/0488 |
Claims
1. A wearable electromyography ("EMG") device comprising: at least
one EMG sensor to in use detect muscle activity of a user of the
wearable EMG device and provide at least one signal in response to
the detected muscle activity; a processor communicatively coupled
to the at least one EMG sensor, the processor to in use determine a
gesture identification flag based at least in part on the at least
one signal provided by the at least one EMG sensor; and an output
terminal communicatively coupled to the processor to in use
transmit the gesture identification flag.
2. The wearable EMG device of claim 1 wherein the gesture
identification flag is independent of any downstream
processor-based device and generic to a variety of end user
applications executable by a variety of downstream processor-based
devices useable with the wearable EMG device.
3. The wearable EMG device of claim 1, further comprising: a
non-transitory processor-readable storage medium communicatively
coupled to the processor, wherein the non-transitory
processor-readable storage medium stores at least a set of gesture
identification flags.
4. The wearable EMG device of claim 3 wherein the non-transitory
processor-readable storage medium stores processor-executable
instructions that embody and/or produce a mapping between at least
one signal provided by the at least one EMG sensor and at least one
gesture identification flag and, when executed by the processor,
the processor-executable instructions cause the processor to
determine a gesture identification flag in accordance with the
mapping.
5. The wearable EMG device of claim 3 wherein the non-transitory
processor-readable storage medium stores processor-executable
instructions that, when executed by the processor, cause the
processor to determine a gesture identification flag based at least
in part on at least one signal provided by the at least one EMG
sensor.
6. The wearable EMG device of claim 1, further comprising: at least
one accelerometer responsive to motion effected by the user of the
wearable EMG device and communicatively coupled to the processor to
provide at least one signal in response to the detected motion, and
wherein the processor determines the gesture identification flag
based at least in part on both the at least one signal provided by
the at least one EMG sensor and the at least one signal provided by
the at least one accelerometer.
7. The wearable EMG device of claim 1 wherein the processor is
selected from the group consisting of: a digital microprocessor, a
digital microcontroller, a digital signal processor, a graphics
processing unit, an application specific integrated circuit, a
programmable gate array, and a programmable logic unit.
8. The wearable EMG device of claim 1 wherein the at least one EMG
sensor includes a plurality of EMG sensors, and wherein the
wearable EMG device further comprises: a set of communicative
pathways to route signals provided by the plurality of EMG sensors
to the processor, wherein each EMG sensor in the plurality of EMG
sensors is communicatively coupled to the processor by at least one
communicative pathway from the set of communicative pathways.
9. The wearable EMG device of claim 8, further comprising: a set of
pod structures that form physically coupled links of the wearable
EMG device, wherein each pod structure in the set of pod structures
is positioned adjacent and physically coupled to at least one other
pod structure in the set of pod structures, and wherein the set of
pod structures comprises at least two sensor pods and a processor
pod, each of the at least two sensor pods comprising a respective
EMG sensor from the plurality of EMG sensors and the processor pod
comprising the processor.
10. The wearable EMG device of claim 9 wherein each pod structure
in the set of pod structures is positioned adjacent and in between
two other pod structures in the set of pod structures and
physically coupled to the two other pod structures in the set of
pod structures, and wherein the set of pod structures forms a
perimeter of an annular configuration.
11. The wearable EMG device of claim 9, further comprising: at
least one adaptive coupler, wherein each respective pod structure
in the set of pod structures is adaptively physically coupled to at
least one adjacent pod structure in the set of pod structures by at
least one adaptive coupler.
12. The wearable EMG device of claim 1 wherein the output terminal
includes at least one of a wireless transmitter and/or a tethered
connector port.
13. The wearable EMG device of claim 1 wherein the at least one EMG
sensor includes at least one capacitive EMG sensor.
14. A method of operating a wearable electromyography ("EMG")
device to provide electromyographic control of an electronic
device, wherein the wearable EMG device includes at least one EMG
sensor, a processor, and an output terminal, the method comprising:
detecting muscle activity of a user of the wearable EMG device by
the at least one EMG sensor; providing at least one signal from the
at least one EMG sensor to the processor in response to the
detected muscle activity; determining, by the processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the processor,
wherein the gesture identification flag is independent of the
electronic device; and transmitting the gesture identification flag
to the electronic device by the output terminal.
15. The method of claim 14 wherein: detecting muscle activity of a
user of the wearable EMG device by the at least one EMG sensor
includes detecting muscle activity of the user of the wearable EMG
device by a first EMG sensor and by at least a second EMG sensor,
providing at least one signal from the at least one EMG sensor to
the processor in response to the detected muscle activity includes
providing at least a first signal from the first EMG sensor to the
processor in response to the detected muscle activity and providing
at least a second signal from the second EMG sensor to the
processor in response to the detected muscle activity, and
determining, by the processor, a gesture identification flag based
at least in part on the at least one signal provided from the at
least one EMG sensor to the processor includes determining, by the
processor, a gesture identification flag based at least in part on
the at least a first signal provided from the first EMG sensor to
the processor and the at least a second signal provided from the at
least a second EMG sensor to the processor.
16. The method of claim 14 wherein the wearable EMG device further
comprises a non-transitory processor-readable storage medium that
stores processor-executable instructions, and wherein determining,
by the processor, a gesture identification flag based at least in
part on the at least one signal provided from the at least one EMG
sensor to the processor includes executing the processor-executable
instructions by the processor to cause the processor to determine a
gesture identification flag based at least in part on the at least
one signal provided from the at least one EMG sensor to the
processor.
17. The method of claim 14 wherein the wearable EMG device further
comprises at least one accelerometer, and wherein the method
further comprises: detecting motion effected by the user of the
wearable EMG device by the at least one accelerometer; and
providing at least one signal from the at least one accelerometer
to the processor in response to the detected motion, and wherein
determining a gesture identification flag based at least in part on
the at least one signal provided from the at least one EMG sensor
to the processor includes: determining, by the processor, a gesture
identification flag based at least in part on both the at least one
signal provided from the at least one EMG sensor to the processor
and the at least one signal provided from the at least one
accelerometer to the processor.
18. The method of claim 17 wherein the wearable EMG device further
comprises a non-transitory processor-readable storage medium that
stores processor-executable instructions, and wherein determining,
by the processor, a gesture identification flag based at least in
part on both the at least one signal provided from the at least one
EMG sensor to the processor and the at least one signal provided
from the at least one accelerometer to the processor includes
executing the processor-executable instructions by the processor to
cause the processor to determine the gesture identification flag
based at least in part on both the at least one signal provided
from the at least one EMG sensor to the processor and the at least
one signal provided from the at least one accelerometer to the
processor.
19. The method of claim 14 wherein the output terminal includes a
wireless transmitter, and wherein transmitting the gesture
identification flag to the electronic device by the output terminal
includes wirelessly transmitting the gesture identification flag to
the electronic device by the wireless transmitter.
20. A system that enables electromyographic control of an
electronic device, the system comprising: a wearable
electromyography ("EMG") device comprising: at least one EMG sensor
responsive to muscle activity of a user of the wearable EMG device
and provide at least one signal in response to a detected muscle
activity, a first processor communicatively coupled to the at least
one EMG sensor, the first processor which in use determines a
gesture identification flag based at least in part on the at least
one signal provided by the at least one EMG sensor, and an output
terminal communicatively coupled to the first processor to transmit
the gesture identification flag; and an electronic device
comprising: an input terminal to receive the gesture identification
flag, and a second processor communicatively coupled to the input
terminal, the second processor which in use determines a function
of the electronic device based at least in part on the gesture
identification flag.
21. The system of claim 20 wherein the gesture identification flag
is independent of the electronic device and generic to a variety of
end user applications executable by the electronic device.
22. The system of claim 20 wherein the wearable EMG device of the
system further comprises: a non-transitory processor-readable
storage medium communicatively coupled to the first processor,
wherein the non-transitory processor-readable storage medium stores
at least a set of gesture identification flags.
23. The system of claim 22 wherein the non-transitory
processor-readable storage medium of the wearable EMG device stores
processor-executable instructions that embody and/or produce a
mapping between at least one signal provided by the at least one
EMG sensor and at least one gesture identification flag and, when
executed by the first processor, the processor-executable
instructions cause the first processor to determine a gesture
identification flag in accordance with the mapping.
24. The system of claim 20 wherein the wearable EMG device of the
system further comprises: a non-transitory processor-readable
storage medium communicatively coupled to the first processor,
wherein the non-transitory processor-readable storage medium stores
processor-executable instructions that, when executed by the first
processor, cause the first processor to determine a gesture
identification flag based at least in part on the at least one
signal provided by the at least one EMG sensor.
25. The system of claim 20 wherein the wearable EMG device of the
system further comprises: at least one accelerometer
communicatively coupled to the first processor, the at least one
accelerometer responsive to motion effected by the user of the
wearable EMG device and which provides at least one signal in
response to a detected motion, and wherein the first processor
determines a gesture identification flag based at least in part on
both the at least one signal provided by the at least one EMG
sensor and the at least on signal provided by the at least one
accelerometer.
26. The system of claim 20 wherein the electronic device of the
system further comprises: a non-transitory processor-readable
storage medium communicatively coupled to the second processor,
wherein the non-transitory processor-readable storage medium stores
at least a set of processor-executable instructions that, when
executed by the second processor, cause the second processor to
determine a function of the electronic device based at least in
part on the gesture identification flag.
27. The system of claim 20 wherein the electronic device of the
system further comprises: a non-transitory processor-readable
storage medium communicatively coupled to the second processor,
wherein the non-transitory processor-readable storage medium
stores: a first application executable by the electronic device; at
least a second application executable by the electronic device; a
first set of processor-executable instructions that, when executed
by the second processor, cause the second processor to determine a
function of the first application based at least in part on a
gesture identification flag; and a second set of
processor-executable instructions that, when executed by the second
processor, cause the second processor to determine a function of
the second application based at least in part on a gesture
identification flag.
28. The system of claim 20 wherein the output terminal of the
wearable EMG device includes a first tethered connector port, the
input terminal of the electronic device includes a second tethered
connector port, and further comprising: a communicative pathway
that in use communicatively couples the first tethered connector
port to the second tethered connector port and to route the gesture
identification flag from the output terminal of the wearable EMG
device to the input terminal of the electronic device.
29. The system of claim 20 wherein the output terminal of the
wearable EMG device includes a wireless transmitter that in use
wirelessly transmits the gesture identification flag and the input
terminal of the electronic device includes a tethered connector
port, and wherein the system further comprises: a wireless receiver
that in use communicatively couples to the tethered connector port
of the electronic device and to in use wirelessly receive the
gesture identification flag from the wireless transmitter of the
wearable EMG device.
30. The system of claim 20 wherein the output terminal of the
wearable EMG device includes a wireless transmitter to wirelessly
transmit the gesture identification flag, and wherein the input
terminal of the electronic device includes a wireless receiver to
wirelessly receive the gesture identification flag from the
wireless transmitter of the wearable EMG device.
31. The system of claim 20 wherein the electronic device is
selected from the group consisting of: a computer, a desktop
computer, a laptop computer, a tablet computer, a mobile phone, a
smartphone, a portable electronic device, an audio player, a
television, a video player, a video game console, a robot, a light
switch, and a vehicle.
32. A method of electromyographically controlling at least one
function of an electronic device by a wearable electromyography
("EMG") device, wherein the wearable EMG device includes at least
one EMG sensor, a first processor, and an output terminal and the
electronic device includes an input terminal and a second
processor, the method comprising: detecting muscle activity of a
user of the wearable EMG device by the at least one EMG sensor;
providing at least one signal from the at least one EMG sensor to
the first processor in response to the detected muscle activity;
determining, by the first processor, a gesture identification flag
based at least in part on the at least one signal provided from the
at least one EMG sensor to the first processor, wherein the gesture
identification flag is independent of the electronic device;
transmitting the gesture identification flag by the output terminal
of the wearable EMG device; receiving the gesture identification
flag by the input terminal of the electronic device; determining,
by the second processor, a function of the electronic device based
at least in part on the gesture identification flag; and performing
the function by the electronic device.
33. The method of claim 32 wherein: detecting muscle activity of a
user of the wearable EMG device by the at least one EMG sensor
includes detecting muscle activity of the user of the wearable EMG
device by a first EMG sensor of the wearable EMG device and by at
least a second EMG sensor of the wearable EMG device, providing at
least one signal from the at least one EMG sensor to the first
processor in response to the detected muscle activity includes
providing at least a first signal from the first EMG sensor to the
first processor in response to the detected muscle activity and
providing at least a second signal from the send EMG sensor to the
first processor in response to the detected muscle activity, and
determining, by the first processor, a gesture identification flag
based at least in part on the at least one signal provided from the
at least one EMG sensor to the first processor includes
determining, by the first processor, a gesture identification flag
based at least in part on the at least a first signal provided from
the first EMG sensor to the first processor and the at least a
second signal provided from the at least a second EMG sensor to the
first processor.
34. The method of claim 32 wherein the wearable EMG device further
comprises a non-transitory processor-readable medium that stores
processor-executable instructions, and wherein determining, by the
first processor, a gesture identification flag based at least in
part on the at least one signal provided from the at least one EMG
sensor to the first processor includes executing the
processor-executable instructions by the first processor to cause
the first processor to determine a gesture identification flag
based at least in part on the at least one signal provided from the
at least one EMG sensor to the first processor.
35. The method of claim 32 wherein the wearable EMG device further
comprises at least one accelerometer, and wherein the method
further comprises: detecting motion effected by the user of the
wearable EMG device by the at least one accelerometer; and
providing at least one signal from the at least one accelerometer
to the first processor in response to the detected motion, and
wherein determining, by the first processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the first
processor includes: determining, by the first processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the first
processor and the at least one signal provided by the at least one
accelerometer to the first processor.
36. The method of claim 32 wherein the output terminal of the
wearable EMG device includes a wireless transmitter and the input
terminal of the electronic device includes a wireless receiver, and
wherein: transmitting the gesture identification flag by the output
terminal of the wearable EMG device includes wirelessly
transmitting the gesture identification flag by the wireless
transmitter of the wearable EMG device, and receiving the gesture
identification flag by the input terminal of the electronic device
includes wirelessly receiving the gesture identification flag by
the wireless receiver of the electronic device.
37. The method of claim 32 wherein the electronic device further
comprises a non-transitory processor-readable storage medium that
stores processor-executable instructions, and wherein determining,
by the second processor, a function of the electronic device based
at least in part on the gesture identification flag includes
executing the processor-executable instructions by the second
processor to cause the second processor to determine a function of
the electronic device based at least in part on the gesture
identification flag.
Description
BACKGROUND
[0001] 1. Technical Field
[0002] The present systems, articles, and methods generally relate
to human-electronics interfaces and particularly relate to
electromyographic control of electronic devices.
[0003] 2. Description of the Related Art
Wearable Electronic Devices
[0004] Electronic devices are commonplace throughout most of the
world today. Advancements in integrated circuit technology have
enabled the development of electronic devices that are sufficiently
small and lightweight to be carried by the user. Such "portable"
electronic devices may include on-board power supplies (such as
batteries or other power storage systems) and may be designed to
operate without any wire-connections to other electronic systems;
however, a small and lightweight electronic device may still be
considered portable even if it includes a wire-connection to
another electronic system. For example, a microphone may be
considered a portable electronic device whether it is operated
wirelessly or through a wire-connection.
[0005] The convenience afforded by the portability of electronic
devices has fostered a huge industry. Smartphones, audio players,
laptop computers, tablet computers, and ebook readers are all
examples of portable electronic devices. However, the convenience
of being able to carry a portable electronic device has also
introduced the inconvenience of having one's hand(s) encumbered by
the device itself. This problem is addressed by making an
electronic device not only portable, but wearable.
[0006] A wearable electronic device is any portable electronic
device that a user can carry without physically grasping,
clutching, or otherwise holding onto the device with their hand(s).
For example, a wearable electronic device may be attached or
coupled to the user by a strap or straps, a band or bands, a clip
or clips, an adhesive, a pin and clasp, an article of clothing,
tension or elastic support, an interference fit, an ergonomic form,
etc. Examples of wearable electronic devices include digital
wristwatches, electronic armbands, electronic rings, electronic
ankle-bracelets or "anklets," head-mounted electronic display
units, hearing aids, and so on.
Human-Electronics Interfaces
[0007] A wearable electronic device may provide direct
functionality for a user (such as audio playback, data display,
computing functions, etc.) or it may provide electronics to
interact with, receive information from, or control another
electronic device. For example, a wearable electronic device may
include sensors that detect inputs effected by a user and transmit
signals to another electronic device based on those inputs.
Sensor-types and input-types may each take on a variety of forms,
including but not limited to: tactile sensors (e.g., buttons,
switches, touchpads, or keys) providing manual control, acoustic
sensors providing voice-control, electromyography sensors providing
gesture control, and/or accelerometers providing gesture
control.
[0008] A human-computer interface ("HCI") is an example of a
human-electronics interface. The present systems, articles, and
methods may be applied to HCIs, but may also be applied to any
other form of human-electronics interface.
Electromyography Devices
[0009] Electromyography ("EMG") is a process for detecting and
processing the electrical signals generated by muscle activity. EMG
devices employ EMG sensors that are responsive to the range of
electrical potentials (typically .mu.V-mV) involved in muscle
activity. EMG signals may be used in a wide variety of
applications, including: medical monitoring and diagnosis, muscle
rehabilitation, exercise and training, prosthetic control, and even
in controlling functions of electronic devices.
[0010] Human-electronics interfaces that employ EMG have been
proposed in the art. For example, U.S. Pat. No. 6,244,873 and U.S.
Pat. No. 8,170,656 describe such systems. Characteristics that are
common to these known proposals will now be described.
[0011] Typically, such systems (including the two examples listed
above) employ a wearable EMG device that exclusively controls
specific, pre-defined functions of a specific, pre-defined
"receiving" electronic device. The term "pre-defined" here refers
to information that is programmed into the wearable EMG device (or
with which the wearable EMG device is programmed) in advance of a
following interaction with a receiving device. The wearable EMG
device typically includes built-in EMG sensors that detect muscle
activity of a user and an on-board processor that determines when
the detected muscle activity corresponds to a pre-defined gesture.
The on-board processor maps each pre-defined gesture to a
particular pre-defined function of the pre-defined receiving
device. In other words, the wearable EMG device stores and executes
pre-defined mappings between detected gestures and receiving device
functions. The receiving device function(s) is/are then controlled
by one or more "command(s)" that is/are output by the wearable EMG
device. Each command that is output by the wearable EMG device has
already been formulated to control (and is therefore limited to
exclusively controlling) a specific function of a specific
receiving device prior to being transmitted by the wearable EMG
device.
[0012] The wearable EMG devices proposed in the art are hard-coded
to map pre-defined gestures to specific, pre-defined commands
controlling specific, pre-defined functions of a specific,
pre-defined receiving device. The wearable EMG devices proposed in
the art are programmed with information about the specific
receiving device (and/or about a specific application within the
specific receiving device) under their control such that the
wearable EMG devices proposed in the art output commands that
include instructions that are specifically formulated for the
specific receiving device (and/or the specific application within
the specific receiving device). Thus, existing proposals for
human-electronics interfaces that employ EMG are limited in their
versatility because they employ a wearable EMG device that is
hard-coded to control a specific electronic device (and/or a
specific application within a specific electronic device). For such
systems, the wearable EMG device needs to be modified/adapted for
each distinct use (e.g., the wearable EMG device needs to be
programmed with command signals that are specific to the receiving
device and/or specific to the application within the receiving
device). Because the outputs (i.e., commands) provided by such
wearable EMG devices are hard-coded with information about the
function(s) of the receiving device(s), a user cannot use such a
wearable EMG device to control any generic electronic device (or
any generic application within an electronic device) without
reprogramming/reconfiguring the wearable EMG device itself. A user
who wishes to control multiple electronic devices (or multiple
applications within a single electronic device, either
simultaneously or in sequence) must use multiple such wearable EMG
devices with each wearable EMG device separately controlling a
different electronic device, or the user must re-program a single
such wearable EMG device in between uses. There is a need in the
art for a human-electronics interface employing EMG that overcomes
these limitations.
BRIEF SUMMARY
[0013] A wearable electromyography ("EMG") device may be summarized
as including: at least one EMG sensor to in use detect muscle
activity of a user of the wearable EMG device and provide at least
one signal in response to the detected muscle activity; a processor
communicatively coupled to the at least one EMG sensor, the
processor to in use determine a gesture identification flag based
at least in part on the at least one signal provided by the at
least one EMG sensor; and an output terminal communicatively
coupled to the processor to in use transmit the gesture
identification flag. The gesture identification flag may be
independent of any downstream processor-based device and generic to
a variety of end user applications executable by a variety of
downstream processor-based devices useable with the wearable EMG
device.
[0014] The wearable EMG device may further include a non-transitory
processor-readable storage medium communicatively coupled to the
processor, wherein the non-transitory processor-readable storage
medium stores at least a set of gesture identification flags. The
non-transitory processor-readable storage medium may store
processor-executable instructions that embody and/or produce/effect
a mapping between at least one signal provided by the at least one
EMG sensor and at least one gesture identification flag and, when
executed by the processor, the processor-executable instructions
may cause the processor to determine a gesture identification flag
in accordance with the mapping. The non-transitory
processor-readable storage medium may store processor-executable
instructions that, when executed by the processor, cause the
processor to determine a gesture identification flag based at least
in part on at least one signal provided by the at least one EMG
sensor.
[0015] The wearable EMG device may further include at least one
accelerometer communicatively coupled to the processor, the at
least one accelerometer to in use detect motion effected by the
user of the wearable EMG device and provide at least one signal in
response to the detected motion, and the processor may in use
determine the gesture identification flag based at least in part on
both the at least one signal provided by the at least one EMG
sensor and the at least one signal provided by the at least one
accelerometer.
[0016] The processor may be selected from the group consisting of:
a digital microprocessor, a digital microcontroller, a digital
signal processor, a graphics processing unit, an application
specific integrated circuit, a programmable gate array, and a
programmable logic unit. The at least one EMG sensor may include a
plurality of EMG sensors, and the wearable EMG device may further
include a set of communicative pathways to route signals provided
by the plurality of EMG sensors to the processor, wherein each EMG
sensor in the plurality of EMG sensors is communicatively coupled
to the processor by at least one communicative pathway from the set
of communicative pathways. The wearable EMG device may further
include a set of pod structures that form physically coupled links
of the wearable EMG device, wherein each pod structure in the set
of pod structures is positioned adjacent and physically coupled to
at least one other pod structure in the set of pod structures, and
wherein the set of pod structures comprises at least two sensor
pods and a processor pod, each of the at least two sensor pods
comprising a respective EMG sensor from the plurality of EMG
sensors and the processor pod comprising the processor. Each pod
structure in the set of pod structures may be positioned adjacent
and in between two other pod structures in the set of pod
structures and physically coupled to the two other pod structures
in the set of pod structures, and the set of pod structures may
form a perimeter of an annular configuration. The wearable EMG
device may further include at least one adaptive coupler, wherein
each respective pod structure in the set of pod structures is
adaptively physically coupled to at least one adjacent pod
structure in the set of pod structures by at least one adaptive
coupler.
[0017] The output terminal of the wearable EMG device may include
at least one of a wireless transmitter and/or a tethered connector
port. The at least one EMG sensor may include at least one
capacitive EMG sensor.
[0018] A method of operating a wearable electromyography ("EMG")
device to provide electromyographic control of an electronic
device, wherein the wearable EMG device includes at least one EMG
sensor, a processor, and an output terminal, may be summarized as
including: detecting muscle activity of a user of the wearable EMG
device by the at least one EMG sensor; providing at least one
signal from the at least one EMG sensor to the processor in
response to the detected muscle activity; determining, by the
processor, a gesture identification flag based at least in part on
the at least one signal provided from the at least one EMG sensor
to the processor, wherein the gesture identification flag is
independent of the electronic device; and transmitting the gesture
identification flag to the electronic device by the output
terminal. Detecting muscle activity of a user of the wearable EMG
device by the at least one EMG sensor may include detecting muscle
activity of the user of the wearable EMG device by a first EMG
sensor and by at least a second EMG sensor. Providing at least one
signal from the at least one EMG sensor to the processor in
response to the detected muscle activity may include providing at
least a first signal from the first EMG sensor to the processor in
response to the detected muscle activity and providing at least a
second signal from the second EMG sensor to the processor in
response to the detected muscle activity. Determining, by the
processor, a gesture identification flag based at least in part on
the at least one signal provided from the at least one EMG sensor
to the processor may include determining, by the processor, a
gesture identification flag based at least in part on the at least
a first signal provided from the first EMG sensor to the processor
and the at least a second signal provided from the at least a
second EMG sensor to the processor.
[0019] The wearable EMG device may further include a non-transitory
processor-readable storage medium that stores processor-executable
instructions, and determining, by the processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the processor
may include executing the processor-executable instructions by the
processor to cause the processor to determine a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the
processor.
[0020] The wearable EMG device may further include at least one
accelerometer, and the method may further include: detecting motion
effected by the user of the wearable EMG device by the at least one
accelerometer; and providing at least one signal from the at least
one accelerometer to the processor in response to the detected
motion. Determining a gesture identification flag based at least in
part on the at least one signal provided from the at least one EMG
sensor to the processor may include determining, by the processor,
a gesture identification flag based at least in part on both the at
least one signal provided from the at least one EMG sensor to the
processor and the at least one signal provided from the at least
one accelerometer to the processor. The wearable EMG device may
include a non-transitory processor-readable storage medium that
stores processor-executable instructions, and determining, by the
processor, a gesture identification flag based at least in part on
both the at least one signal provided from the at least one EMG
sensor to the processor and the at least one signal provided from
the at least one accelerometer to the processor may includes
executing the processor-executable instructions by the processor to
cause the processor to determine the gesture identification flag
based at least in part on both the at least one signal provided
from the at least one EMG sensor to the processor and the at least
one signal provided from the at least one accelerometer to the
processor.
[0021] The output terminal of the wearable EMG device may include a
wireless transmitter, and transmitting the gesture identification
flag to the electronic device by the output terminal may include
wirelessly transmitting the gesture identification flag to the
electronic device by the wireless transmitter.
[0022] A system that enables electromyographic control of an
electronic device may be summarized as including: a wearable
electromyography ("EMG") device comprising: at least one EMG sensor
to in use detect muscle activity of a user of the wearable EMG
device and provide at least one signal in response to the detected
muscle activity, a first processor communicatively coupled to the
at least one EMG sensor, the first processor to in use determine a
gesture identification flag based at least in part on the at least
one signal provided by the at least one EMG sensor, and an output
terminal communicatively coupled to the first processor, the output
terminal to in use transmit the gesture identification flag; and an
electronic device comprising: an input terminal to in use receive
the gesture identification flag, and a second processor
communicatively coupled to the input terminal, the second processor
to in use determine a function of the electronic device based at
least in part on the gesture identification flag. The gesture
identification flag may be independent of the electronic device and
generic to a variety of end user applications executable by the
electronic device.
[0023] The wearable EMG device of the system may further include a
non-transitory processor-readable storage medium communicatively
coupled to the first processor, wherein the non-transitory
processor-readable storage medium stores at least a set of gesture
identification flags. The non-transitory processor-readable storage
medium of the wearable EMG device may store processor-executable
instructions that embody and/or produce/effect a mapping between at
least one signal provided by the at least one EMG sensor and at
least one gesture identification flag and, when executed by the
first processor, the processor-executable instructions may cause
the first processor to determine a gesture identification flag in
accordance with the mapping.
[0024] The wearable EMG device of the system may include a
non-transitory processor-readable storage medium communicatively
coupled to the first processor, wherein the non-transitory
processor-readable storage medium stores processor-executable
instructions that, when executed by the first processor, cause the
first processor to determine a gesture identification flag based at
least in part on the at least one signal provided by the at least
one EMG sensor.
[0025] The wearable EMG device of the system may include at least
one accelerometer communicatively coupled to the first processor,
the at least one accelerometer to in use detect motion effected by
the user of the wearable EMG device and provide at least one signal
in response to the detected motion, and the first processor may in
use determine a gesture identification flag based at least in part
on both the at least one signal provided by the at least one EMG
sensor and the at least on signal provided by the at least one
accelerometer.
[0026] The electronic device of the system may include a
non-transitory processor-readable storage medium communicatively
coupled to the second processor, wherein the non-transitory
processor-readable storage medium stores at least a set of
processor-executable instructions that, when executed by the second
processor, cause the second processor to determine a function of
the electronic device based at least in part on the gesture
identification flag.
[0027] The electronic device of the system may include a
non-transitory processor-readable storage medium communicatively
coupled to the second processor, wherein the non-transitory
processor-readable storage medium stores: a first application
executable by the electronic device; at least a second application
executable by the electronic device; a first set of
processor-executable instructions that, when executed by the second
processor, cause the second processor to determine a function of
the first application based at least in part on a gesture
identification flag; and a second set of processor-executable
instructions that, when executed by the second processor, cause the
second processor to determine a function of the second application
based at least in part on a gesture identification flag.
[0028] The output terminal of the wearable EMG device may include a
first tethered connector port, the input terminal of the electronic
device may include a second tethered connector port, and the system
may further include a communicative pathway to in use
communicatively couple the first tethered connector port to the
second tethered connector port and to route the gesture
identification flag from the output terminal of the wearable EMG
device to the input terminal of the electronic device.
[0029] The output terminal of the wearable EMG device may include a
wireless transmitter to in use wirelessly transmit the gesture
identification flag, the input terminal of the electronic device
may include a tethered connector port, and the system may include a
wireless receiver to in use communicatively couple to the tethered
connector port of the electronic device and to in use wirelessly
receive the gesture identification flag from the wireless
transmitter of the wearable EMG device.
[0030] The output terminal of the wearable EMG device may include a
wireless transmitter to in use wirelessly transmit the gesture
identification flag and the input terminal of the electronic device
may include a wireless receiver to in use wirelessly receive the
gesture identification flag from the wireless transmitter of the
wearable EMG device.
[0031] The electronic device may be selected from the group
consisting of: a computer, a desktop computer, a laptop computer, a
tablet computer, a mobile phone, a smartphone, a portable
electronic device, an audio player, a television, a video player, a
video game console, a robot, a light switch, and a vehicle.
[0032] A method of electromyographically controlling at least one
function of an electronic device by a wearable electromyography
("EMG") device, wherein the wearable EMG device includes at least
one EMG sensor, a first processor, and an output terminal and the
electronic device includes an input terminal and a second
processor, may be summarized as including: detecting muscle
activity of a user of the wearable EMG device by the at least one
EMG sensor; providing at least one signal from the at least one EMG
sensor to the first processor in response to the detected muscle
activity; determining, by the first processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the first
processor, wherein the gesture identification flag is independent
of the electronic device; transmitting the gesture identification
flag by the output terminal of the wearable EMG device; receiving
the gesture identification flag by the input terminal of the
electronic device; determining, by the second processor, a function
of the electronic device based at least in part on the gesture
identification flag; and performing the function by the electronic
device. Detecting muscle activity of a user of the wearable EMG
device by the at least one EMG sensor may include detecting muscle
activity of the user of the wearable EMG device by a first EMG
sensor of the wearable EMG device and by at least a second EMG
sensor of the wearable EMG device. Providing at least one signal
from the at least one EMG sensor to the first processor in response
to the detected muscle activity may include providing at least a
first signal from the first EMG sensor to the first processor in
response to the detected muscle activity and providing at least a
second signal from the send EMG sensor to the first processor in
response to the detected muscle activity. Determining, by the first
processor, a gesture identification flag based at least in part on
the at least one signal provided from the at least one EMG sensor
to the first processor may include determining, by the first
processor, a gesture identification flag based at least in part on
the at least a first signal provided from the first EMG sensor to
the first processor and the at least a second signal provided from
the at least a second EMG sensor to the first processor.
[0033] The wearable EMG device may include a non-transitory
processor-readable medium that stores processor-executable
instructions, and determining, by the first processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the first
processor may include executing the processor-executable
instructions by the first processor to cause the first processor to
determine a gesture identification flag based at least in part on
the at least one signal provided from the at least one EMG sensor
to the first processor.
[0034] The wearable EMG device may include at least one
accelerometer, and the method may include: detecting motion
effected by the user of the wearable EMG device by the at least one
accelerometer; and providing at least one signal from the at least
one accelerometer to the first processor in response to the
detected motion. Determining, by the first processor, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the first
processor may include determining, by the first processor, a
gesture identification flag based at least in part on the at least
one signal provided from the at least one EMG sensor to the first
processor and the at least one signal provided by the at least one
accelerometer to the first processor.
[0035] The output terminal of the wearable EMG device may include a
wireless transmitter and the input terminal of the electronic
device may include a wireless receiver. Transmitting the gesture
identification flag by the output terminal of the wearable EMG
device may include wirelessly transmitting the gesture
identification flag by the wireless transmitter of the wearable EMG
device, and receiving the gesture identification flag by the input
terminal of the electronic device may include wirelessly receiving
the gesture identification flag by the wireless receiver of the
electronic device.
[0036] The electronic device may include a non-transitory
processor-readable storage medium that stores processor-executable
instructions, and determining, by the second processor, a function
of the electronic device based at least in part on the gesture
identification flag may include executing the processor-executable
instructions by the second processor to cause the second processor
to determine a function of the electronic device based at least in
part on the gesture identification flag.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0037] In the drawings, identical reference numbers identify
similar elements or acts. The sizes and relative positions of
elements in the drawings are not necessarily drawn to scale. For
example, the shapes of various elements and angles are not drawn to
scale, and some of these elements are arbitrarily enlarged and
positioned to improve drawing legibility. Further, the particular
shapes of the elements as drawn are not intended to convey any
information regarding the actual shape of the particular elements,
and have been solely selected for ease of recognition in the
drawings.
[0038] FIG. 1 is a perspective view of an exemplary wearable
electromyography device that forms part of a human-electronics
interface in accordance with the present systems, articles and
methods.
[0039] FIG. 2 is an illustrative diagram of a system that enables
electromyographic control of an electronic device in accordance
with the present systems, articles, and methods.
[0040] FIG. 3 is a flow-diagram showing a method of operating a
wearable electromyography device to provide electromyographic
control of an electronic device in accordance with the present
systems, articles, and methods.
[0041] FIG. 4 is a flow-diagram showing a method of operating a
wearable electromyography device to provide both electromyographic
and motion control of an electronic device in accordance with the
present systems, articles, and methods.
[0042] FIG. 5 is a schematic illustration that shows an exemplary
mapping between a set of exemplary gestures and a set of exemplary
gesture identification flags in accordance with the present
systems, articles, and methods.
[0043] FIG. 6 is a flow-diagram showing a method of
electromyographically controlling at least one function of an
electronic device by a wearable electromyography device in
accordance with the present systems, articles, and methods.
[0044] FIG. 7 is a schematic illustration that shows an exemplary
mapping between a set of exemplary gesture identification flags and
a set of exemplary functions of an electronic device in accordance
with the present systems, articles, and methods.
DETAILED DESCRIPTION
[0045] In the following description, certain specific details are
set forth in order to provide a thorough understanding of various
disclosed embodiments. However, one skilled in the relevant art
will recognize that embodiments may be practiced without one or
more of these specific details, or with other methods, components,
materials, etc. In other instances, well-known structures
associated with electronic devices, and in particular portable
electronic devices such as wearable electronic devices, have not
been shown or described in detail to avoid unnecessarily obscuring
descriptions of the embodiments.
[0046] Unless the context requires otherwise, throughout the
specification and claims which follow, the word "comprise" and
variations thereof, such as, "comprises" and "comprising" are to be
construed in an open, inclusive sense, that is as "including, but
not limited to."
[0047] Reference throughout this specification to "one embodiment"
or "an embodiment" means that a particular feature, structures, or
characteristics may be combined in any suitable manner in one or
more embodiments.
[0048] As used in this specification and the appended claims, the
singular forms "a," "an," and "the" include plural referents unless
the content clearly dictates otherwise. It should also be noted
that the term "or" is generally employed in its broadest sense,
that is as meaning "and/or" unless the content clearly dictates
otherwise.
[0049] The headings and Abstract of the Disclosure provided herein
are for convenience only and do not interpret the scope or meaning
of the embodiments.
[0050] The various embodiments described herein provide systems,
articles, and methods for human-electronics interfaces employing a
generalized wearable EMG device that may be readily implemented in
a wide range of applications. The human-electronics interfaces
described herein employ a wearable EMG device that controls
functions of another electronic device not by outputting "commands"
as in the known proposals previously described, but by outputting
generic gesture identification signals, or "flags," that are not
specific to the particular electronic device being controlled. In
this way, the wearable EMG device may be used to control virtually
any other electronic device if, for example, the other electronic
device (or multiple other electronic devices) is (are) programmed
with instructions for how to respond to the gesture identification
flags.
[0051] Throughout this specification and the appended claims, the
term "gesture identification flag" is used to refer to at least a
portion of a data signal (e.g., a bit string) that is defined by
and transmitted from a wearable EMG device in response to the
wearable EMG device identifying that a user thereof has performed a
particular gesture. The gesture identification flag may be received
by a "receiving" electronic device, but the "gesture identification
flag" portion of the data signal does not contain any information
that is specific to the receiving electronic device. A gesture
identification flag is a general, universal, and/or ambiguous
signal that is substantially independent of the receiving
electronic device (e.g., independent of any downstream
processor-based device) and/or generic to a variety of applications
run on any number of receiving electronic devices (e.g., generic to
a variety of end user applications executable by one or more
downstream processor-based device(s) useable with the wearable EMG
device). A gesture identification flag may carry no more
information than the definition/identity of the flag itself. For
example, a set of three gesture identification flags may include a
first flag simply defined as "A," a second flag simply defined as
"B," and a third flag simply defined as "C." Similarly, a set of
four binary gesture identification flags may include a 00 flag, a
01 flag, a 10 flag, and a 11 flag. In accordance with the present
systems, articles, and methods, a gesture identification flag may
be defined and output by a wearable EMG device with little to no
regard for the nature or functions of the receiving electronic
device. The receiving electronic device may be programmed with
specific instructions for how to interpret and/or respond to one or
more gesture identification flag(s). As will be understood by a
person of skill in the art, in some applications a gesture
identification flag may be combined with authentication data,
encryption data, device ID data (i.e., transmitting electronic
device ID data and/or receiving electronic device ID data), pairing
data, and/or any other data to enable and/or facilitate
telecommunications between the wearable EMG device and the
receiving electronic device in accordance with known
telecommunications protocols (e.g., Bluetooth.RTM.). For greater
certainty, throughout this specification and the appended claims,
the term "gesture identification flag" refers to at least a portion
of a data signal that is defined by a wearable EMG device based (at
least in part) on EMG and/or accelerometer data and is
substantially independent of the receiving electronic device. For
the purposes of transmission, a gesture identification flag may be
combined with other data that is at least partially dependent on
the receiving electronic device. For example, a gesture
identification flag may be a 2-bit component of an 8-bit data byte,
where the remaining 6 bits are used for telecommunication purposes,
as in: 00101101, where the exemplary first six bits "001011" may
correspond to telecommunications information such as
transmitting/receiving device IDs, encryption data, pairing data,
and/or the like, and the exemplary last two bits "01" may
correspond to a gesture identification flag. While a bit-length of
two bits is used to represent a gesture identification flag in this
example, in practice a gesture identification flag may comprise any
number of bits (or other measure of signal length of a scheme not
based on bits is employed).
[0052] FIG. 1 is a perspective view of an exemplary wearable EMG
device 100 that may form part of a human-electronics interface in
accordance with the present systems, articles, and methods.
Exemplary device 100 is an armband designed to be worn on the
wrist, forearm, or upper arm of a user, though a person of skill in
the art will appreciate that the teachings described herein may
readily be applied in wearable EMG devices designed to be worn
elsewhere on the body of the user (such as on the finger, leg,
ankle, neck, and/or torso of the user). Exemplary details that may
be included in exemplary wearable EMG device 100 are described in
at least U.S. Provisional Patent Application Ser. No. 61/752,226
(now U.S. Non-Provisional patent application Ser. No. 14/155,107),
U.S. Provisional Patent Application Ser. No. 61/768,322 (now U.S.
Non-Provisional patent application Ser. No. 14/186,889), and U.S.
Provisional Patent Application Ser. No. 61/771,500 (now U.S.
Non-Provisional patent application Ser. No. 14/194,252), each of
which is incorporated herein by reference in its entirety.
[0053] Device 100 includes a set of eight pod structures 101, 102,
103, 104, 105, 106, 107, and 108 that form physically coupled links
of the wearable EMG device 100. Each pod structure in the set of
eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 is
positioned adjacent and in between two other pod structures in the
set of eight pod structures and the set of pod structures forms a
perimeter of an annular or closed loop configuration. For example,
pod structure 101 is positioned adjacent and in between pod
structures 102 and 108 at least approximately on a perimeter of the
annular or closed loop configuration of pod structures, pod
structure 102 is positioned adjacent and in between pod structures
101 and 103 at least approximately on the perimeter of the annular
or closed loop configuration, pod structure 103 is positioned
adjacent and in between pod structures 102 and 104 at least
approximately on the perimeter of the annular or closed loop
configuration, and so on. Each of pod structures 101, 102, 103,
104, 105, 106, 107, and 108 is physically coupled to the two
adjacent pod structures by at least one adaptive coupler (not
visible in FIG. 1). For example, pod structure 101 is physically
coupled to pod structure 108 by an adaptive coupler and to pod
structure 102 by an adaptive coupler. The term "adaptive coupler"
is used throughout this specification and the appended claims to
denote a system, article or device that provides flexible,
adjustable, modifiable, extendable, extensible, or otherwise
"adaptive" physical coupling. Adaptive coupling is physical
coupling between two objects that permits limited motion of the two
objects relative to one another. An example of an adaptive coupler
is an elastic material such as an elastic band. Thus, each of pod
structures 101, 102, 103, 104, 105, 106, 107, and 108 in the set of
eight pod structures may be adaptively physically coupled to the
two adjacent pod structures by at least one elastic band. The set
of eight pod structures may be physically bound in the annular or
closed loop configuration by a single elastic band that couples
over or through all pod structures or by multiple separate elastic
bands that couple between adjacent pairs of pod structures or
between groups of adjacent pairs of pod structures. Device 100 is
depicted in FIG. 1 with the at least one adaptive coupler
completely retracted and contained within the eight pod structures
101, 102, 103, 104, 105, 106, 107, and 108 (and therefore the at
least one adaptive coupler is not visible in FIG. 1). Further
details of adaptive coupling in wearable electronic devices are
described in, for example, U.S. Provisional Application Ser. No.
61/860,063 (now U.S. Non-Provisional patent application Ser. No.
14/276,575), which is incorporated herein by reference in its
entirety.
[0054] Throughout this specification and the appended claims, the
term "pod structure" is used to refer to an individual link,
segment, pod, section, structure, component, etc. of a wearable EMG
device. For the purposes of the present systems, articles, and
methods, an "individual link, segment, pod, section, structure,
component, etc." (i.e., a "pod structure") of a wearable EMG device
is characterized by its ability to be moved or displaced relative
to another link, segment, pod, section, structure component, etc.
of the wearable EMG device. For example, pod structures 101 and 102
of device 100 can each be moved or displaced relative to one
another within the constraints imposed by the adaptive coupler
providing adaptive physical coupling therebetween. The desire for
pod structures 101 and 102 to be movable/displaceable relative to
one another specifically arises because device 100 is a wearable
EMG device that advantageously accommodates the movements of a user
and/or different user forms.
[0055] Device 100 includes eight pod structures 101, 102, 103, 104,
105, 106, 107, and 108 that form physically coupled links thereof.
Wearable EMG devices employing pod structures (e.g., device 100)
are used herein as exemplary wearable EMG device designs, while the
present systems, articles, and methods may be applied to wearable
EMG devices that do not employ pod structures (or that employ any
number of pod structures). Thus, throughout this specification,
descriptions relating to pod structures (e.g., functions and/or
components of pod structures) should be interpreted as being
applicable to any wearable EMG device design, even wearable EMG
device designs that do not employ pod structures (except in cases
where a pod structure is specifically recited in a claim).
[0056] In exemplary device 100 of FIG. 1, each of pod structures
101, 102, 103, 104, 105, 106, 107, and 108 comprises a respective
housing having a respective inner volume. Each housing may be
formed of substantially rigid material and may be optically opaque.
Throughout this specification and the appended claims, the term
"rigid" as in, for example, "substantially rigid material," is used
to describe a material that has an inherent tendency to maintain
its shape and resist malformation/deformation under the moderate
stresses and strains typically encountered by a wearable electronic
device.
[0057] Details of the components contained within the housings
(i.e., within the inner volumes of the housings) of pod structures
101, 102, 103, 104, 105, 106, 107, and 108 are not visible in FIG.
1. To facilitate descriptions of exemplary device 100, some
internal components are depicted by dashed lines in FIG. 1 to
indicate that these components are contained in the inner volume(s)
of housings and may not normally be actually visible in the view
depicted in FIG. 1, unless a transparent or translucent material is
employed to form the housings. For example, any or all of pod
structures 101, 102, 103, 104, 105, 106, 107, and/or 108 may
include electric circuitry. In FIG. 1, a first pod structure 101 is
shown containing electric circuitry 111 (i.e., electric circuitry
111 is contained in the inner volume of the housing of pod
structure 101), a second pod structure 102 is shown containing
electric circuitry 112, and a third pod structure 108 is shown
containing electric circuitry 118. The electric circuitry in any or
all pod structures may be communicatively coupled to the electric
circuitry in at least one other pod structure by at least one
respective communicative pathway (e.g., by at least one
electrically conductive pathway and/or by at least one optical
pathway). For example, FIG. 1 shows a first set of communicative
pathways 121 providing communicative coupling between electric
circuitry 118 of pod structure 108 and electric circuitry 111 of
pod structure 101, and a second set of communicative pathways 122
providing communicative coupling between electric circuitry 111 of
pod structure 101 and electric circuitry 112 of pod structure 102.
Communicative coupling between electric circuitries of pod
structures in device 100 may advantageously include systems,
articles, and methods for signal routing as described in U.S.
Provisional Patent Application Ser. No. 61/866,960 (now U.S.
Non-Provisional patent application Ser. No. 14/461,044) and/or
systems, articles, and methods for strain mitigation as described
in U.S. Provisional Patent Application Ser. No. 61/857,105 (now
U.S. Non-Provisional patent application Ser. No. 14/335,668), both
of which are incorporated by reference herein in their
entirety.
[0058] Throughout this specification and the appended claims the
term "communicative" as in "communicative pathway," "communicative
coupling," and in variants such as "communicatively coupled," is
generally used to refer to an engineered arrangement for
transferring and/or exchanging information. Exemplary communicative
pathways include, but are not limited to, electrically conductive
pathways (e.g., electrically conductive wires, electrically
conductive traces), magnetic pathways (e.g., magnetic media),
and/or optical pathways (e.g., optical fiber), and exemplary
communicative couplings include, but are not limited to, electrical
couplings and/or optical couplings.
[0059] Each individual pod structure within a wearable EMG device
may perform a particular function, or particular functions. For
example, in device 100, each of pod structures 101, 102, 103, 104,
105, 106, and 107 includes a respective EMG sensor 110 (only one
called out in FIG. 1 to reduce clutter) to in use detect muscle
activity of a user and to in use provide electrical signals in
response to the detected muscle activity. Thus, each of pod
structures 101, 102, 103, 104, 105, 106, and 107 may be referred to
as a respective "sensor pod." Throughout this specification and the
appended claims, the term "sensor pod" is used to denote an
individual pod structure that includes at least one sensor to
detect muscle activity of a user. Each EMG sensor may be, for
example, a respective capacitive EMG sensor that detects electrical
signals generated by muscle activity through capacitive coupling,
such as for example the capacitive EMG sensors described in U.S.
Provisional Patent Application Ser. No. 61/771,500 (now U.S.
Non-Provisional patent application Ser. No. 14/194,252).
[0060] Pod structure 108 of device 100 includes a processor 140
that in use processes the signals provided by the EMG sensors 110
of sensor pods 101, 102, 103, 104, 105, 106, and 107 in response to
detected muscle activity. Pod structure 108 may therefore be
referred to as a "processor pod." Throughout this specification and
the appended claims, the term "processor pod" is used to denote an
individual pod structure that includes at least one processor to
process signals. The processor may be any type of processor,
including but not limited to: a digital microprocessor or
microcontroller, an application-specific integrated circuit (ASIC),
a field-programmable gate array (FPGA), a digital signal processor
(DSP), a graphics processing unit (GPU), a programmable gate array
(PGA), a programmable logic unit (PLU), or the like, that in use
analyzes the signals to determine at least one output, action, or
function based on the signals.
[0061] As used throughout this specification and the appended
claims, the terms "sensor pod" and "processor pod" are not
necessarily exclusive. A single pod structure may satisfy the
definitions of both a "sensor pod" and a "processor pod" and may be
referred to as either type of pod structure. For greater clarity,
the term "sensor pod" is used to refer to any pod structure that
includes a sensor and performs at least the function(s) of a sensor
pod, and the term processor pod is used to refer to any pod
structure that includes a processor and performs at least the
function(s) of a processor pod. In device 100, processor pod 108
includes an EMG sensor 110 (not visible in FIG. 1) to sense,
measure, transduce or otherwise detect muscle activity of a user,
so processor pod 108 could be referred to as a sensor pod. However,
in exemplary device 100, processor pod 108 is the only pod
structure that includes a processor 140, thus processor pod 108 is
the only pod structure in exemplary device 100 that can be referred
to as a processor pod. In alternative embodiments of device 100,
multiple pod structures may include processors, and thus multiple
pod structures may serve as processor pods. Similarly, some pod
structures may not include sensors, and/or some sensors and/or
processors may be laid out in other configurations that do not
involve pod structures.
[0062] Processor 140 includes and/or is communicatively coupled to
a non-transitory processor-readable storage medium or memory 141.
As will be described in more detail later, memory 141 may store,
for example, a set of gesture identification flags to be
transmitted by device 100 and/or, for example, processor-executable
instructions to be executed by processor 140. For transmitting
gesture identification flags, a wearable EMG device may include at
least one output terminal communicatively coupled to processor 140.
Throughout this specification and the appended claims, the term
"terminal" is generally used to refer to any physical structure
that provides a telecommunications link through which a data signal
may enter and/or leave a device. The term "output terminal" is used
to describe a terminal that provides at least a signal output link
and the term "input terminal" is used to describe a terminal that
provides at least a signal input link; however unless the specific
context requires otherwise, an output terminal may also provide the
functionality of an input terminal and an input terminal may also
provide the functionality of an output terminal. In general, a
"communication terminal" represents the end (or "terminus") of
communicative signal transfer within a device and the beginning of
communicative signal transfer to/from an external device (or
external devices). As examples, communication terminal 151 of
device 100 may include a wireless transmitter that implements a
known wireless communication protocol, such as Bluetooth.RTM.,
WiFi.RTM., or Zigbee.RTM., while communication terminal 152 may
include a tethered communication port such as Universal Serial Bus
(USB) port, a micro-USB port, a Thunderbolt.RTM. port, and/or the
like.
[0063] For some applications, device 100 may also include at least
one accelerometer 160 (e.g., an inertial measurement unit, or
"IMU," that includes at least one accelerometer and/or at least one
gyroscope) communicatively coupled to processor 140. In use, the at
least one accelerometer may detect, sense, and/or measure motion
effected by a user and provide signals in response to the detected
motion. As will be described in more detail later, signals provided
by accelerometer 160 may be processed together with signals
provided by EMG sensors 110 by processor 140.
[0064] Throughout this specification and the appended claims, the
term "accelerometer" is used as a general example of an inertial
sensor and is not intended to limit (nor exclude) the scope of any
description or implementation to "linear acceleration."
[0065] Throughout this specification and the appended claims, the
term "provide" and variants such as "provided" and "providing" are
frequently used in the context of signals. For example, an EMG
sensor is described as "providing at least one signal" and an
accelerometer is described as "providing at least one signal."
Unless the specific context requires otherwise, the term "provide"
is used in a most general sense to cover any form of providing a
signal, including but not limited to: relaying a signal, outputting
a signal, generating a signal, routing a signal, creating a signal,
transducing a signal, and so on. For example, a capacitive EMG
sensor may include at least one electrode that capacitively couples
to electrical signals from muscle activity. This capacitive
coupling induces a change in a charge or electrical potential of
the at least one electrode which is then relayed through the sensor
circuitry and output, or "provided," by the sensor. Thus, the
capacitive EMG sensor may "provide" an electrical signal by
relaying an electrical signal from a muscle (or muscles) to an
output (or outputs). In contrast, an accelerometer may include
components (e.g., piezoelectric, piezoresistive, capacitive, etc.)
that are used to convert physical motion into electrical signals.
The accelerometer may "provide" an electrical signal by detecting
motion and generating an electrical signal in response to the
motion.
[0066] As previously described, each of pod structures 101, 102,
103, 104, 105, 106, 107, and 108 may include electric circuitry.
FIG. 1 depicts electric circuitry 111 inside the inner volume of
sensor pod 101, electric circuitry 112 inside the inner volume of
sensor pod 102, and electric circuitry 118 inside the inner volume
of processor pod 118. The electric circuitry in any or all of pod
structures 101, 102, 103, 104, 105, 106, 107 and 108 (including
electric circuitries 121, 122, and 128) may include any or all of:
an amplification circuit to in use amplify electrical signals
provided by at least one EMG sensor 110, a filtering circuit to in
use remove unwanted signal frequencies from the signals provided by
at least one EMG sensor 110, and/or an analog-to-digital conversion
circuit to in use convert analog signals into digital signals.
Device 100 may also include a battery (not shown in FIG. 1) to in
use provide a portable power source for device 100.
[0067] Signals that are provided by EMG sensors 110 in device 100
are routed to processor pod 108 for processing by processor 140. To
this end, device 100 employs a set of communicative pathways (e.g.,
121 and 122) to route the signals that are provided by sensor pods
101, 102, 103, 104, 105, 106, and 107 to processor pod 108. Each
respective pod structure 101, 102, 103, 104, 105, 106, 107, and 108
in device 100 is communicatively coupled to at least one other pod
structure by at least one respective communicative pathway from the
set of communicative pathways. Each communicative pathway (e.g.,
121 and 122) may be realized in any communicative form, including
but not limited to: electrically conductive wires or cables, ribbon
cables, fiber-optic cables, optical/photonic waveguides,
electrically conductive traces carried by a rigid printed circuit
board, and/or electrically conductive traces carried by a flexible
printed circuit board.
[0068] The present systems, articles, and methods describe a
human-electronics interface in which a wearable EMG device (e.g.,
device 100) is used to control another electronic device. The
human-electronics interface may be characterized as a system that
enables electromyographic control of an electronic device.
[0069] FIG. 2 is an illustrative diagram of a system 200 that
enables electromyographic control of an electronic device in
accordance with the present systems, articles, and methods. System
200 includes a wearable EMG device 270 and an unspecified
electronic device 280. Wearable EMG device 270 may be, as an
illustrative example, substantially similar to wearable EMG device
100 from FIG. 1. That is, exemplary wearable EMG device 270
includes a set of pod structures 201 (only one called out in FIG. 2
to reduce clutter) that form physically coupled links of device
270, where each pod structure 201 includes a respective EMG sensor
210 (e.g., a respective capacitive EMG sensor) to in use sense,
measure, transduce or otherwise detect muscle activity of a user
and provide electrical signals in response to the muscle activity.
As previously described, however, the present systems, articles,
and methods may be implemented using wearable EMG devices that do
not employ pod structures.
[0070] Each pod structure 201 is electrically coupled to at least
one adjacent pod structure by at least one respective communicative
pathway 220 to route signals in between pod structures (e.g., to
route signals from sensor pods to a processor pod). Each pod
structure 201 is also physically coupled to two adjacent pod
structures 201 by at least one adaptive coupler 260 and the set of
pod structures forms a perimeter of an annular or closed loop
configuration. FIG. 2 shows device 270 in an expanded annular or
closed loop configuration adapted to fit the arm of a larger user
than the contracted annular or closed loop configuration of device
100 from FIG. 1. As a result, adaptive couplers 260 (only one
called out in FIG. 2) providing adaptive physical coupling between
adjacent pairs of pod structures 201 are visible in FIG. 2, whereas
such adaptive couplers 260 are not visible in FIG. 1.
[0071] Each pod structure 201 includes respective electric
circuitry 230 and at least one electric circuitry 230 includes a
first processor 240 (e.g., akin to processor 140 in device 100 of
FIG. 1). At least one electric circuitry 230 may include an IMU
and/or at least one accelerometer. Device 270 also includes an
output terminal 250 to in use interface with unspecified electronic
device 280. For example, device 270 is operative to in use send
gesture identification flags to unspecified electronic device 280
through output terminal 250.
[0072] Unspecified electronic device 280 may be any electronic
device, including but not limited to: a desktop computer, a laptop
computer, a tablet computer, a mobile phone, a smartphone, a
portable electronic device, an audio player, a television, a video
player, a video game console, a robot, a light switch, and/or a
vehicle. Electronic device 280 is denominated as "unspecified"
herein to emphasize the fact that the gesture identification flags
output by wearable EMG device 270 are generic to a variety of
electronic devices and/or applications executed by the electronic
devices. The electronic device 280, its operating characteristics
and/or the operating characteristics of applications executed by
the electronic device 280 may not be a priori known by the EMG
device 270 during use, or even prior to use when a mapping between
signals, gesture flags, and/or gestures is initially defined or
established. As previously described, a data signal output by
device 270 through output terminal 250 may include a gesture
identification flag as a first portion thereof and may also include
at least a second portion to implement known telecommunications
protocols (e.g., Bluetooth.RTM.). Thus, electronic device 280 may
remain "unspecified" with respect to the gesture identification
flag portion(s) of signals output by EMG device 270 but electronic
device 280 may be "specified" by the telecommunications portion(s)
of signals output by EMG device 270 (if such specification is
necessary for signal transfer, e.g., to communicatively "pair"
device 270 and device 280 if required by the telecommunications
protocol being implemented). For example, electronic device 280 may
be and remain "unspecified" while muscle activity is detected by
EMG device 270 and while the processor in EMG device 270 determines
a gesture identification flag based, at least in part, on the
detected muscle activity. After a gesture identification flag is
determined by the processor in EMG device 270, electronic device
280 may become "specified" when the gesture identification flag is
combined with telecommunication data and transmitted to electronic
device 280. In this scenario, the gesture identification flag
itself does not include any information that is specific to
electronic device 280 and therefore electronic device 280 is
"unspecified" in relation to the gesture identification flag.
[0073] Electronic device 280 includes an input terminal 281 to in
use interface with wearable EMG device 270. For example, device 280
may receive gesture identification flags from device 270 through
input terminal 281. Device 280 also includes a second processor 283
to in use process gesture identification flags received from device
270. Second processor 283 may include or be communicatively coupled
to a non-transitory processor-readable storage medium or memory 284
that stores processor-executable instructions to be executed by
second processor 283.
[0074] Wearable EMG device 270 and electronic device 280 are, in
use, communicatively coupled by communicative link 290. More
specifically, output terminal 250 of wearable EMG device 270 is, in
use, communicatively coupled to input terminal 281 of electronic
device 280 by communicative link 290. Communicative link 290 may be
used to route gesture identification flags from wearable EMG device
270 to electronic device 280. Communicative link 290 may be
established in variety of different ways. For example, output
terminal 250 of wearable EMG device 270 may include a first
tethered connector port (e.g., a USB port, or the like), input
terminal 281 of electronic device 280 may include a second tethered
connector port, and communicative link 290 may be established
through a communicative pathway (e.g., an electrical or optical
cable, wire, circuit board, or the like) that communicatively
couples the first connector port to the second connector port to
route gesture identification flags from output terminal 250 to
input terminal 281. Alternatively, output terminal 250 of wearable
EMG device 270 may include a wireless transmitter and communicative
link 290 may be representative of wireless communication between
wearable EMG device 270 and electronic device 280. In this case,
input terminal 281 of electronic device 280 may include a wireless
receiver to in use wirelessly receive gesture identification flags
from the wireless transmitter of wearable EMG device 270 (using,
for example, established wireless telecommunication protocols, such
as Bluetooth.RTM.); or, input terminal 281 may be communicatively
coupled to a wireless receiver 282 (such as a USB dongle
communicatively coupled to a tethered connector port of input
terminal 281) to in use wirelessly receive gesture identification
flags from the wireless transmitter of wearable EMG device 270.
[0075] As previously described, known proposals for
human-electronics interfaces that employ a wearable EMG device are
limited in their versatility because they involve mapping gestures
to functions on-board the wearable EMG device itself. Thus, in
known proposals, the wearable EMG device outputs control signals
(i.e., "commands") that embody pre-defined instructions to effect
pre-defined functions that are specific to a pre-defined receiving
device. If a user wishes to use such a wearable EMG device for a
different purpose (i.e., to control a different receiving device,
or a different application within the same receiving device), then
the definitions of the commands themselves must be re-programmed
within the wearable EMG device. Conversely, the various embodiments
described herein provide systems, articles, and methods for
human-electronics interfaces that employ a wearable EMG device that
controls functions of another electronic device by outputting
generic gesture identification flags that are not specific to the
particular electronic device being controlled. The electronic
device being controlled may include or may access an Application
Programming Interface (i.e., an "API" including instructions and/or
data or information (e.g., library) stored in a non-transitory
processor-readable storage medium or memory) through which a user
may define how gesture identification flags are to be interpreted
by the electronic device being controlled (i.e., where the user may
define how the electronic device responds to gesture identification
flags). The present systems, articles, and methods greatly enhance
the versatility of human-electronics interfaces by employing a
wearable EMG device that outputs the same gesture identification
flags regardless of what it is being used to control, and may
therefore be used to control virtually any electronic receiving
device. The functions or operations that are controlled by the
wearable EMG devices described herein are defined within the
receiving device (or within the applications within the receiving
device) rather than within the wearable EMG device.
[0076] FIG. 3 is a flow-diagram showing a method 300 of operating a
wearable EMG device to provide electromyographic control of an
electronic device in accordance with the present systems, articles,
and methods. The electronic device may be any "unspecified"
electronic device as described previously. For example, the
electronic device may be any downstream processor-based device. The
wearable EMG device may include at least one EMG sensor, a
processor, and an output terminal (i.e., the wearable EMG device
may be substantially similar to wearable EMG device 100 from FIG. 1
and wearable EMG device 270 from FIG. 2). Method 300 includes four
acts 301, 302, 303, and 304, though those of skill in the art will
appreciate that in alternative embodiments certain acts may be
omitted and/or additional acts may be added. Those of skill in the
art will also appreciate that the illustrated order of the acts is
shown for exemplary purposes only and may change in alternative
embodiments.
[0077] At 301, muscle activity of a user (i.e., a wearer of the
wearable EMG device) is sensed, measured, transduced or otherwise
detected by at least one EMG sensor of the wearable EMG device. As
previously described, the at least one EMG sensor may be, for
example, a capacitive EMG sensor and sensing, measuring,
transducing or otherwise detecting muscle activity of the user may
include, for example, capacitively coupling to electrical signals
generated by muscle activity of the user.
[0078] At 302, at least one signal is provided from the at least
one EMG sensor to the processor of the wearable EMG device in
response to the sensed, measured, transduced or otherwise detected
muscle activity. The at least one signal may be an analog signal
that is amplified, filtered, and converted to digital form by
electric circuitry within the wearable EMG device. Providing the at
least one signal from the at least one EMG sensor to the processor
may include routing the at least one signal to the processor
through one or more communicative pathway(s) as described
previously.
[0079] At 303, a gesture identification flag is determined by the
processor of the wearable EMG device, based at least in part on the
at least one signal provided from the at least one EMG sensor to
the processor. The gesture identification flag is substantially
independent of the downstream electronic device. As will be
described in more detail later (e.g., with reference to FIG. 5),
determining a gesture identification flag by the processor may
implement a range of different algorithms, including but not
limited to: a look-up table, a mapping, a machine learning
algorithm, a pattern recognition algorithm, and the like. In some
applications, the wearable EMG device may include a non-transitory
processor-readable medium that stores a set of gesture
identification flags and/or stores processor-executable
instructions that, when executed by the processor of the wearable
EMG device, cause the processor to determine a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the processor.
In such a case, act 303 may include executing the
processor-executable instructions by the processor to cause the
processor to determine a gesture identification flag based at least
in part on the at least one signal provided from the at least one
EMG sensor to the processor.
[0080] At 304, the gesture identification flag is transmitted to
the electronic device by the output terminal of the wearable EMG
device. As previously described, the output terminal of the
wearable EMG device may include a wireless transmitter, and
transmitting the gesture identification flag to the electronic
device may include wirelessly transmitting the gesture
identification flag to the electronic device by the wireless
transmitter.
[0081] As an example, the at least one EMG sensor may include a
first EMG sensor and at least a second EMG sensor, and muscle
activity of the user may be sensed, measured, transduced or
otherwise detected by the first EMG sensor and by at least the
second EMG sensor (at 301). In this case at least a first signal is
provided from the first EMG sensor to the processor of the wearable
EMG device in response to the detected muscle activity (at 302) and
at least a second signal is provided from at least the second EMG
sensor to the processor of the wearable EMG device in response to
the detected muscle activity (at 302). The processer of the
wearable EMG device may then determine (at 303) a gesture
identification flag based at least in part on both the at least a
first signal provided from the first EMG sensor to the processor
and the at least a second signal provided from at least the second
EMG sensor to the processor.
[0082] As previously described, in some applications it may be
advantageous to combine or otherwise make use of both EMG signals
and motion signals sensed, measured or otherwise detected, for
example, by an accelerometer. To this end, the wearable EMG device
may include at least one accelerometer, and an additional method
employing further acts may be combined with acts 301-304 of method
300 to detect and process motion signals.
[0083] FIG. 4 is a flow-diagram showing a method 400 of operating a
wearable EMG device to provide both electromyographic and motion
control of an electronic device in accordance with the present
systems, articles, and methods. Method 400 includes three acts 401,
402, and 403, though those of skill in the art will appreciate that
in alternative embodiments certain acts may be omitted and/or
additional acts may be added. Those of skill in the art will also
appreciate that the illustrated order of the acts is shown for
exemplary purposes only and may change in alternative embodiments.
Method 400 is optionally performed in conjunction with method 300
from FIG. 3 and, if performed, performed using the same wearable
EMG device as that used to perform method 300. For example, while
acts 301 and 302 of method 300 are performed by EMG sensors of the
wearable EMG device, acts 401 and 402 of method 400 may optionally
be performed by at least one accelerometer of the wearable EMG
device.
[0084] At 401, motion effected by the user of the wearable EMG
device is sensed, measured, transduced or otherwise detected by at
least one accelerometer in the wearable EMG device. The at least
once accelerometer may be part of an IMU that includes multiple
accelerometers (such as an MPU-9150 Nine-Axis MEMS
MotionTracking.TM. Device from InvenSense). The motion effected by
the user that may be detected and/or measured may include, e.g.,
translation in one or multiple spatial directions and/or rotation
about one or more axes in one or more spatial directions. The
motion(s) may be detected in terms of a presence or absence of
translation and/or rotation, and/or measured in terms of a speed of
translation and/or rotation and/or acceleration of translation
and/or rotation.
[0085] At 402, at least one signal is provided from the at least
one accelerometer to the processor in response to the sensed,
measured, transduced or otherwise detected motion. The at least one
signal may be an analog signal that is amplified, filtered, and
converted to digital form by electric circuitry within the wearable
EMG device. The at least one signal may be routed to the processor
in the wearable EMG device via one or more communicative pathway(s)
as described previously.
[0086] As previously described, act 303 of method 300 involves
determining, by a processor of the wearable EMG device, a gesture
identification flag based at least in part on the at least one
signal provided from the at least one EMG sensor to the processor
in response to detected muscle activity. In applications where the
wearable EMG device further includes at least one accelerometer and
acts 401 and 402 of method 400 are performed, act 303 of method 300
may be replaced by act 403 of method 400.
[0087] At 403, a gesture identification flag is determined by the
processor, based at least in part on the at least one signal
provided from the at least one EMG sensor to the processor and the
at least one signal provided from the at least one accelerometer to
the processor. The wearable EMG device may include a non-transitory
processor-readable medium (e.g., memory 284 of device 280 from FIG.
2) that stores processor-executable instructions that, when
executed by the processor, cause the processor to determine a
gesture identification flag based on the at least one signal
provided from the at least one EMG sensor to the processor and the
at least one signal provided from the at least one accelerometer to
the processor (i.e., to perform act 403). Thus, act 403 may include
executing the processor-executable instructions stored in the
non-transitory processor-readable medium.
[0088] In some implementations, the at least one signal provided
from the at least one accelerometer to the processor (i.e., at act
402) may be combined with at least one signal provided from at
least one EMG sensor to the processor (i.e., at act 302 of method
300 from FIG. 3) by the processor of the wearable EMG device. Thus,
act 403 requires that acts 401 and 402 from method 400 and acts 301
and 302 from method 300 all be completed. The at least one signal
from the at least one accelerometer and the at least one signal
from the at least one EMG sensor may be summed, concatenated,
overlaid, or otherwise combined in any way by the processor to
produce, provide or output any number of signals, operations,
and/or results.
[0089] After act 403, the gesture identification flag may be
transmitted or output by an output terminal of the wearable EMG
device (i.e., according to act 304 of method 300) to any downstream
electronic device and interpreted or otherwise processed by the
downstream electronic device to cause the downstream electronic
device to perform some function(s) or operation(s), or otherwise
effect an interaction with or response from the downstream
electronic device, in response to the gesture identification
flag.
[0090] In accordance with the present systems, articles, and
methods, at least one signal provided by at least one EMG sensor
(either alone or together with one or more signals provided by one
or more transducers such as an accelerometer or other motion or
acceleration responsive transducers) may represent or be indicative
of a gesture performed by a user of a wearable EMG device.
Determining a gesture identification flag corresponding to that at
least one signal may involve identifying, by a processor, the
gesture performed by the user based at least in part on the at
least one signal(s) from the EMG and/or other sensors or
transducers, and determining, by the processor, a gesture
identification flag that corresponds to that determined gesture.
Unless the specific context requires otherwise, throughout this
specification and the appended claims "a" gesture identification
flag should be interpreted in a general, inclusive sense as "at
least one" gesture identification flag with the understanding that
determining any number of gesture identification flags (e.g.,
determining one gesture identification flag, or determining
multiple gesture identification flags) includes determining "a"
gesture identification flag. Each gesture identification flag may
include, or be represented by, one or more bits of information.
Furthermore, "determining" a gesture identification flag by a
processor may be achieved through a wide variety of different
techniques. For example, a processor may determine a gesture
identification flag by performing or otherwise effecting a mapping
between gestures (e.g., between EMG and/or accelerometer signals
representative of gestures) and gesture identification flags (e.g.,
by invoking a stored look-up table or other form of stored
processor-executable instructions providing and/or effecting
mappings between gestures and gesture identification flags), or a
processor may determine a gesture identification flag by performing
an algorithm or sequence of data processing acts (e.g., by
executing stored processor-executable instructions dictating how to
determine a gesture identification flag based at least in part on
one or more signal(s) provided by at least one EMG sensor and/or at
least one accelerometer).
[0091] FIG. 5 is a schematic illustration showing an exemplary
mapping 500 between a set of exemplary gestures and a set of
exemplary gesture identification flags in accordance with the
present systems, articles, and methods. Mapping 500 may be
representative of processor-executable instructions that are
defined in advance of determining gesture identification flags
based at least in part on at least one EMG signal (and, e.g.,
executed by a processor to perform the act of determining gesture
identification flags based at least in part on at least one EMG
signal), or mapping 500 may be representative of the results (i.e.,
the mapping that is effected) when gesture identification flags are
determined based at least in part on at least one EMG signal. In
other words, mapping 500 characterizes: i) a prescription, embodied
in processor-executable instructions, for or definition of how
gestures (e.g., EMG and/or accelerometer signals that are
representative of gestures) are to be mapped to gesture
identification flags by a processor when determining a gesture
identification flag based at least in part on at least one signal
provided from at least one EMG sensor to the processor; or ii) the
end results when a processor performs an algorithm or series of
data processing steps to determine a gesture identification flag
based at least in part on at least one signal provided from at
least one EMG sensor to the processor. In the former
characterization (i.e., characterization i)), mapping 500 may be
stored as a look-up table or set of defined processor-executable
"mapping instructions" in a non-transitory processor-readable
storage medium and invoked/executed by the processor when
determining a gesture identification flag. In the latter
characterization (i.e., characterization ii)), mapping 500 may not
be stored in a non-transitory processor-readable storage medium
itself, but instead processor-executable instructions to perform an
algorithm or series of data processing acts may be stored in the
non-transitory processor-readable storage medium and mapping 500
may represent the results of executing the stored
processor-executable instructions by the processor when determining
a gesture identification flag. In either case, the present systems,
articles, and methods provide a framework in which a wearable EMG
device is programmed with processor-executable instructions that
embody (i.e., in accordance with characterization i)) and/or
produce/effect (i.e., in accordance with characterization ii)) a
mapping from gestures to gesture identification flags, such as
exemplary mapping 500 from FIG. 5.
[0092] As shown in mapping 500, each gesture identification flag
may, for example, comprise a bit string (e.g., an 8-bit data byte
as illustrated) that uniquely maps to a corresponding gesture
performed by a user. For example, a "gun" or "point" hand gesture
may correspond/map to gesture identification flag 00000001 as
illustrated, a "thumbs up" gesture may correspond/map to gesture
identification flag 00000010 as illustrated, a "fist" gesture may
correspond/map to gesture identification flag 00000011 as
illustrated, and a "rock on" gesture may correspond/map to gesture
identification flag 00000100 as illustrated. A person of skill in
the art will appreciate that an 8-bit data byte can be used to
represent 256 unique gesture identification flags (corresponding to
256 unique gestures). In practice, gesture identification flags
having any number of bits may be used, and if desired, multiple
gestures may map to the same gesture identification flag and/or the
same gesture may map to multiple gesture identification flags. In
accordance with the present systems, articles, and methods, a
gesture identification flag contains only information that
identifies (i.e., maps to) a gesture performed by a user of a
wearable EMG device. A gesture identification flag does not contain
any information about a function or operation that the
corresponding gesture may be used to control. A gesture
identification flag does not contain any information about any
downstream electronic device and/or application that the
corresponding gesture may be used to control. A gesture
identification flag may be appended, adjoined, supplemented, or
otherwise combined with additional data bits as needed for, e.g.,
the purposes of telecommunications.
[0093] Mapping 500 represents gestures with actual illustrations of
hands solely for ease of illustration and description. In practice,
a gesture may be represented by any corresponding configuration of
signals provided by at least one EMG sensor and/or at least one
accelerometer. For example, a gesture may be represented by a
particular signal waveform, a particular signal value, or a
particular configuration/arrangement/permutation/combination of
signal waveforms/values.
[0094] The present systems, articles, and methods describe
human-electronics interfaces. Methods 300 and 400 provide methods
of operating a wearable EMG device to control an unspecified
electronic device (e.g., methods of operating device 100 from FIG.
1 or device 270 from FIG. 2). A complete human-electronics
interface may involve acts performed by both the controller and the
receiver (e.g., methods of operating system 200 from FIG. 2).
[0095] FIG. 6 is a flow-diagram showing a method 600 of
electromyographically controlling at least one function of an
electronic device by a wearable EMG device in accordance with the
present systems, articles, and methods. The wearable EMG device
includes at least one EMG sensor, a first processor, and an output
terminal (with the at least one EMG sensor and the output terminal
each communicatively coupled to the first processor) and the
electronic device includes an input terminal and a second processor
(with the input terminal communicatively coupled to the second
processor). Method 600 includes seven acts 601, 602, 603, 604, 611,
612, and 613, though those of skill in the art will appreciate that
in alternative embodiments certain acts may be omitted and/or
additional acts may be added. Those of skill in the art will also
appreciate that the illustrated order of the acts is shown for
exemplary purposes only and may change in alternative embodiments.
Acts 601, 602, 603, and 604 are performed by the wearable EMG
device to produce and transmit signals and acts 611, 612, and 613
are performed by the electronic device to receive and respond to
the transmitted signals.
[0096] Acts 601, 602, 603, and 604 are substantially similar to
acts 301, 302, 303, and 304 (respectively) of method 300 from FIG.
3. At 601, muscle activity of a user is sensed, measured,
transduced or otherwise detected by at least one EMG sensor of the
wearable EMG device. At 602, at least one signal is provided from
the at least one EMG sensor to a first processor on-board the
wearable EMG device in response to the detected muscle activity. At
603, the first processor determines a gesture identification flag
based at least in part on the at least one signal provided from the
at least one EMG sensor to the first processor. At 604, the gesture
identification flag is transmitted by the output terminal of the
wearable EMG device. In some applications, the wearable EMG device
may include at least one accelerometer and the wearable EMG device
may be used to perform method 400 from FIG. 4. Therefore, act 603
may comprise determining a gesture identification flag based at
least in part on both the at least one signal provided from the at
least one EMG sensor to the first processor and the at least one
signal provided from the at least one accelerometer to the first
processor.
[0097] At 611, the gesture identification flag that is transmitted
or output by the output terminal of the wearable EMG device at 604
is received by the input terminal of the electronic device. As
previously described, transmission of gesture identification flags
between the wearable EMG device and the electronic device may be
through a wired or wireless communicative link (e.g. communicative
link 290 from FIG. 2).
[0098] At 612, a second processor on-board the electronic device
determines a function of the electronic device based at least in
part on the gesture identification flag received by the input
terminal of the electronic device at 611. As described previously,
the electronic device may include a non-transitory
processor-readable storage medium or memory that stores an API or
other information or data structures (e.g., implemented as one or
library(ies)) through which a user may define mappings (i.e.,
processor-executable instructions that embody and/or produce/effect
mappings) between gesture identification flags and functions of the
electronic device, and/or the non-transitory processor-readable
storage medium may store processor-executable instructions that,
when executed by the second processor, cause the second processor
to determine a function of the electronic device based at least in
part on the gesture identification flag.
[0099] At 613, the function determined at 612 is performed by the
electronic device. The function may be any function or operation of
the electronic device. For example, if the electronic device is an
audio and/or video player (or a computer running an application
that performs audio and/or video playback), then the corresponding
function may be a PLAY function that causes the audio/video to
play, a STOP function that causes the audio/video to stop, a REWIND
function that causes the audio/video to rewind, a FAST FORWARD
function that causes the audio/video to fast forward, and so
on.
[0100] Throughout this specification and the appended claims,
reference is often made to "determining a function of an electronic
device based at least in part on a gesture identification flag."
Unless the specific context requires otherwise, throughout this
specification and the appended claims "a" function should be
interpreted in a general, inclusive sense as "at least one"
function with the understanding that determining any number of
functions (e.g., determining one function, or determining multiple
functions) includes determining "a" function. Furthermore,
"determining" a function by a processor may be achieved through a
wide variety of different techniques. For example, a processor may
determine a function by employing a defined mapping between gesture
identification flags and functions (e.g., by invoking a stored
look-up table or other form of stored processor-executable
instructions providing defined mappings between gesture
identification flags and functions), or a processor may determine a
function by performing an algorithm or sequence of data processing
steps (e.g., by executing stored processor-executable instructions
dictating how to determine a function based at least in part on one
or more gesture identification flag(s)).
[0101] FIG. 7 is a schematic illustration showing an exemplary
mapping 700 between a set of exemplary gesture identification flags
and a set of exemplary functions of an electronic device in
accordance with the present systems, articles, and methods. Similar
to mapping 500 from FIG. 5, mapping 700 may be characterized as: i)
a prescription for how gesture identification flags are to be
mapped to functions by a processor when determining a function
based at least in part on a gesture identification flag received
from a wearable EMG device; or ii) the end results when a processor
performs an algorithm or series of data processing acts to
determine a function based at least in part on a gesture
identification flag received from a wearable EMG device. In the
former characterization (i.e., characterization i)), mapping 700
may be stored as a look-up table or set of defined
processor-executable "mapping instructions" in a non-transitory
processor-readable storage medium and invoked by the processor when
determining a function of the electronic device. In the latter
characterization (i.e., characterization ii)), mapping 700 may not
be stored in a non-transitory processor-readable storage medium
itself, but instead processor-executable instructions to perform an
algorithm or series of data processing acts may be stored in the
non-transitory processor-readable storage medium and mapping 700
may represent the results of executing the stored
processor-executable instructions by the processor to determine a
function of the electronic device. In either case, the present
systems, articles, and methods provide a framework in which generic
gesture identification flags are output by a wearable EMG device
and a receiving device is programmed (and/or programmable through,
e.g., an API or other information or data or calls) with
processor-executable instructions that embody and/or produce/effect
a mapping from gesture identification flags to functions of the
electronic device, such as exemplary mapping 700 from FIG. 7.
[0102] For the illustrative example of FIG. 7, the electronic
device is an audio player; however, any electronic device may
include or be communicatively coupled to (or be adapted to include
or be communicatively coupled to) a non-transitory
processor-readable storage medium or memory that stores
processor-executable instructions that embody and/or produce/effect
a mapping such as mapping 700. As shown in mapping 700, each
gesture identification flag may, for example, be a bit string
(e.g., an 8-bit data byte as illustrated) that uniquely maps to a
corresponding function of the electronic device. For example, a
00000001 gesture identification flag may map/correspond to a REWIND
function of an audio player as illustrated, a 00000010 gesture
identification flag may map/correspond to a PLAY function of an
audio player as illustrated, a 00000011 gesture identification flag
may map/correspond to a STOP function of an audio player as
illustrated, and a 00000100 gesture identification flag may
map/correspond to a FAST FORWARD function of an audio player as
illustrated. A person of skill in the art will appreciate that an
8-bit data byte can be used to represent 256 unique gesture
identification flags (corresponding to 256 unique functions). In
practice, gesture identification flags having any number of bits
may be used, multiple gesture identification flags may be mapped to
the same function, and/or a single gesture identification flag may
map to multiple functions.
[0103] In accordance with the present systems, articles, and
methods, processor-executable instructions that embody and/or
produce/effect a mapping from gestures to gesture identification
flags (e.g., mapping 500 from FIG. 5) may be stored in a
non-transitory processor-readable storage medium or memory on-board
a wearable EMG device (e.g., memory 141 of device 100 from FIG. 1)
and communicatively coupled to a first processor (e.g., processor
140 of device 100), and processor-executable instructions that
embody and/or produce/effect a mapping from gesture identification
flags to functions of an electronic device (e.g., mapping 700 from
FIG. 7) may be stored in a non-transitory processor-readable
storage medium or memory on-board an electronic device (e.g.,
memory 282 of device 280 from FIG. 2) and communicatively coupled
to a second processor (e.g., processor 283 of device 280 in FIG.
2). In this way, gesture identification flags may be determined by
the first processor on-board the wearable EMG device based on
signals from one or more sensor(s) (e.g., EMG sensors and/or
inertial sensors) in accordance with, e.g., mapping 500 of FIG. 5;
the gesture identification flags may be transmitted or output to a
receiving device; and then functions of the receiving device may be
determined by the second processor on-board the receiving device
based on the gesture identification flags in accordance with, e.g.,
mapping 700 from FIG. 7. For example, signals corresponding to a
"gun" or "point" gesture (e.g., outwardly extended index finger
with other fingers curled upon themselves) may be processed by the
first processor of the wearable EMG device to determine gesture
identification flag 00000001 according to mapping 500 from FIG. 5,
the 00000001 flag may be transmitted to the electronic device
(through a wired or wireless communicative link), and the 00000001
flag may be processed by the second processor of the electronic
device to determine a REWIND function in accordance with mapping
700.
[0104] In accordance with the present systems, articles, and
methods, an electronic device may store multiple mappings (e.g.,
multiple sets of processor-executable instructions that embody
and/or produce/effect mappings) between gesture identification
flags and functions of the electronic device, and when the
electronic device receives a gesture identification flag it may
perform a corresponding function based on the implementation of one
of the multiple stored mappings (e.g., one or more of the multiple
sets of processor-executable instructions). For example, the
electronic device may be a computer such as a desktop computer, a
laptop computer, a tablet computer, or the like. The computer may
include a non-transitory processor-readable storage medium or
memory that stores multiple mappings (e.g., multiple sets of
processor-executable instructions that embody and/or produce/effect
mappings) between gesture identification flags and functions of the
computer (e.g., multiple variants of mapping 700 from FIG. 7), with
each mapping corresponding to and invoked by a different
application executed by the computer. For example, the
non-transitory processor-readable storage medium may store a first
mapping (e.g., a first set of processor-executable instructions
that embody and/or produce/effect a first mapping) between gesture
identification flags and functions (e.g., a first variant of
mapping 700 from FIG. 7) to be invoked by a first application run
on the computer, a second mapping (e.g., a second set of
processor-executable instructions that embody and/or produce/effect
a second mapping) between gesture identification flags and
functions (e.g., a second variant of mapping 700 from FIG. 7) to be
invoked by a second application run on the computer, a third
mapping (e.g., a third set of processor-executable instructions
that embody and/or produce/effect a third mapping) between gesture
identification flags and functions (e.g., a third variant of
mapping 700 from FIG. 7) to be invoked by a third application run
on the computer, and so on. Each of the first, second, and third
applications may be any application, including but not limited to:
an audio/video playback application, a video game application, a
drawing or modeling application, a control application, a
communication application, a browsing or navigating applications,
and so on. As previously described, the non-transitory
processor-readable medium of the computer may store an API or other
data or information through which a user may program
processor-executable instructions that embody and/or produce/effect
any mapping(s) between gesture identification flags and functions
of any electronic device (including but not limited to the computer
itself). For example, a user may use an API executed by a computer
to define processor-executable instructions that embody and/or
produce/effect mappings between gesture identification flags and
functions of the computer itself (e.g., functions of one or
multiple applications executed by the computer itself), or the user
may use an API executed by a computer to define
processor-executable instructions (such as firmware or embedded
software instructions) that are then ported to, installed on,
loaded in, or otherwise received by a separate electronic device,
where the processor-executable instructions embody and/or
produce/effect mappings between gesture identification flags and
functions of the separate electronic device. In accordance with the
present systems, articles, and methods, virtually any application
run on a computer or any other electronic device may be adapted to
respond to generic gesture identification flags output by a
wearable EMG device. Thus, in some cases, method 600 may include an
additional act performed by the electronic device, the additional
act being selecting and/or initializing a specific application of
the electronic device (e.g., stored in and/or to be executed by the
electronic device) to be controlled by the wearable EMG device.
Selecting and/or initializing a specific application of the
electronic device may include selecting/initializing a first set of
processor-executable instructions that embody and/or produce/effect
a first mapping from multiple sets of processor-executable
instructions that embody and/or produce/effect multiple mappings
(e.g., one set of processor-executable instructions that embody
and/or produce/effect a particular mapping from a plurality of sets
of processor-executable instructions that embody and/or
produce/effect a plurality of respective mappings).
[0105] In accordance with the present systems, articles, and
methods, a wearable EMG device may be used to control multiple
electronic devices, or multiple applications within a single
electronic device. Such is distinct from known proposals for
human-electronics interfaces that employ a wearable EMG device, at
least because the known proposals typically store a direct mapping
from gestures to functions within the wearable EMG device itself,
whereas the present systems, articles, and methods describe an
intermediate mapping from gestures (e.g., from EMG and/or
accelerometer signals representative of gestures) to gesture
identification flags that are stored and executed by the wearable
EMG device and then mappings from gesture identification flags to
functions that are stored and executed by the downstream electronic
device. In accordance with the present systems, articles, and
methods, the mapping from gestures to gesture identification flags
stored and executed by the wearable EMG device is independent of
the downstream electronic device and the same mapping from gestures
to gesture identification flags may be stored and executed by the
wearable EMG device regardless of the nature and/or function(s) of
the downstream electronic device.
[0106] The implementation of gesture identification flags as
described herein enables users to employ the same wearable EMG
device to control a wide range of electronic devices and/or a wide
range of applications within a single electronic device. Since the
gesture identification flags output by the wearable EMG device are
not tied to any specific functions or commands, a user may define
their own mappings (including their own techniques for performing
mappings) between gesture identification flags and electronic
device functions. For example, a user may adapt the
human-electronics interfaces described herein to control virtually
any functions of virtually any electronic device (e.g., to control
virtually any application executed by a computer) by defining
processor-executable instructions that embody and/or produce a
corresponding mapping between gesture identification flags and
electronic device functions (such as mapping 700 from FIG. 7) and
establishing automatic execution of the processor-executable
instructions by the electronic device in response to receiving
gesture identification flags. The processor-executable instructions
may be defined for/within the electronic device itself without
making any modifications to the wearable EMG device.
[0107] The various embodiments described herein provide
human-electronics interfaces in which a wearable EMG device (i.e.,
a controller) provides generic signal "flags" and a downstream
receiving device interprets and responds to the generic flags. The
flags provided by the wearable EMG device are substantially
independent of any downstream receiving device. In accordance with
the present systems, articles, and methods, other forms of
controllers (i.e., controllers that are not wearable and/or
controllers that do not employ EMG sensors) may similarly be
configured to provide generic flags in this way. For example,
instead of or in addition to employing EMG sensors and/or
accelerometers providing gesture control, a controller that
operates in accordance with the present systems, articles, and
methods may employ, for example, tactile sensors (e.g., buttons,
switches, touchpads, or keys) providing manual control, acoustic
sensors providing voice-control, optical/photonic sensors providing
gesture control, or any other type(s) of user-activated sensors
providing any other type(s) of user-activated control. Thus, the
teachings of the present systems, articles, and methods may be
applied using virtually any type of controller employing sensors
(including gesture-based control devices that do not make use of
electromyography or EMG sensors), with the acts described herein as
being performed by "at least one EMG sensor" and/or "at least one
accelerometer" being more generally performed by "at least one
sensor."
[0108] The above description of illustrated embodiments, including
what is described in the Abstract, is not intended to be exhaustive
or to limit the embodiments to the precise forms disclosed.
Although specific embodiments of and examples are described herein
for illustrative purposes, various equivalent modifications can be
made without departing from the spirit and scope of the disclosure,
as will be recognized by those skilled in the relevant art. The
teachings provided herein of the various embodiments can be applied
to other portable and/or wearable electronic devices, not
necessarily the exemplary wearable electronic devices generally
described above.
[0109] For instance, the foregoing detailed description has set
forth various embodiments of the devices and/or processes via the
use of block diagrams, schematics, and examples. Insofar as such
block diagrams, schematics, and examples contain one or more
functions and/or operations, it will be understood by those skilled
in the art that each function and/or operation within such block
diagrams, flowcharts, or examples can be implemented, individually
and/or collectively, by a wide range of hardware, software,
firmware, or virtually any combination thereof. In one embodiment,
the present subject matter may be implemented via Application
Specific Integrated Circuits (ASICs). However, those skilled in the
art will recognize that the embodiments disclosed herein, in whole
or in part, can be equivalently implemented in standard integrated
circuits, as one or more computer programs executed by one or more
computers (e.g., as one or more programs running on one or more
computer systems), as one or more programs executed by on one or
more controllers (e.g., microcontrollers) as one or more programs
executed by one or more processors (e.g., microprocessors), as
firmware, or as virtually any combination thereof, and that
designing the circuitry and/or writing the code for the software
and or firmware would be well within the skill of one of ordinary
skill in the art in light of the teachings of this disclosure.
[0110] When logic is implemented as software and stored in memory,
logic or information can be stored on any computer-readable medium
for use by or in connection with any processor-related system or
method. In the context of this disclosure, a memory is a
computer-readable medium that is an electronic, magnetic, optical,
or other physical device or means that contains or stores a
computer and/or processor program. Logic and/or the information can
be embodied in any computer-readable medium for use by or in
connection with an instruction execution system, apparatus, or
device, such as a computer-based system, processor-containing
system, or other system that can fetch the instructions from the
instruction execution system, apparatus, or device and execute the
instructions associated with logic and/or information.
[0111] In the context of this specification, a "non-transitory
computer-readable medium" can be any element that can store the
program associated with logic and/or information for use by or in
connection with the instruction execution system, apparatus, and/or
device. The computer-readable medium can be, for example, but is
not limited to, an electronic, magnetic, optical, electromagnetic,
infrared, or semiconductor system, apparatus or device. More
specific examples (a non-exhaustive list) of the computer readable
medium would include the following: a portable computer diskette
(magnetic, compact flash card, secure digital, or the like), a
random access memory (RAM), a read-only memory (ROM), an erasable
programmable read-only memory (EPROM, EEPROM, or Flash memory), a
portable compact disc read-only memory (CDROM), digital tape, and
other non-transitory media.
[0112] The various embodiments described above can be combined to
provide further embodiments. To the extent that they are not
inconsistent with the specific teachings and definitions herein,
all of the U.S. patents, U.S. patent application publications, U.S.
patent applications, foreign patents, foreign patent applications
and non-patent publications referred to in this specification
and/or listed in the Application Data Sheet, including but not
limited to: U.S. Provisional Patent Application Ser. No.
61/869,526; U.S. Provisional Patent Application Ser. No. 61/857,105
(now U.S. Non-Provisional patent application Ser. No. 14/335,668);
U.S. Provisional Patent Application Ser. No. 61/752,226 (now U.S.
Non-Provisional patent application Ser. No. 14/155,107); U.S.
Provisional Patent Application Ser. No. 61/768,322 (now U.S.
Non-Provisional patent application Ser. No. 14/186,889); U.S.
Provisional Patent Application Ser. No. 61/771,500 (now U.S.
Non-Provisional patent application Ser. No. 14/194,252); U.S.
Provisional Application Ser. No. 61/860,063 (now U.S.
Non-Provisional patent application Ser. No. 14/276,575), and U.S.
Provisional Application Ser. No. 61/866,960 (now U.S.
Non-Provisional patent application Ser. No. 14/461,044), are
incorporated herein by reference, in their entirety. Aspects of the
embodiments can be modified, if necessary, to employ systems,
circuits and concepts of the various patents, applications and
publications to provide yet further embodiments.
[0113] These and other changes can be made to the embodiments in
light of the above-detailed description. In general, in the
following claims, the terms used should not be construed to limit
the claims to the specific embodiments disclosed in the
specification and the claims, but should be construed to include
all possible embodiments along with the full scope of equivalents
to which such claims are entitled. Accordingly, the claims are not
limited by the disclosure.
* * * * *