U.S. patent application number 16/882673 was filed with the patent office on 2021-11-25 for wearable audio device placement detection.
The applicant listed for this patent is Bose Corporation. Invention is credited to Joseph H. Cattell, Miriam Israelowitz, Jeremy Kemmerer.
Application Number | 20210368254 16/882673 |
Document ID | / |
Family ID | 1000004871137 |
Filed Date | 2021-11-25 |
United States Patent
Application |
20210368254 |
Kind Code |
A1 |
Kemmerer; Jeremy ; et
al. |
November 25, 2021 |
Wearable Audio Device Placement Detection
Abstract
An earbud with an electro-acoustic transducer for producing
sound, a proximity sensor that is configured to detect when the
earbud is close to a user's skin, an orientation sensor that is
configured to detect an orientation of the earbud, and a processor
that is configured to estimate, based on the proximity sensor and
the orientation sensor, whether the earbud has been inserted into
the user's ear canal.
Inventors: |
Kemmerer; Jeremy;
(Holliston, MA) ; Israelowitz; Miriam; (Brookline,
MA) ; Cattell; Joseph H.; (Somerville, MA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Bose Corporation |
Framingham |
MA |
US |
|
|
Family ID: |
1000004871137 |
Appl. No.: |
16/882673 |
Filed: |
May 25, 2020 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 3/011 20130101;
H04R 1/028 20130101; H04R 1/1016 20130101; G06F 3/16 20130101; G06F
3/017 20130101 |
International
Class: |
H04R 1/10 20060101
H04R001/10; H04R 1/02 20060101 H04R001/02; G06F 3/01 20060101
G06F003/01; G06F 3/16 20060101 G06F003/16 |
Claims
1. A wearable audio device, comprising: an electro-acoustic
transducer for producing sound; a proximity sensor that is
configured to detect when the wearable audio device is close to a
user; an orientation sensor that is configured to detect an
orientation of the wearable audio device; an internal microphone
positioned within a housing of the wearable audio device and
configured to detect sound in a cavity formed at least in part by a
portion of the housing; and a processor that is configured to
estimate: (i) based on the proximity sensor and the orientation
sensor, whether the wearable audio device is in place on the user's
body, and (ii) based on the internal microphone, whether the user
has completed handling of the wearable audio device.
2. The wearable audio device of claim 1, wherein the proximity
sensor comprises an infrared sensor.
3. The wearable audio device of claim 1, wherein the orientation
sensor comprises an inertial measurement unit (IMU).
4. The wearable audio device of claim 3, wherein the IMU comprises
an accelerometer that is used to detect the wearable audio device
orientation.
5. The wearable audio device of claim 1, further comprising an
external microphone that is configured to sense sound outside the
housing, and wherein the processor estimation of whether the
wearable audio device is in place on the user's body is further
based on a transfer function between the external microphone and
the internal microphone.
6. The wearable audio device of claim 5, wherein the processor is
further configured to cause the electro-acoustic transducer to
produce sound on which the transfer function is based.
7. The wearable audio device of claim 6, wherein the transfer
function is determined at a frequency of up to 1,000 Hz.
8. The wearable audio device of claim 6, wherein the processor is
further configured to cause the electro-acoustic transducer to
produce sound at two different frequencies, wherein a first
frequency is at least 1,500 Hz and the processor is configured to
determine, based at least in part on the first frequency, if a
wearable audio device nozzle is blocked, and a second frequency is
no more than 1,000 Hz and the processor is configured to determine,
based at least in part on the second frequency, if the wearable
audio device is in place on the user's body.
9. The wearable audio device of claim 6, wherein the produced sound
is part of a wearable audio device startup tone sequence.
10. The wearable audio device of claim 5, wherein the processor
estimation of whether the wearable audio device is in place on the
user's body comprises two sequential steps, a first step based at
least on the proximity sensor, the orientation sensor, and the
internal microphone, and a second step based at least on the
internal microphone and the external microphone.
11. The wearable audio device of claim 10, wherein the processor is
configured to initiate a first group of wearable audio device
functions based on the first step and a second group of wearable
audio device functions based on the second step, wherein the
functions of the second group are different than the functions of
the first group.
12. The wearable audio device of claim 11, wherein the first and
second groups of functions comprise at least one of: telephone call
answering capability, Bluetooth connection, beamforming of
microphones carried by the wearable audio device, playback of audio
received from an external audio source, enablement of user
interface functions, and tuning of an active noise reduction
system.
13. The wearable audio device of claim 10, wherein during at least
the first step the internal microphone is monitored at frequencies
in the 0-20 Hz range.
14. The wearable audio device of claim 1, wherein the proximity
sensor has an output, and the estimation by the processor of
whether the wearable audio device is in place on the user's body is
in part based on a level of the proximity sensor output.
15. The wearable audio device of claim 1, wherein after the
processor estimates whether the wearable audio device is in place
on the user's body the processor is configured to enable
predetermined wearable audio device functions.
16. The wearable audio device of claim 15, wherein the
predetermined wearable audio device functions comprise at least one
of: telephone call answering capability, Bluetooth connection,
beamforming of microphones carried by the wearable audio device,
playback of audio received from an external audio source,
enablement of user interface functions, and tuning of an active
noise reduction system.
17. A method of detecting when a wearable audio device is in place
on a body of a user, wherein the wearable audio device comprises an
electro-acoustic transducer for producing sound, a proximity sensor
that is configured to detect when the wearable audio device is
close to the user, an orientation sensor that is configured to
detect an orientation of the wearable audio device, an internal
microphone positioned within a housing of the wearable audio device
and configured to detect sound in a cavity formed at least in part
by a portion of the housing, and an external microphone that is
configured to sense sound outside the housing, the method
comprising: estimating, based on the proximity sensor, the
orientation sensor, and the internal microphone, whether the
wearable audio device is in place on the user's body; and
estimating, based on the internal microphone, whether the user has
completed handling of the wearable audio device.
18. The method of claim 17, wherein the processor is further
configured to cause the electro-acoustic transducer to produce
sound after the processor has estimated that the wearable audio
device is in place on the user's body, and wherein the processor is
further configured to compute a transfer function between the
external microphone and the internal microphone over a specified
frequency or frequency range of the produced sound.
19. The method of claim 17, wherein the processor estimation of
whether the wearable audio device is in place on the user's body
comprises two sequential steps, a first step based at least on the
proximity sensor, the orientation sensor, and the internal
microphone, and a second step based at least on the internal
microphone and the external microphone, and wherein the processor
is configured to initiate a first group of wearable audio device
functions based on the first step and a second group of wearable
audio device functions based on the second step, wherein the
functions of the second group are different than the functions of
the first group.
Description
BACKGROUND
[0001] This disclosure relates to a wearable audio device.
[0002] Wearable audio devices are designed to function best when
they are in their proper use position on the body, and when the
user has stopped handling the device.
SUMMARY
[0003] All examples and features mentioned below can be combined in
any technically possible way.
[0004] In one aspect a wearable audio device includes an
electro-acoustic transducer for producing sound, a proximity sensor
that is configured to detect when the wearable audio device is
close to a user, an orientation sensor that is configured to detect
an orientation of the wearable audio device, an internal microphone
positioned within a housing of the wearable audio device and
configured to detect sound in a cavity formed at least in part by a
portion of the housing, and a processor that is configured to
estimate: (i) based on the proximity sensor and the orientation
sensor, whether the wearable audio device is in place on the user's
body, and (ii) based on the internal microphone, whether the user
has completed handling of the wearable audio device.
[0005] Some examples include one of the above and/or below
features, or any combination thereof. In an example the proximity
sensor comprises an infrared sensor. In an example the orientation
sensor comprises an inertial measurement unit (IMU). In an example
the IMU comprises an accelerometer that is used to detect the
wearable audio device orientation. In an example the proximity
sensor has an output, and the estimation by the processor of
whether the wearable audio device is in place on the user's body is
in part based on a level of the proximity sensor output. In an
example after the processor estimates whether the wearable audio
device is in place on the user's body the processor is configured
to enable predetermined wearable audio device functions. In an
example the predetermined wearable audio device functions comprise
at least one of: telephone call answering capability, Bluetooth
connection, beamforming of microphones carried by the wearable
audio device, playback of audio received from an external audio
source, enablement of user interface functions, and tuning of an
active noise reduction system.
[0006] Some examples include one of the above and/or below
features, or any combination thereof. In some examples the wearable
audio device further comprises an external microphone that is
configured to sense sound outside the housing, and the processor
estimation of whether the wearable audio device is in place on the
user's body is further based on a transfer function between the
external microphone and the internal microphone. In an example the
processor is further configured to cause the electro-acoustic
transducer to produce sound on which the transfer function is
based. In an example the transfer function is determined at a
frequency of up to 1,000 Hz. In an example the processor is further
configured to cause the electro-acoustic transducer to produce
sound at two different frequencies, wherein a first frequency is at
least 1,500 Hz and the processor is configured to determine, based
at least in part on the first frequency, if a wearable audio device
nozzle is blocked, and a second frequency is no more than 1,000 Hz
and the processor is configured to determine, based at least in
part on the second frequency, if the wearable audio device is in
place on the user's body.
[0007] Some examples include one of the above and/or below
features, or any combination thereof. In an example the produced
sound is part of a wearable audio device startup tone sequence. In
an example the processor estimation of whether the wearable audio
device is in place on the user's body comprises two sequential
steps, a first step based at least on the proximity sensor, the
orientation sensor, and the internal microphone, and a second step
based at least on the internal microphone and the external
microphone. In some examples the processor is configured to
initiate a first group of wearable audio device functions based on
the first step and a second group of wearable audio device
functions based on the second step, wherein the functions of the
second group are different than the functions of the first group.
In an example the first and second groups of functions comprise at
least one of: telephone call answering capability, Bluetooth
connection, beamforming of microphones carried by the wearable
audio device, playback of audio received from an external audio
source, enablement of user interface functions, and tuning of an
active noise reduction system. In an example during at least the
first step the internal microphone is monitored at frequencies in
the 0-20 Hz range.
[0008] In another aspect a method of detecting when a wearable
audio device is in place on the user's body, wherein the wearable
audio device comprises an electro-acoustic transducer for producing
sound, a proximity sensor that is configured to detect when the
wearable audio device is close to a user, an orientation sensor
that is configured to detect an orientation of the wearable audio
device, an internal microphone positioned within a housing of the
wearable audio device and configured to detect sound in a cavity
formed at least in part by a portion of the housing, and an
external microphone that is configured to sense sound outside the
housing, includes estimating, based on the proximity sensor, the
orientation sensor, and the internal microphone, whether the
wearable audio device is in place on the user's body, and
estimating, based on the internal microphone, whether the user has
completed handling of the wearable audio device.
[0009] Some examples include one of the above and/or below
features, or any combination thereof. In an example the processor
is further configured to cause the electro-acoustic transducer to
produce sound after the processor has estimated that the wearable
audio device is in place on the user's body, and wherein the
processor is further configured to compute a transfer function
between the external microphone and the internal microphone over a
specified frequency or frequency range of the produced sound. In an
example the processor estimation of whether the wearable audio
device is in place on the user's body comprises two sequential
steps, a first step based at least on the proximity sensor, the
orientation sensor, and the internal microphone, and a second step
based at least on the internal microphone and the external
microphone, and wherein the processor is configured to initiate a
first group of wearable audio device functions based on the first
step and a second group of wearable audio device functions based on
the second step, wherein the functions of the second group are
different than the functions of the first group.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is perspective view of a wearable audio device.
[0011] FIG. 2 is a partial cross-sectional view of elements of a
wearable audio device.
[0012] FIG. 3 is a flowchart of an operation of an earbud placement
detection methodology.
[0013] FIG. 4 illustrates the transfer function between an earbud
external microphone and internal microphone with the earbud located
in free space and located in the ear.
DETAILED DESCRIPTION
[0014] This disclosure relates to a wearable audio device. Some
non-limiting examples of this disclosure describe a type of
wearable audio device that is known as an earbud. Earbuds generally
include an electro-acoustic transducer for producing sound, and are
configured to deliver the sound directly into the user's ear canal.
Earbuds can be wireless or wired. In examples described herein the
earbuds are wireless and thus carry a power supply (generally, a
rechargeable battery), a wireless communications system (which in
an example is a Bluetooth-based communications system), and any
necessary processing. Other aspects of earbuds that are not
involved in this disclosure are not shown or described.
[0015] Some examples of this disclosure also describe a type of
wearable audio device that is known as an open audio device. Open
audio devices have one or more electro-acoustic transducers (i.e.,
audio drivers) that are located off of the ear canal opening. In
some examples the open audio devices also include one or more
microphones; the microphones can be used to pick up the user's
voice and/or for noise cancellation. Open audio devices are further
described in U.S. Pat. No. 10,397,681, the entire disclosure of
which is incorporated herein by reference for all purposes.
[0016] A headphone refers to a device that typically fits around,
on, or in an ear and that radiates acoustic energy directly or
indirectly into the ear canal. Headphones are sometimes referred to
as earphones, earpieces, headsets, earbuds, or sport headphones,
and can be wired or wireless. A headphone includes an
electro-acoustic transducer (driver) to transduce electrical audio
signals to acoustic energy. The acoustic driver may or may not be
housed in an earcup or in a housing that is configured to be
located on the head or on the ear, or to be inserted directly into
the user's ear canal. A headphone may be a single stand-alone unit
or one of a pair of headphones (each including at least one
acoustic driver), one for each ear. A headphone may be connected
mechanically to another headphone, for example by a headband and/or
by leads that conduct audio signals to an acoustic driver in the
headphone. A headphone may include components for wirelessly
receiving audio signals. A headphone may include components of an
active noise reduction (ANR) system, which may include an internal
microphone within the headphone housing. Headphones may also
include other functionality, such as additional microphones for an
ANR system, or one or more microphones that are used to pick up the
user's voice.
[0017] In an around the ear or on the ear or off the ear headphone,
the headphone may include a headband or other support structure
and/or at least one housing or other structure that contains a
transducer and is arranged to sit on or over or proximate an ear of
the user. The headband can be collapsible or foldable, and can be
made of multiple parts. Some headbands include a slider, which may
be positioned internal to the headband, that provides for any
desired translation of the housing. Some headphones include a yoke
pivotably mounted to the headband, with the housing pivotably
mounted to the yoke, to provide for any desired rotation of the
housing.
[0018] An open audio device includes but is not limited to an
off-ear headphone, i.e., a device that has one or more
electro-acoustic transducers that are coupled to the head or ear
(typically by a support structure) but do not occlude the ear canal
opening. In some examples an open audio device is an off-ear
headphone comprising audio eyeglasses, but that is not a limitation
of the disclosure as in an open audio device the device is
configured to deliver sound to one or both ears of the wearer where
there are typically no ear cups and no ear buds. The wearable audio
systems contemplated herein may include a variety of devices that
include an over-the-ear hook, such as a wireless headset, hearing
aid, eyeglasses, a protective hard hat, and other open ear audio
devices.
[0019] One or more of the devices, systems, and methods described
herein, in various examples and combinations, may be used in a wide
variety of wearable audio devices or systems, including wearable
audio devices in various form factors. Unless specified otherwise,
the term active portion of a wearable audio system, as used in this
document, includes headphones and various other types of wearable
audio devices such as head, shoulder or body-worn acoustic devices
(e.g., audio eyeglasses or other head-mounted audio devices) that
include one more acoustic transducers to receive and/or produce
sound, with or without contacting the ears of a user.
[0020] It should be noted that although specific implementations of
wearable audio devices primarily serving the purpose of
acoustically outputting audio are presented with some degree of
detail, such presentations of specific implementations are intended
to facilitate understanding through provisions of examples and
should not be taken as limiting either the scope of the disclosure
or the scope of the claim coverage.
[0021] In some examples the wearable audio device includes a
proximity sensor that is configured to detect when the earbud is
close to a user. In an example the proximity sensor detects the
user's skin. In an example the proximity sensor is an infrared (IR)
sensor that can detect when the wearable audio device is close to
or against the skin of the ear. In an example the IR sensor output
is within a predetermined range or at least a predetermined
threshold level for an in-use-position decision to be made. In some
examples the wearable audio device also includes an orientation
sensor that is configured to detect an orientation of the wearable
audio device. In an example the orientation is determined in three
mutually orthogonal axes in space. Since the wearable audio device
is designed to be worn in a particular orientation, knowing the
orientation via the orientation sensor can indicate whether the
device is in place. For example when an earbud is in the ear of a
head being held upright, an internal orientation sensor will have a
known orientation in three-dimensional space. The orientation
sensor results can be within a predetermined range of nominal for
the in-use-position decision to be made. The wearable audio device
further includes a processor that is configured to estimate, based
on the proximity sensor and the orientation sensor, whether the
wearable audio device is in place on the user's body. In some
examples where the wearable audio device is an earbud, the proper
location is in the user's ear canal.
[0022] In some examples the wearable audio device also includes an
internal microphone. In an example the internal microphone is
positioned within a housing of the wearable audio device. In an
example the microphone is positioned to detect sound in a cavity
formed at least in part by a portion of the housing. When an
internal microphone is used in the wearable audio device, the
processor can also be configured to estimate whether the user has
completed handling of the wearable audio device. In some examples
the handling comprises putting the wearable audio device in place
on the body or taking it off the body. In some examples the
wearable audio device also includes an external microphone. In an
example the external microphone is positioned to detect sound
external to the wearable audio device housing. When an external
microphone is used in the wearable audio device, the processor can
also be configured to calculate an audio transfer function between
the internal and external microphones, and estimate based on the
transfer function if the wearable audio device is in its proper use
position (e.g., in an ear).
[0023] FIG. 1 is a perspective view of a wireless in-ear earbud 10.
An earbud is a non-limiting example of a wearable audio device.
Earbud 10 includes body or housing 12 that houses the active
components of the earbud. Portion 14 is coupled to body 12 and is
pliable so that it can be inserted into the entrance of the ear
canal. Sound is delivered through opening 15. Retaining loop 16 is
constructed and arranged to be positioned in the outer ear, for
example in the antihelix, to help retain the earbud in the ear.
Earbuds are well known in the field (e.g., as disclosed in U.S.
Pat. No. 9,854,345, the disclosure of which is incorporated herein
by reference in its entirety, for all purposes), and so certain
details of the earbud are not further described herein.
[0024] FIG. 2 is a partial cross-sectional view of only certain
elements of an earbud 20 that are useful to a better understanding
of the present disclosure. Earbud 20 comprises housing 21 that
encloses electro-acoustic transducer (audio driver) 30. Housing 21
comprises front housing portion 50 and rear housing portions 60 and
62. Transducer 30 has diaphragm 32 that is driven in order to
create sound pressure in front cavity 52. Sound pressure is
directed out of front housing portion 50 via sound outlet 54.
Internal microphone 80 is located inside of housing 21. In an
example microphone 80 is in sound outlet 54, as shown in FIG. 2.
External microphone 81 is configured to sense sound external to
housing 21. In an example interior microphone 80 is used as a
feedback microphone for active noise reduction, and exterior
microphone 81 is used as a feed-forward microphone for active noise
reduction. An earbud, such as shown by earbud 10 in FIG. 1,
typically includes a pliable tip (not shown) that is engaged with
neck 51 of housing portion 50, to help direct the sound into the
ear canal. Earbud housing 21 further comprises a rear enclosure
made from rear housing portions 60 and 62, and grille 64. Note that
the details of earbud 20 are exemplary of aspects of earphones and
are not limiting of the scope of this disclosure, as the present
in-ear detection can be used in varied types and designs of earbuds
and earphones and other wearable audio devices.
[0025] Transducer 30 further comprises magnetic structure 34.
Magnetic structure 34 comprises transducer magnet 38 and magnetic
material that functions to confine and guide the magnetic field
from magnet 38, so that the field properly interacts with coil 33
to drive diaphragm 32, as is well known in the electro-acoustic
transducer field. The magnetic material comprises cup 36 and front
plate 35, both of which are preferably made from a material with
relatively high magnetic susceptibility, also as is known in the
field. Transducer printed circuit board (PCB) 40 carries electrical
and electronic components (not shown) that are involved in driving
the transducer. Pads 41 and 42 are locations where wires (not
shown) can be coupled to PCB 40.
[0026] In an example an inertial measurement unit (IMU) 72 is used
to detect the orientation in three-dimensional space of the earbud.
An IMU can include a three-axis accelerometer that can be used to
determine orientation. Using accelerometers to determine an
orientation of a device that includes or carries the accelerometers
is known in the field and so is not further described herein. An
IMU can also include a gyroscope, or three gyroscopes that are
configured to determine rotational velocity about three mutually
orthogonal axes. Gyroscopes can additionally or alternatively be
used to determine the earbud orientation. Using gyroscopes to
determine an orientation of a device that includes or carries the
gyroscope is known in the field and so is not further described
herein. In an example IMU 72 is mounted on PCB 70, although the IMU
could be located elsewhere in or on the earbud.
[0027] In some examples proximity sensor 76 is used to detect when
earbud 20 is close to the user's skin. In an example proximity
sensor 76 can be an infrared (IR) sensor or a capacitive sensor. An
IR sensor can be used to detect close proximity to skin while a
capacitive sensor can detect when the device is touching the skin.
An IR sensor can be used to detect when the earbud is in close
proximity to the skin. Since an earbud needs to be in the ear when
it is in the proper use position, parts of the earbud will be in
contact with the skin, or close to the skin. Locating an IR sensor
in a part of the earbud that will be in contact with or close to
the skin thus allows the IR sensor to be used as a proximity
sensor. In an example an IR sensor is located such that it will
detect the tragus. In another example a distance sensor such as a
time of flight sensor can be used to detect a distance between the
wearable audio device and the desired location on the body (e.g.,
in the ear). Since the wearable audio device is properly located in
a known location on the body (e.g., in the ear, on the ear, or
elsewhere on the head), the distance from the device to the proper
location should be zero or close to zero when the device is in
place.
[0028] Earbud 20 also includes processor 74. In some examples
processor 74 is configured to process outputs of IMU 72, proximity
sensor 76, internal microphone 80, and external microphone 81. Of
course the processor is typically involved in other processing
needed for earbud functionality, such as processing digital sound
files that are to be played by the earbud, as would be apparent to
one skilled in the technical field. In an example the processor is
configured to estimate based on both the proximity sensor and the
orientation sensor whether the wearable audio device is properly in
place on the user's body. In some examples the processor estimation
of whether the wearable audio device is in place on the user's body
is further based on a transfer function between the external
microphone and the internal microphone. In an example the processor
is configured to estimate based on the internal microphone whether
the user has completed handling of the wearable audio device. In
some examples the proximity sensor has an output, and the
estimation by the processor of whether the wearable audio device is
in place on the user's body is in part based on a level of the
proximity sensor output. In an example the estimation by the
processor of whether the wearable audio device is in place on the
user's body is in part based on whether the proximity sensor output
has reached a threshold level. For example, as the wearable audio
device approaches the skin the IR sensor output will increase. For
any particular IR sensor, an output that reaches a predetermined
level can be equated to the sensor being within a predetermined
distance of the skin.
[0029] In some examples the processor estimation of whether the
wearable audio device is in place on the user's body includes
multiple sequential steps. In an example the first step is based on
the proximity sensor and the orientation sensor, a second step is
based on the internal microphone, and a third second step is based
on the internal and external microphones. In an example the
processor is configured to initiate a first group of wearable audio
device functions based on the first and second steps, and a second
group of wearable audio device functions based on the third step.
In an example the functions of the second group are different than
the functions of the first group. In some examples the first and
second groups of functions comprise at least one of: telephone call
answering capability, Bluetooth connection, beamforming of
microphones carried by the wearable audio device, playback of audio
received from an external audio source, enablement of user
interface functions, and tuning of an active noise reduction
system.
[0030] In some examples the processor is configured to cause the
electro-acoustic transducer to produce sound that is used in a
high-confidence determination of whether the wearable audio device
is in place. In an example this high-confidence determination takes
place only after the processor has made a first level, lower
confidence determination that the wearable audio device is in place
on the user's body, and that the user has stopped handling the
device. In an example the processor is configured to compute a
transfer function between the external microphone and the internal
microphone over a specified frequency range. In an example the
produced sound is part of a wearable audio device startup tone
sequence.
[0031] In one example the processor is configured to cause the
electro-acoustic transducer to produce sound at two (or more)
different frequencies for the in-location/out-of-location
decisions. In an example the first frequency is at least 1,500 Hz
(e.g., 1,500-3,000 Hz) and the processor is configured to
determine, based at least in part on that frequency, if a wearable
audio device nozzle is blocked. In an example the second frequency
is less than 1,000 Hz and the processor is configured to determine,
based at least in part on that frequency, if the wearable audio
device is in place on the user's body. When frequencies or
frequency ranges are described herein, it should be understood that
the frequency or range is in many cases approximate. When a
particular frequency is specified, it should be understood that the
actual frequency can be about or approximately the specified
frequency. One reason is that results may not differ dramatically
if the actual frequency differs from the stated frequency.
[0032] The internal and external microphones can be used to detect
sound that is analyzed in different ways and for different
purposes. There can be multiple functions that the microphones are
used for. One is to determine when the user is adjusting the
wearable audio device. The internal microphone can be monitored for
this function. In an example the monitoring can be at frequencies
of up to 20 Hz, and the monitoring is passive; that is, the
microphone is used to monitor sound internal to the wearable audio
device without playing sound using the driver. When the wearable
audio device is an earbud the internal microphone can be a feedback
microphone located in the earbud nozzle. When the earbud is being
handled (i.e., being inserted into the ear or being removed from
the ear), the handling causes sound in the range of up to 20 Hz.
Accordingly detecting sound in this range can be equated to
detecting handling of the earbud. Also, when the earbud is inserted
into the ear the cavity bounded by the driver, the earbud nozzle,
the ear canal, and the eardrum becomes sealed, which causes a DC
pressure spike. The DC pressure spike occurs at 0 Hz, but spreads
out a bit in the frequency range. Accordingly, monitoring the
internal microphone in the 0-20 Hz range is useful for one or both
of estimating when the earbud is inserted into the ear and when it
is no longer being handled.
[0033] A second function that the microphones can be used for is to
make a higher-confidence determination that the wearable audio
device is in place on the user's body. The internal and external
microphones can be used for this function. In an example an audio
transfer function between the external and internal microphones is
determined. The determination can be at a frequency, or frequency
range, where there is good transfer function magnitude value
separation between in-place and out-of-place (e.g., in-ear and
out-of-ear for an earbud). In an example for an earbud, at
frequencies in the range of 0-1,000 Hz the transfer function
magnitude differs between in-ear and out-of-ear states. Determining
the transfer function in this frequency range can thus indicate
with confidence whether the earbud is in the ear or not. The same
principle can be used for other types of earbuds, for headphones,
and for other types of wearable audio devices. The particular
frequency or frequency range where there is good transfer function
magnitude value separation between in-place and out-of-place will
be unique for any given product. The "in-place" decision can then
be made at a frequency or frequency range where there is good
magnitude value separation.
[0034] Another function that the microphones can be used for is to
determine if the nozzle of an earbud is blocked. An earbud nozzle
can be blocked by a finger when the earbud is being handled (e.g.,
when it is being put in place in the ear or removed from the ear).
Thus a determination of a blocked nozzle can be used as a lower
confidence indication that the earbud is being handled and so is
not in its proper use position. In an example, at one frequency
range the audio transfer function between the external and internal
microphones has one value when the nozzle is blocked by a finger
and a different value when the wearable audio device is either in
free air or in proper use position. For example, in the 1,500-3,000
Hz range there is a different response if the earbud nozzle is
being blocked with the thumb, as compared to the earbud being in or
out of the ear (which have a similar response). As another example,
the transfer function for an earbud in the frequency range of
0-1,000 Hz may be low if the earbud is in free air and higher if
the nozzle is blocked (either by a finger or by the earbud being
sealed in the ear canal). Thus the transfer function can be an
indication that the earbud is being handled and can be an
indication that the earbud is in the ear. This determination can be
used together with other determinations described herein to make
the ultimate decision as to whether the wearable audio device is in
its proper use position.
[0035] FIG. 3 illustrates steps of a method 100 of detecting when a
wearable audio device such as that described above is properly in
place on the user's body. The proximity sensor is monitored for a
proximity detection event, step 102. The orientation sensor is
monitored for proper wearable audio device orientation, step 104.
In an example both sensors are monitored at the same time. Once the
proximity and orientation sensors reach a threshold that indicates
a possible in-ear event, an internal microphone is monitored, step
105. In an example the microphone monitoring takes place in the
0-20 Hz range so as to detect when the wearable audio device is
being handled by the user, and when an earbud is inserted into an
ear, as described elsewhere herein. Based on the proximity sensor,
orientation sensor, and internal microphone monitoring, a low
confidence decision is made as to whether the device is in proper
position (i.e., in place on the user's body), step 106. In this
non-limiting example the device is an earbud, and so the decision
is whether the device is in place in the ear. If the low-confidence
in-ear decision is made, a first group of earbud functions are
enabled, step 108. In some examples earbud functions that can be
enabled based on this low-confidence decision include some or all
of automatic power on/off, automatic pause/play of audio, automatic
telephone call answering capability, Bluetooth
connection/reconnection, beamforming of microphones carried by the
wearable audio device, enablement of user interface functions, and
tuning of an active noise reduction system. More specifically, in
an example the functions that are enabled after the low-confidence
decision are background functions, such as turning on processors,
sensors, and Bluetooth. A result is that chips, sensors, and other
aspects of the audio device that use power can remain off until the
first level decision is made, thus saving battery power.
[0036] Method 100 then moves to a second level high confidence
decision of the earbud being in place in the ear. The internal and
external microphones are monitored, step 110. The driver is then
enabled to play a tone or tones, step 112. A purpose of playing a
tone is to receive the tone at the microphones and make a
determination as to whether the receipt of the tone is as it would
be expected to be if the wearable audio device is in proper use
position and the user has stopped handling the wearable audio
device. In the example of an earbud, the microphones that are used
for this purpose include an internal microphone used for
feedback-based active noise reduction and an external microphone
used for feed-forward based active noise reduction. Both types of
microphones are known in the field of active noise reduction for
wearable audio devices and so are not further described herein. The
internal microphone is typically located such that it is able to
sense sound in a cavity formed by the driver, the earbud nozzle,
and the ear canal. An example is microphone 80, FIG. 2. The
external microphone is enabled to sense sound outside of the earbud
housing. An example is microphone 81, FIG. 2. In method 100, the
transfer function (G.sub.sd) between the two microphones is
determined, step 114. G.sub.sd determination is described in U.S.
Pat. No. 10,462,551 entitled "Wearable Audio Device with Head
On/Off State Detection", issued on Oct. 29, 2019, the entire
disclosure of which is incorporated herein by reference for all
purposes. The driver to microphone acoustic coupling (as
exemplified by the transfer function) will change (at least at one
or more frequencies) when the earbud is in the ear vs. out of the
ear. In an example a processor of the wearable audio device is
configured both to cause the electro-acoustic transducer to produce
the sound after the processor has made the first level estimation
that the wearable audio device is in place on the user's body, and
to compute the transfer function. In an example the transfer
function is computed over a specified frequency range, which can be
in the range of 0-1,000 Hz. In some examples the sound(s) that are
played are part of a device startup series of tones of the type
that are common in electronic devices.
[0037] If the transfer function is as expected, a high confidence
in-ear decision is made, step 116. In an example this
high-confidence second-stage decision also involves the first stage
decision (proximity to the user, proper orientation, and a decision
based on a microphone that the user has stopped handling the
wearable audio device). In other words, the device is considered
(with low confidence) to be in its proper use position both when it
is in the correct physical location and the user is not handling
the device (which is expected to happen once the user is satisfied
that the device is in place). In an example, as part of the second
stage high-confidence decision, the blocked nozzle indicator is
calculated at the same time as the G.sub.sd in the 0-1,000 Hz
range. If the blocked nozzle calculation indicates that the nozzle
is blocked then the high-confidence decision can be delayed by a
short time, to ensure that the user has stopped handling the
earbud. If after that time the indication of in-ear state remains,
the high-confidence decision is made. After the second stage
high-confidence decision is made, a second group of earbud
functions are enabled, step 118. In some examples the functions of
the second group are different than the functions of the first
group. In an example the functions of the second group include both
functions that involve interaction with the user (such as switches,
capacitive touch sensors, and microphone beamforming) as well as
tuning of the active noise reduction system. In some examples the
multiple stages can run on different subsystems. For example there
could be a sensor subsystem (using the proximity sensor, the
orientation sensor, and the internal microphone) used in the first
low-confidence stage and a microphone subsystem (using the internal
and external microphones) used in the second stage high-confidence
decision. This would enable the microphone subsystem to be turned
on only when it was needed, to conserve power. This design would
require some messaging between the subsystems.
[0038] In some examples monitoring of the proximity sensor and the
orientation sensor continue during the second stage decision-making
process. One reason is so that the wearable audio device can
determine if it has been removed from its use position. For
example, an earbud can be placed in the ear and then quickly
removed. Or perhaps the user might keep adjusting it for a short
time after it has been placed in the ear. By monitoring the
proximity and/or orientation sensors through the entire startup
sequence, there can be more confidence in the decision that the
device is in place and the user has completed handling it. Also,
continuous or periodic monitoring of the microphone(s) and
calculation of the transfer function can be used to confirm when
the device has been moved from its use position (e.g., removed from
the ear). In some examples the processor can enable a timer that
can be used as part of the in-place/out of place decision. For
example if the high confidence decision remains for at least a
short time (perhaps 1-2 seconds) there can be greater confidence
that the user has stopped adjusting its position. Also, if the
proximity sensor stops sensing proximity (which will happen if the
earbud is removed from the ear) the timer can be used to turn off
earbud functions only after a short time; this can help prevent
erroneous out of position decisions that might be annoying to the
user. For example, if the driver and external microphones are
disabled when the device is removed from its use position, an
erroneous decision can cut off telephone calls or music or podcasts
against the wishes of the user.
[0039] FIG. 4 is a graph 130 that illustrates the transfer function
(G.sub.sd) between an earbud external and internal microphone (as
described above) over a frequency range up to about 10,000 Hz.
Solid line 134 illustrates the transfer function when the earbud is
in the ear, while dotted line 132 illustrates the transfer function
when the earbud is in free space. As can be seen, at frequencies up
to 1,000 Hz the magnitudes of the transfer functions differ.
Accordingly, determining the transfer function in the range of
0-1,000 Hz is indicative of whether or not the earbud is in the
ear.
[0040] When processes are represented or implied in the block
diagram, the steps may be performed by one element or a plurality
of elements. The steps may be performed together or at different
times. The elements that perform the activities may be physically
the same or proximate one another, or may be physically separate.
One element may perform the actions of more than one block. Audio
signals may be encoded or not, and may be transmitted in either
digital or analog form. Conventional audio signal processing
equipment and operations are in some cases omitted from the
drawing.
[0041] Examples of the systems and methods described herein
comprise computer components and computer-implemented steps that
will be apparent to those skilled in the art. For example, it
should be understood by one of skill in the art that the
computer-implemented steps may be stored as computer-executable
instructions on a computer-readable medium such as, for example,
floppy disks, hard disks, optical disks, Flash ROMS, nonvolatile
ROM, and RAM. Furthermore, it should be understood by one of skill
in the art that the computer-executable instructions may be
executed on a variety of processors such as, for example,
microprocessors, digital signal processors, gate arrays, etc. For
ease of exposition, not every step or element of the systems and
methods described above is described herein as part of a computer
system, but those skilled in the art will recognize that each step
or element may have a corresponding computer system or software
component. Such computer system and/or software components are
therefore enabled by describing their corresponding steps or
elements (that is, their functionality), and are within the scope
of the disclosure.
[0042] A number of implementations have been described.
Nevertheless, it will be understood that additional modifications
may be made without departing from the scope of the inventive
concepts described herein, and, accordingly, other examples are
within the scope of the following claims.
* * * * *