U.S. patent application number 15/037302 was filed with the patent office on 2016-10-06 for handsfree beam pattern configuration.
The applicant listed for this patent is APPLE INC.. Invention is credited to John J. Baker, Brad G. Boozer, Afrooz Family, Michael B. Howes, Martin E. Johnson, Craig M. Stanley.
Application Number | 20160295340 15/037302 |
Document ID | / |
Family ID | 51703412 |
Filed Date | 2016-10-06 |
United States Patent
Application |
20160295340 |
Kind Code |
A1 |
Baker; John J. ; et
al. |
October 6, 2016 |
HANDSFREE BEAM PATTERN CONFIGURATION
Abstract
An audio system that adjusts one or more beam patterns emitted
by one or more loudspeaker arrays based on the preferences of
users/listeners is described. The audio system includes an audio
receiver that contains a listener location estimator, a listener
identifier, and a voice command processor. Inputs from the listener
location estimator, the listener identifier, and the voice command
processor are fed into an array processor. The array processor
drives the one or more loudspeaker arrays to emit beam patterns
into the listening area based on inputs from each of these devices.
By examining the location, preferred usage settings, and voice
commands from listeners, the generated beam patterns are customized
to the explicit and implicit preferences of the listeners with
minimal direct input. Other embodiments are also described.
Inventors: |
Baker; John J.; (Cupertino,
CA) ; Johnson; Martin E.; (Los Gatos, CA) ;
Family; Afrooz; (Emerald Hills, CA) ; Howes; Michael
B.; (San Jose, CA) ; Stanley; Craig M.;
(Campbell, CA) ; Boozer; Brad G.; (Saratoga,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
APPLE INC. |
Cupertino |
CA |
US |
|
|
Family ID: |
51703412 |
Appl. No.: |
15/037302 |
Filed: |
September 25, 2014 |
PCT Filed: |
September 25, 2014 |
PCT NO: |
PCT/US2014/057547 |
371 Date: |
May 17, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61907946 |
Nov 22, 2013 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04R 1/403 20130101;
H04S 7/302 20130101; G10L 17/06 20130101; H04R 2430/23 20130101;
H04R 1/08 20130101; H04R 2203/12 20130101; G10L 17/04 20130101 |
International
Class: |
H04S 7/00 20060101
H04S007/00; H04R 1/08 20060101 H04R001/08; H04R 1/40 20060101
H04R001/40; G10L 17/06 20060101 G10L017/06; G10L 17/04 20060101
G10L017/04 |
Claims
1. A method for adjusting sound emitted by a loudspeaker array,
comprising: sensing, by one or more microphone arrays, a voice
command from a user in a listening area; determining a location in
the listening area to steer a beam pattern based on the sensed
voice command; and steering the beam pattern at the determined
location in the listening area.
2. The method of claim 1, wherein determining the location in the
listening area comprises: feeding microphone signals received from
the microphone arrays to a set of beamformers pointing in a
discrete number of directions in the listening area; calculating
the energy output from each beamformer; and determining an angle of
arrival of the voice command based on the beamformer with the
largest energy output, wherein the determined location to steer the
beam pattern is in the direction of the angle of arrival of the
voice command.
3. The method of claim 1, further comprising: receiving microphone
signals from the microphone arrays; and comparing the microphone
signals with one or more speech signatures associated with stored
user profiles to determine a match.
4. The method of claim 3, wherein determining the location in the
listening area comprises: retrieving a preferred location setting
from the matched user profile, wherein the preferred location
setting is set based on historical usage habits by the user.
5. The method of claim 3, further comprising: retrieving one or
more preferred settings from the matched user profile, wherein the
beam pattern is generated based on the one or more preferred
settings.
6. The method of claim 3, further comprising: generating a new user
profile in response to being unable to determine a match between
the microphone signals and the one or more speech signatures
associated with the stored user profiles.
7. The method of claim 1, further comprising: receiving microphone
signals corresponding to the voice command from the microphone
arrays; processing the microphone signals to determine one or more
preferred settings for the beam pattern indicated by the voice
command, wherein the beam pattern is generated based on the
determined one or more preferred settings.
8. The method of claim 7, wherein the one or more preferred
settings indicate a predefined location in the listening area to
steer the beam pattern.
9. The method of claim 7, wherein the one or more preferred
settings indicate one or more of volume, bass, treble, and
reverberation ratio of the beam pattern.
10. An audio system for adjusting sound emitted by one or more
loudspeaker arrays, comprising: a plurality of microphones to sense
a voice command from a listener; a voice command processor to
determine a first setting to apply to a beam pattern emitted by one
of the one or more loudspeaker arrays based on signals representing
the sensed voice command received from the plurality of
microphones; and an array processor to generate a set of driving
signals to drive each transducer in the one of the one or more
loudspeaker arrays to emit the beam pattern based on the determined
first setting.
11. The audio system of claim 10, further comprising: a listener
location estimator to estimate the location of the listener based
on the signals representing the sensed voice command received from
the plurality of microphones, wherein the array processor generates
the driving signals to steer the beam pattern at the estimated
location.
12. The audio system of claim 11, further comprising: a user
profile database that stores speech signatures for one or more
known listeners; and a listener identifier to compare the signals
representing the sensed voice command received from the plurality
of microphones with each of the stored speech signatures to
identify a user profile corresponding to the listener.
13. The audio system of claim 12, wherein the listener identifier
feeds a second setting associated with the user profile
corresponding to the listener to the array processor, wherein the
array processor generates the driving signals to emit the beam
pattern based on the second setting.
14. The audio system of claim 13, wherein when the listener
location estimator is unable to estimate the location of the
listener, the second setting is a preferred location corresponding
to the listener to direct the beam pattern.
15. The audio system of claim 10, wherein the plurality of
microphones form one or more microphone arrays and each microphone
array is integrated in a separate loudspeaker array.
16. The audio system of claim 10, wherein each of the plurality of
microphones are integrated in a separate loudspeaker array.
17. An article of manufacture for adjusting sound emitted by a
loudspeaker array, comprising: a non-transitory machine-readable
storage medium that stores instructions which, when executed by a
processor in a computing device, detect, from microphone signals, a
voice command from a user in a listening area; determine a location
in the listening area to steer a beam pattern based on the detected
voice command; and steer the beam pattern at the determined
location in the listening area.
18. The article of manufacture of claim 17, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: feed the
microphone signals to a set of beamformers pointing in a discrete
number of directions in the listening area; calculate the energy
output from each beamformer; and determine an angle of arrival of
the voice command based on the beamformer with the largest energy
output, wherein the determined location to steer the beam pattern
is in the direction of the angle of arrival of the voice
command.
19. The article of manufacture of claim 17, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: compare the
microphone signals with one or more speech signatures associated
with stored user profiles to determine a match.
20. The article of manufacture of claim 19, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: retrieve a
preferred location setting from the matched user profile, wherein
the preferred location setting is set based on historical usage
habits by the user.
21. The article of manufacture of claim 19, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: retrieve one or
more preferred settings from the matched user profile, wherein the
beam pattern is generated based on the one or more preferred
settings.
22. The article of manufacture of claim 19, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: generate a new
user profile in response to being unable to determine a match
between the microphone signals and the one or more speech
signatures associated with the stored user profiles.
23. The article of manufacture of claim 17, wherein the
non-transitory machine-readable storage medium stores further
instruction which when executed by the processor: process the
microphone signals to determine one or more preferred settings for
the beam pattern indicated by the voice command, wherein the beam
pattern is generated based on the determined one or more preferred
settings.
24. The article of manufacture of claim 23, wherein the one or more
preferred settings indicate a predefined location in the listening
area to steer the beam pattern.
25. The article of manufacture of claim 23, wherein the one or more
preferred settings indicate one or more of volume, bass, treble,
and reverberation ratio of the beam pattern.
Description
RELATED MATTERS
[0001] This application claims the benefit of the earlier filing
date of U.S. provisional application No. 61/907,946, filed Nov. 22,
2013.
FIELD
[0002] A system and method for configuring and adjusting beam
patterns output by a speaker system in response to voice commands
is described. Other embodiments are also described.
BACKGROUND
[0003] Loudspeaker arrays may generate beam patterns that focus
sound in a specific direction. For example, sets of transducers in
a loudspeaker array may be individually and separately driven
according to different parameters and settings, including delays
and energy levels, to generate one or more beam patterns in a
listening area. The beam patterns may focus sound at a particular
object or individual in the listening area.
[0004] Although beam patterns allow sound to be focused in
different directions and/or at specific objects or individuals,
configuring beam patterns is often a complex and arduous process.
For example, as noted above, configuration of a beam pattern may
require individually and separately adjusting delays and energy
levels of driving signals for each transducer in a loudspeaker
array to achieve a desired result.
SUMMARY
[0005] One embodiment of the invention is directed to an audio
system that adjusts one or more beam patterns emitted by one or
more loudspeaker arrays based on the preferences of
users/listeners. In one embodiment, the audio system includes an
audio receiver that is comprised of a listener location estimator,
a listener identifier, and a voice command processor. The listener
location estimator estimates the location of one or more listeners
in a listening area based on sensed voice commands received from
one or more microphone arrays. The listener identifier attempts to
associate the one or more listeners with user profiles based on
comparisons of the voice commands with stored speech signatures.
The user profiles are associated with preferred settings, which
have been personalized for each identified listener based on
previous use of the audio system. The voice command processor
determines settings designated by listeners in each voice
command.
[0006] Inputs from the listener location estimator, the listener
identifier, and the voice command processor are fed into an array
processor. The array processor drives the one or more loudspeaker
arrays to emit beam patterns into the listening area based on
inputs from each of these devices. By examining the location,
historical preferred usage settings, and voice commands from
listeners, the generated beam patterns are customized to the
explicit and implicit preferences of the listeners with minimal
direct input.
[0007] The above summary does not include an exhaustive list of all
aspects of the present invention. It is contemplated that the
invention includes all systems and methods that can be practiced
from all suitable combinations of the various aspects summarized
above, as well as those disclosed in the Detailed Description below
and particularly pointed out in the claims filed with the
application. Such combinations have particular advantages not
specifically recited in the above summary.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The embodiments of the invention are illustrated by way of
example and not by way of limitation in the figures of the
accompanying drawings in which like references indicate similar
elements. It should be noted that references to "an" or "one"
embodiment of the invention in this disclosure are not necessarily
to the same embodiment, and they mean at least one.
[0009] FIG. 1 shows an overhead view of an audio system according
to one embodiment of the invention.
[0010] FIG. 2A shows a loudspeaker array housed in a single cabinet
according to one embodiment.
[0011] FIG. 2B shows a loudspeaker array housed in a single cabinet
according to another embodiment.
[0012] FIG. 3 shows a functional unit block diagram and some
constituent hardware components of an audio receiver according to
one embodiment.
[0013] FIGS. 4A-4D shows the estimated location of a listener in
the listening area.
[0014] FIG. 5 shows a method for adjusting sound emitted by the
loudspeaker array according to one embodiment.
[0015] FIGS. 6A and 6B show examples of a listener speaking
separate voice commands.
[0016] FIGS. 7A and 7B show beam patterns generated for listeners
based on example voice commands.
DETAILED DESCRIPTION
[0017] Several embodiments are described with reference to the
appended drawings are now explained. While numerous details are set
forth, it is understood that some embodiments of the invention may
be practiced without these details. In other instances, well-known
circuits, structures, and techniques have not been shown in detail
so as not to obscure the understanding of this description.
[0018] FIG. 1 shows an overhead view of an audio system 1 according
to one embodiment of the invention. The audio system 1 may include
an external audio source 2, an audio receiver 3, and a loudspeaker
array 4. The audio system 1 outputs sound program content into a
room or listening area 6 in which one or more intended listeners 5A
and 5B are located. The listeners 5A and 5B may be seated at
various positions in the listening area 6 and the audio receiver 3
may adjust sound output by the loudspeaker array 4 according to
commands and preferences of the listeners 5A and 5B as will be
described in further detail below.
[0019] The external audio source 2 may be any device capable of
transmitting one or more audio streams representing sound program
content to the audio receiver 3 for processing. For example, the
external audio source 2 in the system 1 of FIG. 1 is a laptop
computer that transmits one or more audio streams representing
sound program content to the audio receiver 3 for processing either
through a wired or wireless connection. In other embodiments, the
external audio source 2 may instead be one or more of a desktop
computer, a laptop computer, a tablet computer, a mobile device
(e.g., a mobile phone or mobile music player), and a remote media
server (e.g., an Internet streaming music or movie service), a set
top box, a television, a game system, a personal video recorder, a
DVD player, a Bluray player, etc.
[0020] As shown in FIG. 1, the components of the audio system 1 are
distributed and contained in separate units. In other embodiments,
the audio receiver 3 is integrated within the loudspeaker array 4
to provide a standalone unit. In this embodiment, the loudspeaker
array 4 receives one or more audio streams representing sound
program content directly from the external audio source 2 either
through wired or wireless connections.
[0021] Although described as receiving audio streams from the
external audio source 2, the audio receiver 3 may access audio
streams stored in a local storage medium. In this embodiment, the
audio receiver 3 retrieves the audio streams from the local storage
medium for processing without interaction with an external audio
source 2.
[0022] As will be described in further detail below, the audio
receiver 3 may be any type of device or set of devices for
processing streams of audio and driving one or more loudspeaker
arrays 4. For example, the audio receiver 3 may be a laptop
computer, a desktop computer, a tablet computer, a mobile device, a
home theatre audio receiver, or a set of hardware processors and
logic structures capable of processing audio signals.
[0023] Turning now to the loudspeaker array 4, FIG. 2A shows one
loudspeaker array 4 with multiple transducers 7 housed in a single
cabinet 8. In this example, ten transducers 7 may be aligned in a
single row in the cabinet 8 to form a sound-bar style loudspeaker
array 4. In another example shown in FIG. 2B, a loudspeaker array 4
may have thirty-two distinct transducers 7 evenly aligned in eight
rows and four columns within the cabinet 8. In other embodiments,
different numbers of transducers 7 may be used with uniform or
non-uniform spacing. Although shown as aligned in a flat plane or
straight line, the transducers 7 may be aligned in a curved fashion
along an arc.
[0024] The transducers 7 may be any combination of full-range
drivers, mid-range drivers, subwoofers, woofers, and tweeters. Each
of the transducers 7 may use a lightweight diaphragm, or cone,
connected to a rigid basket, or frame, via a flexible suspension
that constrains a coil of wire (e.g., a voice coil) to move axially
through a cylindrical magnetic gap. When an electrical audio signal
is applied to the voice coil, a magnetic field is created by the
electric current in the voice coil, making it a variable
electromagnet. The coil and the transducers' 7 magnetic system
interact, generating a mechanical force that causes the coil (and
thus, the attached cone) to move back and forth, thereby
reproducing sound under the control of the applied electrical audio
signal coming from a source (e.g., a signal processor, a computer,
and an audio receiver).
[0025] Each transducer 7 may be individually and separately driven
to produce sound in response to separate and discrete audio
signals. By allowing the transducers 7 in the loudspeaker array 4
to be individually and separately driven according to different
parameters and settings (including delays and energy levels), the
loudspeaker array 4 may produce numerous directivity/beam patterns
to simulate or better represent respective channels of sound
program content played according to the preferences of the
listeners 5. For example, beam patterns of different
widths/directivities and angles may be emitted by the loudspeaker
array 4.
[0026] As shown in FIG. 1, the loudspeaker array 4 may include
wires or conduit for connecting to the audio receiver 3. For
example, the loudspeaker array 4 may include multiple wiring points
and the audio receiver 3 may include complementary wiring points.
The wiring points may be binding posts or spring clips on the back
of the loudspeaker array 4 and the audio receiver 3, respectively.
The wires are separately wrapped around or are otherwise coupled to
respective wiring points to electrically couple the loudspeaker
array 4 to the audio receiver 3.
[0027] In other embodiments, the loudspeaker array 4 may be coupled
to the audio receiver 3 using wireless protocols such that the
array 4 and the audio receiver 3 are not physically joined but
maintain a radio-frequency connection. For example, the loudspeaker
array 4 may include a WiFi receiver for receiving audio signals
from a corresponding WiFi transmitter in the audio receiver 3. In
some embodiments, the loudspeaker array 4 may include integrated
amplifiers for driving the transducers 7 using the wireless audio
signals received from the audio receiver 3. As noted above, the
loudspeaker array 4 may be a standalone unit that includes
components for signal processing and for driving each transducer 7
according to the techniques described below.
[0028] Although shown in FIG. 1 as including a single loudspeaker
array 4, the audio system 1 may include any number of loudspeaker
arrays 4 that are coupled to the audio receiver 3 through wireless
or wired connections. For example, the audio system 1 may include
six loudspeaker arrays 4 that respectively represent a front left
channel, a front center channel, a front right channel, a rear
right surround channel, a rear left surround channel, and a low
frequency channel (e.g., a subwoofer) of a piece of sound program
content. In another embodiment, the audio system 1 may include two
loudspeaker arrays 4 that represent front left and front right
channels of a piece of stereo sound program content.
[0029] FIG. 3 shows a functional unit block diagram and some
constituent hardware components of the audio receiver 3 according
to one embodiment. The components shown in FIG. 3 are
representative of elements included in the audio receiver 3 and
should not be construed as precluding other components. Each
element of FIG. 3 will be described by way of example below.
[0030] The audio receiver 3 may include multiple inputs 9 for
receiving one or more channels of sound program content using
electrical, radio, or optical signals from one or more external
audio sources 2. The inputs 9 may be a set of digital inputs 9A and
9B and analog inputs 9C and 9D including a set of physical
connectors located on an exposed surface of the audio receiver 3.
For example, the inputs 9 may include a High-Definition Multimedia
Interface (HDMI) input, an optical digital input (Toslink), a
coaxial digital input, and a phono input. In one embodiment, the
audio receiver 3 receives audio signals through a wireless
connection with the external audio source 2. In this embodiment,
the inputs 9 include a wireless adapter for communicating with the
external audio source 2 using wireless protocols. For example, the
wireless adapter may be capable of communicating using Bluetooth,
IEEE 802.11x, cellular Global System for Mobile Communications
(GSM), cellular Code division multiple access (CDMA), or Long Term
Evolution (LTE) protocols.
[0031] In one embodiment, the external audio source 2 and the audio
receiver 3 are integrated in one indivisible unit. In this
embodiment, the loudspeaker array 4 may also be integrated into the
same unit. For example, the external audio source 2 and the audio
receiver 3 may be in one computing unit with transducers 7
integrated in left and right sides of the unit.
[0032] Returning to the audio receiver 3, general signal flow from
the inputs 9 will now be described. Looking first at the digital
inputs 9A and 9B, upon receiving a digital audio signal through the
input 9A and/or 9B, the audio receiver 3 uses a decoder 10A or 10B
to decode the electrical, optical, or radio signals into a set of
audio channels representing sound program content. For example, the
decoder 10A may receive a single signal containing six audio
channels (e.g., a 5.1 signal) and decode the signal into six audio
channels. The decoders 10 may be capable of decoding an audio
signal encoded using any codec or technique, including Advanced
Audio Coding (AAC), MPEG Audio Layer II, MPEG Audio Layer III, and
Free Lossless Audio Codec (FLAC).
[0033] Turning to the analog inputs 9C and 9D, each analog signal
received by analog inputs 9C and 9D may represent a single audio
channel of the sound program content. Accordingly, multiple analog
inputs 9C and 9D may be needed to receive each channel of a piece
of sound program content. The audio channels may be digitized by
respective analog-to-digital converters 11A and 11B to form digital
audio channels.
[0034] The digital audio channels from each of the decoders 10A and
10B and the analog-to-digital converters 11A and 11B are fed to the
multiplexer 12. The multiplexer 12 selectively outputs a set of
audio channels based on a control signal 13. The control signal 13
may be received from a control circuit or processor in the audio
receiver 3 or from an external device. For example, a control
circuit controlling a mode of operation of the audio receiver 3 may
output the control signal 13 to the multiplexer 12 for selectively
outputting a set of digital audio channels.
[0035] The multiplexer 12 feeds the selected digital audio channels
to an array processor 14. The channels output by the multiplexer 12
are processed by the array processor 14 to produce a set of
processed driving signals. The processing may operate in both the
time and frequency domains using transforms such as the Fast
Fourier Transform (FFT). The array processor 14 may be a special
purpose processor such as application-specific integrated circuit
(ASIC), a general purpose microprocessor, a field-programmable gate
array (FPGA), a digital signal controller, or a set of hardware
logic structures (e.g., filters, arithmetic logic units, and
dedicated state machines). The array processor 14 generates the set
of signals for driving the transducers 7 in the loudspeaker array 4
based on inputs from a listener location estimator 15, a listener
identifier 16, and/or a voice command processor 17.
[0036] The listener location estimator 15 estimates the location of
one or more human listeners 5 in the listening area 6. For example,
the location estimator 15 may estimate the physical coordinates of
a listener 5 in the listening area 6 or angle of a listener 5
relative to the loudspeaker array 4. FIG. 4A shows the estimated
location of the listener 5A in the listening area 6. The estimated
location is defined by coordinates x, y, relative to a side of the
loudspeaker array 4. Although shown in FIG. 4A as Cartesian
coordinates, the estimated location of the listener 5A may be
represented as an angle relative to the loudspeaker array 4 as
shown in FIG. 4B (e.g., listener 5A is degrees to the left of the
longitudinal axis of the loudspeaker array 4). Although described
in relation to a single listener 5, the location estimator 15 may
estimate the location of multiple listeners 5 in the listening area
6 (e.g., the listeners 5A and 5B).
[0037] The listener location estimator 15 may use any device or
algorithm for estimating the location of the listeners 5. In one
embodiment, one or more microphones 18A-18D may be communicatively
coupled to the listener location estimator 15 for assisting in
determining the location of one or more listeners 5 in the
listening area 6. In one embodiment, the microphones 18A-18D are
directly coupled to an audio codec 19. The audio codec 19 may be
used for coding or decoding a data stream or signals received from
the microphones 18A-18D. In one embodiment, the audio codec 19
performs conversion between the analog domain and the digital
domain for the microphone signals produced by the microphones
18A-18D, in addition to digital audio signal processing.
[0038] In one embodiment, the microphones 18A-18D are integrated in
the loudspeaker array 4 and microphone signals corresponding to
sensed sounds are transmitted to the audio receiver 3 using one or
more wireless protocols (e.g., Bluetooth and IEEE 802.11x). For
example, as shown in FIGS. 1, 4A, and 4B, the microphones 18A-18D
are integrated into the loudspeaker array 4. The microphones
18A-18D may be any type of acoustic-to-electric transducer or
sensor, including a MicroElectrical-Mechanical System (MEMS)
microphone, a piezoelectric microphone, an electret condenser
microphone, or a dynamic microphone. The microphones 18A-18D may
provide a range of polar patterns, such as cardioid,
omnidirectional, and figure-eight. In one embodiment, the polar
patterns of the microphones 18A-18D may vary continuously over
time.
[0039] In one embodiment, the microphones 18A-18D may form a
microphone array 18. The listener location estimator 15 may receive
inputs from the microphone array 18 and estimate the location of a
listener 5 based on these inputs. In one embodiment, the microphone
array 18 may sense a voice command from a listener 5 in the
listening area 6 and estimate the location of the listener 5 based
on the sensed voice command. For example, the listener 5 may
command the audio system 1 to output sound by saying "Play music."
In response to sensing by the microphone array 18 this command, the
location estimator 15 may begin attempting to locate the speaking
listener 5 in the listening area 6. In another example, the
listener 5 may instigate the location estimator 15 to locate the
listener 5 in the listening area 6 by greeting/addressing the audio
system 1 (e.g., "Hello System" where the audio system 1 is named
"System").
[0040] In one embodiment, the listener location estimator 15
estimates the location of one or more of the listeners 5 in the
listening area 6 by determining the angle of arrival for the sensed
sounds (i.e., the sensed voice command). For example, the location
estimator 15 may estimate the angle of arrival by running sensed
microphone signals received from the microphone array 18 through a
number of beamformers pointing in a discrete number of
angles/directions. The energy output from each beamformer is
calculated and the direction with the largest energy is chosen as
the angle of arrival. Although described in relation to the angle
or direction of the listener 5 to the loudspeaker array 4, in one
embodiment the location estimator 15 may also estimate the distance
of the listener 5 from the loudspeaker array 4 based on the sensed
microphone signals received from the microphone array 18.
[0041] In one embodiment, multiple microphone arrays 18 may be used
to estimate the location of one or more of the listeners 5 in the
listening area 6. For example, as shown in FIG. 4C, multiple
loudspeaker arrays 4A and 4B may each include separate microphone
arrays 18 composed of microphones 18A-18D and 18E-18H,
respectively. Knowing the relative geometry of the loudspeaker
arrays 4A and 4B, the location of one or more listeners 5 may be
estimated using triangulation. In this example, the estimated
location of the listeners 5 may be used to adjust beam patterns
generated by loudspeaker arrays 4A and 4B for angle and distance.
For example, the loudspeaker array 4A that is farther from the
listener 5B than loudspeaker array 4B may have its gain increased
such that the audio system 1 sounds more balanced to the listener
5B.
[0042] In another embodiment, multiple individual microphones
18A-18C may be used to estimate the location of one or more of the
listeners 5 in the listening area 6. For example, as shown in FIG.
4D, multiple loudspeaker arrays 4A-4C may each include separate
microphones 18A-18C. Knowing the relative geometry of the
loudspeaker arrays 4A-4C, the location of one or more listeners 5
may be estimated using time delays of arrival of the sound at each
microphone 18A-18C. For example, if the microphone 18B senses a
voice command a time d.sub.AB later than the microphone 18A, then
the listener 5B sits on a constant delay line AB. Similarly, if
microphone 18B senses the voice command a time d.sub.BC later than
the microphone 18C, then the listener 5B sits on a constant delay
line BC. The intersection of the delay lines AB and BC represents
an estimate of the location of the listener 5B. The estimated
location of the listener 5B may be used to adjust beam patterns
generated by loudspeaker arrays 4A-4C.
[0043] In one embodiment, the audio receiver 3 may also include a
listener identifier 16 for determining the identity of the
listeners 5 in the listening area 6. In one embodiment, the
listener identifier 16 receives signals from the microphone array
18. The signals may represent voice commands or other speech spoken
by a listener 5. The listener identifier 16 compares these speech
signals against patterns corresponding to known users/listeners.
For example, the audio receiver 3 may include a user profile
database 20 which stores speech patterns of known users/listeners.
The stored speech patterns may be recorded during normal use of the
audio system 1 or during a configuration of the audio system 1.
[0044] Based on the comparison, the listener identifier 16
associates speech from a listener 5 with a known user profile. The
user profile may include one or more preferences for the identified
listener 5. For example, the preferences may include preferred
volume settings, preferred bass level settings, preferred treble
level settings, preferred reverberation level settings, preferred
equalization settings, a common seating spot in the listening area
6, and/or other similar settings/preferences. In one embodiment,
upon failure to match speech from a listener 5 with stored speech
patterns associated with user profiles, the listener identifier 16
may generate a new user profile for the new listener 5. The newly
created user profile may be initialized with default settings. As
the new listener 5 uses the audio system 1 over time and alters
settings based on preference (e.g., using voice commands processed
by the voice command processor 17), the user profile settings may
adapt to match these preferences.
[0045] For example, in response to the listener 5A's request to
increase the volume, the listener 5A's user profile may be updated
to indicate a preference for higher volume. Accordingly, during
subsequent use of the audio system 1 by the listener 5A, the volume
of audio output by the audio receiver 3 and the loudspeaker array 4
may begin at a higher volume. Similar adjustments may be made to
other user profile preferences based on the use of the audio system
1 over time by each of the listeners 5.
[0046] In one embodiment, user profile settings are content based
such that each setting in a listener 5's user profile may have
separate values for different content types. For example, a volume
setting may have separate preferred values for music, movies,
television, etc. These content delineations may be further divided
based on genre (e.g., separate volume settings for comedy movies,
honor movies, and drama movies).
[0047] In one embodiment, the user profile setting may be similarly
divided by time of day. For example, preferred volume settings may
be based on time of day such that during the morning hours a
preferred volume setting is at a first value (e.g., 15 dB) and
during the afternoon hours the preferred volume setting is at a
second value (e.g., 20 dB). Although described in relation to
volume settings, each user profile setting for each listener 5 may
be similarly divided.
[0048] In one embodiment, the audio receiver 3 may also include a
voice command processor 17. The voice command processor 17 receives
signals from the microphone array 18 either directly or indirectly
through the audio codec 19. The microphone signals may represent
voice commands spoken by a listener 5. The voice command processor
17 processes these microphone signals to determine the intended
command from the listener 5 and transmits corresponding control
signals to the array processor 14 to carry out the command. For
example, the microphone signals may correspond to the listener 5A
stating "Hi, System!" or "System, add me!" In response to receipt
of these microphone signals, the voice command processor 17 may
transmit control signals to cause the array processor 14 to
generate a beam pattern directed at the listener 5A. In this
example, the beam pattern may be generated also based on inputs
from the listener location estimator 15 and the listener identifier
16 such that the beam pattern is focused on the current location of
the listener 5A and according to preferences of the listener
5A.
[0049] In one embodiment, the listener 5A may focus a beam pattern
at preset zones, positions, or spots in the listening area 6
instead of the estimated location of the listener 5 determined by
the listener location estimator 15. For example, the listener 5A
may state "Spot 2 at half volume." In this example, spot 2 may be
preset to be the right and forward-most seat on a couch in the
listening area 6 where the listener 5B is located. In response to
this request from listener 5A, the voice command processor 17
transmits control signals such that the array processor 14 drives
the loudspeaker array 4 to generate a beam pattern at spot 2 in the
listening area 6 with half volume.
[0050] In one embodiment, a beam pattern may be directed at an
identified listener 5 based on historical usage of the audio system
1. For example, after continual use of the audio system 1 by
listener 5A at 6:00 PM for several days with sound beams directed
at predefined spots 1 and 2, subsequent use of the audio system 1
by listener 5A at or around 6:00 PM will default to directing sound
beams at predefined spots 1 and 2. In one embodiment, the audio
system 1 may default to the last known position of one more
listeners 5 upon the listener location estimator 15 being unable to
estimate the location of the listeners 5. The last know locations
of the listeners 5 may be stored in the user profile database 20
along with common/preferred seating locations in the listening area
6.
[0051] As described above, the voice command processor 17 analyzes
voice commands to determine operations to perform. The voice
commands may be preset operations. For example, the listener 5A may
state "Equalize for trance music." In response to this request, the
voice command processor 17 transmits control signals which cause
the array processor 17 to equalize the input audio based on preset
equalization settings for trance music. In another example, the
listener 5A may state "System, flood the room!" In response to this
request, the voice command processor 17 transmits control signals
which cause the array processor 17 to drive the loudspeaker array 4
to generate a wide beam pattern that encompasses the entire
listening area 6.
[0052] As described above, based on inputs from the listener
location estimator 15, the listener identifier 16, and/or the voice
command processor 17 one or more audio channels of a piece of sound
program content are modified by the array processor 14 to generate
beam patterns according to preferences of the listeners 5. The
processed segments of the sound program content are passed from the
array processor 14 to the one or more digital-to-analog converters
21 to produce one or more distinct analog signals. The analog
signals produced by the digital-to-analog converters 21 are fed to
the power amplifiers 22 to drive selected transducers 7 of the
loudspeaker array 4 to produce the desired beam patterns.
[0053] Turning now to FIG. 5, a method 30 for adjusting sound
emitted by the loudspeaker array 4 will be described. The
operations of method 30 may be performed by one or more components
of the audio receiver 3, the loudspeaker array 4, and other devices
of the audio system 1.
[0054] In one embodiment, the method 30 begins at operation 31 with
a listener 5 speaking a voice command. The voice command may
indicate the listener 5's desire to have a sound beam focused on
him/her or a designated location in the listening area 6, adjust
sound emitted by the loudspeaker array 4 (e.g., volume,
equalization, and reverberation), and/or other similar
modifications. FIG. 6A and FIG. 6B show examples of the listener 5A
speaking separate voice commands.
[0055] Following a listener uttering a voice command, operation 32
senses the voice command using one or more microphone arrays 18.
The microphone arrays 18 may be incorporated into one or more
loudspeaker arrays 4, respectively, or coupled directly to the
audio receiver 3. Microphone signals corresponding to the sensed
voice command may be relayed to the audio receiver 3 for further
processing as will be described in further detail below.
[0056] At operation 33, the microphone signals are compared against
one or more speech patterns/signatures associated with user
profiles to identify the speaking listener 5. The comparison may
use any known technique for performing voice recognition such that
the speaking listener 5 is identified. In one embodiment, upon
failure to match speech from a listener 5 with stored speech
patterns/signatures associated with user profiles, operation 33 may
generate a new user profile for the new listener 5. The newly
created user profile may be initialized with default settings. As
the new listener 5 uses the audio system 1 over time and alters
settings based on preference (e.g., voice commands), the user
profile settings may adapt to match these preferences.
[0057] At operation 34, the method 30 determines settings to apply
to a beam pattern that will be generated by the audio receiver 3
and the loudspeaker array 4. The settings may be based on
indications in the sensed voice command, the estimated location of
the speaking/identified listener 5, and/or preferences stored in a
user profile associated with the speaking/identified listener 5.
For example, operation 35 may determine an estimated location of
the speaking/identified listener 5 using the listener location
estimator 15 as described above. Using this estimated location,
operation 35 may determine a direction setting for steering the
beam pattern. In another example, the sensed voice command may
state "Reduce sound to half volume." In response to this voice
command, operation 35 may determine a volume setting for the beam
pattern. In still another example, operation 35 may determine
equalization settings for audio emitted through the beam pattern
based on stored preferences in an associated user profile.
[0058] Based on the example voice command in FIG. 6A, operation 34
may determine settings for a first beam pattern directed to spot 1
with half volume and second beam pattern directed to spot 2 with
full volume. Other settings may be determined based on the identity
of listeners 5A and 5B, the time of day, the audio content, etc.
Based on the example voice command in FIG. 6A, operation 34 may
determine settings for a first beam pattern directed at the
estimated location of the listener 5A and a second beam pattern
directed at the estimated location of the listener 5B. Other
settings may be determined based on the identity of listeners 5A
and 5B, the time of day, the audio content, etc.
[0059] At operation 35, the method 30 applies the determined
settings to generate tailored beam patterns without significant
overhead and interaction by the listeners 5A and 5B. In particular,
the generated beam patterns are customized to the explicit and
implicit preferences of the listeners 5A and/or 5B with minimal
input from the listeners 5A and 5B. FIGS. 7A and 7B show beam
patterns generated for the listeners 5A and 5B based on the voice
commands in FIGS. 6A and 6B, respectively. In one embodiment, the
method 30 is continually performed to modify beam patterns for
listeners 5 as preferences and positions of the listeners 5 change
over time. In one embodiment, operation 35 updates preferences for
respective user profiles stored in the user profile database 20
based on currently used settings. This adjustment ensures that
listener settings are up-to-date and reflective of the current
preferences of each listener 5.
[0060] As explained above, an embodiment of the invention may be an
article of manufacture in which a machine-readable medium (such as
microelectronic memory) has stored thereon instructions which
program one or more data processing components (generically
referred to here as a "processor") to perform the operations
described above. In other embodiments, some of these operations
might be performed by specific hardware components that contain
hardwired logic (e.g., dedicated digital filter blocks and state
machines). Those operations might alternatively be performed by any
combination of programmed data processing components and fixed
hardwired circuit components.
[0061] While certain embodiments have been described and shown in
the accompanying drawings, it is to be understood that such
embodiments are merely illustrative of and not restrictive on the
broad invention, and that the invention is not limited to the
specific constructions and arrangements shown and described, since
various other modifications may occur to those of ordinary skill in
the art. The description is thus to be regarded as illustrative
instead of limiting.
* * * * *