U.S. patent number 3,665,105 [Application Number 05/017,747] was granted by the patent office on 1972-05-23 for method and apparatus for simulating location and movement of sound.
This patent grant is currently assigned to The Board of Trustees of the Leland Stanford Junior University. Invention is credited to John M. Chowning.
United States Patent |
3,665,105 |
Chowning |
May 23, 1972 |
METHOD AND APPARATUS FOR SIMULATING LOCATION AND MOVEMENT OF
SOUND
Abstract
A method and system for generating or processing music or sound
signal information to provide, in addition to the musical
parameters of pitch, loudness, time, timbre, control over the
apparent location and movement of the sound. The method and system
controls the distribution of energy between loudspeakers to provide
amplitude changes for the direct and reverberant signals and
doppler shift to give the illusion of motion or arbitrary location
in space of the sound produced by the loudspeakers.
Inventors: |
Chowning; John M. (Stanford,
CA) |
Assignee: |
The Board of Trustees of the Leland
Stanford Junior University (Stanford, CA)
|
Family
ID: |
21784310 |
Appl.
No.: |
05/017,747 |
Filed: |
March 9, 1970 |
Current U.S.
Class: |
381/17;
338/128 |
Current CPC
Class: |
H04R
3/12 (20130101); H04S 3/00 (20130101) |
Current International
Class: |
H04R
3/12 (20060101); H04S 3/00 (20060101); H04r
005/00 (); H03h 007/30 () |
Field of
Search: |
;179/1G,1GP,1M,1DM,1J,1AT |
References Cited
[Referenced By]
U.S. Patent Documents
Primary Examiner: Claffy; Kathleen H.
Assistant Examiner: Olms; Douglas W.
Claims
I claim:
1. The method of processing signals representative of sound
information to provide signals in two or more channels which can be
reproduced by two or more loudspeakers comprising the steps of
varying the frequency of the signals to provide an output signal
having an apparent doppler shift corresponding to the apparent
velocity of the change in distance from the listener, varying the
amplitude of the signal in inverse proportion to the apparent
distance from the listener, applying the frequency and amplitude
modulated signals to two or more channels, and individually
amplitude modulating the signals on each channel in proportion to
the desired direction of sound from the listener whereby to provide
sound having apparent movement and direction.
2. The method as in claim 1 wherein said signal is amplitude
modulated in inverse proportion to the square of the distance.
3. The method as in claim 1 additionally including means for
obtaining a fraction of the frequency modulated signal to provide a
reverberant signal, varying the amplitude of the reverberant signal
in inverse proportion to the desired apparent distance from the
listener, and adding said signal equally to the two or more
channels.
4. The method as in claim 3 including the steps of developing a
reverberant signal whose amplitude for each channel varies with
direction and distance and adding said signal to the signals in
said one or more channels.
5. The method as in claim 1 including the steps of developing a
reverberant signal whose amplitude for each channel varies with
direction and distance and adding said signal to the signals in
said one or more channels.
6. The method of processing signals representing sound information
to provide reverberant signals in two or more channels which can be
reproduced by two or more loudspeakers which comprises varying the
amplitude of said signals in inverse proportion to the desired
apparent distance from a listener, obtaining a fraction of the
amplitude varied signals and varying the amplitude of said fraction
in inverse proportion to the distance from the listener, delaying
the amplitude varied signals and fraction, and adding the delayed
signals equally to the undelayed signals in said two or more
channels.
7. The method as in claim 6 including the step of developing a
reverberant signal whose amplitude for each channel varies in
proportion to direction and distance and adding said signal to the
signals in said two or more channels.
8. Apparatus for processing signals having a predetermined
frequency representative of sound information to provide signals
which can be applied to two or more loudspeakers for reproduction
including means for receiving said signals and providing an output
signal whose frequency is modulated in accordance with apparent
rate of change of distance away from a listener, means for
receiving said frequency signal and providing a plurality of
signals each having amplitude corresponding to the direction of the
signals from the listener, and means for forming a recording of
said signals for reproduction.
9. Apparatus as in claim 8 including means for obtaining a fraction
of said signal and adding the same equally to the plurality of
signals.
10. Apparatus as in claim 9 including additionally means for
receiving said fraction of said signal and providing a plurality of
signals having amplitude corresponding to direction of direct
signal and adding said signal to the plurality of signals.
11. Apparatus for processing signals having a predetermined
frequency representative of sound information to provide signals
which can be applied to two or more loudspeakers for reproduction
including means for receiving said signals and providing an output
signal whose frequency is modulated in accordance with apparent
rate of change of distance away from a listener, and means for
receiving said frequency signal and providing a plurality of
signals each having amplitude corresponding to the direction of the
signals from the listener.
12. The method as in claim 5 wherein said signals are amplitude
modulated in proportion to direction and the amplitude modulated
signals are added to said channels.
Description
BACKGROUND OF THE INVENTION
This invention relates to a system and method for generating or
processing music or sound so that when the music or sound is
produced by loudspeakers, it will have location and apparent
movement in space.
The normal experience in listening to music is to have a continuum
of sound source locations. This includes direct sounds from the
location of the sources and reverberations from the surrounding
environment. In addition, if the sound source moves, there is a
frequency shift corresponding to the doppler frequency shift or
effect.
However, when music including electronic music, recorded music,
computer music and synthesized music is reproduced by loudspeakers,
the number of sound locations is limited by the number of available
loudspeakers. In conventional stereophonic music, there are two
loudspeakers spaced apart to give the effect of the music emanating
from any point between the two locations.
In order to locate any real sound source in an enclosed space, the
listener requires two types of information: the angular location of
the source and the distance information. The cues for the angular
location are the energy ratio to the two ears resulting from the
shadow effect of the head and the phase shift resulting from the
different arrival times of the signals to the two ears. The cue to
the distance of a sound source is the ratio of direct sound to the
indirect or reverberant sound where the energy of the direct sound
reaching the listener falls off more quickly with distance than the
reverberant sound. In addition, when the sound source is moving,
there is a frequency shift in the received signal which corresponds
to the doppler shift.
It is, therefore, desirable to provide a system and method for
processing sound or music to be reproduced by a plurality of
loudspeakers in which the signals applied to each of the
loudspeakers are so processed as to have frequencies and amplitudes
to give the effect of the sound or music being generated in a
spatial continuum.
SUMMARY OF THE INVENTION AND OBJECTS
The present invention relates to a system and method for processing
signals representative of sound information to produce a plurality
of signals which can be produced by two or more loudspeakers to
provide sound having apparent movement and location within a
selected space environment. Briefly, the system and method comprise
frequency modulating the signals to provide an output signal having
a frequency variation which corresponds to the doppler shift due to
the radial velocity of the sound source with respect to the
listener, amplitude variations inversely proportional to the
distance and distributed between the loudspeakers in accordance
with angular location of the source. In addition, the system
includes control of the reverberant signal such that the
distribution of the total reverberant energy to the plurality of
speakers is a function of the distance of the direct signal from
the listener. When the direct signal is in greatest proximity to
the listener, the reverberant signal is distributed equally between
the plurality of speakers, and when the direct signal is in the
least proximity to the listener, the reverberant signal is
distributed among the speakers in the same ratio as the direct
signal.
It is a general object of the present invention to provide a system
and method for generating or processing sound and music signals so
that when the signals are reproduced, the sound or music has
illusory location and movement.
It is another object of the present invention to provide a system
and method for generating or processing sound or music in which the
music may be reproduced by a plurality of loudspeakers and yet give
the effect of a continuum of possible sound sources where the sound
source has angular and radial spatial displacement and movement as
well as reverberations corresponding to a selected sound
environment.
The foregoing and other objects of the invention will become more
clearly apparent from the following description taken in connection
with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic representation showing a single sound source
moving within en enclosed space with respect to a listener and the
location of the speakers to reproduce the sound.
FIG. 2 shows one embodiment of a system for processing sound or
music signals in accordance with the present invention.
FIG. 3 shows another embodiment of a system for processing sound or
music signals in accordance with the invention.
FIG. 4 shows a means for arranging the potentiometers shown in FIG.
3 so as to be automatically operated when sound signal location is
selected.
FIG. 5 is a sectional view taken along line 5--5 of FIG. 4.
FIG. 6 is a schematic representation of a system for recording and
reproducing signal information generated or processed in accordance
with the invention.
DESCRIPTION OF PREFERRED EMBODIMENT
Referring to FIG. 1, there is illustrated an enclosed space or
environment 11 such as an auditorium or concert hall. A sound
source 12 is disposed in the space and adapted to move along the
line 13. A listener 14 is shown in the center of the space 11. A
plurality of speakers 1, 2, 3 and 4 are shown in relation to the
listener as the speakers would be located in a reproduction
system.
It is recognized that the space 11 may have different effects upon
the moving sound. For example, if the space 11 is a marble room,
the level of reverberations from the sound source will be high,
whereas if the walls of the space 11 are acoustically damped,
reverberations will be low. Furthermore, if the space is large, the
reverberations will be low, and if the space represents the
outdoors, there will be fewer reverberations.
The listener 14 listening to the sound from the source will first
obtain distance information D. Assuming that the source is at the
point a, the direct and reverberant sound will appear to come
primarily from locations directly ahead with each of the listener's
ears receiving the same amplitude of sound. In reproduction, the
sound should emanate primarily from speakers 1 and 2 if the sound
is to simulate to the condition just described. However, as the
sound source moves to the point b along the line 13, the sound
source is substantially primarily from the location b and at a
different distance. In reproduction, the direct sound should come
primarily from speaker 2. Furthermore, if the source travels
between a and b along the line 13, there will be a radial component
of velocity which will introduce a doppler frequency shift. As the
sound reaches the point c, it will appear to come entirely from the
right of the listener. The direction changes as the sound source
moves along the line 13 from the points a through f.
In accordance with the present invention, it is desired to either
synthesize a sound or process a sound so that it can be reproduced
on two or more loudspeakers, loudspeakers 1, 2, 3 and 4, and have
the same effect on the listener as if the sound were moving in the
space 11. The reproduced sound will have amplitude which changes
with distance. The power of the sound varies in proportion to the
inverse square of the distance. The signals applied to the speakers
will, therefore, have a corresponding amplitude change. In
addition, however, the sound will reverberate from all walls and
the amount of reverberation will depend upon the characteristics of
the space 11 and the distance from the source of reverberation. In
general, as the distance from the listener increases, the distance
from some of the wall surfaces decreases and the reverberant signal
increases in that direction. The reverberant signal can be
approximated by introducing a factor representative of the
characteristics of the enclosure and varying the strength from any
direction in proportion to the inverse of the distance. In
addition, as the location of the sound source is closer to one of
the walls, the amount of reverberant sound from that particular
location will increase. For example, the reverberations from the
location b will be primarily from that location and, therefore,
there will be an additional intensity of reverberant signal at this
location which can be introduced by the loudspeaker 2 to give the
effect of the sound being located in that general position.
In summary, the movement of the sound can be simulated by a
frequency shift corresponding to movement toward or away from the
listener; the location of the sound by the ratio of amplitude of
sounds emanating from the various loudspeakers; the distance by the
ratio of amplitudes between the direct and reverberant signals; and
the overall acoustics of the enclosure by the reverberations.
A signal may be processed to provide a four channel output for
application to the speakers to simulate location, movement and
environment by a system including frequency and amplitude
modulators. A suitable processing system is shown in FIG. 2. The
signal to be processed is applied along the line 21 to a frequency
modulator 22. The frequency of the signal is modulated by applying
a voltage which corresponds to the radial velocity dD/dt. The
output from the frequency modulator 22, line 23, therefore,
includes the doppler frequency shift. Suitable means for generating
voltages representative of distance D and angular position .theta.
will be described with reference to FIGS. 4 and 5.
The signal is applied to an amplitude modulator 24 which modulates
the amplitude of the signal in proportion to the factor 1/D.sup.2.
The output of the amplitude modulator 24, line 26, therefore, has
an amplitude which corresponds to the distance of the direct signal
from the listener. This signal is applied to four amplitude
modulators 31 - 34. These four modulators are connected to the four
loudspeakers 1, 2, 3 and 4, respectively, through summing networks
76, 77, 78 and 79. For example, if the sound is to emanate from the
direction of loudspeaker 1, only that channel will have direct
signals. Thus, the modulating signal applied to the amplitude
modulator 31 via channel 1 will be maximum and the modulating
signals applied to channels 2, 3 and 4 will be zero. Similarly, if
the signal is to appear to emanate from the position a, FIG. 1,
then the modulating signals applied to amplitude modulators 31 and
32 from channels 1 and 2 will be equal and less than maximum so
that the output on lines 36 and 37 is equal and the level of sound
to the listener from speakers 1 and 2 will be equal. No signal is
applied to the amplitude modulators 33 and 34 so that there is no
output on the lines 38 and 39. Thus, the signals in the four
channels are amplitude modulated corresponding to the angular
position of the sound source. The system thus far described
provides for velocity and angular location of the direct signal
within the listening space 11 together with some distance
information.
If the sound were emanating from a place having no reverberations
whatsoever, the signals applied to the speakers 1, 2, 3 and 4 would
then correspond to the signals on the lines 36, 37, 38 and 39.
Generally, the sound source is within some space or environment 11.
The system of the present invention includes means for introducing
into each of the channels or speakers 1, 2, 3 and 4 a signal which
produces reverberant sound on which the distance information partly
depends.
The frequency modulated signal on the line 23 is applied to an
attenuator 41 wherein the signal is reduced by a given percent, %,
to provide at the output line 42 a signal which is a percent of the
signal on the line 23. If the enclosure is assumed to be highly
reverberant, then the percentage is relatively high while the
percentage is zero for an infinite space and no output signal would
appear on the line 42.
Thus, the output on the line 42 is a fraction, % SIG, of the
frequency modulated signal on line 23. The signal is amplitude
modulated in an amplitude modulator 43 by a signal which
corresponds to about 1/.sqroot.D. The actual relation for
reverberation is arbitrary. It need be a scaling function that
attenuates the portion of the signal to be reverberated at a rate
that is not as extreme with distance as is the direct signal. As
previously described, the amplitude of the reverberant signal falls
off less with distance than the amplitude of the direct signal.
The output from the amplitude modulator appearing on line 44 is %
SIG/.sqroot.D. This signal is multiplied or modulated at 27 by the
distance function 1/D which determines the amount of "global"
reverberation. The output on line 28 goes equally to the four
reverberators' summing networks 46, 47, 48 and 49. The global
reverberation decreases with increased distance of the direct
signal. The signal on line 44 is modulated or multiplied at 29 by 1
- 1/D whereby this portion of the total reverberant signal
increases with distance of the source away from the listener. This
reverberant signal is distributed in location according to the same
functions which control the direct signal by applying it to
modulators 61, 62, 63 and 64 controlled by the signal which
controls modulators 31, 32, 33 and 34. The output of these
modulators is then added to the global reverberation at summing
networks 46, 47, 48 and 49, and applied to reverberators 71, 72, 73
and 74 which introduce reverberation. The reverberators 71, 72, 73
and 74 may be of the type described by Schroeder in the Journal of
the Audio Engineering Society, July 1962, Volume 10, No. 3,
pp.219-223. The effect of the system just described is to cause an
increase in the localization of the reverberant signal as the
direct signal moves away from the listener.
The output of the reverberators 71, 72, 73 and 74 is applied to
adders 76, 77, 78 and 79 where it is added to the direct signals
from modulators 31, 32, 33 and 34.
Making reference to the first example, where the signal is located
in a direction between speakers 1 and 2, the signal is equal for
the two speakers. The signal that is applied to the amplitude
modulators 31 and 32 is also supplied to the multipliers 61 and 62
and serves to provide equal signals to the adders 46 and 47 while
zero signals appear at the output of the modulators 63 and 64 for
addition at the adders 48 and 49. The signal output from adders 46
and 47 is reverberated and added to the output from the amplitude
modulators 31 and 32 appearing on lines 36 and 37. Thus, the signal
applied to speakers 1 and 2 includes a component which corresponds
to the overall reverberation of the space or global reverberation,
line 28, together with a component which corresponds to the direct
or local reverberations and the direct signal information. The
signals applied to speakers 3 and 4 have only global reverberation,
line 28.
Thus, the outputs of lines 1, 2, 3 and 4 will contain various
signal components including the doppler shift information, distance
information, directional reverberant and global reverberant
information to thereby simulate dynamic sound in space.
It should be noted that the reason for the independent
reverberators is to achieve different reverberations from the four
directions. Each reverberator has its delays and gains in an
incommensurate relationship to achieve the different (though
slight) reverberations. This method is effective in creating a
diffused sound space which minimizes the point source location of
the loudspeakers.
In FIGS. 3, 4 and 5, there is shown a circuit and apparatus for
operating on the signals in accordance with the invention.
Referring first to FIGS. 4 and 5, the apparatus includes an arm 81
which carries a movable slider 82. One end of the arm 81 is secured
to a shaft 83 to pivot about the shaft and rotate the shaft. The
arm, shaft and slider are shown in FIG. 1 where the arm and slider
are moved to trace the path 13 of the movement of the sound within
the space 11.
The arm is provided with wound resistors or resistive films 86, 87,
88, 89 and 90 on three faces. The slider includes sliders 91, 92,
93, 94 and 95 which ride along the resistive elements to form
contact therewith. A voltage is applied to one end of the resistive
film and the other end is grounded. The slider and resistive film
act as a potentiometer whereby movement of the slider picks off a
portion of the voltage which appears on the leads connected to the
sliders.
Referring particularly to the left-hand side of the circuit diagram
in FIG. 3, the resistive elements 86, 87, 88, 89 and 90 are shown
in association with the sliders 91, 92, 93, 94 and 95. The
resistive element 86 has applied thereto a fixed voltage +V. The
contactor 91 picks up a predetermined portion of said voltage which
varies in proportion to radial distance D and applies it to a
potentiometer resistor 96. The contactor 97 is positioned to scale
the doppler for distance. The output on the contactor 97 is applied
to a differentiating circuit 98 which provides an output voltage
dD/dt which is applied to a voltage controlled oscillator (VCO) 99
which provides an output signal having a variable frequency. This
signal is applied to a magnetic tape transport which drives the
tape at a variable speed.
As the slider is moved along the line 13, FIG. 1, there will be
radial movement corresponding to the distance D and this causes a
change in voltage at slider 91 and, in turn, a change in the speed
of the capstan drive of the tape transport and, therefore, the tape
speed. The reproduced signal from the tape 101 appearing on the
line 23a is frequency modulated to the desired doppler shift. This
signal is applied to one end of the potentiometer resistor 87 which
has its other end grounded. The variable tap 92 picks off a
proportion of this signal. The resistive element 87 is tailored so
that the voltage on the line 92 varies as the factor 1/D.sup.2,
which voltage then appears on the line 26. Thus, the amplitude of
the signal is varied inversely proportional to the square of the
distance D. The voltage on line 23a is also applied to a
potentiometer which includes resistor 102 and variable tap 103.
This potentiometer serves to obtain a fraction of the frequency
modulated signal (% SIG) and apply the same to the line 42. This
potentiometer is employed to reduce the signal in accordance with
the reverberant character-istics of the enclosure or surrounds. The
signal on line 42 is applied to the potentiometer resistor 88 which
is proportioned to give an output equal to 1/.sqroot.D times the
input which scales the reverberant signal as previously described.
The output on tap 93 is applied to the potentiometer resistors 89
and 90 which are proportioned to give a signal inversely
proportional to distance. The potentiometer 90 is reversed to give
1 - 1/D.
Thus, there is provided the signal for the line 28a which is
applied to adders 46a, 47a, 48a and 49a and the signal which is
applied to modulators 61a, 62a, 63a and 64a.
The amplitude modulators are in the form of potentiometers, FIG. 4,
and rotation of shaft 83 serves to rotate the contactors in each of
the potentiometers to pick off the output voltage. The
potentiometers 31a, 32a, 33a and 34a are each staggered 90.degree.
with respect to the adjacent potentiometer as shown. The voltage
along the line 26a is applied to the center of the potentiometer
resistor. Thus, for example, when the pointer 12 is at the point a,
voltage is derived at potentiometers 31a and 32a on lines 36 and 37
in equal amplitude, while zero voltage is derived at potentiometers
33a and 34a on lines 38 and 39. When the pointer is at point b,
full voltage appears at potentiometer 32a on line 37 and zero
voltage at all other potentiometers.
The potentiometers 61a, 62a, 63a and 64 are likewise arranged in
90.degree. relationship with the voltage on the line 30 applied at
the center of the resistors as shown. The percent direct
reverberant signal is varied in the same manner as the direct
signal as described above.
The output of the potentiometers 31a-34a appearing on the lines
36-39 is applied to adders 76a, 77a, 78a and 79a. The sum appearing
at the adders 46a, 47a, 48a and 49a is applied to reverberators 71,
72, 73, 74, and thence to the adders 76a-79a. The added signals are
then either applied to speakers 1, 2, 3 and 4 or are applied to a
four-track recorder-reproducer where the signals can be stored and
thereafter reproduced and applied to speakers 1, 2, 3 and 4. Thus,
there is shown a simple apparatus including potentiometric means
for performing substantially all of the functions described with
respect to the circuits shown in FIG. 2.
FIG. 6 shows the signal outputs 1, 2, 3 and 4 applied to means for
recording and reproducing the signals whereby they can be played
back through the associated loudspeakers. The recording means may
be magnetic tape, records or the like.
The two embodiments just described are two-dimensional; however, it
will be apparent that the invention is applicable to
three-dimensional sound reproduction and creation. For example, the
speakers can be arranged in tetrahedron or eight in square and
signal processing provided to take into account the
three-dimensional direction of the sound. The percentage values and
powers of distance given in the foregoing discussion were
empirically determined and can be varied to provide different sound
effects.
Furthermore, it should be apparent that the processes shown and
described in connection with the embodiment of FIG. 2 and the
embodiment of FIGS. 3, 4 and 5 can be carried out by a suitably
programmed digital or analog computer which processes the signal
information applied thereto in accordance with programmed
instructions.
* * * * *