US3665105A

US3665105A - Method and apparatus for simulating location and movement of sound

Info

Publication number: US3665105A
Application number: US17747A
Authority: US
Inventors: John M Chowning
Original assignee: Leland Stanford Junior University
Current assignee: Leland Stanford Junior University
Priority date: 1970-03-09
Filing date: 1970-03-09
Publication date: 1972-05-23
Anticipated expiration: 1989-05-23

Abstract

A method and system for generating or processing music or sound signal information to provide, in addition to the musical parameters of pitch, loudness, time, timbre, control over the apparent location and movement of the sound. The method and system controls the distribution of energy between loudspeakers to provide amplitude changes for the direct and reverberant signals and doppler shift to give the illusion of motion or arbitrary location in space of the sound produced by the loudspeakers.

Description

United States Patent Chowning 51 May 23, 1972 i541 METHOD AND APPARATUS FOR SIMULATING LOCATION AND MOVEMENT OF SOUND [72] Inventor: John M. Chowning, Stanford, Calif.

[73] Assignee: The Board of Trustees of the Leland Stanford Junior University, Stanford, Calif.

[22] Filed: Mar. 9, 1970 [21] Appi.No.: 17,747

521 U.S.Cl ..l79/1G,l79/1.i 51 lnt.Cl ..H04r5/00,H03h7/30 [58] FieldofSearch ..179/lG,lGP,lM,.lDM,lJ, 179/1 AT dD aflDOPPLER) [56] References Cited UNITED STATES PATENTS 3,322,899 5/1967 Renwick, Jr. ..l79/l GP Primary Examiner-Kathleen H. Claffy Assistant xaminer-Douglas W. Olms Att0rneyFlehr, Hohbach, Test, Albritton & Herbert ABSTRACT A method and system for generating or processing music or sound signal information to provide, in addition to the musical parameters of pitch, loudness, time, timbre, control over the apparent location and movement of the sound. The method and system controls the distribution of energy between loudspeakers to provide amplitude changes for the direct and reverberant signals and doppler shift to give the illusion of motion or arbitrary location in space of the sound produced by the loudspeakers.

12 Claims, 6 Drawing Figures REVERB.

CH. l

REVERB.

Patented May 23, 1972 3,665,105

3 Sheets-Sheet 1 (l RECORDER REPRODUCING INVENTOR.

JOHN M. CHOWNING m, 40.1 W ATTORNEYS METHOD AND APPARATUS FOR SIMULATING LOCATION AND MOVEMENT OF SOUND BACKGROUND OF THE INVENTION This invention relates to a system and method for generating or processing music or sound so that when the music or sound is produced by loudspeakers, it will have location and apparent movement in space.

The normal experience in listening to music is to have a continuum of sound source locations. This includes direct sounds from the location of the sources and reverberations from the surrounding environment. In addition, if the sound source moves, there is a frequency shift corresponding to the doppler frequency shift or effect. I

However, when music including electronic music, recorded music, computer music and synthesized music is reproduced by loudspeakers, the number of sound locations is limited by the number of available loudspeakers. In conventional stereophonic music, there are two loudspeakers spaced apart to give the effect of the music emanating from any point between the two locations.

In order to locate any real sound source in an enclosed space, the listener requires two types of information: the angular location of the source and the distance information. The cues for the angular location are the energy ratio to the two ears resulting from the shadow effect of the head and the phase shift resulting from the difierent arrival times of the signals to the two ears. The cue to the distance of a sound source is the ratio of direct sound to the indirect or reverberant sound where the energy of the direct sound reaching the listener falls off more quickly with distance than the reverberant sound. In addition, when the sound source is moving, there is a frequency shift in the received signal which corresponds to the doppler shift.

It is, therefore, desirable to provide a system and method for processing sound or music to be reproduced by a plurality of loudspeakers in which the signals applied to each of the loudspeakers are so processed as to have frequencies and amplitudes to give the effect of the sound or music being generated in a spatial continuum.

SUMMARY OF THE INVENTION AND OBJECTS The present invention relates to a system and method for processing signals representative of sound information to angular location of the source. In addition, the system includes control of the reverberant signal such that the distribution of the total reverberant energy to the plurality of speakers is a function of the distance of the direct signal from the listener. When the direct signal is in greatest proximity to the listener, the reverberant signal is distributed equally between the plurality of speakers, and when the direct signal is in the least proximity to the listener, the reverberant signal is distributed among the speakers in the same ratio as the direct signal.

It is a general object of the present invention to provide a system and method for generating or processing sound and music signals so that when the signals are reproduced, the sound or music has illusory location and movement.

It is another object of the present invention to provide a system and method for generating or processing sound or music in which the music may be reproduced by a plurality of loudspeakers and yet give the effect of a continuum of possible sound sources where the sound source has angular and radial spatial displacement and movement as well as reverberations corresponding to a selected sound environment.

The foregoing and other objects of the invention will become more clearly apparent from the following description taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic representation showing a single sound source moving within en enclosed space with respect to a listener and the location of the speakers to reproduce the sound.

FIG. 2 shows one embodiment of a system for processing sound or music signals in accordance with the present invention.

FIG. 3 shows another embodiment of a system for processing sound or music signals in accordance with the invention.

FIG. 4 shows a means for arranging the potentiometers shown in FIG. 3 so as to be automatically operated when sound signal location is selected.

FIG. 5 is a sectional view taken along line 5-5 of FIG. 4.

FIG. 6 is a schematic representation of a system for recording and reproducing signal information generated or processed in accordance with the invention.

DESCRIPTION OF PREFERRED EMBODIMENT Referring to FIG. 1, there is illustrated an enclosed space or environment 11 such as an auditorium or concert hall. A sound source 12 is disposed in the space and adapted to move along the line 13. A listener 14 is shown in the center of the space 11. A plurality of

speakers

1, 2, 3 and 4 are shown in relation to the listener as the speakers would be located in a reproduction system.

It is recognized that the space -1 1 may have different effects upon the moving sound. For example, if the space 1 l is a marble room, the level of reverberations from the sound source will be high, whereas if the walls of the space 11 are acoustically damped, reverberations will be low. Furthermore, if the space is large, the reverberations will be low, and if the space represents the outdoors, there will be fewer reverberations.

The listener 14 listening to the sound from the source will first obtain distance information D. Assuming that the source is at the point a, the direct and reverberant sound will appear to come primarily from locations directly ahead with each of the listeners ears receiving the same amplitude of sound. In reproduction, the sound should emanate primarily from

speakers

1 and 2 if the sound is to simulate to the condition just described. However, as the sound source moves to the point b along the line 13, the sound source is substantially primarily from the location b and at a different distance. In reproduction, the direct sound should come primarily from speaker 2. Furthermore, if the source travels between a and b along the line 13, there will be a radial component of velocity which will introduce a doppler frequency shift. As the sound reaches the point c, it will appear to come entirely from the right of the listener. The direction changes as the sound source moves along the line 13 from the points a through f.

In accordance with the present invention, it is desired to either synthesize a sound or process a sound so that it can be reproduced on two or more loudspeakers, loudspeakers l, 2, 3 and 4, and have the same effect on the listener as if the sound were moving in the space 11. The reproduced sound will have amplitude which changes with distance. The power of the sound varies in proportion to the inverse square of the distance. The signals applied to the speakers will, therefore, have a corresponding amplitude change. In addition, however, the sound will reverberate from all walls and the amount of reverberation will depend upon the characteristics of the space 11 and the distance from the source of reverberation. In general, as the distance from the listener increases, the distance from some of the wall surfaces decreases and the reverberant signal increases in that direction. The reverberant signal can be approximated by introducing a factor representative of the characteristics of the enclosure and varying the strength from any direction in proportion to the inverse of the distance. In addition, as the location of the sound source is closer to one of the walls, the amount of reverberant sound from that particular location will increase. For example, the reverberations from the location b will be primarily from that location and, therefore, there will be an additional intensity of reverberant signal at this location which can be introduced by the loudspeaker 2 to give the effect of the sound being located in that general position.

In summary, the movement of the sound can be simulated by a frequency shift corresponding to movement toward or away from the listener; the location of the sound by the ratio of amplitude of sounds emanating from the various loudspeakers; the distance by the ratio of amplitudes between the direct and reverberant signals; and the overall acoustics of the enclosure by the reverberations.

A signal may be processed-to provide a four channel output for application to the speakers to simulate location, movement and environment by a system including frequency and amplitude modulators. A suitable processing system is shown in F 10.2. The signal to be processed is applied along the line 21 to a frequency modulator 22. The frequency of the signal is modulated by applying a voltage which corresponds to the radial velocity dD/dt. The output from the frequency modulator 22, line 23, therefore, includes the doppler frequency shift. Suitable means for generating voltages representative of distance D and angular position will be described with reference to FIGS. 4 and 5.

The signal is applied to an amplitude modulator 24 which modulates the amplitude of the signal in proportion to the factor l/D The output of the amplitude modulator 24, line 26, therefore, has an amplitude which corresponds to the distance of the direct signal from the listener. This signal is applied to four amplitude modulators 31 34. These four modulators are connected to the four

loudspeakers

1, 2, 3 and 4, respectively,

through summing

networks

76, 77, 78 and 79. For example, if the sound is to emanate from the direction of loudspeaker 1, only that channel will have direct signals. Thus, the modulating signal applied to the amplitude modulator 31 via channel 1 will be maximum and the modulating signals applied to

channels

2, 3 and 4 will be zero. Similarly, if the signal is to appear to emanate from the position a, FIG. 1, then the modulating signals applied to amplitude modulators 31 and'32 from channels l and 2 will be equal and less than maximum so that the output on

lines

36 and 37 is equal and the level of sound to the listener from

speakers

1 and 2 will be equal. No signal is applied to the

amplitude modulators

33 and 34 so that there is no output on the

lines

38 and 39. Thus, the signals in the four channels are amplitude modulated corresponding to the angular position of the sound source. The system thus far described provides for velocity and angular location of the direct signal within the listening space 11 together with some distance information.

If the sound were emanating from a place having no reverberations whatsoever, the signals applied to the speakers I, 2,

i 3 and 4 would then correspond to the signals on the lines 36,

37, 38 and 39. Generally, the sound source is within some space or environment 1 1. The system of the present invention includes means for introducing into each of the channels or

speakers

1, 2, 3 and 4 a signal which produces reverberant sound on which the distance information partly depends.

The frequency modulated signal on the line 23 is applied to an attenuator 41 wherein the signal is reduced by a given percent, to provide at the output line 42 a signal which is a percent of the signal on the line 23. If the enclosure is assumed to be highly reverberant, then the percentage is relatively high while the percentage is zero for an infinite space and no output signal would appear on the line 42.

Thus, the output on the line 42 is a fraction, SIG, of the frequency modulated signal on line 23. The signal is amplitude modulated in an amplitude modulator 43 by a signal which corresponds to about 1/ The actual relation for reverberation is arbitrary. It need be a scaling function that attenuates the portion of the signal to be reverberated at a rate that is not as extreme with distance as is the direct signal. As previously described, the amplitude of the reverberant signal falls off less with distance than the amplitude of the direct signal.

The output from the amplitude modulator appearing on line 44 is SIG/ This signal is multiplied or modulated at 27 by the distance function l/D which determines the amount of global" reverberation. The output on line 28 goes equally to the four

reverberators summing networks

46, 47, 48 and 49. The global reverberation decreases with increased distance of the direct signal. The signal on line 44 is modulated or multiplied at 29 by 1 1/D whereby this portion of the total reverberant signal increases with distance of the source away from the listener. This reverberant signal is distributed in location according to the same functions which control the direct signal by applying it to modulators 61, 62, 63 and 64 controlled by the signal which controls

modulators

31, 32, 33 and 34. The output of these modulators is then added to the global reverberation at summing

networks

46, 47, 48 and 49, and applied to reverberators 71, 72, 73 and 74 which introduce reverberation. The

reverberators

71, 72, 73 and 74 may be of the type described by Schroeder in the Journal of the Audio Engineering Society, July 1962, Volume 10, No. 3, pp.2 19-223. The effect of the system just described is to cause an increase in the localization of the reverberant signal as the direct signal moves away from the listener.

The output of the

reverberators

71, 72, 73 and 74 is applied to

adders

76, 77, 78 and 79 where it is added to the direct signals from

modulators

31, 32, 33 and 34.

Making reference to the first example, where the signal is located in a direction between

speakers

1 and 2, the signal is equal for the two speakers. The signal that is applied to the

amplitude modulators

31 and 32 is also supplied to the

multipliers

61 and 62 and serves to provide equal signals to the

adders

46 and 47 while zero signals appear at the output of the

modulators

63 and 64 for addition at the

adders

48 and 49. The signal output from

adders

46 and 47 is reverberated and added to the output from the

amplitude modulators

31 and 32 appearing on

lines

36 and 37. Thus, the signal applied to

speakers

1 and 2 includes a component which corresponds to the overall reverberation of the space or global reverberation, line 28, together with a component which corresponds to the direct or local reverberations and the direct signal information. The signals applied to

speakers

3 and 4 have only global reverberation, line 28.

Thus, the outputs of lines], 2, 3 and 4 will contain various signal components including the doppler shift information, distance information, directional reverberant and global reverberant information to thereby simulate dynamic sound in space.

It should be noted that the reason for the independent reverberators is to achieve different reverberations from the four directions. Each reverberator has its delays and gains in an incommensurate relationship to achieve the different (though slight) reverberations. This method is effective in creating a diffused sound space which minimizes the point source location of the loudspeakers.

In FIGS. 3, 4 and 5, there is shown a circuit and apparatus for operating on the signals in accordance with the invention. Referring first to FIGS. 4 and 5, the apparatus includes an arm 81 which carries a movable slider 82. One end of the arm 81 is secured to a shaft 83 to pivot about the shaft and rotate the shaft. The arm, shaft and slider are shown in FIG. 1 where the arm and slider are moved to trace the path 13 of the move- Referring particularly to the left-handside of the circuit diagram in FIG. 3, the

resistive elements

86, 87, 88, 89 and 90 are shown in association with the

sliders

91, 92, 93, 94 and 95. The resistive element 86 has applied thereto a fixed voltage +V. The contactor 91 picks up a predetermined portion of said voltage which varies in proportion to radial distance D and applies it to a potentiometer resistor 96. The contactor 97 is positioned to scale the doppler for distance. The output on the contactor 97 is applied to a differentiating circuit 98 which provides an output voltage dD/dt which is applied to a voltage controlled oscillator (VCO) 99 which provides an output signal having a variable frequency. This signal is applied to a magnetic tape transport which drives the tape at a variable speed.

As the slider is moved along the line 13, FIG. 1, there will be radial movement corresponding to the distance D and this causes a change in voltage at slider 91 and, in turn, a change in the speed of the capstan drive of the tape transport and, therefore, the tape speed. The reproduced signal from the tape 101 appearing on the line 23a is frequency modulated to the desired doppler shift. This signal is applied to one end of the potentiometer resistor 87 which has its other end grounded. The variable tap 92 picks off a proportion of this signal. The

' resistive element 87 is tailored so that the voltage on the line 92 varies as the factor 1 /D which voltage then appears on the line 26. Thus, the amplitude of the signal is varied inversely proportional to the square of the distance D. The voltage on line 23a is also applied to a potentiometer which includes resistor 102 and variable tap 103. This potentiometer serves to obtain a fraction of the frequency modulated signal SIG) and apply the same to the line 42. This potentiometer is employed to reduce the signal in accordance with the reverberant character-istics of the enclosure or surrounds. The signal on line 42 is applied to the potentiometer resistor 88 which is proportioned to give an output equal to 1/ times the input which scales the reverberant signal as previously described. The output on tap 93 is applied to the

potentiometer resistors

89 and 90 which are proportioned to give a signal inversely proportional to distance. The potentiometer 90 is reversed to give 1 l/D.

Thus, there is provided the signal for the line 28a which is applied to

adders

46a, 47a, 48a and 49a and the signal which is applied to

modulators

61a, 62a, 63a and 64a.

The amplitude modulators are in the form of potentiometers, FIG. 4, and rotation of shaft 83 serves to rotate the contactors in each of the potentiometers to' pick off the output voltage. The

potentiometers

31a, 32a, 33a and 34a are each staggered 90 with respect to the adjacent potentiometer as shown. The voltage along the line 26a is applied to the center of the potentiometer resistor. Thus, for example, when the pointer 12 is at the point a, voltage is derived at potentiometers 31a and 32a on

lines

36 and 37 in equal amplitude, while zero voltage is derived at

potentiometers

33a and 34a on

lines

38 and 39. When the pointer is at point b, full voltage appears at potentiometer 32a on line 37 and zero voltage at all other potentiometers.

The

potentiometers

61a, 62a, 63a and 64 are likewise arranged in 90 relationship with the voltage on the line applied at the center of the resistors as shown. The percent direct reverberant signal is varied in the same manner as the direct signal as described above.

The output of the potentiometers 31a-34a appearing on the lines 36-39 is applied to

adders

76a, 77a, 78a and 79a. The sum appearing at the

adders

46a, 47a, 48a and 49a is applied to

reverberators

71, 72, 73, 74, and thence to the adders 76a-79bq. The added signals are then either applied to

speakers

1, 2, 3 and 4 or are applied to a four-track recorderreproducer where the signals can be stored and thereafter reproduced and applied to

speakers

1,2, 3 and 4. Thus, there is shown a simple apparatus including potentiometric means for performing substantially all of the functions described with respect to the circuits shown in FIG. 2.

FIG. 6 shows the signal outputs l, 2, 3 and 4 applied to means for recording and reproducing the signals whereby they can be played back through the associated loudspeakers. The recording means may be magnetic tape, records or the like.

The two embodiments just described are two-dimensional; however, it will be apparent that the invention is applicable to three-dimensional sound reproduction and creation. For example, the speakers can be arranged in tetrahedron or eight in square and signal processing provided to take into account the three-dimensional direction of the sound. The percentage values and powers of distance given in the foregoing discussion were empirically determined and can be varied to provide different sound effects.

Furthermore, it should be apparent that the processes shown and described in connection with the embodiment of FIG. 2 and the embodiment of FIGS. 3, 4 and 5 can be carried out by a suitably programmed digital or analog computer which processes the signal information applied thereto in accordance with programmed instructions.

I claim:

1. The method of processing signals representative of sound information to provide signals in two or more channels which can be reproduced by two or more loudspeakers comprising the steps of varying the frequency of the signals to provide an output signal having an apparent doppler shift corresponding to the apparent velocity of the change in distance from the listener, varying the amplitude of the signal in inverse proportion to the apparent distance from the listener, applying the frequency and amplitude modulated signals to two or more channels, and individually amplitude modulating the signals on each channel in proportion to the desired direction of sound from the listener whereby to provide sound having apparent movement and direction.

2. The method as in claim 1 wherein said signal is amplitude modulated in inverse proportion to the square of the distance.

3. The method as in claim 1 additionally including means for obtaining a fraction of the frequency modulated signal to provide a reverberant signal, varying the amplitude of the reverberant signal in inverse proportion to the desired apparent distance from the listener, and adding said signal equally to the two or more channels.

4. The method as in claim 3 including the steps of developing a reverberant signal whose amplitude for each channel varies with direction and distance and adding said signal to the signals in said one or more channels.

5. The method as in claim 1 including the steps of developing a reverberant signal whose amplitude for each channel varies with direction and distance and adding said signal to the signals in said one or more channels.

6. The method of processing signals representing sound information to provide reverberant signals in two or more channels which can be reproduced by two or more loudspeakers which comprises varying the amplitude of said signals in inverse proportion to the desired apparent distance from a listener, obtaining a fraction of the amplitude varied signals and varying the amplitude of said fraction in inverse proportion to the distance from the listener, delaying the amplitudevaried signals and fraction, and adding the delayed signals equally to the undelayed signals in said two or more channels.

7. The method as in claim 6 including the step of developing a reverberant signal whose amplitude for each channel varies in proportion to direction and distance and adding said signal to the signals in said two or more channels.

8. Apparatus for processing signals having a predetermined frequency representative of sound information to provide signals which can be applied to two or more loudspeakers for reproduction including means for receiving said signals and providing an output signal whose frequency is modulated in accordance with apparent rate of change of distance away from a listener, means for receiving said frequency signal and providing a plurality of signals each having amplitude corresponding to the direction of the signals from the listener, and means for forming a recording of said signals for reproduction.

for reproduction including means for receiving said signals and providing an output signal whose frequency is modulated in accordance with apparent rate of change of distance away from a IistenerQ and means for receiving said frequency signal and providing a plurality of signals each having amplitude corresponding to the direction of the signals from the listener.

12. The method as in claim 5 wherein said signals are amplitude modulated in proportion to direction and the amplitude modulated signals are added to said channels.

Claims

6. The method of processing signals representing sound information to provide reverberant signals in two or more channels which can be reproduced by two or more loudspeakers which comprises varying the amplitude of said signals in inverse proportion to the desired apparent distance from a listener, obtaining a fraction of the amplitude varied signals and varying the amplitude of said fraction in inverse proportion to the distance from the listener, delaying the amplitude varied signals and fraction, and adding the delayed signals equally to the undelayed signals in said two or more channels.

9. Apparatus as in claim 8 including means for obtaining a fraction of said signal and adding the same equally to the plurality of signals.

10. Apparatus as in claim 9 including additionally means for receiving said fraction of said signal and providing a plurality of signals having amplitude corresponding to direction of direct signal and adding said signal to the plurality of signals.

11. Apparatus for processing signals having a predetermined frequency representative of sound information to provide signals which can be applied to two or more loudspeakers for reproduction including means for receiving said signals and providing an output signal whose frequency is modulated in accordance with apparent rate of change of distance away from a listener, and means for receiving said frequency signal and providing a plurality of signals each having amplitude corresponding to the direction of the signals from the listener.