Targeted audio in multi-dimensional space

• US 8,190,438 B1
• Filed: 10/14/2009
• Issued: 05/29/2012
• Est. Priority Date: 10/14/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nelissen, Marco
• Primary Examiner(s): Azad, Abul
• Application Number:
  US12/579,037
• Time in Patent Office: 958 Days
• Field of Search: 704/270, 704/274, 381300-310, 381/85, 381/89, 381/332
• US Class Current: 704/274
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

In general, the subject matter described in this specification can be embodied in methods, system, and program products for providing sound from multiple audio sources. A first audio stream is played from a first source through a multi-speaker audio system. A signal indicating a need to play a second audio stream is received from a second source. Adjustments to be made to the first audio stream to move an acoustic image of the first audio stream in multi-dimensional space from a first location to a second location are determined. Adjustments are applied to a signal of the first audio stream in order to move the acoustic image of the first audio stream from the first location to the second location. The second audio stream is played so that an acoustic image of the second audio stream is located in a third location. The first audio stream is simultaneously played.

Claims

1. A computer-implemented method for providing sound from multiple audio sources, the method comprising:

playing a first audio stream from a first source through a multi-speaker audio system;

receiving a signal indicating a need to play a second audio stream from a second source through the multi-speaker audio system;

determining adjustments to be made to the first audio stream to move an acoustic image of the first audio stream in multi-dimensional space from a first location to a second location, wherein an acoustic image is an apparent location of a source of an audio stream; and

applying adjustments to a signal of the first audio stream in order to move the acoustic image of the first audio stream from the first location to the second location; and

playing the second audio stream so that an acoustic image of the second audio stream is located in a third location that is different from the second location, and simultaneously playing the first audio stream in the second location.

2. The computer-implemented method of claim 1, wherein the first audio stream comprises a stream of playing music, and the second audio stream comprises a portion of a spoken conversation.

3. The computer-implemented method of claim 2, wherein the spoken conversation is carried on between a synthesized voice warning unit and a user listening to the first and second audio streams.

4. The computer-implemented method of claim 2, wherein the spoken conversation is a two-way spoken conversation between an incoming telephone caller and a user listening to the first and second audio streams.

5. The computer-implemented method of claim 1, wherein applying the adjustments comprises adjusting phase and volume of the signal of the first audio stream independent of other audio streams.

6. The computer-implemented method of claim 5, wherein adjusting the phase and volume of the signal comprises performing a lookup in a spatial targeting table that identifies values for positioning a sound at a determined location in three-dimensional space.

7. The computer-implemented method of claim 1,

wherein moving the acoustic image of the first audio stream from the first location to the second location comprises moving the acoustic image of the first audio stream to a location substantially behind a user who is listening to the first audio stream, and

wherein playing the second audio stream so that an acoustic image of the second audio stream is located in a third location that is different from the second location comprises placing the acoustic image of the second audio stream at a location substantially in front of the user.

8. The computer-implemented method of claim 1, further comprising:

receiving user input indicating a need to rotate the locations of the acoustic images; and

applying adjustments to signals of the first audio stream and the second audio stream in order to rotate the locations acoustic images substantially about a position of a user listening to the first and second audio streams.

9. The computer-implemented method of claim 1, further comprising:

receiving a second signal indicating a need to play a third audio stream from a third source;

in response to receiving the second signal;

applying adjustments to the signal of the first audio stream in order to move the acoustic image of the first audio stream to a new location; and

applying adjustments to a signal of the second audio stream in order to move the acoustic image of the second audio stream to a new location; and

playing the third audio stream so that an acoustic image of the third audio stream is located in a location that is different from the new locations of the acoustic images for the first and second audio streams.

10. The computer-implemented method of claim 1, further comprising presenting data for displaying icons corresponding to the first and second audio streams on a visual display of a computing device, and where the icons are located at locations on the display corresponding to locations of the acoustic images in multi-dimensional space.

11. The computer-implemented method of claim 1, wherein the acoustic images for the first audio stream and the second audio stream are controlled virtual sound sources that appear to a listener to be separate from the locations of the speakers in the multi-speaker audio system.

12. A system for providing positional sound from multiple audio sources, the system comprising:

a multi-channel electronic amplifier arranged to produce signals for producing audible sound on multiple independent channels;

a plurality of individually addressable speakers, with each speaker connected to one of the channels of the electronic amplifier; and

an acoustic image generation unit programmed to modify a signal for a first audio stream upon identifying an arrival of a second audio stream, so as to provide one or more signals to the amplifier so that an acoustic image for the first audio stream is positioned at a location in multi-dimensional space that is separated from a location in multi-dimensional space for an acoustic image of the second audio stream when the first and second audio streams are reproduced simultaneously by the speakers, wherein an acoustic image is an apparent location of a source of an audio stream.

13. The system of claim 12, further comprising:

a microphone amplifier arranged to receive signals from a microphone and provide the signals for transmission as an outgoing channel of a two-way spoken communication session, wherein at least one of the audio streams includes received audio from an incoming channel of the two-way communication; and

a sound cancellation unit to cancel noise received at the microphone amplifier that is from audio streams that do not include the incoming channel of the two-way communication.

14. The system of claim 12, further comprising:

an electronic device arranged to wirelessly transmit an audio stream; and

a wireless interface adapted to wirelessly receive the second audio stream and identify arrival of the second audio stream to the acoustic image generation unit.

15. The system of claim 12, further comprising:

a display device arranged to display icons at locations in the display device that represent a spatial arrangement of the acoustic images in multi-dimensional space.

16. The system of claim 15, further comprising:

an input device arranged to (a) receive user input selecting an audio stream from a collection of audio streams, and (b) assign the selected audio stream to a location in the displayed representation of the spatial arrangement of the acoustic images;

wherein the acoustic image generation unit modifies a signal for the selected audio stream, so as to provide one or more signals to the amplifier so that an acoustic image for the selected audio stream is positioned at a location in multi-dimensional space that corresponds to the assigned location in the displayed representation of acoustic images.

17. The system of claim 12, wherein the acoustic image generation unit is further programmed to modify the signal for the first audio stream upon identifying the arrival of the second audio stream, so as to provide one or more signals to the amplifier in order to move the acoustic image of the first audio stream from a first location to the positioned location.

18. The system of claim 12, wherein providing the one or more signals to the amplifier includes adjusting phase and volume of the signal for the first audio stream independent of the second audio stream.

19. The system of claim 12, wherein the acoustic image generation unit is further programmed to modify the signal for the first audio stream and a signal for the second audio stream upon identifying an arrival of a third audio stream, so as to provide one or more signals to the amplifier so that the acoustic images for the first audio stream and second audio stream are positioned at locations in multi-dimensional space that are separated from a location in multi-dimensional space for an acoustic image of the third audio stream when the first, second, and third audio streams are reproduced simultaneously by the speakers.

20. A system for providing positional sound from multiple audio sources, the system comprising:

a multi-channel electronic amplifier arranged to produce signals for producing audible sound on multiple independent channels;

a plurality of individually addressable speakers, with each speaker connected to one of the channels of the electronic amplifier; and

means for modifying a signal for a first audio stream upon identifying an arrival of a second audio stream, so as to provide one or more signals to the amplifier so that an acoustic image for the first audio stream is positioned at a location in multi-dimensional space that is separated from a location in multi-dimensional space for an acoustic image of the second audio stream when the first and second audio streams are reproduced simultaneously by the speakers, wherein an acoustic image is an apparent location of a source of an audio stream.