METHOD FOR CONTROLLING A SPEAKER ARRAY TO PROVIDE SPATIALIZED, LOCALIZED, AND BINAURAL VIRTUAL SURROUND SOUND

• US 20140064526A1
• Filed: 11/15/2011
• Published: 03/06/2014
• Est. Priority Date: 11/15/2010
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California), University of Southampton
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Otto, Peter, Kamdar, Suketu, Toshiro, Yamada, Fazi, Fillippo M.
• Granted Patent: US 9,578,440 B2
• Time in Patent Office: Days
• US Class Current: 381/300
• CPC Class Codes: H04R 1/403 loud-speakers
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 2420/13 Application of wave-field synthesis in stereophonic audio systems
• CPC Class Codes: H04S 5/00 Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

A system and method for producing a binaural and localized audio signal to a user is provided. A signal processing method is provided for delivering spatialized sound in various ways using highly optimized inverse filters to deliver narrow localized beams of sound from the included speaker array. The inventive method can be used to provide private listening areas in a public space and provide spatialization of source material for a single user to create a virtual surround sound effect. In a binaural mode, a speaker array provides two targeted beams aimed towards the primary user'"'"'s ears—one discrete beam for the left ear and one discrete beam for the right ear. In a privacy mode, a privacy zone could be created in which a primary audio beam would deliver a signal of interest to the user while secondary beams would be aimed at different angles to provide a masking noise.

Claims

1. A method for producing binaural sound from a speaker array, comprising:

receiving a plurality of audio signals from a plurality of sources;

filtering each audio signal through a left Head-Related Transfer Function (HRTF) and a right HRTF, wherein the left HRTF is calculated based on an angle at which the plurality of audio signals will be transmitted to a left ear of a user; and

wherein the right HRTF is calculated based on an angle at which the plurality of audio signals will be transmitted to a right ear of a user;

merging the audio signals filtered through the left HRTF into a left total binaural signal, and merging the audio signals filtered through the right HRTF into a right total binaural signal;

filtering the left total binaural signal through a set of left spatialization filters, wherein a separate left spatialization filter is provided for each speaker in the speaker array;

filtering the right total binaural signal through a set of right spatialization filters, wherein a separate right spatialization filter is provided for each speaker in the speaker array;

summing the filtered left total binaural signal and filtered right total binaural signal for each respective speaker into a speaker signal;

feeding the speaker signal to the respective speaker in the speaker array; and

transmitting the speaker signal through the respective speaker to the user.

2. The method of claim 1, wherein the left HRTF and right HRTF are computed in real-time using a binaural processor.

3. The method of claim 1, wherein the spatialization filters are finite impulse response (FIR) filters.

4. The method of claim 3, wherein two control points are used to compute the FIR filters, and wherein the distance between the control points is approximately 0.1 meters (m) to approximately 0.3 m.

5. The method of claim 1, further comprising adapting the spatialization filters in real-time based on a change in the location of the user.

6. The method of claim 1, further comprising pre-filtering the plurality of audio signals with a Psychoacoustic Bandwidth Extension Processor (PBEP).

7. The method of claim 6, further comprising matching the loudness of the pre-filtered audio signals using a Dynamic Range Compressor and Expander (DRCE).

8. A method for producing a localized sound from a speaker array, comprising:

receiving at least one audio signal;

filtering each audio signal through a set of spatialization filters, wherein a separate spatialization filter is provided for each speaker in the speaker array;

summing the filtered audio signals for each respective speaker into a speaker signal;

transmitting each speaker signal to the respective speaker in the speaker array; and

delivering each speaker signal to one or more regions of space occupied by one or more users.

9. The method of claim 8, further comprising delivering at least one secondary audio signal to an area around the one or more users which masks the speaker signal in the area not occupied by the one or more users.

10. The method of claim 9, wherein the masking signal is a musical signal.

11. The method of claim 9, further comprising dynamically adjusting the amplitude and time of the masking signals.

12. The method of claim 8, wherein the spatialization filters are finite impulse response (FIR) filters.

13. The method of claim 8, further comprising adapting the spatialization filters in real-time based on a change in the location of the one or more users.

14. The method of claim 8, further comprising pre-filtering the plurality of audio signals with a Psychoacoustic Bandwidth Extension Processor (PBEP).

15. The method of claim 14, further comprising matching the loudness of the pre-filtered audio signals using a Dynamic Range Compressor and Expander (DRCE).

16. A speaker array system for producing localized sound, comprising:

an input which receives a plurality of audio signals from at least one source;

a processor and a memory which determines whether the plurality of audio signals should be processed by a binaural processing system or a beamforming processing system; and

a speaker array comprising a plurality of loudspeakers;

wherein the binaural processing system comprises;

at least one filter which filters each audio signal through a left Head-Related Transfer Function (HRTF) and a right HRTF, wherein the left HRTF is calculated based on an angle at which the plurality of audio signals will be transmitted to a left ear of a user; and

wherein the right HRTF is calculated based on an angle at which the plurality of audio signals will be transmitted to a right ear of a user;

a left combiner which combines all of the audio signals from the left HRTF into a left total binaural signal;

a right combiner which combines all of the audio signals from the right HRTF into a right total binaural signal;

at least one left spatialization filter which filters the left total binaural signal, wherein a separate left spatialization filter is provided for each loudspeaker in a speaker array;

at least one right spatialization filter which filters the right total binaural signal, wherein a separate right spatialization filter is provided for each loudspeaker in the speaker array; and

a binaural combiner which sums the filtered left total binaural signal and filtered right total binaural signal into a binaural speaker signal for each respective loudspeaker and transmits each binaural speaker signal to the respective loudspeaker;

wherein the beamforming processing system comprises;

a plurality of beamforming spatialization filters which filters each audio signal, wherein a separate spatialization filter is provided for each loudspeaker in the speaker array; and

a beamforming combiner which sums the filtered audio signals for each respective loudspeaker into a beamforming speaker signal and transmits each beamforming speaker signal to the respective speaker in the speaker array;

wherein the speaker array delivers the respective binaural speaker signal or the beamforming speaker signal through the plurality of loudspeakers to one or more users.

17. The speaker array system of claim 16, wherein the plurality of audio signals can be processed by the beamforming processing system and the binaural processing system before being delivered to the one or more users through the plurality of loudspeakers.

18. The speaker array system of claim 16, further comprising a user tracking unit which adjusts the binaural processing system and beamforming processing system based on a change in a location of the one or more users.

19. The speaker array system of claim 16, wherein the binaural processing system further comprises a binaural processor which computes the left HRTF and right HRTF in real-time.