Immersive audio rendering system

• US 10,034,113 B2
• Filed: 07/16/2015
• Issued: 07/24/2018
• Est. Priority Date: 01/04/2011
• Status: Active Grant

• Current Assignee: DTS, Inc. (Adeia Inc.)
• Original Assignee: DTS, Inc. (Adeia Inc.)
• Inventors: Kraemer, Alan D., Tracey, James, Katsianos, Themis
• Primary Examiner(s): Lee, Ping
• Application Number:
  US14/801,652
• Publication Number:
  US 20160044431A1
• Time in Patent Office: 1,104 Days
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/07 Generation or adaptation of the Low Frequency Effect [LFE] channel, e.g. distribution or signal processing
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

A depth processing system can employ stereo speakers to achieve immersive effects. The depth processing system can advantageously manipulate phase and/or amplitude information to render audio along a listener'"'"'s median plane, thereby rendering audio along varying depths. In one embodiment, the depth processing system analyzes left and right stereo input signals to infer depth, which may change over time. The depth processing system can then vary the phase and/or amplitude decorrelation between the audio signals over time to enhance the sense of depth already present in the audio signals, thereby creating an immersive depth effect.

Claims

1. A method of processing audio signals, the method comprising:

receiving left and right audio signals, the left and right audio signals each comprising information about a spatial position of a sound source relative to a listener;

calculating depth steering information using the left and right audio signals, the depth steering information based at least partly on the spatial position of the sound source and corresponding to an amount of decorrelation to be performed on the left and right audio signals;

decorrelating the left and right audio signals by an amount that depends at least partly on the depth steering information to produce decorrelated left and right audio signals;

calculating difference information in the decorrelated left and right audio signals;

applying at least one perspective filter to the difference information to produce first left and right output signals;

applying crosstalk cancellation to the first left and right output signals to reduce backwave crosstalk and obtain second left and right output signals; and

providing the second left and right output signals for playback,

wherein the method is performed by one or more hardware processors.

2. The method of claim 1, further comprising performing at least one of:

detecting an envelope of the difference information or smoothing the difference information.

3. The method of claim 2, further comprising modulating the application of the at least one perspective filter based at least in part on one or both of the envelope of the difference information and the smoothed difference information.

4. The method of claim 3, further comprising normalizing the difference information based at least in part on signal levels of the decorrelated left and right audio signals.

5. The method of claim 4, wherein the modulating comprises modulating the application of the at least one perspective filter based at least in part on the normalized difference information.

6. The method of claim 4, wherein the normalizing comprises computing a geometric mean of the left and right audio signals and dividing the difference information with the computed geometric mean.

7. The method of claim 1, wherein decorrelating the left and right audio signals comprises dynamically adjusting one or both of a delay and a gain applied to the left and right audio signals.

8. The method of claim 1, wherein calculating the depth steering information comprises decorrelating the left and right audio signals.

9. An audio signal processing system comprising:

a signal analyzer configured to;

receive left and right audio signals, the left and right audio signals each comprising information about a spatial position of a sound source relative to a listener,

calculate depth steering information using the left and right audio signals, the depth steering information based at least partly on the spatial position of the sound source and corresponding to an amount of decorrelation to be performed on the left and right audio signals,

decorrelate the left and right audio signals by an amount that depends at least partly on the depth steering information to produce decorrelated left and right audio signals, and

calculate a difference signal in the decorrelated left and right audio signals; and

a surround processor configured to;

apply at least one perspective filter to the difference signal to produce first left and right output signals, wherein the surround processor comprises one or more processors,

apply crosstalk cancellation to the first left and right output signals to obtain second left and right output signals, and

provide the second left and right output signals for playback;

wherein the signal analyzer and the surround processor are implemented at least partially in electronic hardware.

10. The system of claim 9, wherein the signal analyzer is further configured perform at least one of:

detect an envelope of the difference signal or smooth the difference signal.

11. The system of claim 10, wherein the surround processor is further configured to modulate the application of the at least one perspective filter based at least in part on one or both of the envelope of the difference signal and the smoothed difference signal.

12. The system of claim 9, wherein the signal analyzer is further configured to normalize the difference signal based at least in part on signal levels of the left and right audio signals.

13. The system of claim 12, wherein the surround processor is further configured to modulate the application of the at least one perspective filter based at least in part on the normalized difference signal.

14. The system of claim 12, wherein the signal analyzer is further configured to normalize the difference signal by at least computing a geometric mean of the left and right audio signals and dividing the difference signal with the computed geometric mean.

15. The system of claim 9, wherein the signal analyzer is configured to decorrelate the left and right audio signals by dynamically adjusting one or both of a delay and a gain applied to the left and right audio signals.

16. The system of claim 9, wherein the signal analyzer is configured to calculate the depth steering information based on decorrelating the left and right audio signals.

17. Non-transitory physical computer storage comprising instructions stored therein configured to implement, in one or more hardware processors, operations for processing an audio signal, the operations comprising:

receiving left and right audio signals, the left and right audio signals each comprising information about a spatial position of a sound source relative to a listener;

calculating first difference information using the left and right audio signals, the first difference information based at least partly on the spatial position of the sound source and corresponding to an amount of decorrelation to be performed on the left and right audio signals;

decorrelating the left and right audio signals by an amount that depends at least partly on the first difference information to produce decorrelated left and right audio signals;

calculating second difference information in the decorrelated left and right audio signals;

applying at least one perspective filter to the second difference information to produce first left and right output signals;

applying crosstalk cancellation to the first left and right output signals to obtain second left and right output signals; and

providing the second left and right output signals for playback.

18. The storage of claim 17, wherein the operations further comprise normalizing the second difference information and modulating the application of the at least one perspective filter based at least in part on the normalized second difference information.