Closed loop quantization of higher order ambisonic coefficients

• US 9,959,876 B2
• Filed: 05/14/2015
• Issued: 05/01/2018
• Est. Priority Date: 05/16/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kim, Moo Young, Peters, Nils Gnther, Sen, Dipanjan
• Primary Examiner(s): Huber, Paul
• Application Number:
  US14/712,638
• Publication Number:
  US 20150332681A1
• Time in Patent Office: 1,083 Days
• Field of Search: None
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/038 Vector quantisation, e.g. TwinVQ audio
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2420/11 Application of ambisonics in stereophonic audio systems
• CPC Class Codes: H04S 3/02 of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround

Abstract

In general, techniques are described for closed loop quantization of HOA coefficients that provide a three-dimensional representation of the sound field. An audio encoding device may perform closed loop quantization of an audio object based at least in part on a result of performing quantization of directional information associated with the audio object. An audio decoding device may obtain an audio object that has been closed loop quantized based at least in part on a result of performing quantization of directional information associated with the audio object, and may dequantize the audio object.

Claims

1. A method for quantizing a foreground audio signal, comprising:

receiving, by at least one processor, audio data indicative of Higher Order Ambisonics (HOA) coefficients captured by a microphone;

decomposing, by the at least one processor, an audio object and directional information associated with the audio object from the HOA coefficients; and

performing, by the at least one processor, closed loop quantization of the audio object based at least in part on a result of performing quantization of the directional information associated with the audio object.

2. The method of claim 1, wherein performing the closed loop quantization of the audio object further comprises:

performing quantization of the directional information associated with the audio object; and

performing quantization of a the audio object based at least in part on a result of performing quantization of the directional information associated with the audio object.

3. The method of claim 2, wherein performing quantization of the audio object further comprises:

performing quantization of the audio object based at least in part on a quantization error resulting from performing quantization of the directional information associated with the audio object.

4. The method of claim 3, wherein performing quantization of the audio object based at least in part on the quantization error resulting from performing quantization of the directional information associated with the audio object further comprises:

compensating for the quantization error resulting from performing quantization of the directional information associated with the audio object.

5. The method of claim 4, wherein compensating for the quantization error resulting from performing quantization of the directional information associated with the audio object further comprises:

determining a quantization-compensated audio object based at least in part on a pseudoinverse of a result of performing quantization of the directional information associated with the audio object; and

performing quantization of the quantization-compensated audio object.

6. The method of claim 5, wherein determining the quantization-compensated audio object based at least in part on the pseudoinverse of the result of performing quantization of the directional information associated with the audio object further comprises:

determining the quantization-compensated audio object as a product of the HOA coefficients and the pseudoinverse of the result of performing quantization of the directional information associated with the audio object.

7. The method of claim 1, wherein:

the audio object comprises a product of a U matrix representative of left-singular vectors of a plurality of spherical harmonic coefficients and a S matrix representative of singular values of the plurality of spherical harmonic coefficients; and

the directional information associated with the audio object comprises a V matrix representative of right-singular vectors of the plurality of spherical harmonic coefficients.

8. The method of claim 1, further comprising:

capturing, by the microphone, the audio data indicative of the HOA coefficients.

9. A device for quantizing a foreground audio signal, comprising:

at least one processor configured to;

receive audio data indicative of Higher Order Ambisonics (HOA) coefficients captured by a microphone;

decompose an audio object and directional information associated with the audio object from the HOA coefficients; and

perform closed loop quantization of the audio object based at least in part on a result of performing quantization of the directional information associated with the audio object; and

a memory configured to store the audio object and the directional information associated with the audio object.

10. The device of claim 9, wherein the at least one processor is further configured to:

perform quantization of the directional information associated with the audio object; and

perform quantization of the audio object based at least in part on a result of performing quantization of the directional information associated with the audio object.

11. The device of claim 10, wherein performing quantization of the audio object further comprises:

perform quantization of the audio object based at least in part on a quantization error resulting from performing quantization of the directional information associated with the audio object.

12. The device of claim 11, wherein the at least one processor is further configured to:

compensate for the quantization error resulting from performing quantization of the directional information associated with the audio object.

13. The device of claim 12, wherein the at least one processor is further configured to:

determine a quantization-compensated audio object based at least in part on a pseudoinverse of a result of performing quantization of the directional information associated with the audio object; and

perform quantization of the quantization-compensated audio object.

14. The device of claim 13, wherein the at least one processor is further configured to:

determine the audio object as a product of the HOA coefficients and the pseudoinverse of the result of performing quantization of the directional information associated with the audio object.

15. The device of claim 9, further comprising:

a microphone configured to capture the audio data indicative of HOA coefficients.

16. A method for dequantizing an audio object, comprising:

obtaining, by at least one processor, an audio object that has been closed loop quantized based at least in part on a result of performing quantization of directional information associated with the audio object; and

dequantizing, by the at least one processor, the audio object;

rendering, by the at least one processor using the dequantized audio object, loudspeaker feeds; and

outputting, by the at least one processor, the loudspeaker feeds to drive one or more speakers to playback the loudspeaker feeds.

17. The method of claim 16, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing the audio object based at least in part on a result of quantizing the directional information associated with the audio object.

18. The method of claim 16, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing

the audio object based at least in part on a quantization error resulting from quantizing the directional information associated with the audio object.

19. The method of claim 16, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing the audio object based at least in part on a quantization error resulting from quantizing of the directional information associated with the audio object, including compensating for the quantization error resulting from performing quantization of the directional information associated with the audio object.

20. The method of claim 16 wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object, determining a quantization-compensated audio object based at least in part on a pseudoinverse of a result of quantizing the directional information associated with the audio object, and quantizing the quantization-compensated audio object.

21. The method of claim 20, wherein the audio object is close looped quantized by determining the quantization-compensated audio object as a product of Higher Order Ambisonics (HOA) coefficients and the pseudoinverse of the result of performing quantization of the directional information associated with the audio object.

22. The method of claim 16, wherein:

the audio object and the directional information are decomposed from higher order ambisonic coefficients;

the audio object comprises a product of a U matrix representative of left-singular vectors of a plurality of spherical harmonic coefficients and a S matrix representative of singular values of the plurality of spherical harmonic coefficients; and

the directional information associated with the audio object comprises a V matrix representative of right-singular vectors of the plurality of spherical harmonic coefficients.

23. The method of claim 16, further comprising:

receiving a bitstream; and

decoding the bitstream to obtain the closed loop quantized audio object and the quantized directional information.

24. The method of claim 16, further comprising:

playing back, by the one or more speakers, the loudspeaker feeds rendered from the dequantized audio object.

25. A device for dequantizing a foreground audio signal, comprising:

a memory configured to store an audio object;

at least one processor configured to;

obtain the audio object that has been closed loop quantized based at least in part on a result of performing quantization of directional information associated with the audio object;

dequantize the audio object;

render, using the dequantized audio object, loudspeaker feeds; and

output the loudspeaker feeds to drive one or more speakers to playback the loudspeaker feeds; and

the one or more speakers configured to playback the loudspeaker feeds rendered from the dequantized audio object.

26. The device of claim 25, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing the audio object based at least in part on a result of quantizing the directional information associated with the audio object.

27. The device of claim 25, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing the audio object based at least in part on a quantization error resulting from quantizing the directional information associated with the audio object.

28. The device of claim 25, wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object and quantizing the audio object based at least in part on a quantization error resulting from quantizing of the directional information associated with the audio object, including compensating for the quantization error resulting from performing quantization of the directional information associated with the audio object.

29. The device of claim 25 wherein the audio object is close looped quantized by quantizing the directional information associated with the audio object, determining a quantization-compensated audio object based at least in part on a pseudoinverse of a result of quantizing the directional information associated with the audio object, and quantizing the quantization-compensated audio object.

30. The device of claim 29, wherein the audio object is close looped quantized by determining the quantization-compensated audio object as a product of Higher Order Ambisonics (HOA) coefficients and the pseudoinverse of the result of performing quantization of the directional information associated with the audio object.

31. The device of claim 25, wherein the at least one processor is further configured to:

receiving a bitstream; and

decoding the bitstream to obtain the closed loop quantized audio object and the quantized directional information.

32. The device of claim 25, further comprising:

the one or more speakers configured to playback the loudspeaker feeds rendered from the dequantized audio object.