HIERARCHICAL DECCORELATION OF MULTICHANNEL AUDIO

• US 20140112481A1
• Filed: 10/18/2012
• Published: 04/24/2014
• Est. Priority Date: 10/18/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: KLEIJN, Willem Bastiaan, LI, Minyue, SKOGLUND, Jan
• Granted Patent: US 9,396,732 B2
• Time in Patent Office: Days
• US Class Current: 381/23
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/035 Scalar quantisation
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding

Abstract

Provided are methods, systems, and apparatus for hierarchical decorrelation of multichannel audio. A hierarchical decorrelation algorithm is designed to adapt to possibly changing characteristics of an input signal, and also preserves the energy of the original signal. The algorithm is invertible in that the original signal can be retrieved if needed. Furthermore, the proposed algorithm decomposes the decorrelation process into multiple low-complexity steps. The contribution of these steps is generally in a decreasing order, and thus the complexity of the algorithm can be scaled.

Claims

1. A method for decorrelating channels of an audio signal, the method comprising:

selecting a plurality of the channels of the audio signal based on at least one criterion;

performing a unitary transform on the selected plurality of channels, yielding a plurality of decorrelated channels;

combining the plurality of decorrelated channels with remaining channels of the audio signal other than the selected plurality; and

determining whether to further decorrelate the combined channels based on computational complexity.

2. The method of claim 1, further comprising:

responsive to determining not to further decorrelate the combined channels, passing the combined channels as output.

3. The method of claim 1, wherein the at least one criterion is one of:

number of bits saved for audio compression, degree of energy concentration, and remaining correlation.

4. The method of claim 1, wherein selecting the plurality of channels includes identifying one or more of the channels of the audio signal having a higher energy concentration than the remaining channels.

5. The method of claim 1, wherein selecting the plurality of channels includes identifying one or more of the channels of the audio signal that saves the most bits for audio compression.

6. The method of claim 1, wherein selecting the plurality of channels includes identifying one or more of the channels of the audio signal that minimizes remaining correlation.

7. The method of claim 1, wherein the unitary transform is a Karhunen-Loé

ve transform (KLT).

8. The method of claim 1, wherein the plurality of channels is two.

9. A method for encoding an audio signal comprised of a plurality of channels, the method comprising:

segmenting the audio signal into frames;

transforming each of the frames into a frequency domain representation;

estimating, for each frame, a signal model;

quantizing the signal model for each frame; and

performing hierarchical decorrelation using the frequency domain representation and the quantized signal model for each of the frames; and

quantizing an outcome of the hierarchical decorrelation using a quantizer.

10. The method of claim 9, wherein the estimated signal model for each frame yields a spectral matrix.

11. The method of claim 9, wherein performing the hierarchical decorrelation includes:

selecting a set of channels, of the plurality of channels of the audio signal, based on number of bits saved for audio compression;

performing a unitary transform on the selected set of channels, yielding a set of decorrelated channels; and

combining the set of decorrelated channels with remaining channels of the plurality other than the selected set.

12. The method of claim 11, further comprising:

determining whether to further decorrelate the combined channels based on computational complexity; and

responsive to determining not to further decorrelate the combined channels, passing the combined channels as output.

13. The method of claim 11, wherein the unitary transform is calculated from the quantized signal model.

14. The method of claim 11, wherein the unitary transform is a Karhunen-Loé

ve transform (KLT).

15. The method of claim 11, wherein the selected set of channels is two.

16. A method for suppressing noise in an audio signal comprised of a plurality of channels, the method comprising:

segmenting the audio signal into frames;

transforming each of the frames into a frequency domain representation;

estimating, for each frame, a signal model;

quantizing the signal model for each frame;

performing hierarchical decorrelation using the frequency domain representation and the quantized signal model for each of the frames to produce a plurality of decorrelated channels;

setting one or more of the plurality of decorrelated channels with low energy to zero;

performing inverse hierarchical decorrelation on the plurality of decorrelated channels; and

transforming the plurality of decorrelated channels to the time domain to produce a noise-suppressed signal.

17. The method of claim 16, wherein the estimated signal model for each frame yields a spectral matrix.

18. The method of claim 16, wherein performing the hierarchical decorrelation includes:

selecting a set of channels, of the plurality of channels of the audio signal, based on degree of energy concentration; and

performing a unitary transform on the selected set of channels, yielding a set of decorrelated channels.

19. The method of claim 18, wherein the unitary transform is calculated from the quantized signal model.

20. The method of claim 18, wherein the unitary transform is a Karhunen-Loé

ve transform (KLT).

21. The method of claim 18, wherein the selected set of channels is two.

22. A method for separating sources of an audio signal comprised of a plurality of channels, the method comprising:

segmenting the audio signal into frames;

estimating, for each frame, a signal model;

performing hierarchical decorrelation using the audio signal and the signal model for each of the frames to produce a plurality of decorrelated channels;

reordering the plurality of decorrelated channels based on energy of each decorrelated channel; and

combining the frames to obtain a source separated version of the audio signal.

23. The method of claim 22, wherein the estimated signal model for each frame yields a spectral matrix.

24. The method of claim 22, wherein performing the hierarchical decorrelation includes:

selecting a set of channels, of the plurality of channels of the audio signal, based on minimizing remaining correlation across the plurality of channels; and

performing a unitary transform on the selected set of channels, yielding a set of decorrelated channels.

25. The method of claim 24 wherein the unitary transform is calculated from the signal model.

26. The method of claim 24, wherein the unitary transform is a Karhunen-Loé

ve transform (KLT).

27. The method of claim 24, wherein the selected set of channels is two.

28. A method for encoding an audio signal comprised of a plurality of channels, the method comprising:

segmenting the audio signal into frames;

normalizing each of the frames of the audio signal to obtain a constant signal-to-noise ratio (SNR) in each of the plurality of channels;

performing hierarchical decorrelation on the frames using a unitary transform in time domain, yielding a plurality of decorrelated channels;

transforming the plurality of decorrelated channels to frequency domain;

applying one or more weighting terms to the plurality of decorrelated channels;

quantizing the plurality of decorrelated channels with the weighting terms to obtain a quantized audio signal; and

encoding the quantized audio signal using an entropy coder to produce an encoded bit stream.

29. The method of claim 28, further comprising extracting power spectral densities (PSDs) for the plurality of decorrelated channels.

30. The method of claim 29, wherein the one or more weighting terms are based on the PSDs of the plurality of decorrelated channels.

31. The method of claim 29, wherein the quantized audio signal is encoded using a probabilistic model based on the PSDs of the plurality of decorrelated channels.

32. The method of claim 28, wherein normalizing each of the frames of the audio signal includes applying a normalization factor based on temporal characteristics of the audio signal.

33. The method of claim 28, wherein each of the frames of the audio signal is normalized against an excitation power for the frame.

34. The method of claim 28, wherein transforming the plurality of decorrelated channels to the frequency domain includes applying a discrete Fourier transform (DFT) to the plurality of decorrelated channels.

35. The method of claim 28, wherein the steps of applying, quantizing, and encoding are performed in the frequency domain.

36. A system for encoding a multichannel audio signal, the system comprising:

one or more mono audio coders; and

a hierarchical decorrelation component, the hierarchical decorrelation component configured to;

select a plurality of channels of the audio signal based on at least one criterion;

perform a unitary transform on the selected plurality of channels, yielding a plurality of decorrelated channels;

combine the plurality of decorrelated channels with remaining channels of the audio signal other than the selected plurality; and

output the combined channels to the one or more mono audio coders.

37. The system of claim 36, wherein the unitary transform is realized in the time domain.

38. The system of claim 36, wherein the unitary transform includes a gain and a delay factor.

39. The system of claim 37, wherein the hierarchical decorrelation component is further configured to:

determine whether the combined channels should be further decorrelated based on computational complexity; and

responsive to determining that the combined channels should not be further decorrelated, pass the combined channels as output to the one or more audio coders.

40. The system of claim 39, wherein the hierarchical decorrelation component is further configured to stop decorrelating the combined channels when a predefined maximum cycle is reached.

41. The system of claim 39, wherein the hierarchical decorrelation component is further configured to stop decorrelating the combined channels when the gain factor at a cycle is close to zero.

42. The system of claim 36, wherein the one or more mono audio coders is configured to:

receive the combined channels from the hierarchical decorrelation component in the time domain;

transform the combined channels to frequency domain;

apply one or more weighting terms to the combined channels;

quantize the combined channels with the weighting terms to obtain a quantized audio signal; and

encode the quantized audio signal to produce an encoded bit stream.

43. The system of claim 42, wherein the one or more mono audio coders are further configured to extract power spectral densities (PSDs) for the combined channels.

44. The system of claim 43, wherein the one or more weighting terms are based on the PSDs of the combined channels.

45. The system of claim 42, wherein the one or more mono audio coders is further configured to encode the quantized audio signal using a probabilistic model based on the PSDs of the combined channels.

46. The system of claim 42, wherein the one or more mono audio coders is further configured to transform the combined channels to the frequency domain by applying a discrete Fourier transform (DFT) to the combined channels.