Compatible multi-channel coding/decoding by weighting the downmix channel

• US 7,447,317 B2
• Filed: 10/02/2003
• Issued: 11/04/2008
• Est. Priority Date: 10/02/2003
• Status: Active Grant

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Hilpert, Johannes, Hölzer, Andreas, Herre, Jürgen, Geyersberger, Stefan, Spenger, Claus
• Primary Examiner(s): Edouard; Patrick N.
• Assistant Examiner(s): SHAH, PARAS D
• Application Number:
  US10/679,085
• Publication Number:
  US 20050074127A1
• Time in Patent Office: 1,860 Days
• Field of Search: 704/500, 704/200, 704/200.1, 381/77, 381/317, 381/56, 381/57, 381/307, 381/27, 381/300, 381/23, 381/17, 381/19, 704/501, 700/94
• US Class Current: 381/23
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: H04S 2400/03 Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 H04S2400/01 takes precedence
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 3/02 of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other

Abstract

In processing a multi-channel audio signal having at least three original channels, a first downmix channel and a second downmix channel are provided, which are derived from the original channels. For a selected original channel of the original channels, channel side information are calculated such that a downmix channel or a combined downmix channel including the first and the second downmix channels, when weighted using the channel side information, results in an approximation of the selected original channel. The channel side information and the first and second downmix channels form output data to be transmitted to a decoder, which, in case of a low level decoder only decodes the first and second downmix channels or, in case of a high level decoder provides a full multi-channel audio signal based on the downmix channels and the channel side information.

Claims

1. Apparatus for processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, comprising:

means for providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

means for calculating channel side information for a selected original channel of the original signals, the means for calculating being operative to calculate the channel side information such that a downmix channel or a combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel;

the means for calculating channel side information being operative to perform joint stereo coding using a downmix channel as a carrier channel and using, as an input channel, the selected original channel, to generate joint stereo parameters as channel side information for the selected original channel; and

means for generating output data, the output data including the channel side information.

2. Apparatus in accordance with claim 1, in which the means for generating is operative to generate the output data such that the output data additionally include the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel.

3. Apparatus in accordance with claim 1, in which the means for calculating is operative to determine the channel side information as parametric data not including time domain samples or spectral values.

4. Apparatus in accordance with claim 3, in which the means for calculating is operative to perform intensity stereo coding or binaural cue coding, such that the channel side information represent an energy distribution or binaural cue parameters for the selected original channel, wherein a downmix channel or a combined downmix channel is usable as a carrier channel.

5. Apparatus in accordance with claim 1,

in which the multi-channel audio signal includes a left channel, a left surround channel, a right channel and a right surround channel,

in which the means for providing is operative to providethe first downmix channel as a left downmix channel and toprovide the second downmix channel as a right downmix channel,the left and the right downmix channels being formed such that a result, when played, is a stereo representation of the multi-channel audio signal, and

in which the means for calculating is operative to calculate the channel side information for the left channel as the selected original channel using the left downmix channel,

to calculate the channel side information for the right channel as the selected original channel using the right downmix channel,

to calculate the channel side information for the left surround channel as the selected original channel using the left downmix channel, and

to calculate the channel side information for the right surround channel as the selected original channel using the right downmix channel.

6. Apparatus in accordance with claim 1,

in which the original channels include a center channel,

which further includes a combiner for combining the first downmix channel and the second downmix channel to obtain the combined downmix channel;

and

wherein the means for calculating the channel side information for the center channel as the selected original channel is operative to calculate the channel side information such that the combined downmix channel when weighted using the channel side information results in an approximation of the original center channel.

7. Apparatus in accordance with claim 1, in which the means for providing is operative to derive the first downmix channel and the second downmix channel from the original channels using a first predetermined linear weighted combination for the first downmix channel and using a second predetermined linear weighted combination for the second downmix channel.

8. Apparatus in accordance with claim 6, in which the first predetermined linear weighted combination is defined as follows:

Lc=t·

(L+a·

Ls+b·

c);

or

in which the predetermined second linear weightedcombination is defined as follows;

Rc=t·

(R+a·

Rs +b·

C),

wherein Lc is the first downmix channel, wherein Rc is the second downmix channel, wherein t, a and b are weighting factors smaller than 1, wherein L is an original left channel, wherein C is an original center channel, wherein R is an original right channel, wherein Ls is an original left surround channel, and wherein Rs is an original right surround channel.

9. Apparatus in accordance with claim 1, in which the means for providing is operative to receive externally supplied first and second downmix channels.

10. Apparatus in accordance with claim 1, in which the first downmix channel and the second downmix channel are generated by combining the original channels in varying degrees, wherein the means for calculating is operative, to use, for calculating the channel side information, the downmix channel among both downmix channels,which is stronger influenced by the selected original channelwhen compared to the other downmix channel.

11. Apparatus in accordance with claim 1, inwhich the means for generating is operative to form the output data such that the output data are in compliance with an output data syntax to be used by a low level decoder for processing the first downmix channel or a signal derived from the first downmix channel or the second downmix channel or a signal derived from the second downmix channel to obtain a decoded stereo representation of the multi-channel audio signal.

12. Apparatus in accordance with claim 11, in which the output data syntax is structured such that same includes a special data field to be ignored by a low level decoder, and in which the means for generating is operative to insert the channel side information into the special data field.

13. Apparatus in accordance with claim 12, in which the syntax is mp3 syntax and the special data field is an ancillary data field.

14. Apparatus in accordance with claim 11, in which the means for generating is operative to insert the channel side information into the output data such that the channel side information are only used by a high level decoder but are ignored by the low level decoder.

15. Apparatus in accordance with claim 2, which further comprises an encoder for encoding the first downmix channel to obtain the signal derived from the first downmix channel or for encoding the second downmix channel to obtain the signal derived from the second downmix channel.

16. Apparatus in accordance with claim 15, in which the encoder is a perceptual encoder which includes means for converting a signal to be encoded into a spectral representation, means for quantizing the spectral representation using a psychoacoustic model and means for entropy encoding a quantized spectral representation to obtain an entropy encoded quantized spectral representation as the signal derived from the first downmix channel or the signal derived from the second downmix channel.

17. Apparatus in accordance with claim 16, in which the perceptual encoder is an encoder in accordance with MPEG-½

layer III (mp3) or MPEG- 2/4 advanced audio coding (AAC)

18. Apparatus in accordance with claim 1, in which the means for calculating is operative to calculate downmix energy values for the downmix channel or the combined downmix channel,

to calculate an original energy value for the selected original channel, and

to calculate a gain factor as the channel side information, the gain factor being derived from the downmix energy value and the original energy value.

19. Apparatus in accordance with claim 1, in which the means for calculating is operative to calculate frequency dependent channel side information parameters such that for a plurality of frequency bands, a plurality of different channel side information parameters are obtained.

20. Method of processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, the original channels including a center channel, the method comprising:

providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

combining the first downmix channel and the second downmix channel to obtain a combined downmix channel;

calculating channel side information for a selected original channel of the original signals such that a downmix channel or the combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, wherein the channel side information for the center channel as the selected original channel is calculated, and wherein the channel side information for the center channel is calculated such that the combined downmix channel, when weiqhted using the channel side information, results in an approximation of the original center channel; and

generating output data, the output data including the channel side information.

21. Apparatus for inverse processing of input data, the input data including channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multi-channel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel including the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, the apparatus comprising:

an input data reader for reading the input data to obtain the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information;

a channel reconstructor for reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to obtain the approximation of the selected original channel; and

the channel reconstructor being operative to reconstruct an approximation for the center channel using channel side information for the center channel and the combined downmix channel.

22. Apparatus in accordance with claim 21, further comprising a decoder for decoding the signal derived from the first downmix channel to obtain the decoded version of the first downmix channel and for decoding the signal derived from the second downmix channel to obtain a decoded version of the second downmix channel.

23. Apparatus in accordance with claim 21, further comprising a combiner for combining the first downmix channel and the second downmix channel to obtain the combined downmix channel.

24. Apparatus in accordance with claim 21,

in which the original audio signal includes a left channel, a left surround channel, a right channel, a right surround channel and center channel,

wherein the first downmix channel and the second downmix channel are a left downmix channel and a right downmix channel, respectively, and

wherein the input data include channel side information for at least three of the left channel, the left surround channel, the right channel, the right surround channel and the center channel,

wherein the channel reconstructor is operative

to reconstruct an approximation of the left channel using channel side information for the left channel and the left downmix channel,

to reconstruct an approximation for the left surround channel using channel side information for the left surround channel and the left downmix channel,

to reconstruct an approximation for the right channel using channel side information for the right channel and the right downmix channel, and

to reconstruct an approximation for the right surround channel using channel side information for the right surround channel and the right downmix channel.

25. Method of inverse processing of input data, the input data including channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multi-channel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel including the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, the method comprising:

reading the input data to obtain the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information;

reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to obtain the approximation of the selected original channel; and

the reconstructing step further including reconstructingan approximation for the center channel using channel side information for the center channel and the combined downmix channel.

26. A computer implemented method of processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, the original channels including a center channel, the method comprising:

providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

combining the first downmix channel and the second downmix channel to obtain a combined downmix channel;

calculating channel side information for a selected original channel of the original signals such that a downmix channel or the combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, wherein the channel side information for the center channel as the selected original channel is calculated, and wherein the channel side information for the center channel is calculated such that the combined downmix channel when weighted using the channel side information results in an approximation of the original center channel; and

generating output data, the output data including the channel side information.

27. A computer implemented method for inverse processing of input data, the input data including channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multi-channel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel including the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, the method comprising:

reading the input data to obtain the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information;

reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to obtain the approtimation of the selected original channel; and

the reconstructing step further including reconstructingan approximation for the center channel using channel side information for the center channel and the combined downmix channel.

28. An apparatus for processing a multi-channel audio signal having at least three original channels, including a left channel, a left surround channel a right channel and a right surround channel, the apparatus comprising:

means for providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

said means for providing being operative to provide the first downmix channel as a left downmix channel and to provide the second downmix channel as a right downmix channel, the left and the right downmix channels being formed such that are sult, when played, is a stereo representation of the multi-channel audio signal;

means for calculating channel. side information for a selected original channel of the original signals, the means for calculating being operative to calculate the channel side information such that a downmix channel or a combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel;

said means for calculating being operative to;

calculate the channel side information for the left channel as the selected original channel using the left downmix channel;

calculate the channel side information for the right channel as the selected original channel using the right downmix channel;

calculate the channel side information for the left surround channel as the selected original channel using the left downmix channel; and

calculate the channel side information for the right surround channel as the selected original channel using the right downmix channel; and

means for generating output data, the output data including the channel side information.

29. An apparatus for processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, the original channels including a center channel, the apparatus comprising:

means for providing a first dowrxmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

a combiner for combining the first downmix channel and the second downmix channel to obtain a combined downmix channel;

means for calculating channel side information for a selected original channel of the original signals, the means for calculating being operative to calculate the channel side information such that a downmix channel or the combined downmix channel including the first and the second downmix channels, when weighted using the channel side information, results in an approximation of the selected original channel, wherein the means for calculating the channel side information is operative to calculate the channel side information for the center channel as the selected original channel, and wherein the means for calculating the channel side information is operative to calculate the channel side information for the center channel such that the combined downmix channel, when weighted using the channel side information, results in an approximation of the original center channel; and

means for generating output data, the output datain cluding the channel side information.

30. A method of processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, the multi-channel audio signal including a left channel, a left surround channel, a right channel and a right surround channel, the method comprising:

providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels, wherein the first downmix channel is provided as a left downmix channel and the second downmix channel is provided as a right downmix channel, the left and the right downmix channels being formed such that a result, when played, is a stereo representation of the multi-channel audio signal;

calculating channel side information for a selected original channel of the original signals such that a downmix channel or a combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel,

wherein the step of calculating farther includes;

calculating the channel side information for the left channel as the selected original channel using the left downmix channel;

calculating the channel side information for the right channel as the selected original channel using the right downmix channel;

calculating the channel side information for the left surround channel as the selected original channel using the left downmix channel; and

calculating the channel side information for the right surround channel as the selected original channel using the right downmix channel; and

generating output data, the output data including the channel side information.

31. A method of processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, the method comprising:

providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels;

calculating channel side information for a selected original channel of the original signals such that a downmix channel or a combined downmix channel including the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel;

the calculating step further including performing joint stereo coding using a downmix channel as a carrier channel and using, as an input channel, the selected original channel, togenerate joint stereo parameters as channel side information for the selected original channel; and

generating output data, the output data including the channel side information.