Support of a multichannel audio extension

• US 7,787,632 B2
• Filed: 03/21/2003
• Issued: 08/31/2010
• Est. Priority Date: 03/04/2003
• Status: Active Grant

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Nokia Corporation
• Inventors: Ojanpera, Juha
• Primary Examiner(s): Faulk; Devona E
• Application Number:
  US10/548,227
• Publication Number:
  US 20070165869A1
• Time in Patent Office: 2,720 Days
• Field of Search: 381/1, 381/2, 381 17- 23, 704/E19.005, 704/200.1, 704/203, 704/205, 704/500
• US Class Current: 381/23
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence

Abstract

The invention relates to methods and units supporting a multichannel audio extension. In order to allow an efficient extension requiring a low computational complexity, it is proposed that at an encoding end, at least state information is provided as side information for a provided mono audio signal (M) generated out of a multichannel audio signal. The state information indicates for each of a plurality of frequency bands how a predetermined or equally provided gain value is to be applied in the frequency domain to the mono audio signal (M) for obtaining first and a second channel signals (L,R) of a reconstructed multichannel audio signal.

Claims

1. A method for supporting a multichannel audio extension at an encoding end of a multichannel audio coding system, said method comprising:

transforming a first channel signal of a multichannel audio signal into the frequency domain, resulting in a spectral first channel signal;

transforming a second channel signal of said multichannel audio signal into the frequency domain, resulting in a spectral second channel signal; and

determining for each of a plurality of adjacent frequency bands of said transformed multichannel audio signals whether said spectral first channel signal, said spectral second channel signal or none of said spectral channel signals is dominant in the respective frequency band and providing a corresponding state information for each of said adjacent frequency bands.

2. The method according to claim 1, further comprising in case it was determined that one of said spectral first channel signal and said spectral second channel signal is dominant in at least one of said frequency bands calculating providing at least one gain value representative of the degree of said dominance.

3. The method according to claim 2, wherein said at least one gain value comprises a dedicated gain value for each of said frequency bands, each dedicated gain value being representative of the degree of the determined dominance of the respective dominant one of said spectral first channel signal and said spectral second channel signal in the respective frequency band.

4. The method according to claim 2, wherein said at least one gain value comprises a common gain value representing an average degree of a dominance of said spectral first channel signal and said spectral second channel signal in all of said frequency bands.

5. The method according to claim 1, wherein said state information is coded according to one of several coding schemes, the coding scheme being selected at least partly depending on which one of said spectral first channel signal and said spectral second channel signal is more frequently dominant in all of said frequency bands.

6. The method according to claim 1, further comprising

generating a reconstructed spectral first channel signal and a reconstructed spectral second channel signal based on said state information and on a mono channel version of said first channel signal and said second channel signal;

and

generating and providing for those frequency bands, for which said state information indicates that one of said channel signals is dominant, an enhancement information which reflects on a sample basis the difference between said reconstructed spectral first and second channel signals on the one hand and said original spectral first and second channel signals on the other hand.

7. The method according to claim 6, wherein generating said enhancement information comprises quantizing said difference on a frequency band basis sample-by-sample to a predetermined range by adjusting a quantization gain for the respective frequency band, said quantizing resulting in quantized spectral enhancement samples, wherein said quantization gain employed for a respective frequency band are provided as part of said enhancement information.

8. The method according to claim 6, further comprising providing an information on a bitrate employed for providing at least said state information and said enhancement information, said information on said bitrate being provided as part of said enhancement information.

9. A method for supporting a multichannel audio extension at a decoding end of a multichannel audio coding system, said method comprising:

transforming a received mono audio signal into the frequency domain, resulting in a spectral mono audio signal; and

generating a spectral first channel signal and a spectral second channel signal out of said spectral mono audio signal by weighting said spectral mono audio signal separately in each of a plurality of adjacent frequency bands for each of said spectral first channel signal and said spectral second channel signal based on at least one gain value and in accordance with a received state information, said state information indicating for each of said adjacent frequency bands whether said spectral first channel signal, said spectral second channel signal or none of said spectral channel signals is to be dominant within the respective frequency band.

10. The method according to claim 9, comprising

generating said spectral first channel signal within each of said frequency bands by multiplying one of said at least one gain values valid for a respective frequency band with samples of said spectral mono audio signal within said respective frequency band in case said state information indicates for said respective frequency band a dominance of said first channel signal, by multiplying the reciprocal value of said gain value with samples of said spectral mono audio signal within said respective frequency band in case said state information indicates for said respective frequency band a dominance of said second channel signal, and by taking over said spectral mono audio signal within said respective frequency band otherwise;

and

generating said spectral second channel signal within each of said frequency bands by multiplying one of said at least one gain values valid for a respective frequency band with samples of said spectral mono audio signal within said respective frequency band in case said state information indicates for said respective frequency band a dominance of said second channel signal, by multiplying the weighted or not-weighted reciprocal value of said gain value with samples of said spectral mono audio signal within said respective frequency band in case said state information indicates for said respective frequency band a dominance of said first channel signal, and by taking over said spectral mono audio signal within said respective frequency band otherwise.

11. The method according to claim 9, wherein said at least one gain value comprises a dedicated gain value for each of said plurality of frequency bands.

12. The method according to claim 11, wherein said mono audio signal is arranged in frames, wherein said gain values are smoothed at the start of each frame by averaging the gain value valid for the respective frequency band and the gain value valid for the respective next lower frequency band, and wherein said gain values are smoothed at the end of each frame by averaging the gain value valid for the respective frequency band and the gain value valid for the respective next higher frequency band.

13. The method according to claim 9, wherein for obtaining said state information, a received state information bitstream is decoded, which state information bitstream comprises at least partly in addition to said state information a coding scheme information, said coding scheme information indicating a coding scheme which has been employed for encoding said state information, said state information being decoded based on said coding scheme information.

14. The method according to claim 9, further comprising

receiving enhancement information which reflects at least for some spectral sample of those frequency bands, for which said state information indicates that one of said channel signals is dominant, on a sample basis the difference between said generated spectral first and second channel signals on the one hand and original spectral first and second channel signals on the other hand;

generating enhanced spectral first and second channel signals by taking into account on a sample-by-sample basis said difference reflected by said enhancement information; and

transforming said enhanced spectral first and second channel signals into the time domain, resulting in a first channel signal and a second channel signal of a reconstructed multichannel audio signal.

15. An apparatus comprising a stereo extension encoder implemented at least partly in hardware,

said stereo extension encoder configured to transform a first channel signal of a multichannel audio signal into the frequency domain, resulting in a spectral first channel signal;

said stereo extension encoder configured to transform a second channel signal of said multichannel audio signal into the frequency domain, resulting in a spectral second channel signal;

said stereo extension encoder configured to determine for each of a plurality of adjacent frequency bands of said transformed multichannel audio signals whether said spectral first channel signal, said spectral second channel signal or none of said spectral channel signals is dominant in the respective frequency band; and

said stereo extension encoder configured to provide a corresponding state information for each of said adjacent frequency bands.

16. The apparatus according to claim 15, wherein said stereo extension encoder is further configured to calculate and provide at least one gain value representative of the degree of said dominance, in case it was determined that one of said spectral first channel signal and said spectral second channel signal is dominant in at least one of said frequency bands.

17. The apparatus according to claim 16, wherein said at least one gain value comprises a dedicated gain value for each of said frequency bands, each dedicated gain value being representative of the degree of the determined dominance of the respective dominant one of said spectral first channel signal and said spectral second channel signal in the respective frequency band.

18. The apparatus according to claim 17, wherein said stereo extension encoder is configured to calculate channel weights for said spectral first channel signal and for said spectral second channel signal separately for each of said frequency bands based on the levels of spectral samples in said spectral channel signals, and wherein said stereo extension encoder is configured to determine said dedicated gain value for a particular frequency band to correspond to the ratio between the higher weight calculated for one of said spectral channel signals for said particular frequency band and the lower weight calculated for the respective other one of said spectral channel signals for said particular frequency band.

19. The apparatus according to claim 16, wherein said at least one gain value comprises a common gain value representing an average degree of a dominance of said spectral first channel signal and said spectral second channel signal in all of said frequency bands.

20. The apparatus according to claim 19, wherein said stereo extension encoder is configured to calculate channel weights for said spectral first channel signal and for said spectral second channel signal separately for each of said frequency bands based on the levels of spectral samples in said spectral channel signals, and wherein said stereo extension encoder is configured to determine a preliminary dedicated gain value for each frequency band to correspond to the ratio between the higher weight calculated for one of said spectral channel signals for a respective frequency band and the lower weight calculated for the respective other one of said spectral channel signals for said respective frequency band, and to determine said common gain value to be the average of said preliminary dedicated gain values.

21. The apparatus according to claim 15, wherein the stereo extension encoder is further configured to code said state information according to one of several coding schemes, the coding scheme being selected at least partly depending on which one of said spectral first channel signal and said spectral second channel signal is more frequently dominant in all of said frequency bands.

22. The apparatus according to claim 15, wherein said stereo extension encoder is configured to calculate channel weights for said spectral first channel signal and for said spectral second channel signal separately for each of said frequency bands based on the levels of spectral samples in said spectral channel signals, and to assume the presence of a dominance in a particular one of said frequency bands in case the ratio between the higher channel weight resulting for said frequency band and the lower channel weight resulting for said frequency band reaches or exceeds a predetermined threshold value.

23. The apparatus according to claim 15, further comprising an enhancement layer encoder implemented at least partly in hardware,

said enhancement layer encoder configured to generate a reconstructed spectral first channel signal and a reconstructed spectral second channel signal based on said state information and on a mono channel version of said first channel signal and said second channel signal;

and

said enhancement layer encoder configured to generate and provide for those frequency bands, for which said state information indicates that one of said channel signals is dominant, an enhancement information which reflects on a sample basis the difference between said reconstructed spectral first and second channel signals on the one hand and said original spectral first and second channel signals on the other hand.

24. The apparatus according to claim 23, wherein for generating and providing said enhancement information said enhancement layer encoder is configured to quantize said difference on a frequency band basis sample-by-sample to a predetermined range by adjusting a quantization gain for the respective frequency band, said quantizing resulting in quantized spectral enhancement samples, and to provide said quantization gain employed for a respective frequency band as part of said enhancement information.

25. The apparatus according to claim 24, wherein said enhancement layer encoder is configured to provide said quantized spectral enhancement samples for said enhancement information only for those frequency bands for which quantized spectral enhancement samples having non-zero values are available and which frequency bands require a quantization gain exceeding a specific threshold, and to provide an identification of those frequency bands for which said quantized spectral enhancement samples are provided for said enhancement information as part of said enhancement information.

26. The apparatus according to claim 24, wherein for generating and providing said enhancement information said enhancement layer encoder is further configured to assign said quantized spectral enhancement samples in groups of a predetermined number of samples to a respective codebook index, and to provide said codebook indices as part of said enhancement information.

27. The apparatus according to claim 23, wherein said enhancement layer encoder is further configured to provide an information on a bitrate employed for providing at least said state information and said enhancement information, said information on said bitrate being provided as part of said enhancement information.

28. The apparatus according to claim 15, wherein said apparatus is one of:

a multichannel audio encoder, a multichannel extension encoder for a multichannel audio encoder and a mobile terminal.

41. A multichannel audio coding system comprising the apparatus according to claim 15 and an apparatus comprising a stereo extension decoder implemented at least partly in hardware:

said stereo extension decoder configured to transform a received mono audio signal into the frequency domain, resulting in a spectral mono audio signal; and

said stereo extension decoder configured to generate a spectral first channel signal and a spectral second channel signal out of said spectral mono audio signal by weighting said spectral mono audio signal separately in each of a plurality of adjacent frequency bands for each of said spectral first channel signal and said spectral second channel signal based on at least one gain value and in accordance with a received state information, said state information indicating for each of said adjacent frequency bands whether said spectral first channel signal, said spectral second channel signal or none of said spectral channel signals is to be dominant within the respective frequency band.

29. An apparatus comprising a stereo extension decoder implemented at least partly in hardware,

said stereo extension decoder configured to transform a received mono audio signal into the frequency domain, resulting in a spectral mono audio signal;

and

said stereo extension decoder configured to generate a spectral first channel signal and a spectral second channel signal out of said spectral mono audio signal by weighting said spectral mono audio signal separately in each of a plurality of adjacent frequency bands for each of said spectral first channel signal and said spectral second channel signal based on at least one gain value and in accordance with a received state information, said state information indicating for each of said adjacent frequency bands whether said spectral first channel signal, said spectral second channel signal or none of said spectral channel signals is to be dominant within the respective frequency band.

30. The apparatus according to claim 29, wherein said stereo extension decoder is configured to delay said mono audio signal before being transformed into the time domain, in case said mono audio signal is not time-aligned with an original multichannel audio signal which is to be reconstructed.

31. The apparatus according to claim 29, wherein said at least one gain value comprises a dedicated gain value for each of said plurality of frequency bands.

32. The apparatus according to claim 31, wherein said mono audio signal is arranged in frames, and wherein said stereo extension decoder is configured to smooth said gain values at the start of each frame by averaging the gain value valid for the respective frequency band and the gain value valid for the respective next lower frequency band, and to smooth said gain values at the end of each frame by averaging the gain value valid for the respective frequency band and the gain value valid for the respective next higher frequency band.

33. The apparatus according to claim 29, wherein said stereo extension decoder is configured to decode a received state information bitstream for obtaining said state information, which state information bitstream comprises at least partly in addition to said state information a coding scheme information, said coding scheme information indicating a coding scheme which has been employed for encoding said state information, and wherein said stereo extension decoder configured to decode said state information based on said coding scheme information.

34. The apparatus according to claim 29, further comprising an enhancement layer decoder implemented at least partly in hardware,

said enhancement layer decoder being configured to receive enhancement information which reflects at least for some spectral sample of those frequency bands, for which said state information indicates that one of said channel signals is dominant, on a sample basis the difference between said generated spectral first and second channel signals on the one hand and original spectral first and second channel signals on the other hand;

said enhancement layer decoder being configured to generate enhanced spectral first and second channel signals by taking into account on a sample-by-sample basis said difference reflected by said enhancement information; and

said enhancement layer decoder being configured to transform said enhanced spectral first and second channel signals into the time domain, resulting in a first channel signal and a second channel signal of a reconstructed multichannel audio signal.

35. The apparatus according to claim 34, wherein said enhancement layer decoder is configured to obtain said difference by dequantizing quantized spectral enhancement samples obtained from said received enhancement information, said dequantizing employing a dedicated quantization gain for each frequency band for which quantized spectral enhancement samples are available, wherein said quantization gains are indicated in said enhancement information.

36. The apparatus according to claim 35, wherein said received enhancement information identifies in addition those frequency bands among all frequency bands for which said state information indicates that one of said channel signals is dominant, for which frequency bands quantized spectral enhancement samples are available, and wherein said enhancement layer decoder is configured to take said identification of frequency bands into account in generating said enhanced spectral first and second channel signals.

37. The apparatus according to claim 35, wherein said enhancement layer decoder is configured to obtain said quantized spectral enhancement samples from said received enhancement information by an inverse codebook mapping of codebook indices comprised in said received enhancement information to values of a respective group of a predetermined number of quantized spectral enhancement samples.

38. The apparatus according to claim 37, wherein said received enhancement information comprises only codebook indices for selected groups of samples, wherein said enhancement information further comprises an identification of said groups for which codebook indices are comprised, and wherein said enhancement layer decoder is configured to take said identification of groups into account in generating said enhanced spectral first and second channel signals.

39. The apparatus according to claim 34, wherein said enhancement information further comprises an indication of a bitrate with which at least said state information and said enhancement information are provided, and wherein said enhancement layer decoder is configured to employ said bitrate indication for determining the amount of received enhancement information.

40. The apparatus according to claim 29, wherein said apparatus is one of:

a multichannel audio decoder, a multichannel extension decoder for a multichannel audio decoder and a mobile terminal.