Audio coding system using spectral hole filling

• US 20030233234A1
• Filed: 06/17/2002
• Published: 12/18/2003
• Est. Priority Date: 06/17/2002
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Truman, Michael Mead, Vinton, Mark Stuart, Robinson, Charles Quito, Davidson, Grant Allen, Fellers, Matthew Conrad, Watson, Matthew Aubrey
• Granted Patent: US 7,447,631 B2
• Time in Patent Office: Days
• US Class Current: 704/203
• CPC Class Codes: G10L 19/035 Scalar quantisation
• CPC Class Codes: G10L 21/038 using band spreading techniques

Abstract

Audio coding processes like quantization can cause spectral components of an encoded audio signal to be set to zero, creating spectral holes in the signal. These spectral holes can degrade the perceived quality of audio signals that are reproduced by audio coding systems. An improved decoder avoids or reduces the degradation by filling the spectral holes with synthesized spectral components. An improved encoder may also be used to realize further improvements in the decoder.

Claims

1. A method for generating audio information, wherein the method comprises:

receiving an input signal and obtaining therefrom a set of subband signals each having one or more spectral components representing spectral content of an audio signal;

identifying within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold, and in which a plurality of spectral components have a zero value;

generating synthesized spectral components that correspond to respective zero-valued spectral components in the particular subband signal and that are scaled according to a scaling envelope less than or equal to the threshold;

generating a modified set of subband signals by substituting the synthesized spectral components for corresponding zero-valued spectral components in the particular subband signal; and

generating the audio information by applying a synthesis filterbank to the modified set of subband signals.

2. The method of claim 1 wherein the scaling envelope is uniform.

3. The method of claim 1 wherein the synthesis filterbank is implemented by a block transform that has spectral leakage between adjacent spectral components and the scaling envelope varies at a rate substantially equal to a rate of roll off of the spectral leakage of the block transform.

4. The method of claim 1 wherein the synthesis filterbank is implemented by a block transform and the method comprises:

applying a frequency-domain filter to one or more spectral components in the set of subband signals; and

deriving the scaling envelope from an output of the frequency-domain filter.

5. The method of claim 4 that comprises varying the response of the frequency-domain filter as a function of frequency.

6. The method of claim 1 that comprises:

obtaining a measure of tonality of the audio signal represented by the set of subband signals; and

adapting the scaling envelope in response to the measure of tonality.

7. The method of claim 6 that obtains the measure of tonality from the input signal.

8. The method of claim 6 that comprises deriving the measure of tonality from the way in which the zero-valued spectral components are arranged in the particular subband signal.

9. The method of claim 1 wherein the synthesis filterbank is implemented by a block transform and the method comprises:

obtaining a sequence of sets of subband signals from the input signal;

identifying a common subband signal in the sequence of sets of subband signals where, for each set in the sequence, one or more spectral components have a non-zero value and a plurality of spectral components have a zero value;

identifying a common spectral component within the common subband signal that has a zero value in a plurality of adjacent sets in the sequence that are either preceded or followed by a set with the common spectral components having a non-zero value;

scaling the synthesized spectral components that correspond to the zero-valued common spectral components according to the scaling envelope that varies from set to set in the sequence according to temporal masking characteristics of the human auditory system;

generating a sequence of modified sets of subband signals by substituting the synthesized spectral components for the corresponding zero-valued common spectral components in the sets; and

generating the audio information by applying the synthesis filterbank to the sequence of modified sets of subband signals.

10. The method of claim 1 wherein the synthesis filterbank is implemented by a block transform and the method generates the synthesized spectral components by spectral translation of other spectral components in the set of subband signals.

11. The method of claim 1 wherein the scaling envelope varies according to temporal masking characteristics of the human auditory system.

12. A method for generating an output signal, wherein the method comprises:

generating a set of subband signals each having one or more spectral components representing spectral content of an audio signal by quantizing information that is obtained by applying an analysis filterbank to audio information;

identifying within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold, and in which a plurality of spectral components have a zero value;

deriving scaling control information from the spectral content of the audio signal, wherein the scaling control information controls scaling of synthesized spectral components to be synthesized and substituted for the spectral components having a zero value in a receiver that generates audio information in response to the output signal; and

generating the output signal by assembling the scaling control information and information representing the set of subband signals.

13. The method according to claim 12 that comprises:

obtaining a measure of tonality of the audio signal represented by the set of subband signals; and

deriving the scaling control information from the measure of tonality.

14. The method according to claim 12 that comprises:

obtaining an estimated psychoacoustic masking threshold of the audio signal represented by the set of subband signals; and

deriving the scaling control information from the estimated psychoacoustic masking threshold.

15. The method according to claim 12 that comprises:

obtaining two measures of spectral levels for portions of the audio signal represented by the non-zero-valued and the zero-valued spectral components; and

deriving the scaling control information from the two measures of spectral levels.

16. An apparatus for generating audio information, wherein the apparatus comprises:

a deformatter that receives an input signal and obtains therefrom a set of subband signals each having one or more spectral components representing spectral content of an audio signal;

a decoder coupled to the deformatter that identifies within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold, and in which a plurality of spectral components have a zero value, that generates synthesized spectral components that correspond to respective zero-valued spectral components in the particular subband signal and are scaled according to a scaling envelope less than or equal to the threshold, and that generates a modified set of subband signals by substituting the synthesized spectral components for corresponding zero-valued spectral components in the particular subband signal; and

a synthesis filterbank coupled to the decoder that generates the audio information in response to the modified set of subband signals.

17. The apparatus of claim 16 wherein the scaling envelope is uniform.

18. The apparatus of claim 16 wherein the synthesis filterbank is implemented by a block transform that has spectral leakage between adjacent spectral components and the scaling envelope varies at a rate substantially equal to a rate of roll off of the spectral leakage of the block transform.

19. The apparatus of claim 16 wherein the synthesis filterbank is implemented by a block transform and the decoder:

applies a frequency-domain filter to one or more spectral components in the set of subband signals; and

derives the scaling envelope from an output of the frequency-domain filter.

20. The apparatus of claim 19 wherein the decoder varies the response of the frequency-domain filter as a function of frequency.

21. The apparatus of claim 16 wherein the decoder:

obtains a measure of tonality of the audio signal represented by the set of subband signals; and

adapts the scaling envelope in response to the measure of tonality.

22. The apparatus of claim 21 that obtains the measure of tonality from the input signal.

23. The apparatus of claim 21 wherein the decoder derives the measure of tonality from the way in which the zero-valued spectral components are arranged in the particular subband signal.

24. The apparatus of claim 16 wherein the synthesis filterbank is implemented by a block transform and:

the deformatter obtains a sequence of sets of subband signals from the input signal;

the decoder identifies a common subband signal in the sequence of sets of subband signals where, for each set in the sequence, one or more spectral components have a non-zero value and a plurality of spectral components have a zero value, identifies a common spectral component within the common subband signal that has a zero value in a plurality of adjacent sets in the sequence that are either preceded or followed by a set with the common spectral components having a non-zero value, scales the synthesized spectral components that correspond to the zero-valued common spectral components according to the scaling envelope that varies from set to set in the sequence according to temporal masking characteristics of the human auditory system; and

generates a sequence of modified sets of subband signals by substituting the synthesized spectral components for the corresponding zero-valued common spectral components in the sets; and

the synthesis filterbank generates the audio information in response to the sequence of modified sets of subband signals.

25. The apparatus of claim 16 wherein the synthesis filterbank is implemented by a block transform and the decoder generates the synthesized spectral components by spectral translation of other spectral components in the set of subband signals.

26. The apparatus of claim 16 wherein the scaling envelope varies according to temporal masking characteristics of the human auditory system.

27. An apparatus for generating an output signal, wherein the apparatus comprises:

an analysis filterbank that generates in response to audio information a set of subband signals each having one or more spectral components representing spectral content of an audio signal;

quantizers coupled to the analysis filterbank that quantize the spectral components;

an encoder coupled to the quantizers that identifies within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold and in which a plurality of spectral components have a zero value, derives scaling control information from the spectral content of the audio signal, wherein the scaling control information controls scaling of synthesized spectral components to be synthesized and substituted for the spectral components having a zero value in a receiver that generates audio information in response to the output signal; and

a formatter coupled to the encoder that generates the output signal by assembling the scaling control information and information representing the set of subband signals.

28. The apparatus according to claim 27 that:

obtains a measure of tonality of the audio signal represented by the set of subband signals; and

derives the scaling control information from the measure of tonality.

29. The apparatus according to claim 27 comprising a modeling component that:

obtains an estimated psychoacoustic masking threshold of the audio signal represented by the set of subband signals; and

derives the scaling control information from the estimated psychoacoustic masking threshold.

30. The apparatus according to claim 27 that:

obtains two measures of spectral levels for portions of the audio signal represented by the non-zero-valued and the zero-valued spectral components; and

derives the scaling control information from the two measures of spectral levels.

31. A medium that conveys a program of instructions and is readable by a device for executing the program of instructions to perform a method for generating audio information, wherein the method comprises:

receiving an input signal and obtaining therefrom a set of subband signals each having one or more spectral components representing spectral content of an audio signal;

identifying within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold, and in which a plurality of spectral components have a zero value;

generating synthesized spectral components that correspond to respective zero-valued spectral components in the particular subband signal and that are scaled according to a scaling envelope less than or equal to the threshold;

generating a modified set of subband signals by substituting the synthesized spectral components for corresponding zero-valued spectral components in the particular subband signal; and

generating the audio information by applying a synthesis filterbank to the modified set of subband signals.

32. The medium of claim 31 wherein the scaling envelope is uniform.

33. The medium of claim 31 wherein the synthesis filterbank is implemented by a block transform that has spectral leakage between adjacent spectral components and the scaling envelope varies at a rate substantially equal to a rate of roll off of the spectral leakage of the block transform.

34. The medium of claim 31 wherein the synthesis filterbank is implemented by a block transform and the method comprises:

applying a frequency-domain filter to one or more spectral components in the set of subband signals; and

deriving the scaling envelope from an output of the frequency-domain filter.

35. The medium of claim 34 wherein the method comprises varying the response of the frequency-domain filter as a function of frequency.

36. The medium of claim 31 wherein the method comprises:

obtaining a measure of tonality of the audio signal represented by the set of subband signals; and

adapting the scaling envelope in response to the measure of tonality.

37. The medium of claim 36 wherein the method obtains the measure of tonality from the input signal.

38. The medium of claim 36 wherein the method comprises deriving the measure of tonality from the way in which the zero-valued spectral components are arranged in the particular subband signal.

39. The medium of claim 31 wherein the synthesis filterbank is implemented by a block transform and the method comprises:

obtaining a sequence of sets of subband signals from the input signal;

identifying a common subband signal in the sequence of sets of subband signals where, for each set in the sequence, one or more spectral components have a non-zero value and a plurality of spectral components have a zero value;

identifying a common spectral component within the common subband signal that has a zero value in a plurality of adjacent sets in the sequence that are either preceded or followed by a set with the common spectral components having a non-zero value;

scaling the synthesized spectral components that correspond to the zero-valued common spectral components according to the scaling envelope that varies from set to set in the sequence according to temporal masking characteristics of the human auditory system;

generating a sequence of modified sets of subband signals by substituting the synthesized spectral components for the corresponding zero-valued common spectral components in the sets; and

generating the audio information by applying the synthesis filterbank to the sequence of modified sets of subband signals.

40. The medium of claim 31 wherein the synthesis filterbank is implemented by a block transform and the method generates the synthesized spectral components by spectral translation of other spectral components in the set of subband signals.

41. The medium of claim 31 wherein the scaling envelope varies according to temporal masking characteristics of the human auditory system.

42. A medium that conveys a program of instructions and is readable by a device for executing the program of instructions to perform a method for generating an output signal, wherein the method comprises:

generating a set of subband signals each having one or more spectral components representing spectral content of an audio signal by quantizing information that is obtained by applying an analysis filterbank to audio information;

identifying within the set of subband signals a particular subband signal in which one or more spectral components have a non-zero value and are quantized by a quantizer having a minimum quantizing level that corresponds to a threshold, and in which a plurality of spectral components have a zero value;

deriving scaling control information from the spectral content of the audio signal, wherein the scaling control information controls scaling of synthesized spectral components to be synthesized and substituted for the spectral components having a zero value in a receiver that generates audio information in response to the output signal; and

generating the output signal by assembling the scaling control information and information representing the set of subband signals.

43. The medium according to claim 42 wherein the method comprises:

obtaining a measure of tonality of the audio signal represented by the set of subband signals; and

deriving the scaling control information from the measure of tonality.

44. The medium according to claim 42 wherein the method comprises:

obtaining an estimated psychoacoustic masking threshold of the audio signal represented by the set of subband signals; and

deriving the scaling control information from the estimated psychoacoustic masking threshold.

45. The medium according to claim 42 wherein the method comprises:

obtaining two measures of spectral levels for portions of the audio signal represented by the non-zero-valued and the zero-valued spectral components; and

deriving the scaling control information from the two measures of spectral levels.