Methods and systems for adaptive time-frequency resolution in digital data coding

• US 9,008,811 B2
• Filed: 09/16/2011
• Issued: 04/14/2015
• Est. Priority Date: 09/17/2010
• Status: Active Grant

• Current Assignee: Xiph.Org Foundation
• Original Assignee: Xiph.Org Foundation
• Inventors: Valin, Jean-Marc, Terriberry, Timothy B.
• Primary Examiner(s): Flanders, Andrew C
• Application Number:
  US13/235,190
• Publication Number:
  US 20120069898A1
• Time in Patent Office: 1,306 Days
• Field of Search: 700/94, 704500-504
• US Class Current: 700/94
• CPC Class Codes: G10L 19/0204 using subband decomposition

Abstract

Embodiments are described for a system and method for implementing an adaptive time-frequency resolution in audio and video coding systems. A method of adaptively transforming the time-frequency resolution for a defined spectrum comprises dividing the spectrum of the input signal into a into plurality of bands; determining, for each band of the plurality of bands, a characteristic of the content (e.g., tonal or transient content); modifying the time-frequency resolution value to one or more bands of the plurality of bands to increase either a time resolution of the band or a frequency resolution of the band depending on the characteristic of the content; determining a cost associated with modifying the time-frequency resolution value of the one more bands based on an entropy measure of the bands, and altering the modified time-frequency resolution values in a manner that accounts for the coding cost.

Claims

1. A method of adaptively transforming the time-frequency resolution of a signal containing content over a defined spectrum, comprising:

separating the received signal into a plurality of bands by grouping sub-bands obtained by a first transform process;

determining, for each band of the plurality of bands, a desired change of the time-frequency resolution of each band;

applying a specific time-frequency (T-F) transform value to at least one of the bands to increase either a time (T) resolution of the respective band or a frequency (F) resolution of the respective band depending on the desired change of the time-frequency resolution of the respective band; and

applying a second T-F transform value to at least another of the bands to increase the other of wither a time (T) resolution of the respective band or a frequency (F) resolution of the respective band depending on the desired change of the time-frequency resolution of each band.

2. The method of claim 1, wherein the content comprises audio content and wherein the dominant characteristic comprises one of tonal content or transient content, the method further comprising:

increasing the frequency resolution of a band if the band has predominantly tonal content; and

increasing the time resolution of a band if the band has predominantly transient content.

3. The method of claim 1 wherein the specific time-frequency transform to increase the T or F resolution is a DCT (Discrete Cosine Transform) function.

4. The method of claim 1 wherein the specific time-frequency transform to increase the T or F resolution is a binary-basis function comprising an approximation of a DCT (DCT) function.

5. The method of claim 1 wherein the binary-basis function comprises a Hadamard transform function.

6. The method of claim 1 wherein the first transform process is one of:

a filter bank selection process, a lapped transform (LT), or a discrete cosine transform (DCT).

7. The method of claim 1 wherein the T-F transform value comprises a binary value pair, the method further comprising coding the T-F transform using a variable rate coding scheme to compress information representing multiple bands of the plurality of bands having the same T-F transform value.

8. The method of claim 7 wherein the variable rate coding scheme comprises arithmetic/range coding.

9. The method of claim 7 wherein the T-F transform value is selected from a selection of two possible binary value pairs.

10. The method of claim 7 further comprising:

determining an initial entropy value for a given T-F resolution value;

determining a change in the entropy value for a change in the give T-F resolution value; and

selecting the modified T-F resolution value based on the changed entropy value.

11. The method of claim 10 further comprising using a Viterbi Trellis algorithm for selection of the T-F transform value using the entropy factors.

12. The method of claim 1 wherein the signal comprises one of an audio signal, an image signal, and a video signal.

13. The method of claim 12 wherein the signal comprises an audio signal, and further wherein the bands are based on a Bark scale division of the audio spectrum.

18. The method of claim 1 further comprising encoding the time-frequency transform value for each band in a bit-stream for transmission to a decoder.

19. The method of claim 18 wherein:

if a band of the plurality of bands has predominantly tonal content, the frequency resolution of the band is increased; and

if a band of the plurality of bands has predominantly transient content, the time resolution of the band is increased.

14. A method of coding the time-frequency resolution for a defined spectrum, comprising:

defining an initial time-frequency (T-F) resolution value for the spectrum as a whole based on a measure of tonal content versus transient content of the spectrum;

dividing an input signal into a plurality of bands that comprise the spectrum;

modifying the time-frequency resolution value of one or more bands of the plurality of bands to increase either a time (T) resolution of the band or a frequency (F) resolution of the band depending on the relative transient content or tonal content in the band;

determining a cost associated with modifying the time-frequency resolution value of the one more bands based on an entropy measure of the bands;

altering the one or more modified time-frequency resolution values to minimize the cost and to generate a selected time-frequency resolution value for each band; and

modifying the time frequency resolution value of one or more other bands of the plurality of bands to increase the other of either a time (T) resolution of the band or a frequency (F) resolution of the band depending on the relative transient content or tonal content in the band.

15. The method of claim 14 wherein the bitstream comprises quantized filter output signals each band and the selected T-F resolution value for each band.

16. The method of claim 15 further comprising decoding the bitstream in the decoder to apply the selected T-F resolution values for each band to the input signal in order to suppress compression artifacts generated by compressing the input signal in a codec upon playback of the input signal.

17. The method of claim 16 wherein the input signal comprises an audio signal and further wherein the bands are based on a Bark scale division of the audio spectrum.

20. The method of claim 14 wherein the time-frequency modification value is applied using a process comprising one of:

a DCT function, a binary-basis function to approximate a DCT function, and a Hadamard transform.

21. The method of claim 14 wherein the T-F transform value comprises a binary value pair, the method further comprising coding the T-F transform using a variable rate coding scheme to compress information representing multiple bands of the plurality of bands having the same T-F transform value, and wherein the T-F transform value is selected from a selection of two or more possible binary value pairs.

22. The method of claim 21 wherein the T-F transform value is selected based on an entropy measure, the method further comprising:

determining an initial entropy value for a given T-F resolution value;

determining a change in the entropy value for a change in the give T-F resolution value; and

selecting the modified T-F resolution value if the changed entropy value is lower than the initial entropy value.

23. The method of claim 22 further comprising using a Viterbi Trellis algorithm for selection of the T-F transform value using the entropy factors.

24. A system for adaptively transforming the time-frequency resolution of a signal containing content over a defined spectrum, comprising:

a filter bank component separating the received signal into a plurality of bands by subdividing the defined spectrum;

a content analyzer component determining a desired characteristic of the content for each band of the plurality of bands;

a time-frequency resolution component applying a specific time-frequency (T-F) transform value to each band to increase either a time (T) resolution of the band or a frequency (F) resolution of the band depending on the desired characteristic, wherein at least one band is transformed for increased time resolution and at least another band is transformed from increased frequency resolution; and

a transmission line configured to transmit the transformed signal containing content.

25. The system of claim 24 further comprising an encoder stage encoding the time-frequency transform value for each band in a bitstream for transmission to a decoder.

26. The system of claim 25 wherein the bitstream comprises quantized filter output signals each band.

27. The system of claim 26 wherein the decoder decodes the bitstream to apply the selected T-F resolution values for each band to the input signal in order to suppress compression artifacts generated by compressing the input signal in a codec upon playback of the input signal.

28. The system of claim 27 wherein the input signal comprises an audio signal and further wherein the bands are based on a Bark scale division of the audio spectrum.

29. The system of claim 28 wherein the desired characteristic comprises tonal content or transient content of the signal, and further wherein:

if a band of the plurality of bands has predominant tonal content, the frequency resolution of the band is increased; and

if a band of the plurality of bands has predominant transient content, the time resolution of the band is increased.

30. The system of claim 24 wherein the T-F resolution value is transformed using a process comprising one of:

an MDCT function, a binary-basis function to approximate an MDCT function, and a Hadamard transform.

31. The system of claim 30 wherein the T-F transform value comprises a binary value pair, the method further comprising coding the T-F transform using a variable rate coding scheme to compress information representing multiple bands of the plurality of bands having the same T-F transform value, and wherein the T-F transform value is selected from a selection of two or more possible binary value pairs.

32. The system of claim 31 wherein the T-F transform value is selected based on an entropy metric, the method further comprising:

determining an initial entropy value for a given T-F resolution value;

determining a change in the entropy value for a change in the give T-F resolution value; and

selecting the modified T-F resolution value if the changed entropy value is lower than the initial entropy value.

33. A method comprising:

receiving a bitstream from an encoder, wherein the bitstream includes a quantized output of a time-frequency (T-F) resolution change for at least two groups of sub-bands processed by the encoder, wherein at least one group of sub-bands is processed from increased time resolution and at least another group of sub-bands is processed for increased frequency resolution;

applying an inverse T-F filter bank process to each of the group of sub-bands; and

processing each of the groups of sub-bands through a windowed overlap-add process to produce an output encapsulating information regarding a relative time resolution versus frequency resolution for each of the groups of sub-bands.

34. The method of claim 33 wherein the bitstream is encoded in the encoder by:

separating an original received content signal into a plurality of bands by grouping sub-bands obtained by a first transform process;

determining, for each band of the plurality of bands, a desired change of the time-frequency resolution of each band; and

applying a specific time-frequency (T-F) transform value to at least one of the bands to increase either a time (T) resolution of the respective band or a frequency (F) resolution of the respective band depending on the desired change of the time-frequency resolution of each band.

35. The method of claim 34 wherein the encoder includes a process for determining a cost associated with modifying the time-frequency resolution value of the one more bands based on an entropy measure of the bands, and altering the modified time-frequency resolution values to minimize the cost and to generate a selected time-frequency resolution value for each band.

36. The method of claim 35 wherein the encoder further includes a process for:

determining an initial entropy value for a given T-F resolution value;

determining a change in the entropy value for a change in the give T-F resolution value; and

selecting the modified T-F resolution value based on the changed entropy value.

37. The method of claim 33 wherein the encoder includes a process that

defines an initial time-frequency (T-F) resolution value for the spectrum as a whole based on a measure of tonal content versus transient content of the spectrum;

divides an input signal into a plurality of bands that comprise the spectrum;

modifies the time-frequency resolution value of one or more bands of the plurality of bands to increase either a time (T) resolution of the band or a frequency (F) resolution of the band depending on the relative transient content or tonal content in the band;

determines a cost associated with modifying the time-frequency resolution value of the one more bands based on an entropy measure of the bands; and

alters the modified time-frequency resolution values to minimize the cost and to generate a selected time-frequency resolution value for each band.

38. A system comprising:

a transmission line configured to receive a transformed signal containing content;

a decoder stage receiving a bitstream from the transmission line, wherein the bitstream includes a quantized output of a time-frequency (T-F) resolution change for at least two groups of sub-bands processed by the encoder, wherein at least one group of sub-bands is processed for increased time resolution and at least another group of sub-bands is processed for increased frequency resolution;

an inverse T-F filter bank component applying and inverse T-F filter bank process to each of the group of sub-bands; and

a window overlap-add component processing each of the group of sub-bands to produce an output encapsulating information regarding a relative time resolution versus frequency resolution for each of the groups of sub-bands.

39. The system of claim 38 wherein the bitstream is encoded in the encoder by:

a grouping component separating an original received content signal into a plurality of bands by grouping sub-bands obtained by a first transform process;

a time-resolution determination component determining, for each band of the plurality of bands, a desired change of the time-frequency resolution of each band; and

a transform component applying a specific time-frequency (T-F) transform value to at least one of the bands to increase either a time (T) resolution of the respective band or a frequency (F) resolution of the respective band depending on the desired change of the time-frequency resolution of each band.

40. The system of claim 39 wherein the encoder component includes a cost determination module determining a cost associated with modifying the time-frequency resolution value of the one more bands based on an entropy measure of the bands, and altering the modified time-frequency resolution values to minimize the cost and to generate a selected time-frequency resolution value for each band.