Coding generic audio signals at low bitrates and low delay

• US 9,015,038 B2
• Filed: 10/25/2011
• Issued: 04/21/2015
• Est. Priority Date: 10/25/2010
• Status: Active Grant

• Current Assignee: VoiceAge EVS LLC (FIG LLC (d/b/a Fortress Investment Group LLC))
• Original Assignee: VoiceAge Corporation
• Inventors: Vaillancourt, Tommy, Jelinek, Milan
• Primary Examiner(s): VO, HUYEN X
• Application Number:
  US13/280,707
• Publication Number:
  US 20120101813A1
• Time in Patent Office: 1,274 Days
• Field of Search: 704/203, 704/205, 704/219, 704/223, 704500-504, 704229-230, 704/206, 704/221, 704/227
• US Class Current: 704/206
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/08 Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding

Abstract

A mixed time-domain/frequency-domain coding device and method for coding an input sound signal, wherein a time-domain excitation contribution is calculated in response to the input sound signal. A cut-off frequency for the time-domain excitation contribution is also calculated in response to the input sound signal, and a frequency extent of the time-domain excitation contribution is adjusted in relation to this cut-off frequency. Following calculation of a frequency-domain excitation contribution in response to the input sound signal, the adjusted time-domain excitation contribution and the frequency-domain excitation contribution are added to form a mixed time-domain/frequency-domain excitation constituting a coded version of the input sound signal. In the calculation of the time-domain excitation contribution, the input sound signal may be processed in successive frames of the input sound signal and a number of sub-frames to be used in a current frame may be calculated.

Claims

1. A mixed time-domain/frequency-domain coding device for coding an input sound signal, comprising:

a calculator of a time-domain excitation contribution in response to the input sound signal;

a calculator of a cut-off frequency for the time-domain excitation contribution in response to the input sound signal;

a filter responsive to the cut-off frequency for adjusting a frequency extent of the time-domain excitation contribution;

a calculator of a frequency-domain excitation contribution in response to the input sound signal; and

an adder of the filtered time-domain excitation contribution and the frequency-domain excitation contribution to form a mixed time-domain/frequency-domain excitation constituting a coded version of the input sound signal.

2. A mixed time-domain/frequency-domain coding device according to claim 1, wherein the time-domain excitation contribution includes (a) only an adaptive codebook contribution, or (b) the adaptive codebook contribution and a fixed codebook contribution.

3. A mixed time-domain/frequency-domain coding device according to claim 2, wherein the calculator of time-domain excitation contribution uses a Code-Excited Linear Prediction coding of the input sound signal.

4. A mixed time-domain/frequency-domain coding device according to claim 3, wherein the calculator of frequency-domain excitation contribution comprises a calculator of a difference between a frequency representation an LP residual of the input sound signal and a filtered frequency representation of the time-domain excitation contribution.

5. A mixed time-domain/frequency-domain coding device according to claim 3, wherein the calculator of frequency-domain excitation contribution performs a frequency transform of a LP residual obtained from an LP analysis of the input sound signal to produce a frequency representation of the LP residual.

6. A mixed time-domain/frequency-domain coding device according to claim 5, wherein the calculator of cut-off frequency comprises a computer of cross-correlation, for each of a plurality of frequency bands, between the frequency representation of the LP residual and a frequency representation of the time-domain excitation contribution, and the coding device comprises a finder of an estimate of the cut-off frequency in response to the cross-correlation.

7. A mixed time-domain/frequency-domain coding device according to claim 5, comprising a smoother of the cross-correlation through the frequency bands to produce a cross-correlation vector, a calculator of an average of the cross-correlation vector over the frequency bands, and a normalizer of the average of the cross-correlation vector, wherein the finder of the estimate of the cut-off frequency determines a first estimate of the cut-off frequency by finding a last frequency of one of the frequency bands which minimizes a difference between said last frequency and the normalized average of the cross-correlation vector multiplied by a spectrum width value.

8. A mixed time-domain/frequency-domain coding device according to claim 7, wherein the calculator of cut-off frequency comprises a finder of one of the frequency bands in which a harmonic computed from the time-domain excitation contribution is located, and a selector of the cut-off frequency as the higher frequency between said first estimate of the cut off-frequency and a last frequency of the frequency band in which said harmonic is located.

9. A mixed time-domain/frequency-domain coding device according to claim 5, wherein the calculator of frequency-domain excitation contribution comprises a calculator of a difference between the frequency representation of the LP residual and a frequency representation of the time-domain excitation contribution up to the cut-off frequency to form a first portion of a difference vector.

10. A mixed time-domain/frequency-domain coding device according to claim 9, comprising a downscale factor applied to the frequency representation of the time-domain excitation contribution in a determined frequency range following the cut-off frequency to form a second portion of the difference vector.

11. A mixed time-domain/frequency-domain coding device according to claim 10, wherein the difference vector is formed by the frequency representation of the LP residual for a third remaining portion above the determined frequency range.

12. A mixed time-domain/frequency-domain coding device according to claim 9, comprising a quantizer of the difference vector.

13. A mixed time-domain/frequency-domain coding device according to claim 12, wherein the adder adds, in the frequency domain, the quantized difference vector and a frequency-transformed version of the filtered, time-domain excitation contribution to form the mixed time-domain/frequency-domain excitation.

14. A mixed time-domain/frequency-domain coding device according to claim 2, comprising a calculator of a number of sub-frames to be used in a current frame, wherein the calculator of time-domain excitation contribution uses in the current frame the number of sub-frames determined by the sub-frame number calculator for said current frame.

15. A mixed time-domain/frequency-domain coding device according to claim 14, wherein the calculator of the number of sub-frames in the current frame is responsive to at least one of an available bit budget and a high frequency spectral dynamic of the input sound signal.

16. A mixed time-domain/frequency-domain coding device according to claim 1, comprising a calculator of a frequency transform of the time-domain excitation contribution.

17. A decoder for decoding a sound signal coded using the mixed time-domain/frequency-domain coding device of claim 16, comprising:

a converter of the mixed time-domain/frequency-domain excitation in time-domain; and

a synthesis filter for synthesizing the sound signal in response to the mixed time-domain/frequency-domain excitation converted in time-domain.

18. A decoder according to claim 17, wherein the converter uses an inverse discrete cosine transform.

19. A decoder according to claim 17, wherein the synthesis filter is a LP synthesis filter.

20. A mixed time-domain/frequency-domain coding device according to claim 1, wherein the filter comprises a zeroer of frequency bins which forces the frequency bins of a plurality of frequency bands above the cut-off frequency to zero.

21. A mixed time-domain/frequency-domain coding device according to claim 1, wherein the filter comprises a zeroer of frequency bins which forces all the frequency bins of a plurality of frequency bands to zero when the cut-off frequency is lower than a given value.

22. A mixed time-domain/frequency-domain coding device according to claim 1, wherein the adder adds the time-domain excitation contribution and the frequency-domain excitation contribution in the frequency domain.

23. A mixed, time-domain/frequency-domain coding device according to claim 1, comprising means for dynamically allocating a bit budget between the time-domain excitation contribution and the frequency-domain excitation contribution.

24. An encoder using a time-domain and frequency-domain model, comprising:

a classifier of an input sound signal as speech or non-speech;

a time-domain only coder;

the mixed time-domain/frequency-domain coding device of claim 1; and

a selector of one of the time-domain only coder and the mixed time-domain/frequency-domain coding device for coding the input sound signal depending on the classification of the input sound signal.

25. An encoder as defined in claim 24, wherein the time-domain only coder is a Code-Excited Linear Prediction coder.

26. An encoder as defined in claim 24, comprising a selector of a memory-less time-domain coding mode which, when the classifier classifies the input sound signal as non-speech and detects a temporal attack in the input sound signal, forces the memory-less time-domain coding mode for coding the input sound signal in the time-domain only coder.

27. An encoder as defined in claim 24, wherein the mixed time-domain/frequency-domain coding device uses sub-frames of a variable length in the calculation of a time-domain contribution.

28. A mixed time-domain/frequency-domain coding device for coding an input sound signal, comprising:

a calculator of a time-domain excitation contribution in response to the input sound signal, wherein the calculator of time-domain excitation contribution processes the input sound signal in successive frames of said input sound signal and comprises a calculator of a number of sub-frames to be used in a current frame of the input sound signal, wherein the sub-frame number calculator is responsive to at least one of an available bit budget and a high frequency spectral dynamic of the input sound signal and wherein the calculator of time-domain excitation contribution uses in the current frame the number of sub-frames determined by the sub-frame number calculator for said current frame;

a calculator of a frequency-domain excitation contribution in response to the input sound signal; and

an adder of the time-domain excitation contribution and the frequency-domain excitation contribution to form a mixed time-domain/frequency-domain excitation constituting a coded version of the input sound signal.

29. A decoder for decoding a sound signal coded using the mixed time-domain/frequency-domain coding device of claim 28, comprising:

a converter of the mixed time-domain/frequency-domain excitation in time-domain; and

a synthesis filter for synthesizing the sound signal in response to the mixed time-domain/frequency-domain excitation converted in time-domain.

30. A mixed time-domain/frequency-domain coding method for coding an input sound signal, comprising:

calculating a time-domain excitation contribution in response to the input sound signal;

calculating a cut-off frequency for the time-domain excitation contribution in response to the input sound signal;

in response to the cut-off frequency, adjusting a frequency extent of the time-domain excitation contribution;

calculating a frequency-domain excitation contribution in response to the input sound signal; and

adding the adjusted time-domain excitation contribution and the frequency-domain excitation contribution to form a mixed time-domain/frequency-domain excitation constituting a coded version of the input sound signal.

31. A mixed time-domain/frequency-domain coding method according to claim 30, wherein the time-domain excitation contribution includes (a) only an adaptive codebook contribution, or (b) the adaptive codebook contribution and a fixed codebook contribution.

32. A mixed time-domain/frequency-domain coding method according to claim 31, wherein calculating the time-domain excitation contribution comprises using a Code-Excited Linear Prediction coding of the input sound signal.

33. A mixed time-domain/frequency-domain coding method according to claim 32, wherein calculating the frequency-domain excitation contribution comprises calculating a difference between a frequency representation an LP residual of the input sound signal and a filtered frequency representation of the time-domain excitation contribution.

34. A mixed time-domain/frequency-domain coding method according to claim 32, wherein calculating the frequency-domain excitation contribution comprises performing a frequency transform of a LP residual obtained from an LP analysis of the input sound signal to produce a frequency representation of the LP residual.

35. A mixed time-domain/frequency-domain coding method according to claim 34, wherein calculating the cut-off frequency comprises computing a cross-correlation, for each of a plurality of frequency bands, between the frequency representation of the LP residual and a frequency representation of the time-domain excitation contribution, and the coding method comprises finding an estimate of the cut-off frequency in response to the cross-correlation.

36. A mixed time-domain/frequency-domain coding method according to claim 35, comprising smoothing the cross-correlation through the frequency bands to produce a cross-correlation vector, calculating an average of the cross-correlation vector over the frequency bands, and normalizing the average of the cross-correlation vector, wherein finding the estimate of the cut-off frequency comprises determining a first estimate of the cut-off frequency by finding a last frequency of one of the frequency bands which minimizes a difference between said last frequency and the normalized average of the cross-correlation vector multiplied by a spectrum width value.

37. A mixed time-domain/frequency-domain coding method according to claim 36, wherein calculating the cut-off frequency comprises finding one of the frequency bands in which a harmonic computed from the time-domain excitation contribution is located, and selecting the cut-off frequency as the higher frequency between said first estimate of the cut off-frequency and a last frequency of the frequency band in which said harmonic is located.

38. A mixed time-domain/frequency-domain coding method according to claim 34, wherein calculating the frequency-domain excitation contribution comprises calculating a difference between the frequency representation of the LP residual and a frequency representation of the time-domain excitation contribution up to the cut-off frequency to form a first portion of a difference vector.

39. A mixed time-domain/frequency-domain coding method according to claim 38, comprising applying a downscale factor to the frequency representation of the time-domain excitation contribution in a determined frequency range following the cut-off frequency to form a second portion of the difference vector.

40. A mixed time-domain/frequency-domain coding method according to claim 39, comprising forming the difference vector with the frequency representation of the LP residual for a third remaining portion above the determined frequency range.

41. A mixed time-domain/frequency-domain coding method according to claim 38, comprising quantizing the difference vector.

42. A mixed time-domain/frequency-domain coding method according to claim 41, wherein adding the adjusted time-domain excitation contribution and the frequency-domain excitation contribution to form the mixed time-domain/frequency-domain excitation comprises adding, in the frequency domain, the quantized difference vector and a frequency-transformed version of the adjusted, time-domain excitation contribution.

43. A mixed time-domain/frequency-domain coding method according to claim 31, comprising calculating a number of sub-frames to be used in a current frame, wherein calculating the time-domain excitation contribution comprises using in the current frame the number of sub-frames determined for said current frame.

44. A mixed time-domain/frequency-domain coding method according to claim 43, wherein calculating the number of sub-frames in the current frame is responsive to at least one of an available bit budget and a high frequency spectral dynamic of the input sound signal.

45. A mixed time-domain/frequency-domain coding method according to claim 30, comprising calculating a frequency transform of the time-domain excitation contribution.

46. A method of decoding a sound signal coded using the mixed time-domain/frequency-domain coding method of claim 45, comprising:

converting the mixed time-domain/frequency-domain excitation in time-domain; and

synthesizing the sound signal through a synthesis filter in response to the mixed time-domain/frequency-domain excitation converted in time-domain.

47. A method of decoding according to claim 46, wherein converting the mixed time-domain/frequency-domain excitation in time-domain comprises using an inverse discrete cosine transform.

48. A method of decoding according to claim 46, wherein the synthesis filter is a LP synthesis filter.

49. A mixed time-domain/frequency-domain coding method according to claim 30, wherein adjusting the frequency extent of the time-domain excitation contribution comprises zeroing frequency bins to force the frequency bins of a plurality of frequency bands above the cut-off frequency to zero.

50. A mixed time-domain/frequency-domain coding method according to claim 30, wherein adjusting the frequency extent of the time-domain excitation contribution comprises zeroing frequency bins to force all the frequency bins of a plurality of frequency bands to zero when the cut-off frequency is lower than a given value.

51. A mixed time-domain/frequency-domain coding method according to claim 30, wherein adding the adjusted time-domain excitation contribution and the frequency-domain excitation contribution to form the mixed time-domain/frequency-domain excitation comprises adding the time-domain excitation contribution and the frequency-domain excitation contribution in the frequency domain.

52. A mixed, time-domain/frequency-domain coding method according to claim 30, comprising dynamically allocating a bit budget between the time-domain excitation contribution and the frequency-domain excitation contribution.

53. A method of encoding using a time-domain and frequency-domain model, comprising:

classifying an input sound signal as speech or non-speech;

providing a time-domain only coding method;

providing the mixed time-domain/frequency-domain coding method of claim 30; and

selecting one of the time-domain only coding method and the mixed time-domain/frequency-domain coding method for coding the input sound signal depending on the classification of the input sound signal.

54. A method of encoding as defined in claim 53, wherein the time-domain only coding method is a Code-Excited Linear Prediction coding method.

55. A method of encoding as defined in claim 53, comprising selecting a memory-less time-domain coding mode which, when the input sound signal is classified as non-speech and a temporal attack in the input sound signal is detected, forces the memory-less time-domain coding mode for coding the input sound signal using the time-domain only coding method.

56. A method of encoding as defined in claim 53, wherein the mixed time-domain/frequency-domain coding method comprises using sub-frames of a variable length in the calculation of a time-domain contribution.

57. A mixed time-domain/frequency-domain coding method for coding an input sound signal, comprising:

calculating a time-domain excitation contribution in response to the input sound signal, wherein calculating the time-domain excitation contribution comprises processing the input sound signal in successive frames of said input sound signal and calculating a number of sub-frames to be used in a current frame of the input sound signal, wherein calculating the number of sub-frames in the current frame is responsive to at least one of an available bit budget and a high frequency spectral dynamic of the input sound signal and wherein calculating the time-domain excitation contribution also comprises using in the current frame the number of sub-frames calculated for said current frame;

calculating a frequency-domain excitation contribution in response to the input sound signal; and

adding the time-domain excitation contribution and the frequency-domain excitation contribution to form a mixed time-domain/frequency-domain excitation constituting a coded version of the input sound signal.

58. A method of decoding a sound signal coded using the mixed time-domain/frequency-domain coding method of claim 57, comprising:

converting the mixed time-domain/frequency-domain excitation in time-domain; and

synthesizing the sound signal through a synthesis filter in response to the mixed time-domain/frequency-domain excitation converted in time-domain.