Audio coding systems and methods using spectral component coupling and spectral component regeneration

• US 7,318,035 B2
• Filed: 05/08/2003
• Issued: 01/08/2008
• Est. Priority Date: 05/08/2003
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Truman, Michael Mead, Williams, Philip Anthony, Vernon, Stephen Decker, Andersen, Robert Loring
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): Han; Qi
• Application Number:
  US10/434,449
• Publication Number:
  US 20040225505A1
• Time in Patent Office: 1,706 Days
• Field of Search: 704/206, 704/500, 704/200.1, 704/260, 704/503
• US Class Current: 704/500
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 21/038 using band spreading techniques

Abstract

An audio encoder discards spectral components of an input signal and uses channel coupling to reduce the information capacity requirements of an encoded signal. Channel coupling represents selected spectral components of multiple channels of signals in a composite form. An audio decoder synthesizes spectral components to replace the discarded spectral components and generates spectral components for individual channel signals from the coupled-channel signal. The encoder provides scale factors in the encoded signal that improve the efficiency of the decoder to generate output signals that substantially preserve the spectral energy of the original input signals.

Claims

1. A method for encoding a plurality of input audio signals, wherein the method comprises:

receiving the plurality of input audio signals and obtaining therefrom one or more baseband signals and one or more residual signals, wherein spectral components of a baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and spectral components in an associated residual signal represent spectral components of the respective input audio signal in a second set of frequency subbands that are not represented by the baseband signal;

obtaining energy measures of at least some spectral components of one or more synthesized signals to be generated during decoding, wherein the one or more synthesized signals have spectral components within the second set of frequency subbands;

obtaining energy measures of at least some spectral components of each residual signal;

obtaining from the plurality of input audio signals a coupled-channel signal having spectral components representing a composite of spectral components of two or more of the input audio signals in a third sec of frequency subbands;

obtaining energy measures of at least some spectral components of the coupled-channel signal;

obtaining energy measures of at least some of the spectral components of the two or more input audio signals represented by the coupled-channel signal in the third set of frequency subbands;

calculating coupling scale factors by obtaining square roots of ratios of the energy measures of spectral components in the two or more input audio signals to the energy measures of spectral energy in the coupled-channel signal, square roots of ratios of the energy measures of spectral energy in the coupled-channel signal to the energy measures of spectral components in the two or more input audio signals, ratios of square roots of the energy measures of spectral components in the two or more input audio signals to square roots of the energy measures of spectral energy in the coupled-channel signal, or ratios of square roots of the energy measures of spectral energy in the coupled-channel signal to square roots of the energy measures of spectral components in the two or more input audio signals;

calculating scale factors by obtaining square roots of ratios of the energy measures of spectral components in the residual signals to the energy measures of spectral components in the one or more synthesized signals, square roots of ratios of the energy measures of spectral components in the one or more synthesized signals to the energy measures of spectral components in the residual signals, ratios of square roots of the energy measures of spectral components in the residual signals to square roots of the energy measures of spectral components in the one or more synthesized signals, or ratios of square roots of the energy measures of spectral components in the one or more synthesized signals to square roots of the energy measures of spectral components in the residual signals; and

assembling signal information and scaling information into an encoded signal, wherein the signal information represents the spectral components in the one or more baseband signals and the spectral components in the coupled-channel signal, and wherein the scaling information represents the scale factors and the coupling scale factors.

2. The method according to claim 1 wherein the one or more synthesized signals are to be generated at least in part by frequency translation of at least some of the spectral components in the one or more baseband signals.

3. The method according to claim 2 wherein the spectral components of synthesized signals are to be generated by frequency translation that maintains phase coherence.

4. The method according to claim 1 wherein the one or more synthesized signals are to be generated at least in part by a combination of a frequency translation of at least some of the spectral components in the one or more baseband signals and a generation of one or more noise-like signals having spectral levels adapted according to spectral levels in the one or more baseband signals, and wherein the energy measures of spectral components in the one or more synthesized signals is obtained without regard to spectral levels in the noise-like signals.

5. The method according to claim 1 wherein the one or more synthesized signals are to be generated at least in part by generation of one or more noise-like signals.

6. The method according to claim 1 wherein the energy measures of spectral components of the residual signals are obtained from values representing magnitudes of the spectral components.

7. The method according to claim 6 that comprises:

applying a first analysis filterbank to the plurality of input audio signals to obtain the one or more baseband signals and the one or more residual signals; and

applying a second analysis filterbank to the plurality of input audio signals to obtain second spectral components;

wherein the energy measures of spectral components in the residual signals are calculated from the spectral components of the residual signals and one or more of the second spectral components.

8. The method according to claim 1 wherein the scaling information represents the scale factors normalized with respect to one or more normalizing values, and wherein the scaling information includes a representation of the one or more normalizing values.

9. The method according to claim 8 wherein the one or more normalizing values are selected from a set of values.

10. The method according to claim 8 wherein the one or more normalizing values comprise a maximum allowable value for scale factors.

11. The method according to claim 1 that calculates a scale factor for one or more of the frequency subbands for the respective residual signals.

12. The method according to claim 11 wherein frequency extents of one or more of the sets of frequency subbands are adapted, and wherein the method assembles into the encoded signal an indication of the adapted frequency extents.

13. The method according to claim 12 wherein the frequency extents are adapted by selecting from a set of extents.

14. The method according to claim 1 wherein the one or more synthesized signals are to be generated at least in part by frequency translation of at least some of the spectral components of the input audio signals in the third set of frequency subbands.

15. The method according to claim 1 that comprises:

detecting one or more characteristics of the plurality of input audio signals;

adapting frequency extents of the first set of frequency subbands, the second set of frequency subbands, or the third set of frequency subbands in response to the detected characteristics; and

assembling into the encoded signal an indication of the adapted frequency extents.

16. The method according to claim 1 that comprises:

detecting one or more characteristics of the plurality of input audio signals;

adapting frequency extents of the first set of frequency subbands or the second set of frequency subbands in response to the detected characteristics; and

assembling into the encoded signal an indication of the adapted frequency extents.

44. A computer-readable medium recording a program of instructions executable by a computer, wherein execution of the program of instructions causes the computer to perform the method of claim 1.

60. A computer-readable medium recording encoded information representing one or more input audio signals, wherein the encoded information was generated by the method of claim 1.

61. The medium according to claim 60 wherein the one or more synthesized signals are to be generated at least in part by frequency translation of at least some of the spectral components of the input audio signals in the third set of frequency subbands.

62. The medium according to claim 60 wherein the method comprises:

detecting one or more characteristics of the one or more input audio signals;

adapting frequency extents of the first set of frequency subbands or the second set of frequency subbands in response to the detected characteristics; and

assembling into the encoded signal an indication of the adapted frequency extents.

17. An encoder for encoding a plurality of input audio signals, wherein the encoder has processing circuitry that performs a method for encoding a plurality of input audio signals that comprises:

receiving the plurality of input audio signals and obtaining therefrom one or more baseband signals and one or more residual signals, wherein spectral components of a baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and spectral components in an associated residual signal represent spectral components of the respective input audio signal in a second set of frequency subbands that are not represented by the baseband signal;

obtaining energy measures of at least some spectral components of one or more synthesized signals to be generated during decoding, wherein the one or more synthesized signals have spectral components within the second set of frequency subbands;

obtaining energy measures of at least some spectral components of each residual signal;

obtaining from the plurality of input audio signals a coupled-channel signal having spectral components representing a composite of spectral components of two or more of the input audio signals in a third set of frequency subbands;

obtaining energy measures of at least some spectral components of the coupled-channel signal;

obtaining energy measures of at least some of the spectral components of the two or more input audio signals represented by the coupled-channel signal in the third set of frequency subbands;

calculating coupling scale factors by obtaining square roots of ratios of the energy measures of spectral components in the two or more input audio signals to the energy measures of spectral energy in the coupled-channel signal, square roots of ratios of the energy measures of spectral energy in the coupled-channel signal to the energy measures of spectral components in the two or more input audio signals, ratios of square roots of the energy measures of spectral components in the two or more input audio signals to square roots of the energy measures of spectral energy in the coupled-channel signal, or ratios of square roots of the energy measures of spectral energy in the coupled-channel signal to square roots of the energy measures of spectral components in the two or more input audio signals;

calculating scale factors by obtaining square roots of ratios of the energy measures of spectral components in the residual signals to the energy measures of spectral components in the one or more synthesized signals, square roots of ratios of the energy measures of spectral components in the one or more synthesized signals to the energy measures of spectral components in the residual signals, ratios of square roots of the energy measures of spectral components in the residual signals to square roots of the energy measures of spectral components in the one or more synthesized signals, or ratios of square roots of the energy measures of spectral components in the one or more synthesized signals to square roots of the energy measures of spectral components in the residual signals; and

assembling signal information and scaling information into an encoded signal, wherein the signal information represents the spectral components in the one or more baseband signals and the spectral components in the coupled-channel signal, and wherein the scaling information represents the scale factors and the coupling scale factors.

18. A method for decoding an encoded signal representing a plurality of input audio signals, wherein the method comprises:

obtaining from the encoded signal signal information, a coupled-channel signal and scaling information,

the signal information representing spectral components for one or more baseband signals, the spectral components in each baseband signal representing spectral components of a respective input audio signal in a first set of frequency subbands,

the coupled-channel signal having spectral components representing a composite of two or more of the plurality of input audio signals in a third set of frequency subbands, and

the scaling information representing scale factors calculated from square roots of ratios of energy measures of spectral components or ratios of square roots of energy measures of spectral components, and representing coupling scale factors calculated from square roots of ratios of energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands to the energy measures of spectral energy in the coupled-channel signal, square roots of ratios of the energy measures of spectral energy in the coupled-channel signal to the energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands, ratios of square roots of the energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands to square roots of the energy measures of spectral energy in the coupled-channel signal, or ratios of square roots of the energy measures of spectral energy in the coupled-channel signal to square roots of the energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands;

generating for each respective baseband signal an associated synthesized signal having spectral components in a second set of frequency subbands that are not represented by the respective baseband signal, wherein the spectral components in the associated synthesized signal are scaled by multiplication or division according to one or more of the scale factors;

generating from the coupled-channel signal a respective decoupled signal for each of the two or more input audio signals represented by the coupled-channel signal, wherein the decoupled signals have spectral components in the third set of frequency subbands that are scaled by multiplication or division according to one or more of the coupling scale factors; and

generating a plurality of output audio signals, each output audio signal representing a respective input audio signal and generated from the spectral components in a respective baseband signal and its associated synthesized signal and from the spectral components in respective decoupled signals.

19. The method according to claim 18 wherein the associated synthesized signal is generated at least in part by frequency translation of at least some of the spectral components in the respective baseband signal.

20. The method according to claim 19 wherein the frequency translation maintains phase coherence.

21. The method according to claim 18 wherein the associated synthesized signal is generated at least in part by generation of a noise-like signal having spectral levels adapted according to one or more of the scale factors.

22. The method according to claim 18 that obtains from the encoded signal one or more normalizing values and reverses normalization of the scale factors with respect to the one or more normalizing values.

23. The method according to claim 22 wherein the one or more normalizing values are conveyed in the encoded signal by scaling information that represents selected values in a set of values.

24. The method according to claim 22 wherein the one or more normalizing values comprise a maximum allowable value for scale factors.

25. The method according to claim 18 wherein frequency subbands of the associated synthesized signal are associated with a respective scale factor.

26. The method according to claim 25 that adapts the generation of the associated synthesized signal in response to subband information conveyed in the encoded signal that specifies frequency extents of the frequency subbands.

27. The method according to claim 26 wherein the subband information represents selected frequency extents in a set of extents.

28. The method according to claim 18 wherein the associated synthesized signal is generated at least in part by frequency translation of at least some of the spectral components in the third set of frequency subbands.

29. The method according to claim 18 that comprises:

obtaining from the encoded signal an indication of frequency extents of the first, second or third sets of frequency subbands; and

adapting the generation of synthesized signals and decoupled signals in response to the indication.

30. The method according to claim 18 that comprises:

obtaining from the encoded signal an indication of frequency extents of the first or second sets of frequency subbands; and

adapting the generation of synthesized signals and decoupled signals in response to the indication.

45. A computer-readable medium recording a program of instructions executable by a computer, wherein execution of the program of instructions causes the computer to perform the method of claim 18.

46. The medium according to claim 45 wherein the associated synthesized signal is generated at least in part by frequency translation of the least some of the spectral components in the respective baseband signal.

47. The medium according to claim 46 wherein the frequency translation maintains phase coherence.

48. The medium according to claim 45 wherein the associated synthesized signal is generated at least in part by generation of a noise-like signal having spectral levels adapted according to one or more of the scale factors.

49. The medium according to claim 45 wherein the method obtains from the encoded signal one or more normalizing values and reverses normalization of the scale factors with respect to the one or more normalizing values.

50. The medium according to claim 49 wherein the one or more normalizing values are conveyed in the encoded signal by scaling information that represents selected values in a set of values.

51. The medium according to claim 49 wherein the one or more normalizing values comprise a maximum allowable value for scale factors.

52. The medium according to claim 45 wherein frequency subbands of the associated synthesized signal are associated with a respective scale factor.

53. The medium according to claim 52 wherein the method adapts the generation of the associated synthesized signal in response to subband information conveyed in the encoded signal that specifies frequency extents of the frequency subbands.

54. The medium according to claim 53 wherein the subband information represents selected frequency extents in a set of extents.

55. The medium according to claim 45 wherein the associated synthesized signal is generated at least in part by frequency translation of at least some of the spectral components in the third sat of frequency subbands.

56. The medium according to claim 45 wherein the method comprises:

obtaining from the encoded signal an indication of frequency extents of the first or second sets of frequency subbands; and

adapting the generation of synthesized signals and decoupled signals in response to the indication.

31. A decoder for decoding an encoded signal representing a plurality of input audio signals, wherein the decoder has processing circuitry that performs a method for decoding an encoded signal representing a plurality of input audio signals that comprises:

obtaining from the encoded signal signal information, a coupled-channel signal and scaling information,

the signal information representing spectral components for one or more baseband signals, the spectral components in each baseband signal representing spectral components of a respective input audio signal in a first set of frequency subbands,

the coupled-channel signal having spectral components representing a composite of two or more of the plurality of input audio signals in a third set of frequency subbands, and

the scaling information representing scale factors calculated from square roots of ratios of energy measures of spectral components or ratios of square roots of energy measures of spectral components, and representing coupling scale factors calculated from square roots of ratios of energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands to the energy measures of spectral energy in the coupled-channel signal, square roots of ratios of the energy measures of spectral energy in the coupled-channel signal to the energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands, ratios of square roots of the energy measures of spectral components of the two or more input audio signals in the third set of frequency subbands to square roots of the energy measures of spectral energy in the coupled-channel signal, or ratios of square roots of the energy measures of spectral energy in the coupled-channel signal to square roots of the energy measures of spectral components of the two or more input audio signals in the third set of frequency subband;

generating for each respective baseband signal an associated synthesized signal having spectral components in a second set of frequency subbands that are not represented by the respective baseband signal, wherein the spectral components in the associated synthesized signal are scaled by multiplication or division according to one or more of the scale factors;

generating from the coupled-channel signal a respective decoupled signal for each of the two or more input audio signals represented by the coupled-channel signal, wherein the decoupled signals have spectral components in the third set of frequency subbands that are scaled by multiplication or division according to one or more of the coupling scale factors; and

generating a plurality of output audio signals, each output audio signal representing a respective input audio signal and generated from the spectral components in a respective baseband signal and its associated synthesized signal and from the spectral components in respective decoupled signals.

32. A method for encoding a plurality of input audio signals, wherein the method comprises:

receiving the plurality of input audio signals and obtaining therefrom a plurality of baseband signals, a plurality of residual signals and a coupled-channel signal, wherein spectral components of a baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and spectral components of an associated residual signal represent spectral components of the respective input audio signal in a second set of frequency subbands that are not represented by the baseband signal, and wherein spectral components of the coupled-channel signal represent a composite of spectral components of two or more of the input audio signals in a third set of frequency subbands;

obtaining energy measures of at least some spectral components of each residual signal and the two or more input audio signals represented by the coupled-channel signal; and

assembling control information and, signal information into an encoded signal, wherein the control information is derived from the energy measures and wherein the signal information represents the spectral components in the plurality of baseband signals and the coupled-channel signal.

33. The method according to claim 32 that comprises;

obtaining energy measures of at least some spectral components of one or more synthesized signals to be generated during decoding, wherein the one or more synthesized signals have spectral components within the second set of frequency subbands; and

deriving at least some of the control information by calculating square roots of ratios of the energy measures or ratios of square roots of the energy measures.

34. The method or claim 33 wherein at least some of the spectral components of the one or more synthesized signals are to be synthesized from spectral components in the third set of frequency subbands.

35. The method according to claim 32 wherein frequency extents of the sets of frequency subbands are adapted, and wherein the method assembles into the encoded signal an indication of the adapted frequency extents.

59. A computer-readable medium recording a program of instructions executable by a computer, wherein execution of the program of instructions causes the computer to perform the method of claim 32.

63. A computer-readable medium recording encoded information representing a plurality of input audio signals, wherein the encoded information, was generated by the method of claim 32.

64. The medium according to claim 63 wherein the method comprises:

obtaining energy measures of at least some spectral components of one or more synthesized signals to be generated during decoding, wherein the one or more synthesized signals have spectral components within the second set of frequency subbands and at least some of the spectral components of the one or more synthesized signals are to be synthesized from spectral components in the third set of frequency subbands; and

deriving at least some of the control information by calculating square roots of ratios of the energy measures or ratios of square toots of the energy measures.

65. The medium according to claim 63 wherein frequency extents of the sets of frequency subbands are adapted, and wherein the method assembles into the encoded signal an indication of the adapted frequency extents.

36. A method for decoding an encoded signal representing a plurality of input audio signals, wherein the method comprises:

obtaining control information and signal information from the encoded signal, wherein the control information is derived from energy measures of spectral components and the signal information represents spectral components of a plurality of baseband signals and a coupled-channel signal, wherein the spectral components in each baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and the spectral components of the coupled-channel signal represent a composite of spectral components in a third set of frequency subbands of two or more of the plurality of input audio signals;

generating for each respective baseband signal an associated synthesized signal having spectral components in a second set of frequency subbands that are not represented by the respective baseband signal, wherein the spectral components in the associated synthesized signal are scaled according to the control information;

generating from the coupled-channel signal a respective decoupled signal for each of the two or more input audio signals represented by the coupled-channel signal, wherein the decoupled signals have spectral components in the third set of frequency subbands that are sealed according to the control information; and

generating a plurality of output audio signals, wherein each output audio signal represents a respective input audio signal and is generated from the spectral components in a respective baseband signal and its associated synthesized signal, and wherein output audio signals representing the two or more audio signals are also generated from the spectral components in the respective decoupled signals.

37. The method according to claim 36 wherein the control information conveys a representation of scale factors calculated from square roots of ratios of energy measures or ratios of square roots of the energy measures, and wherein some of the energy measures in the ratios represent energy of at least some spectral components of the synthesized signals.

38. The method of claim 37 wherein at least some of the spectral components of the one or more synthesized signals are synthesized from spectral components in the third set of frequency subbands.

39. The method according to claim 36 wherein frequency extents of one or more of the sets of frequency subbands are adapted in response to the control information.

40. A computer-readable medium recording a program of instructions executable by a computer, wherein execution of the program of instructions causes the computer to perform the method of claim 36.

41. The medium according to claim 40 wherein the control information conveys a representation of scale factors calculated from square roots of ratios of energy measures or ratios of square roots of the energy measures, and wherein some of the energy measures in the ratios represent energy of at least some spectral components of the synthesized signals.

42. The medium according to claim 41 wherein at least some of the spectral components of the one or more synthesized signals are synthesized from spectral components in the third set of frequency subbands.

43. The medium according to claim 40 wherein frequency extents of one or more of the sets of frequency subbands are adapted in response to the control information.

57. An encoder for encoding a plurality of input audio signals, wherein the encoder has processing circuitry that performs a method for encoding a plurality of input audio signals that comprises:

receiving the plurality of input audio signals and obtaining therefrom a plurality of baseband signals, a plurality of residual signals and a coupled-channel signal, wherein spectral components of a baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and spectral components of an associated residual signal represent spectral components of the respective input audio signal in a second set of frequency subbands that are not represented by the baseband signal, and wherein spectral components of the coupled-channel signal represent a composite of spectral components of two or more of the input audio signals in a third set of frequency subbands;

obtaining energy measures of at least some spectral components of each residual signal and the two or more input audio signals represented by the coupled-channel signal; and

assembling control information and signal information into an encoded signal, wherein the control information is derived from the energy measures and wherein the signal information represents the spectral components in the plurality of baseband signals and the coupled-channel signal.

58. A decoder for decoding an encoded signal representing a plurality of input audio signals, wherein the decoder has processing circuitry that performs a method for decoding an encoded signal representing a plurality of input audio signals that comprises;

obtaining control information and signal information from the encoded signal, wherein the control information is derived from energy measures of spectral components and the signal information represents spectral components of a plurality of baseband signals and a coupled-channel signal, wherein the spectral components in each baseband signal represent spectral components of a respective input audio signal in a first set of frequency subbands and the spectral components of the coupled-channel signal represent a composite of spectral components in a third set of frequency subbands of two or more of the plurality of input audio signals;

generating for each respective baseband signal an associated synthesized signal having spectral components in a second net of frequency subbands that are not represented by the respective baseband signal, wherein the spectral components in the associated synthesized signal are scaled according to the control information;

generating from the coupled-channel signal a respective decoupled signal for each of the two or more input audio signals represented by the coupled-channel signal, wherein the decoupled signals have spectral components in the third set of frequency subbands that are scaled according to the control information; and

generating a plurality of output audio signals, wherein each output audio signal represents a respective input audio signal and is generated from the spectral components in a respective baseband signal and its associated synthesized signal, and wherein output audio signals representing the two or more audio signals are also generated from the spectral components in the respective decoupled signals.