System and method for embedded audio coding with implicit auditory masking

• US 20030187634A1
• Filed: 03/28/2002
• Published: 10/02/2003
• Est. Priority Date: 03/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jin
• Granted Patent: US 7,110,941 B2
• Time in Patent Office: Days
• US Class Current: 704/200.1
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders

Abstract

The embedded audio coder (EAC) is a fully scalable psychoacoustic audio coder which uses a novel perceptual audio coding approach termed “implicit auditory masking” which is intermixed with a scalable entropy coding process. When encoding and decoding an audio file using the EAC, auditory masking thresholds are not sent to a decoder. Instead, the masking thresholds are automatically derived from already coded coefficients. Furthermore, in one embodiment, rather than quantizing the audio coefficients according to the auditory masking thresholds, the masking thresholds are used to control the order that the coefficients are encoded. In particular, in this embodiment, during the scalable coding, larger audio coefficients are encoded first, as the larger components are the coefficients that contribute most to the audio energy level and lead to a higher auditory masking threshold.

Claims

1. A method for coding audio data comprising of using a computing device to:

transform an audio input to produce at least one set of transform coefficients;

split separate bits representing transform coefficients into at least one embedded coding unit (ECU);

set an initial auditory masking threshold; and

sequentially entropy encode each ECU, wherein a first ECU is encoded using the initial masking threshold, and each subsequent entropy encoded ECU is entropy encoded using an auditory masking threshold which is automatically derived from a previously encoded coefficient.

2. The method of claim 1 wherein the audio input is transformed using a modulated lapped transform to produce the at least one set of transform coefficients.

3. The method of claim 1 wherein the audio input is transformed using wavelet transforms to produce the at least one set of transform coefficients.

4. The method of claim 1 wherein the initial auditory masking threshold is set to a quiet threshold of a human psychoacoustic masking model.

5. The method of claim 1 wherein the initial auditory masking threshold is set to a predetermined constant value.

6. The method of claim 1 wherein the audio input is multiplexed prior to transforming the audio to provide at least one separate audio channel.

7. The method of claim 6 wherein transforming the audio input comprises individually transforming each separate audio channel.

8. The method of claim 1 wherein each ECU consists of one bit of a transform coefficient.

9. The method of claim 1 wherein each ECU consists of more than one bit of a transform coefficient.

10. The method of claim 1 wherein each ECU is individually entropy encoded in order of each ECU'"'"'s overall contribution to perceptual audio quality, with those ECUs providing a greater contribution to perceptual audio quality being encoded prior to those ECUs providing a lesser contribution to perceptual audio quality.

11. The method of claim 1 wherein the set of transform coefficients is automatically split into at least two critical bands.

12. The method of claim 11 wherein each ECU consists of bits of a same sub-bitplane of the same critical band.

13. The method of claim 1 wherein each ECU is automatically reordered prior to entropy encoding, and wherein the reordering ensures that those ECUs providing a greater contribution to perceptual audio quality are encoded prior to those ECUs providing a lesser contribution to perceptual audio quality.

14. The method of claim 1, wherein

the transform coefficients are split into at least two sections;

the bits of each section of coefficients are further split into at least one ECUs, which are sequentially encoded; and

a compressed bitstream of the sections is assembled according to each section'"'"'s overall contribution to perceptual audio quality.

15. The method of claim 11, wherein sequentially entropy encoding ECUs further comprises performing the following steps:

a. calculate a maximum bitplane for all audio coefficients;

b. set progress indicators for all critical bands to a predicted insignificance sub-bitplane of the maximum bitplane,

c. determine a next ECU to be encoded by calculating a gap between each progress indicator and the masking threshold of critical band, with the smallest gaps among all critical bands representing a current gap, and choosing the critical band with a gap value the same as the current gap to be encoded, and choosing the ECU to be the one in the chosen critical band, with a sub-bitplane pointed to by the progress indicator,

d. encode the ECU by encoding individual bits using a context sensitive entropy coder,

e. update the progress indicator to identify a next sub-bitplane to be encoded,

f. update the masking threshold based on the already coded audio coefficients if the progress indicator has reached a predetermined checkpoint,

g. determine whether a predetermined end criteria has been met, and

h. iteratively repeat steps (b) through (g) until the predetermined end criterion is reached.

16. The method of claim 11 wherein automatically deriving the auditory masking threshold from a previously encoded coefficient comprises:

calculating an adjusted energy value for each critical band;

calculating an intra-band masking threshold from the adjusted energy value; and

calculating a combined masking threshold from the intra-band masking thresholds of individual critical bands for deriving the auditory masking threshold.

17. The method of claim 16, wherein the calculation of the adjusted energy value of each critical band is accomplished by:

initializing the adjusted energy value of each critical band to zero;

performing one incremental operation per significant bit ‘

1’

encoded;

performing one shift, one decrement and one addition operation per refinement bit ‘

1’

encoded; and

performing one shift operation per entire bitplane of the critical band have been encoded.

18. The method of claim 16, wherein the calculation of the intra-band masking threshold of the critical band from the adjusted energy value is accomplished by one logarithm and two addition operations.

19. The method of claim 16, wherein the calculation of the combined masking threshold from the intra-band masking thresholds of individual critical band is achieved by a set of maximum operations.

20. The method of claim 1 wherein the auditory masking threshold which is automatically derived from the previously encoded coefficient is updated only after a predetermined checkpoint has been reached.

21. The method of claim 2 wherein the modulated lapped transform is a fully reversible modulated lapped transform with integer calculation, and wherein the entropy encoding of the audio data is lossless.

22. The method of claim 1 wherein the encoded ECUs of each set of coefficients is assembled into an assembled bitstream.

23. The method of claim 22 further comprising streaming the assembled bitstream from a server computer to a remote client computer.

24. The method of claim 22 wherein the assembled bitstream is decoded by automatically deriving auditory masking thresholds directly from the encoded coefficients in the assembled bitstream without the use of an auditory mask and performing a reverse transform on the encoded coefficients using the automatically derived auditory masking thresholds to generate decoded audio components.

25. The method of claim 24 wherein the decoded audio components are combined to generate a decoded copy of the encoded audio data.

26. The method of claim 1 wherein sequentially entropy encoding ECUs continues until all bits of all coefficients have been encoded.

27. The method of claim 1 wherein sequentially entropy encoding ECUs continues until a predetermined coding bitrate has been reached.

28. The method of claim 1 wherein sequentially entropy encoding ECUs continues until a predetermined coding quality has been reached.

29. A system for psychoacoustic audio coding comprising:

transforming at least one channel of audio data to produce at least one set of transform coefficients;

setting an initial auditory masking threshold;

dividing bits of each transform coefficient into at least one coding group; and

sequentially entropy encoding each coding group, wherein each coding group is entropy encoded using an auditory masking threshold which is sequentially derived from a previously encoded coding group, beginning with a first entropy encoded coding group that is entropy encoded using the initial masking threshold.

30. The system of claim 29 wherein the at least one channel of audio data is transformed using a modulated lapped transform to produce the at least one set of transform coefficients.

31. The system of claim 29 wherein the at least one channel of audio data is transformed using wavelet transforms to produce the at least one set of transform coefficients.

32. The system of claim 31 wherein the initial auditory masking threshold is set to a quiet threshold of a human psychoacoustic masking model.

33. The system of claim 31 wherein the initial auditory masking threshold is set to a predetermined constant value.

34. The system of claim 29 wherein the audio data is multiplexed prior to transforming the at least one channel to provide separate audio channels to be transformed.

35. The system of claim 29 wherein each encoded coding group is automatically assembled into a bitstream as it is entropy encoded.

36. The system of claim 29 wherein each coding group consists of at least one bit of a transform coefficient.

37. The system of claim 29 wherein each coding group is individually entropy encoded in order of each coding groups overall contribution to perceptual audio quality, with those coding groups providing a greater contribution to perceptual audio quality being encoded prior to those coding groups providing a lesser contribution to perceptual audio quality.

38. The system of claim 30 wherein the modulated lapped transform is a fully reversible modulated lapped transform with integer calculation, and wherein the entropy encoding of the audio data is lossless.

39. The system of claim 29 wherein each coefficient is automatically split into at least two sections prior to entropy encoding, with each section representing a predetermined portion of a frequency spectrum.

40. The system of claim 29 wherein each coefficient is split into a number of auditory critical bands, and wherein each critical band is separately entropy encoded using the automatically derived coefficients.

41. The system of claim 40 wherein at least one critical band of a coefficient is not encoded where the critical band of the coefficient would not produce a perceptual improvement in audio quality.

42. The system of claim 29 wherein sequentially entropy encoding at least one coding group continues until all coefficients have been encoded.

43. The system of claim 29 wherein sequentially entropy encoding at least one coding group continues until a predetermined coding bitrate has been reached.

44. The system of claim 29 wherein sequentially entropy encoding at least one coding group continues until a predetermined coding quality has been reached.

45. The system of claim 29 wherein the auditory masking threshold includes temporal audio masking.

46. The system of claim 40 further comprising determining a significance of each auditory critical band.

47. The system of claim 46 wherein any critical band which is determined to be insignificant is not encoded.

48. The system of claim 40 wherein a half harmonic of each coefficient is used to determine whether the coefficient is significant.

49. A system for entropy coding an audio signal, comprising:

transforming at least one channel of audio data to produce at least one set of transform coefficients; and

entropy coding at least one transform coefficient using half harmonic frequency components of the at least one transform coefficient to determine a context for significant identification of at least one bit of at least one transform coefficient.

50. The system of claim 49 wherein at least one neighborhood context lookup table is used increase a speed of the entropy coding by providing a significant identification based on the half harmonic frequency components.

51. A computer-implemented process for decoding audio data encoded using psychoacoustic masking, comprising using a computing device to:

automatically derive auditory masking thresholds directly from entropy coded coefficients in encoded audio data without explicitly receiving an auditory mask;

perform a reverse transform on the encoded coefficients to generate decoded audio components; and

combine the decoded audio components to generate a decoded copy of the encoded audio data.

52. The computer-implemented process of claim 51 wherein the encoded audio data is transmitted over a network from a server computer to at least one remote client computer.

53. The computer-implemented process of claim 51 wherein the combined audio components are demultiplexed to provide a composite audio signal.

54. A computer-readable medium having computer executable instructions for psychoacoustic encoding of audio data, said computer executable instructions comprising:

inputting an audio signal into the computer;

multiplexing the audio signal to separate individual audio channel components;

transforming each audio channel component to produce a set of coefficients for each audio channel component;

splitting bits of coefficients into at least one embedded coding unit (ECU); and

performing the following steps;

(a) initializing an entropy encoder with an initial masking threshold,

(b) determining a next ECU of the audio signal to be encoded,

(c) entropy encoding the next ECU of the audio signal,

(d) updating the initial masking threshold by automatically deriving a new masking threshold from the entropy encoded ECU that was encoded in step (c), and

(e) repeating steps (b) through (d) until a desired endpoint is reached.

55. The computer-readable medium of claim 54 wherein a bitstream representing each encoded coefficient section is automatically combined into an assembled bitstream as it is entropy encoded.

56. The computer-readable medium of claim 55 further comprising streaming the assembled bitstream from a server computer to a remote client computer.

57. The computer-readable medium of claim 55 wherein the assembled bitstream is decoded by automatically deriving auditory masking thresholds directly from the encoded coefficients in the assembled bitstream without the use of an auditory mask and performing a reverse transform on the encoded coefficients using the automatically derived auditory masking thresholds to generate decoded audio components.

58. The computer-readable medium of claim 57 wherein the decoded audio components are combined to generate a decoded copy of the encoded audio data.

59. The computer-readable medium of claim 54 wherein the desired endpoint is that all coefficients have been encoded.

60. The computer-readable medium of claim 54 wherein the desired endpoint is that a desired coding bitrate has been reached.

61. The computer-readable medium of claim 54 wherein the desired endpoint is that a desired coding quality has been reached.

62. A method for entropy encoding audio signal comprising using a computing device to:

transform the audio signal to provide at least one set of transform coefficients, each set of transform coefficients comprising a number of critical bands;

divide bits comprising each critical band of each transform coefficient into at least one coding unit;

entropy encode the coding units by using the computer to perform the following steps;

a. initialize a context sensitive entropy coder by calculating a maximum bitplane of all coefficients, setting progress indicators for all coefficients to a predicted insignificance sub-bitplane of the maximum bitplane, setting an initial masking threshold and marking all critical bands of each coefficient as insignificant,

b. determine a next critical band to be encoded by calculating a gap between each gaps progress indicator and the masking threshold, with the smallest gaps among all segments representing a current gap, and choosing the critical bands with a gap value the same as the current gap to be encoded,

c. encode the sub-bitplane of the coding unit by encoding individual bits in the chosen significant critical band using the context sensitive entropy coder,

d. update the progress indicator to identify a next sub-bitplane to be encoded,

e. update the masking threshold based on the already coded audio coefficients if the progress indicator has reached a predetermined checkpoint,

f. determine whether a predetermined end criteria has been met, and

g. iteratively repeat steps (b) through (f) until the predetermined end criterion is reached.

63. The method of claim 62 wherein the audio signal is multiplexed to provide separate audio channels prior to transforming the audio signal to provide transform coefficients.

64. The method of claim 62 wherein the initial masking threshold is set to a quiet threshold of the critical band being encoded.

65. The method of claim 62 further comprising skipping the encoding of at least one critical band where the at least one critical band is determined to be insignificant.

66. The method of claim 62 wherein each critical band is separated into individual bits, and wherein the individual bits are reordered into coding units in order of highest significance to lowest significance.

67. The method of claim 62 wherein the calculation of the auditory masking threshold is computed using an adjusted energy value.