Unified lossy and lossless audio compression

• US 20040044521A1
• Filed: 07/14/2003
• Published: 03/04/2004
• Est. Priority Date: 09/04/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Wei-Ge, He, Chao
• Granted Patent: US 7,424,434 B2
• Time in Patent Office: Days
• US Class Current: 704/200.1
• CPC Class Codes: G10L 19/0017 Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error G10L19/24 takes precedence
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/06 Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A unified lossy and lossless audio compression scheme combines lossy and lossless audio compression within a same audio signal. This approach employs mixed lossless coding of a transition frame between lossy and lossless coding frames to produce seamless transitions. The mixed lossless coding performs a lapped transform and inverse lapped transform to produce an appropriately windowed and folded pseudo-time domain frame, which can then be losslessly coded. The mixed lossless coding also can be applied for frames that exhibit poor lossy compression performance.

Claims

1. In an audio encoder, a method of encoding an input audio signal, the method comprising:

receiving information input by a user of the audio encoder designating one or more portions of the input audio signal for lossless compression;

encoding a portion of the input audio signal other than the designated portion using lossy coding; and

encoding the designated portions using lossless coding.

2. The method of claim 1 comprising:

encoding a transition portion of the input audio signal between the portion encoded using the lossy coding and one of the portions encoded using lossless coding using lossless coding with part rectangular and part non-rectangular windowing of the transition portion.

3. The method of claim 2 wherein said lossy coding is based on a lapped transform, and encoding the transition portion comprises:

applying the lapped transform used in the lossy coding to the transition portion;

applying an inverse of the lapped transform; and

encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

4. The method of claim 2 wherein encoding the transition portion comprises:

applying a processing equivalent of a lapped transform used in the lossy coding and inverse lapped transform to the transition portion; and

encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

5. The method of claim 1 wherein said lossy coding is based on a lapped transform with overlapping non-rectangular windowing, and said lossless coding is based on linear predictive coding using an adaptive filter with rectangular windowing.

6. A computer-readable program-carrying medium having computer-executable instructions carried thereon for an audio encoding program, the audio encoding program being executable on a computer to perform a method of encoding an input audio signal, the method comprising:

receiving information input by a user of the audio encoder designating a portion of the input audio signal for lossless compression;

encoding a portion of the input audio signal other than the designated portion using lossy coding; and

encoding the designated portion using lossless coding.

7. The computer-readable program-carrying medium of claim 6 comprising:

encoding a transition portion of the input audio signal between the portion encoded using lossy coding and the portion encoded using lossless coding using lossless coding with non-rectangular windowing of the transition portion.

8. The computer-readable program-carrying medium of claim 7 wherein said lossy coding is based on a lapped transform, and encoding the transition portion comprises:

applying the lapped transform used in the lossy coding to the transition portion;

applying an inverse of the lapped transform; and

encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

9. The computer-readable program-carrying medium of claim 7 wherein encoding the transition portion comprises:

applying a processing equivalent of a lapped transform used in the lossy coding and inverse lapped transform to the transition portion; and

encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

10. The computer-readable program-carrying medium of claim 6 wherein said lossy coding is based on a lapped transform with overlapping non-rectangular windowing, and said lossless coding is based on linear predictive coding using an adaptive filter with rectangular windowing.

11. A method for encoding an input audio signal using unified lossy and lossless compression in accordance with a user designation of at least a portion of an audio signal to be encoded in one of a lossy compression mode or a lossless compression mode, the method comprising:

for a frame of the input audio signal, determining responsive to the user designation whether to encode the frame in the lossy compression mode or the lossless compression mode;

if in the lossy compression mode, encoding the frame using a lossy coding with a non-rectangular windowing;

if in the lossless compression mode, encoding the frame using a lossless coding with a rectangular windowing;

except, encoding a frame between frames encoded using the lossy coding and the lossless coding using lossless coding with a transition windowing.

12. The method of claim 11 further comprising, if in the lossy compression mode, encoding the frame using a better performing of the lossy coding with the non-rectangular windowing or a mixed lossless coding with the same windowing.

13. The method of claim 11 further comprising, if in the lossy compression mode,

determining whether the lossy coding with the non-rectangular windowing of the frame would meet a compression performance criteria;

if the criteria is met, encoding the frame using the lossy coding with the non-rectangular windowing; and

otherwise, determining which of the lossy coding with the non-rectangular windowing or a mixed lossless coding with the same windowing achieves better compression performance for the frame and encoding the frame using the better performing coding for the frame.

14. The method of claim 11 further comprising, if in the lossless compression mode,

determining whether the frame begins or ends a set of consecutive frames to be encoded using the lossless coding with a rectangular windowing;

if so, encoding the frame using the lossless coding with a transition windowing;

otherwise, encoding the frame using the lossless coding with a rectangular windowing.

15. A computer-readable program carrying medium having a program carried thereon for causing a computer to execute a method for encoding an input audio signal using unified lossy and lossless compression in accordance with a user designation of a portion of an audio signal to be encoded in one of a lossy compression mode or a lossless compression mode, the method comprising:

for a frame of the input audio signal, determining responsive to the user designation whether to encode the frame in the lossy compression mode or the lossless compression mode;

if in the lossy compression mode, encoding the frame using a lossy coding with a non-rectangular windowing;

if in the lossless compression mode, encoding the frame using a lossless coding with a rectangular windowing;

except, encoding a frame between frames encoded using the lossy coding and the lossless coding using mixed lossless coding with a transition windowing.

16. The computer-readable program-carrying medium of claim 15 further comprising, if in the, lossy compression mode, encoding the frame using a better performing of the lossy coding with the non-rectangular windowing or mixed lossless coding with the same windowing.

17. The computer-readable program-carrying medium of claim 15 further comprising, if in the lossy compression mode,

determining whether the lossy coding with the non-rectangular windowing of the frame would meet a compression performance criteria;

if the criteria is met, encoding the frame using the lossy coding with the non-rectangular windowing; and

otherwise, determining which of the lossy coding with the non-rectangular windowing or mixed lossless coding with the same windowing achieves better compression performance for the frame and encoding the frame using the better performing coding for the frame.

18. The computer-readable program-carrying medium of claim 15 further comprising, if in the lossless compression mode,

determining whether the frame begins or ends a set of consecutive frames to be encoded using the lossless coding with a rectangular windowing;

if so, encoding the frame using the mixed lossless coding with a transition windowing;

otherwise, encoding the frame using the lossless coding with a rectangular windowing.

26. The method of claim 15 wherein said the time domain based codec is a codec which is not based on a transform that converts the signal from time domain to frequency domain.

27. The method of claim 15 wherein said the non-lapped-transform based codec is based on a non-overlapping transform that converts the signal from time domain to frequency domain.

28. The method of claim 15 wherein said the non-lapped-transform based codec is a time domain codec.

29. The method of claim 15 wherein the transition windowing function is half rectangular and half non-rectangular, the rectangular half connecting with the first part of the signal that is compressed by the non-lapped transform based codec and the non-rectangular half connecting with the second part of the signal that is compressed by the lapped transform based codec.

19. A method of audio signal encoding unifying lossy and lossless compression of separate portions of an audio signal, the method comprising:

encoding at least a designated portion of the audio signal using lossless compression with rectangular-shaped windowing of coding frames within the designated portion;

encoding other portions of the audio signal about the designated portion using lossy compression with non-rectangular shaped windowing of coding frames within the portions; and

encoding transition frames between the designated portion and the other portions of the audio signal, wherein encoding the transition frames comprises;

performing a part rectangular, part non-rectangular shaped windowing of a coding frame for a respective transition frame; and

losslessly compressing the windowed coding frame for the respective transition frame.

20. The method of claim 19 wherein the non-rectangular shaped windowing is sine shaped windowing, and the part rectangular, part non-rectangular shaped windowing is part sine shaped and part rectangular windowing.

21. The method of claim 19 wherein said lossy compression is a transform based lossy coding and said encoding the transition frames further comprises processing the windowed coding frame to effect the transform and an inverse of the transform on the windowed coding frame.

22. The method of claim 21 wherein the transform is a lapped transform.

23. The method of claim 22 wherein the lapped transform is the modulated discrete cosine transform.

24. A method using a pseudo-time domain signal for a smooth coding transit from a lapped transform based codec to a non-lapped-transform based codec or vice versa, the method comprising:

applying the lapped transform based codec with overlapping non-rectangular windowing function on a first part of a signal;

applying a transition windowing function on a transition part of the signal between the first part and a second part;

applying a lapped transform and its inverse transform on the transition part of the signal to generate the pseudo time domain signal;

compressing the pseudo-time domain signal using a time domain codec; and

applying the non-lapped-transform based codec with a non-overlapping rectangular windowing function on the second part of the signal.

25. The method of claim 24 further comprising reversing the order of applying the lapped transform based codec and applying the non-lapped transform based codec to transit back to the lapped transform based codec.

30. The method of claim 25 for further connecting two different overlapped transform based codecs seamlessly and smoothly in a single bit stream, the method further comprising:

transiting the coding from the first lapped transform based codec to non-lapped-transform based codec via the actions recited in claim 24;

applying a non-lapped-transform based codec for a short time;

transiting the coding from the non-lapped-transform based codec to the second lapped transform based codec via the actions recited in claim 25.

31. The method of claim 30 for further applying multiple different lapped transform based codecs and non-lapped-transform based codecs on different parts of the signal and seamlessly and efficiently generating a single unified bit stream, the method comprising:

transiting from the lapped transform based codec to the non-lapped-transform based codec via the actions recited in claim 24;

transiting from the non-lapped-transform based codec to the lapped transform based codec again via the actions recited in claim 25;

transiting from the non-lapped-transform based codec to another non-lapped-transform based codec directly; and

transiting from the lapped transform based codec to another lapped transform based codec via the non-lapped-transform based codec via the actions recited in claim 30.

32. An audio encoder for encoding an input audio signal using unified lossy and lossless compression in accordance with a user designation of a portion of an audio signal to be encoded in one of a lossy compression mode or a lossless compression mode, the audio encoder comprising:

means for determining, for a frame of the input audio signal, whether to encode the frame in the lossy compression mode or the lossless compression mode responsive to the user designation;

a lossy compressor for encoding the frame using a lossy coding with a non-rectangular windowing if in the lossy compression mode;

a lossless compressor for encoding the frame using a lossless coding with a rectangular windowing if in the lossless compression mode;

a transition frame encoder for encoding a frame between frames encoded using the lossy coding and the lossless coding using lossless coding with a transition windowing.

33. The audio encoder of claim 32 further comprising the lossy compressor encoding the frame, if in the lossy compression mode, using a better performing of the lossy coding with the non-rectangular windowing or mixed lossless coding with the same windowing.

34. The audio encoder of claim 32 further comprising:

means for determining, if in the lossy compression mode, whether the lossy coding with the non-rectangular windowing of the frame would meet a compression performance criteria;

means for determining which of the lossy coding with the non-rectangular windowing or mixed lossless coding with the same windowing achieves better compression performance for the frame if the criteria is not met; and

the lossy compressor operating to encode the frame using the lossy coding with the non-rectangular windowing if the criteria is met or the lossy coding achieves better compression performance for the frame; and

the transition frame encoder operating to encode the frame using the better performing coding for the frame if the criteria is not met and the mixed lossless coding achieves better compression performance for the frame.

35. The audio encoder of claim 32 further comprising, if in the lossless compression mode,

means for determining whether the frame begins or ends a set of consecutive frames to be encoded using the lossless coding with a rectangular windowing;

the transition frame encoder operating to encode the frame using the mixed lossless coding with a transition windowing if the frame begins or ends the set;

the lossless compressor otherwise operating to encode the frame.

36. A unified lossy and lossless compressor comprising:

a lossless signal encoder for encoding at least a designated portion of the audio signal using lossless compression with rectangular-shaped windowing of coding frames within the designated portion;

a lossy signal encoder for encoding two other portions of the audio signal about the designated portion using lossy compression with non-rectangular shaped windowing of coding frames within the portions; and

a transition encoder for encoding transition frames between the designated portion and the two other portions of the audio signal, wherein encoding the transition frames comprises performing a part rectangular, part non-rectangular shaped windowing of a coding frame for a respective transition frame, and losslessly compressing the windowed coding frame for the respective transition frame.

37. The unified lossy and lossless compressor of claim 36 wherein the non-rectangular shaped windowing is sine shaped windowing, and the part rectangular, part non-rectangular shaped windowing is part sine shaped and part rectangular windowing.

38. The unified lossy and lossless compressor of claim 36 wherein said lossy compression is a transform based lossy coding and said encoding of the transition frames in the transition encoder further comprises processing the windowed coding frame to effect the transform and an inverse of the transform on the windowed coding frame.

39. The unified lossy and lossless compressor of claim 38 wherein the transform is a lapped transform.

40. The unified lossy and lossless compressor of claim 39 wherein the lapped transform is the modulated discrete cosine transform.

41. A computer readable medium having a program carried thereon executable on a computer to perform a method of audio signal encoding unifying lossy and lossless compression of separate portions of an audio signal, the method comprising:

encoding at least a designated portion of the audio signal using lossless compression with rectangular-shaped windowing of coding frames within the designated portion;

encoding two other portions of the audio signal about the designated portion using lossy compression with non-rectangular shaped windowing of coding frames within the portions; and

encoding transition frames between the designated portion and the two other portions of the audio signal, wherein encoding the transition frames comprises;

performing a part rectangular, part non-rectangular shaped windowing of a coding frame for a respective transition frame; and

losslessly compressing the windowed coding frame for the respective transition frame.

42. The computer readable medium of claim 41 wherein the non-rectangular shaped windowing is sine shaped windowing, and the part rectangular, part non-rectangular shaped windowing is part sine shaped and part rectangular windowing.

43. The computer readable medium of claim 41 wherein said lossy compression is a transform based lossy coding and said encoding the transition frames further comprises processing the windowed coding frame to effect the transform and an inverse of the transform on the windowed coding frame.

44. The computer readable medium of claim 43 wherein the transform is a lapped transform.

45. The computer readable medium of claim 44 wherein the lapped transform is the modulated discrete cosine transform.

46. A digital signal processor comprising:

an input for receiving a user designation of a portion of the input signal for lossless compression;

a lossy encoder for encoding a portion of the input signal other than the designated portion using lossy coding; and

a lossless encoder for encoding the designated portion using lossless coding.

47. The digital signal processor of claim 46 comprising:

a mixed lossless encoder for encoding a transition portion of the input signal between the portion encoded using lossy coding and the portion encoded using lossless coding using lossless coding with part rectangular and part non-rectangular windowing of the transition portion.

48. The digital signal processor of claim 47 wherein said lossy coding is based on a lapped transform, and the mixed lossless encoder comprises:

a signal transform unit for applying the lapped transform used in the lossy coding to the transition portion, and applying an inverse of the lapped transform; and

a lossless encoder unit for encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

49. The digital signal processor of claim 47 wherein the mixed lossless encoder comprises:

a signal transform unit for applying a processing equivalent of a lapped transform used in the lossy coding and inverse lapped transform to the transition portion; and

a lossless encoder unit for encoding pseudo-time domain results of the lapped transform and inverse lapped transform using lossless coding.

50. The digital signal processor of claim 46 wherein said lossy coding is based on a lapped transform with overlapping non-rectangular windowing, and said lossless coding is based on linear predictive coding using an adaptive filter with rectangular windowing.