System and method for multiresolution scalable audio signal encoding

• US 5,886,276 A
• Filed: 01/16/1998
• Issued: 03/23/1999
• Est. Priority Date: 01/16/1997
• Status: Expired due to Fees

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Verma, Tony S., Levine, Scott N.
• Primary Examiner(s): Witkowski, Stanley J.
• Application Number:
  US09/007,995
• Time in Patent Office: 431 Days
• Field of Search: 84/603, 84/621, 84/661, 84/683, 84/691, 84/699, 84/700, 84/DIG. 9, 704/205-210, 704/220, 704/258-269
• US Class Current: 84/603
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 2240/051 AC3, i.e. Audio Codec 3, Dolby Digital
• CPC Class Codes: G10H 2240/066 MPEG audio-visual compression file formats, e.g. MPEG-4 for coding of audio-visual objects MP3 G10H2240/061
• CPC Class Codes: G10H 2240/295 Packet switched network, e.g. token ring circuit-switched networks, e.g. traditional analog telephone transmission G10H2240/241
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2250/031 Spectrum envelope processing
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10H 2250/545 Aliasing, i.e. preventing, eliminating or deliberately using aliasing noise, distortions or artifacts in sampled or synthesised waveforms, e.g. by band limiting, oversampling or undersampling, respectively
• CPC Class Codes: G10H 2250/571 Waveform compression, adapted for music synthesisers, sound banks or wavetables audio compression G10L19/00
• CPC Class Codes: G10H 7/105 using Fourier coefficients
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: Y10S 84/09 Filtering

Abstract

An audio signal analyzer and encoder is based on a model that considers audio signals to be composed of deterministic or sinusoidal components, transient components representing the onset of notes or other events in an audio signal, and stochastic components. Deterministic components are represented as a series of overlapping sinusoidal waveforms. To generate the deterministic components, the input signal is divided into a set of frequency bands by a multi-complementary filter bank. The frequency band signals are oversampled so as to suppress cross-band aliasing energy in each band. Each frequency band is analyzed and encoded as a set of spectral components using a windowing time frame whose length is inversely proportional to the frequency range in that band. Low frequency bands are encoded using longer time frames than higher frequency bands. Transient components are represented by parameters denoting sinusoidal shaped waveforms produced when the transient components are transformed into a real valued frequency domain waveform. Stochastic or noise components are represented as a series of spectral envelopes. The parameters representing the three signal components compose a stream of compressed encoded audio data that can be further compressed so as to meet a specified transmission bandwidth limit by the deleting the least significant bits of quantized parameter values, reducing the update rates of parameters, and/or deleting the parameters used to encode higher frequency bands until the bandwidth of the compressed audio data meets the bandwidth requirement. Signal quality degrades in a graduated manner with successive reductions in the transmitted data rate.

Claims

1. An audio signal encoder, comprising:

means for filtering a digitally sampled audio signal with a multi-complementary filter bank that splits the audio signal into a plurality of band signals, where the plurality of band signals contain contiguous frequency range portions of the audio signal and wherein the band signals are oversampled so as to suppress cross-band aliasing energy in each of the band signals; and

means for analyzing each of the band signals, using for each respective band signal a respective windowing time whose length is inversely proportional to the frequency range of the associated band signal, to identify spectral peaks within each band signal and to generate encoded parameters representing each of the identified spectral peaks.

2. The audio signal encoder of claim 1, further including:

a sinusoidal signal synthesizer for generating a set of sinusoidal waveforms corresponding to the encoded parameters generated by the band signal analyzing means;

a signal subtracter means that subtracts the set of sinusoidal waveforms from the audio signal so as to generate a residual signal; and

a transient component analyzer for analyzing and encoding transient signal components in the residual signal with a set of transient component signal parameters.

3. The audio signal encoder of claim 2, the transient component analyzer including:

a transform means for transforming frames of the residual signal into real valued frequency domain frames; and

an analyzer for identifying spectral peaks in respective ones of the frequency domain frames and encoding the identified spectral peaks so as to generate the set of transient component signal parameters for the respective ones of the frequency domain frames.

4. The audio signal encoder of claim 3, further including:

a transient signal synthesizer for generating a reconstructed transient signal from the transient component signal parameters;

a second signal subtracter for subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

a noise component encoder for generating a set of noise modeling parameters representing spectral components of the second residual signal.

5. The audio signal encoder of claim 4, further including:

means for assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

means for reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

6. The audio signal encoder of claim 5, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.

7. The audio signal encoder of claim 2, further including:

a transient signal synthesizer for generating a reconstructed transient signal from the transient component signal parameters;

a second signal subtracter for subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

a noise component encoder for generating a set of noise modeling parameters representing spectral components of the second residual signal.

8. The audio signal encoder of claim 7, further including:

means for assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

means for reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

9. The audio signal encoder of claim 8, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks in a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many data bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.

10. A method of encoding an audio signal, comprising:

filtering a digitally sampled audio signal with a multi-complementary filter bank that splits the audio signal into a plurality of band signals, where the plurality of band signals contain contiguous frequency range portions of the audio signal and wherein the band signals are oversampled so as to suppress cross-band aliasing energy in each of the band signals; and

analyzing each of the band signals, using for each respective band signal a respective windowing time whose length is inversely proportional to the frequency range of the associated band signal, to identify spectral peaks within each band signal and to generate encoded parameters representing each of the identified spectral peaks.

11. The method of claim 10, further including:

generating a set of sinusoidal waveforms corresponding to the encoded parameters representing the identified spectral peaks;

subtracting the set of sinusoidal waveforms from the audio signal so as to generate a residual signal; and

analyzing and encoding transient signal components in the residual signal with a set of transient component signal parameters.

12. The method of claim 11, the transient signal component analyzing and encoding step including:

transforming frames of the residual signal into real valued frequency domain frames; and

identifying spectral peaks in respective ones of the frequency domain frames and encoding the identified spectral peaks so as to generate the set of transient component signal parameters for the respective ones of the frequency domain frames.

13. The method of claim 12, further including:

generating a reconstructed transient signal from the transient component signal parameters;

subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

generating a set of noise modeling parameters representing spectral components of the second residual signal.

14. The method of claim 13, further including:

assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

15. The method of claim 14, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.

16. The method of claim 11, further including:

generating a reconstructed transient signal from the transient component signal parameters;

subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

generating a set of noise modeling parameters representing spectral components of the second residual signal.

17. The method of claim 16, further including:

assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

18. The method of claim 17, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks in a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many data bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.

19. A computer program product for use in conjunction with a computer system, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:

instructions for filtering a digitally sampled audio signal with a multi-complementary filter bank that splits the audio signal into a plurality of band signals, where the plurality of band signals contain contiguous frequency range portions of the audio signal and wherein the band signals are oversampled so as to suppress cross-band aliasing energy in each of the band signals; and

instructions for analyzing each of the band signals, using for each respective band signal a respective windowing time whose length is inversely proportional to the frequency range of the associated band signal, to identify spectral peaks within each band signal and to generate encoded parameters representing each of the identified spectral peaks.

20. The computer program product of claim 19 further including:

instructions for generating a set of sinusoidal waveforms corresponding to the encoded parameters generated by the band signal analyzing means;

instructions that subtract the set of sinusoidal waveforms from the audio signal so as to generate a residual signal; and

instructions for analyzing and encoding transient signal components in the residual signal with a set of transient component signal parameters.

21. The computer program product of claim 20, including:

instructions for transforming frames of the residual signal into real valued frequency domain frames; and

instructions for identifying spectral peaks in respective ones of the frequency domain frames and encoding the identified spectral peaks so as to generate the set of transient component signal parameters for the respective ones of the frequency domain frames.

22. The computer program product of claim 21, further including:

instructions for generating a reconstructed transient signal from the transient component signal parameters;

instructions for subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

noise encoding instructions for generating a set of noise modeling parameters representing spectral components of the second residual signal.

23. The audio signal encoder of claim 22, further including:

instructions for assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

instructions for reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

24. The audio signal encoder of claim 23, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.

25. The audio signal encoder of claim 20, further including:

instructions for generating a reconstructed transient signal from the transient component signal parameters;

instructions for subtracting the reconstructed transient signal from the residual signal to generate a second residual signal; and

noise encoding instructions for generating a set of noise modeling parameters representing spectral components of the second residual signal.

26. The audio signal encoder of claim 25, further including:

instructions for assembling a parameter stream from the encoded parameters representing the identified spectral peaks in the band signals, the transient component signal parameters and the noise modeling parameters; and

instructions for reducing transmission bandwidth associated with the parameter stream by performing a subset of a predefined set of bandwidth reduction actions.

27. The audio signal encoder of claim 26, wherein the predefined set of bandwidth reduction actions includes a plurality of actions selected from the set consisting of deleting from the parameter stream a subset of the encoded parameters representing the identified spectral peaks in the band signals, reducing how often the noise modeling parameters are included in the parameter stream, deleting from the parameter stream all encoded parameters representing the identified spectral peaks in a highest frequency one of the band signals, reducing how often the encoded parameters are included in the parameter stream for a second highest frequency one of the band signals, reducing how many data bits are used to represent the encoded parameters in the parameter stream, and deleting a subset of the transient component signal parameters.