System and method of encoding and decoding a layered bitstream by re-applying psychoacoustic analysis in the decoder

• US 6,092,041 A
• Filed: 08/22/1996
• Issued: 07/18/2000
• Est. Priority Date: 08/22/1996
• Status: Expired due to Term

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Schnurr, Otto, Pan, Davis
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): SOFOCLEOUS, MICHAEL D
• Application Number:
  US08/701,293
• Time in Patent Office: 1,426 Days
• Field of Search: 704/229, 704/230, 704/206
• US Class Current: 704/229
• CPC Class Codes: G10L 19/002 Dynamic bit allocation for perceptual audio coders G10L19/032
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding

Abstract

The invention provides a device, method (400,500,600), and system (100) to improve compression efficiency when coding audio for bitrate scalability. It includes at least one of an encoder and a decoder and is applicable when utilizing perceptual coding for an upper bitrate. The encoder includes a hybrid psychoacoustic modeling unit, coupled to receive lowband audio and diffband audio, for determining psychoacoustic data, and a quantizer control and zero-flagging unit, coupled to receive psychoacoustic data and diffband audio, for determining explicit quantizer stepsize parameters and at least one of: 1) implicit quantizer stepsize parameters and 2) implicit zero-flags. The decoder includes a lowband psychoacoustic model, coupled to receive lowband audio samples, for determining lowband psychoacoustic data, and a implicit quantizer stepsize and zero-flag computer, coupled to receive lowband psychoacoustic data for determining at least one of: 1) implicit quantizer stepsize parameters and 2) implicit zero-flags.

Claims

1. A scalable bitrate audio compression system comprising at least one of A-B:

A) an encoder, comprising;

A1) a coding delay compensation unit, coupled to receive audio samples, for providing delayed audio samples for synchronizing the audio samples with an output of a low bitrate decoding unit;

A2) a low bitrate coding unit, coupled to receive the audio samples, for coding the audio samples to provide a low bitrate audio bitstream;

A3) the low bitrate decoding unit, coupled to the low bitrate coding unit, for generating decoded lowband audio samples;

A4) a difference unit, coupled to the coding delay compensation unit and the low bitrate decoding unit, for generating diffband audio samples by subtracting the decoded lowband audio from the delayed audio samples;

A5) a time-to-frequency analysis unit, coupled to the difference unit, for generating diffband frequency coefficients;

A6) a quantizer and sample coding unit, coupled to the time-to-frequency unit and a hybrid psychoacoustic modeling and quantizer control unit, for quantizing and coding the diffband frequency coefficients to provide coded diffband frequency coefficients wherein to improve coding efficiency, lowband frequency coefficients are compared against predetermined lowband masking thresholds, lowband frequency coefficients with values below a corresponding predetermined lowband masking threshold are zero-flagged, zero-flagged lowband frequency coefficients are replaced with zero, and the quantizer and sample coding unit omits coding of zero-flagged lowband frequency coefficients when coding the diffband frequency coefficients;

A7) the hybrid psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, the difference unit and the time-to-frequency analysis unit, for providing to the bitstream coding and formatting unit and to the quantizer and sample coding unit, explicit quantizer stepsize parameters and for providing to the quantizer and sample coding unit,

A7a) implicit quantizer stepsize parameters; and

A7b) implicit zero-flags;

A8) a bitstream and coding formatting unit, coupled to the quantizer and sample coding unit, the hybrid psychoacoustic modeling and quantizer control unit and the low bitrate coding unit, for generating at least one of;

A8a) a low bitrate audio bitstream of coded lowband audio from the low bitrate coding unit; and

A8b) a supplemental audio bitstream for enhancing audio fidelity of the low bitrate audio bitstream, wherein the bitstream and coding formatting unit provides a hybrid bitstream comprising the low bitrate audio bitstream and the supplemental audio bitstream;

B) a decoder, comprising;

B1) a bitstream decoding unit, coupled to receive at least one of;

the supplemental bitstream and the low bitrate audio bitstream, for redirecting the low bitrate audio bitstream to the low bitrate decoding unit and for separating the supplemental bitstream into explicit quantizer stepsize parameters and coded diffband frequency coefficients wherein the bitstream decoding unit separates the hybrid bitstream into explicit quantizer stepsize parameters, coded diffband frequency coefficients and the low bitrate audio bitstream;

B2) a low bitrate decoding unit, coupled to receive the low bitrate audio bitstream from the bitstream decoding unit, for generating decoded lowband audio samples wherein the low bitrate decoding unit further sample rate converts the decoded bitstream to match a sample rate of the audio samples;

B3) a lowband psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, for generating;

B3a) implicit quantizer stepsize parameters; and

B3b) implicit zero-flags;

B4) a sample decoding unit and requantizer, coupled to the bitstream decoding unit and the lowband psychoacoustic modeling and quantizer control unit, for decoding and requantizing requantized diffband frequency coefficients wherein, where zero-flagging mode is selected, the sample decoding unit and requantizer reconstructs requantized diffband frequency coefficients from coded diffband frequency coefficients and explicit quantizer stepsize parameters, both from the bitstream decoding unit, and at least one of;

     1) implicit quantizer stepsize parameters; and

     2) implicit zero-flags provided by the lowband psychoacoustic modeling and quantizer control unit and reconstructs zero-flagged diffband frequency coefficients with zero values;

B5) a frequency-to-time synthesis unit, coupled to the sample decoding unit and requantizer, for converting the requantized diffband frequency coefficients into requantized diffband audio samples;

B6) a time alignment unit, coupled to the low bitrate decoding unit, for synchronizing the output of the low bitrate decoding unit with the requantized diffband audio samples;

B7) a summer, coupled to the time-to-frequency synthesis unit and the time alignment unit, for summing the time-aligned, decoded, lowband audio samples with requantized diffband audio samples to provide fullband audio samples.

2. The scalable bitrate audio compression system of claim 1 wherein the low bitrate coding unit and the low bitrate decoding units further provide additional scalable bitrates.

3. A method for using a computer processor for providing scalable bitrate audio compression parameters, comprising:

A) generating, using a decoded lowband audio signal and a diffband audio signal, by a hybrid psychoacoustic modeling unit, psychoacoustic data that is composed of at least one of;

signal-to-mask ratios, lowband frequency coefficients and lowband masking thresholds,

wherein the hybrid psychoacoustic modeling unit performs scalable bitrate audio compression using the steps of at least one of A1-A2;

A1) in an encoder;

A1a) using a coding delay compensation unit for providing delayed audio samples for synchronizing the audio samples with an output of a low bitrate decoding unit;

A1b) using a low bitrate coding unit for coding the audio samples to provide a low bitrate audio bitstream;

A1c) using the low bitrate decoding unit for generating decoded lowband audio samples;

A1d) using a difference unit for generating diffband audio samples by subtracting the decoded lowband audio from the delayed audio samples;

A1e) using a time-to-frequency analysis unit for generating diffband frequency coefficients;

A1f) using a quantizer and sample coding unit for quantizing and coding the diffband frequency coefficients to provide coded diffband frequency coefficients wherein, where zero-flagging is implemented to improve coding efficiency, lowband frequency coefficients are compared against predetermined lowband masking thresholds, lowband frequency coefficients with values below a corresponding predetermined lowband masking threshold are zero-flagged, zero-flagged lowband frequency coefficients are replaced with zero, and the quantizer and sample coding unit omits coding of zero-flagged lowband frequency coefficients when coding the diffband frequency coefficients;

A1g) using a hybrid psychoacoustic modeling and quantizer control unit for providing to the bitstream coding and formatting unit and to the quantizer and sample coding unit, explicit quantizer stepsize parameters and for providing to the quantizer and sample coding unit,

A1g1) implicit quantizer stepsize parameters; and

A1g2) implicit zero-flags;

A1h) using a bitstream and coding formatting unit for generating at least one of;

A1h1) a low bitrate audio bitstream of coded lowband audio from the low bitrate coding unit; and

A1h2) a supplemental audio bitstream for enhancing audio fidelity of the low bitrate audio bitstream, wherein the bitstream and coding formatting unit provides a hybrid bitstream comprising the low bitrate audio bitstream and the supplemental audio bitstream;

A2) in a decoder;

A2a) using a bitstream decoding unit for redirecting the low bitrate audio bitstream to the low bitrate decoding unit and for separating the supplemental bitstream into explicit quantizer stepsize parameters and coded diffband frequency coefficients wherein the bitstream decoding unit separates the hybrid bitstream into explicit quantizer stepsize parameters, coded diffband frequency coefficients and the low bitrate audio bitstream;

A2b) using a low bitrate decoding unit for generating decoded lowband audio samples wherein the low bitrate decoding unit further sample rate converts the decoded bitstream to match a sample rate of the audio samples;

A2c) using a lowband psychoacoustic modeling and quantizer control unit for generating at least one of;

A2c1) implicit quantizer stepsize parameters; and

A2c2) implicit zero-flags;

A2d) using a sample decoding unit and requantizer for decoding and requantizing requantized diffband frequency coefficients wherein, where zero-flagging mode is selected, the sample decoding unit and requantizer reconstructs requantized diffband frequency coefficients from coded diffband frequency coefficients and explicit quantizer stepsize parameters, both from the bitstream decoding unit, and

     1) implicit quantizer stepsize parameters; and

     2) implicit zero-flags provided by the lowband psychoacoustic modeling and quantizer control unit and reconstructs zero-flagged diffband frequency coefficients with zero values;

A2e) using a frequency-to-time synthesis unit for converting the requantized diffband frequency coefficients into requantized diffband audio samples;

A2f) using a time alignment unit for synchronizing the output of the low bitrate decoding unit with the requantized diffband audio samples;

A2g) using a summer for summing the time-aligned, decoded, lowband audio samples with requantized diffband audio samples to provide fullband audio samples; and

B) generating, by a quantizer control unit and zero-flagging unit, explicit quantizer stepsize parameters and at least one of;

implicit quantizer stepsize parameters and implicit zero-flags.

4. The method of claim 3 wherein the method is implemented by a computer program for providing scalable bitrate audio compression parameters, wherein the computer program is implemented/embodied in a tangible medium of at least one of:

A) a memory;

B) an application specific integrated circuit;

C) a digital signal processor; and

D) a field programmable gate array.

5. A hybrid psychoacoustic device for providing scalable bitrate audio compression parameters, wherein the hybrid psychoacoustic device includes a scalabitrate audio compression system comprising at least one of A-B:

A) an encoder, comprising;

A1) a coding delay compensation unit, coupled to receive audio samples, for providing delayed audio samples for synchronizing the audio samples with an output of a low bitrate decoding unit;

A2) a low bitrate coding unit, coupled to receive the audio samples, for coding the audio samples to provide a low bitrate audio bitstream;

A3) the low bitrate decoding unit, coupled to the low bitrate coding unit, for generating decoded lowband audio samples;

A4) a difference unit, coupled to the coding delay compensation unit and the low bitrate decoding unit, for generating diffband audio samples by subtracting the decoded lowband audio from the delayed audio samples;

A5) a time-to-frequency analysis unit, coupled to the difference unit, for generating diffband frequency coefficients;

A6) a quantizer and sample coding unit, coupled to the time-to-frequency unit and a hybrid psychoacoustic modeling and quantizer control unit, for quantizing and coding the diffband frequency coefficients to provide coded diffband frequency coefficients wherein, where zero-flagging is selected to improve coding efficiency, lowband frequency coefficients are compared against predetermined lowband masking thresholds, lowband frequency coefficients with values below a corresponding predetermined lowband masking threshold are zero-flagged, zero-flagged lowband frequency coefficients are replaced with zero, and the quantizer and sample coding unit omits coding of zero-flagged lowband frequency coefficients when coding the diffband frequency coefficients;

A7) the hybrid psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, the difference unit and the time-to-frequency analysis unit, for providing to the bitstream coding and formatting unit and to the quantizer and sample coding unit, explicit quantizer stepsize parameters and for providing to the quantizer and sample coding unit,

A7a) implicit quantizer stepsize parameters; and

A7b) implicit zero-flags;

A8) a bitstream and coding formatting unit, coupled to the quantizer and sample coding unit, the hybrid psychoacoustic modeling and quantizer control unit and the low bitrate coding unit, for generating at least one of;

A8a) a low bitrate audio bitstream of coded lowband audio from the low bitrate coding unit; and

A8b) a supplemental audio bitstream for enhancing audio fidelity of the low bitrate audio bitstream, wherein the bitstream and coding formatting unit provides a hybrid bitstream comprising the low bitrate audio bitstream and the supplemental audio bitstream;

B) a decoder, comprising;

B1) a bitstream decoding unit, coupled to receive at least one of;

the supplemental bitstream and the low bitrate audio bitstream, for redirecting the low bitrate audio bitstream to the low bitrate decoding unit and for separating the supplemental bitstream into explicit quantizer stepsize parameters and coded diffband frequency coefficients wherein the bitstream decoding unit separates the hybrid bitstream into explicit quantizer stepsize parameters, coded diffband frequency coefficients and the low bitrate audio bitstream;

B2) a low bitrate decoding unit, coupled to receive the low bitrate audio bitstream from the bitstream decoding unit;

for generating decoded lowband audio samples wherein the low bitrate decoding unit further sample rate converts the decoded bitstream to match a sample rate of the audio samples;

B3) a lowband psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, for generating;

B3a) implicit quantizer stepsize parameters; and

B3b) implicit zero-flags;

B4) a sample decoding unit and requantizer, coupled to the bitstream decoding unit and the lowband psychoacoustic modeling and quantizer control unit, for decoding and requantizing requantized diffband frequency coefficients wherein, where zero-flagging mode is selected, the sample decoding unit and requantizer reconstructs requantized diffband frequency coefficients from coded diffband frequency coefficients and explicit quantizer stepsize parameters, both from the bitstream decoding unit, and at least one of;

     1) implicit quantizer stepsize parameters; and

     2) implicit zero-flags provided by the lowband psychoacoustic modeling and quantizer control unit and reconstructs zero-flagged diffband frequency coefficients with zero values;

B5) a frequency-to-time synthesis unit, coupled to the sample decoding unit and requantizer, for converting the requantized diffband frequency coefficients into requantized diffband audio samples;

B6) a time alignment unit, coupled to the low bitrate decoding unit, for synchronizing the output of the low bitrate decoding unit with the requantized diffband audio samples;

B7) a summer, coupled to the time-to-frequency synthesis unit and the time alignment unit, for summing the time-aligned, decoded, lowband audio samples with requantized diffband audio samples to provide fullband audio samples.

6. A computer having a hybrid psychoacoustic device for providing scalable bitrate audio compression parameters, wherein the hybrid psychoacoustic device includes a scalabitrate audio compression system comprising at least one of A-B:

A) an encoder, comprising;

A1) a coding delay compensation unit, coupled to receive audio samples, for providing delayed audio samples for synchronizing the audio samples with an output of a low bitrate decoding unit;

A2) a low bitrate coding unit, coupled to receive the audio samples, for coding the audio samples to provide a low bitrate audio bitstream;

A3) the low bitrate decoding unit, coupled to the low bitrate coding unit, for generating decoded lowband audio samples;

A4) a difference unit, coupled to the coding delay compensation unit and the low bitrate decoding unit, for generating diffband audio samples by subtracting the decoded lowband audio from the delayed audio samples;

A5) a time-to-frequency analysis unit, coupled to the difference unit, for generating diffband frequency coefficients;

A6) a quantizer and sample coding unit, coupled to the time-to-frequency unit and a hybrid psychoacoustic modeling and quantizer control unit, for quantizing and coding the diffband frequency coefficients to provide coded diffband frequency coefficients wherein to improve coding efficiency, lowband frequency coefficients are compared against predetermined lowband masking thresholds, lowband frequency coefficients with values below a corresponding predetermined lowband masking threshold are zero-flagged, zero-flagged lowband frequency coefficients are replaced with zero, and the quantizer and sample coding unit omits coding of zero-flagged lowband frequency coefficients when coding the diffband frequency coefficients;

A7) the hybrid psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, the difference unit and the time-to-frequency analysis unit, for providing to the bitstream coding and formatting unit and to the quantizer and sample coding unit, explicit quantizer stepsize parameters and for providing to the quantizer and sample coding unit,

A7a) implicit quantizer stepsize parameters; and

A7b) implicit zero-flags;

A8) a bitstream and coding formatting unit, coupled to the quantizer and sample coding unit, the hybrid psychoacoustic modeling and quantizer control unit and the low bitrate coding unit, for generating at least one of;

A8a) a low bitrate audio bitstream of coded lowband audio from the low bitrate coding unit; and

A8b) a supplemental audio bitstream for enhancing audio fidelity of the low bitrate audio bitstream, wherein the bitstream and coding formatting unit provides a hybrid bitstream comprising the low bitrate audio bitstream and the supplemental audio bitstream;

B) a decoder, comprising;

B1) a bitstream decoding unit, coupled to receive at least one of;

the supplemental bitstream and the low bitrate audio bitstream, for redirecting the low bitrate audio bitstream to the low bitrate decoding unit and for separating the supplemental bitstream into explicit quantizer stepsize parameters and coded diffband frequency coefficients wherein the bitstream decoding unit separates the hybrid bitstream into explicit quantizer stepsize parameters, coded diffband frequency coefficients and the low bitrate audio bitstream;

B2) a low bitrate decoding unit, coupled to receive the low bitrate audio bitstream from the bitstream decoding unit, for generating decoded lowband audio samples wherein the low bitrate decoding unit further sample rate converts the decoded bitstream to match a sample rate of the audio samples;

B3) a lowband psychoacoustic modeling and quantizer control unit, coupled to the low bitrate decoding unit, for generating;

B3a) implicit quantizer stepsize parameters; and

B3b) implicit zero-flags;

B4) a sample decoding unit and requantizer, coupled to the bitstream decoding unit and the lowband psychoacoustic modeling and quantizer control unit, for decoding and requantizing requantized diffband frequency coefficients wherein, where zero-flagging mode is selected, the sample decoding unit and requantizer reconstructs requantized diffband frequency coefficients from coded diffband frequency coefficients and explicit quantizer stepsize parameters, both from the bitstream decoding unit, and at least one of;

     1) implicit quantizer stepsize parameters; and

     2) implicit zero-flags provided by the lowband psychoacoustic modeling and quantizer control unit and reconstructs zero-flagged diffband frequency coefficients with zero values;

B5) a frequency-to-time synthesis unit, coupled to the sample decoding unit and requantizer, for converting the requantized diffband frequency coefficients into requantized diffband audio samples;

B6) a time alignment unit, coupled to the low bitrate decoding unit, for synchronizing the output of the low bitrate decoding unit with the requantized diffband audio samples;

B7) a summer, coupled to the time-to-frequency synthesis unit and the time alignment unit, for summing the time-aligned, decoded, lowband audio samples with requantized diffband audio samples to provide fullband audio samples.