ADAPTIVE TIME/FREQUENCY-BASED AUDIO ENCODING AND DECODING APPARATUSES AND METHODS

• US 20070106502A1
• Filed: 09/26/2006
• Published: 05/10/2007
• Est. Priority Date: 11/08/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Junghoe, Oh, Eunmi, Son, Changyong, Choo, Kihyun
• Granted Patent: US 8,548,801 B2
• Time in Patent Office: Days
• US Class Current: 704/207
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/12 the excitation function being a code excitation, e.g. in code excited linear prediction [CELP] vocoders
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding

Abstract

Adaptive time/frequency-based audio encoding and decoding apparatuses and methods. The encoding apparatus includes a transformation & mode determination unit to divide an input audio signal into a plurality of frequency-domain signals and to select a time-based encoding mode or a frequency-based encoding mode for each respective frequency-domain signal, an encoding unit to encode each frequency-domain signal in the respective encoding mode, and a bitstream output unit to output encoded data, division information, and encoding mode information for each respective frequency-domain signal. In the apparatuses and methods, acoustic characteristics and a voicing model are simultaneously applied to a frame, which is an audio compression processing unit. As a result, a compression method effective for both music and voice can be produced, and the compression method can be used for mobile terminals that require audio compression at a low bit rate.

Claims

1. An adaptive time/frequency-based audio encoding apparatus, comprising:

a transformation &

mode determination unit to divide an input audio signal into a plurality of frequency-domain signals and to select a time-based encoding mode or a frequency-based encoding mode for each respective frequency-domain signal;

an encoding unit to encode each frequency-domain signal in the respective encoding modes selected by the transformation &

mode determination unit; and

a bitstream output unit to output encoded data, division information, and encoding mode information for each respective encoded frequency-domain signal.

2. The apparatus of claim 1, wherein the transformation &

mode determination unit comprises;

a frequency-domain transform unit to transform the input audio signal into a full frequency-domain signal; and

an encoding mode determination unit to divide the full frequency-domain signal into the frequency-domain signals according to a preset standard and to determine the time-based encoding mode or the frequency-based encoding mode for each respective frequency-domain signal.

3. The apparatus of claim 2, wherein the full frequency-domain signal is divided into the frequency-domain signals suitable for the time-based encoding mode or the frequency-based encoding mode based on at least one of a spectral tilt, a size of signal energy of each frequency domain, a change in signal energy between sub-frames, and a voicing level determination, and the respective encoding mode for each frequency-domain signal is determined accordingly.

4. The apparatus of claim 1, wherein the encoding unit comprises:

a time-based encoding unit to perform an inverse frequency-domain transform on a first frequency-domain signal determined to be encoded in the time-based encoding mode and to perform time-based encoding on the first frequency-domain signal on which the inverse frequency-domain transform is performed; and

a frequency-based encoding unit to perform frequency-based encoding on a second frequency-domain signal determined to be encoded in the frequency-based encoding mode.

5. The apparatus of claim 4, wherein the time-based encoding unit selects the encoding mode for the first input frequency-domain signal based on at least one of a linear coding gain, a spectral change between linear prediction filters of adjacent frames, a predicted pitch delay, and a predicted long-term prediction gain, continues to perform the time-based encoding on the first frequency-domain signal when the time-based encoding unit determines that the time-based encoding mode is suitable for the first frequency-domain signal, and stops performing the time-based encoding on the first frequency-domain signal and transmits a mode conversion control signal to the transformation &

mode determination unit when the time-based encoding unit determines that the frequency-based encoding mode is suitable for the first frequency-domain signal, and the transformation &

mode determination unit outputs the first frequency-domain signal again, which was provided to the time-based encoding unit, to the frequency-based encoding unit in response to the mode conversion control signal.

6. The apparatus of claim 2, wherein the frequency-domain transform unit performs the frequency-domain transform using a frequency varying modulated lapped transform (MLT).

7. The apparatus of claim 4, wherein the time-based encoding unit quantizes a residual signal obtained from linear prediction and dynamically allocates bits to the quantized residual signal according to importance.

8. The apparatus of claim 4, wherein the time-based encoding unit transforms a residual signal obtained from a linear prediction into a frequency-domain signal, quantizes the frequency-domain signal, and dynamically allocates bits to the quantized signal according to importance.

9. The apparatus of claim 7, wherein the importance is determined based on a voicing model.

10. The apparatus of claim 8, wherein the importance is determined based on a voicing model.

11. The apparatus of claim 4, wherein the frequency-based encoding unit determines a quantization step size of an input frequency-domain signal according to a psychoacoustic model and quantizes the frequency-domain signal.

12. The apparatus of claim 4, wherein the frequency-based encoding unit extracts important frequency components from an input frequency-domain signal according to a psychoacoustic model, encodes the extracted important frequency components, and encodes remaining signals using noise modeling.

13. The apparatus of claim 8, wherein the residual signal is obtained using a code excited linear prediction (CELP) algorithm.

14. An audio data encoding apparatus, comprising:

a transformation and mode determination unit to divide a frame of audio data into first audio data and second audio data; and

an encoding unit to encode the first audio data in a time domain and to encode the second audio data in a frequency domain.

15. The apparatus of claim 14, wherein the transformation and mode determination unit divides the frame based on estimations of frequency characteristics of the first and second audio data in the frame into voice audio data and non-voice audio data, respectively.

16. The apparatus of claim 15, wherein the encoding unit performs a time based encoding operation on the voice data, determines whether the estimation of the first audio data as the voice data is accurate during the time based encoding operation by evaluating intermediate data generated during the time based encoding operation, continues the time based encoding operation on the first audio data when the estimation of the first audio data as the voice data is accurate, and stops the time based encoding operation when the estimation of the first audio data as the voice data is not accurate.

17. The apparatus of claim 16, wherein the encoding unit performs a frequency based encoding operation on the first audio data when the estimation of the first audio data as the voice data is not accurate.

18. The apparatus of claim 14, wherein the transformation and mode determination unit approximates the first audio data in a frame as voice audio data and approximates the second audio data in the frame as non-voice audio data.

19. The apparatus of claim 18, wherein the transformation and mode determination unit performs the approximation using at least one of a spectral measuring method, an energy measuring method, a long term prediction estimation method, and a voicing level determination method.

20. The apparatus of claim 14, further comprising:

a collection and inverse transform unit to transform one of the first and second audio data such that the first and second audio data are in the same one of the time and frequency domains and to combine the first and second audio data into an output bitstream.

21. The apparatus of claim 20, wherein the output bitstream comprises at least one of encoding mode information about encoding processes used to encode the first and second audio data and division information about how the first and second audio data are divided with respect to each other in the same frame.

22. The apparatus of claim 14, wherein the first and second audio data each includes one or more sub-frames that correspond to different frequency bands within a full frequency domain of the same frame.

23. An adaptive time/frequency-based audio decoding apparatus, comprising:

a bitstream sorting unit to extract encoded data for each frequency band, division information, and encoding mode information for each frequency band from an input bitstream;

a decoding unit to decode the encoded data for each frequency domain based on the division information and the respective encoding mode information; and

a collection &

inverse transform unit to collect decoded data in a frequency domain and to perform an inverse frequency-domain transform on the collected data.

24. The apparatus of claim 23, wherein the decoding unit comprises:

a time-based decoding unit to perform time-based decoding on first encoded data based on the division information and respective first encoding mode information; and

a frequency-based decoding unit to perform frequency-based decoding on second encoded data based on the division information and respective second encoding mode information.

25. The apparatus of claim 24, wherein the time-based decoding unit decodes the first encoded data using a CELP algorithm.

26. The apparatus of claim 23, wherein the collection &

inverse transform unit performs envelope smoothing on the decoded data in the frequency domain and then performs the inverse frequency-domain transform on the decoded data such that the decoded data maintains continuity in the frequency domain.

27. The apparatus of claim 23, wherein a final audio signal is generated using a frequency-varying MLT after the decoded data is collected in the frequency domain.

28. An audio data decoding apparatus, comprising:

a bitstream sorting unit to extract encoded audio data of a frame; and

a decoding unit to decode the audio data of the frame into first audio data in a time domain and second audio data in a frequency domain.

29. An adaptive time/frequency-based audio encoding method, comprising:

dividing an input audio signal into a plurality of frequency-domain signals and selecting a time-based encoding mode or a frequency-based encoding mode for each respective frequency-domain signal;

encoding each frequency-domain signal in the respective encoding mode; and

outputting encoded data, division information, and encoding mode information of each respective frequency-domain signal.

30. The method of claim 29, wherein the division of the input audio signal comprises:

transforming the input audio signal into a full frequency-domain signal; and

dividing the full frequency-domain signal into the frequency-domain signals according to a preset standard and selecting the time-based encoding mode or the frequency-based encoding mode for each respective frequency-domain signal.

31. The method of claim 30, wherein the division of the full frequency-domain signal comprises:

dividing the full frequency-domain signal into the frequency-domain signals suitable for the time-based encoding mode or the frequency-based encoding mode based on at least one of a spectral tilt, a size of signal energy of each frequency domain, a change in signal energy between sub-frames and a voicing level determination; and

selecting the encoding mode for each respective frequency-domain signal.

32. The method of claim 29, wherein the encoding of each frequency-domain signal comprises:

performing the time-based encoding on a first frequency-domain signal determined to be encoded in the time-based encoding mode; and

performing frequency-based encoding on a second frequency-domain signal determined to be encoded in the frequency-based encoding mode.

33. An audio data encoding method, comprising:

dividing a frame of audio data into first audio data and second audio data; and

encoding the first audio data in a time domain and encoding the second audio data in a frequency domain.

34. An adaptive time/frequency-based audio decoding method, comprising:

extracting encoded data for each frequency band from an input bitstream, division information, and encoding mode information for each frequency band;

decoding the encoded data for each frequency domain based on the division information and the respective encoding mode information; and

collecting decoded data in a frequency domain and performing an inverse frequency-domain transform on the collected data.

35. A computer-readable recording medium having a software program to execute an adaptive time/frequency-based audio encoding method, the method comprising:

dividing an input audio signal into a plurality of frequency-domain signals and selecting a time-based encoding mode or a frequency-based encoding mode for each respective frequency-domain signal;

encoding each frequency-domain signal in the respective encoding mode; and

outputting encoded data, division information, and encoding mode information of each respective frequency-domain signal.