Adaptive time and/or frequency-based encoding mode determination apparatus and method of determining encoding mode of the apparatus

• US 20070174051A1
• Filed: 09/21/2006
• Published: 07/26/2007
• Est. Priority Date: 01/24/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Oh, Eun Mi, Son, Chang Yong, Choo, Ki Hyun, Kim, Jung-Hoe
• Granted Patent: US 8,744,841 B2
• Time in Patent Office: Days
• US Class Current: 704/211
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/22 Mode decision, i.e. based on audio signal content versus external parameters

Abstract

An adaptive time/frequency-based encoding mode determination apparatus including a time domain feature extraction unit to generate a time domain feature by analysis of a time domain signal of an input audio signal, a frequency domain feature extraction unit to generate a frequency domain feature corresponding to each frequency band generated by division of a frequency domain corresponding to a frame of the input audio signal into a plurality of frequency domains, by analysis of a frequency domain signal of the input audio signal, and a mode determination unit to determine any one of a time-based encoding mode and a frequency-based encoding mode, with respect to the each frequency band, by use of the time domain feature and the frequency domain feature.

Claims

1. An adaptive time and/or frequency-based encoding mode determination apparatus comprising:

a time domain feature extraction unit to generate a time domain feature by analyzing a time domain signal of an input audio signal;

a frequency domain feature extraction unit to generate a frequency domain feature corresponding to each frequency band generated by dividing a frequency domain corresponding to a frame of the input audio signal into a plurality of frequency domains, by analyzing a frequency domain signal of the input audio signal; and

a mode determination unit to determine one of a time-based encoding mode and a frequency-based encoding mode as an encoding mode, with respect to the each frequency band, according to the time domain feature and the frequency domain feature.

2. The apparatus of claim 1, wherein, when the frequency domain feature extraction unit analyzes a frequency domain signal of a current frame of the input audio signal, the time domain feature extraction unit analyzes a time domain signal corresponding to the frequency domain signal of either the current frame or a next frame of the input audio signal.

3. The apparatus of claim 2, further comprising:

a long-term feature extraction unit to generate a time domain long-term feature and a frequency domain long-term feature by analyzing the time domain feature and the frequency domain feature,

wherein;

the time domain feature is a time domain short-term feature of the input audio signal;

the frequency domain feature is a frequency domain short-term feature corresponding to the each frequency band; and

the mode determination unit determines the encoding mode according to the time domain long-term feature and the frequency domain long-term feature.

4. The apparatus of claim 3, wherein, when the mode determination unit determines the encoding mode with respect to the current frame, a result of analyzing the time domain with respect to the next frame is used to calculate a short-term/long-term prediction gain with respect to a previous, the current, and the next frame via a frame feature buffer.

5. The apparatus of claim 3, wherein the time domain short-term feature comprises a transition extent and a short-term/long-term prediction gain, and the frequency domain short-term feature comprises a voicing probability.

6. The apparatus of claim 5, wherein the time domain long-term feature comprises a continuity of periodicity, a frequency spectral tilt, and/or a frame energy, and the frequency domain long-term feature comprises a correlation between channels.

7. The apparatus of claim 6, wherein the mode determination unit determines the encoding mode to be the frequency-based encoding mode according to at least one of:

a first condition in which a stereo extent of the input audio signal is more than a predetermined level;

a second condition in which a transition extent is less than a predetermined level;

a third condition in which the short-term/long-term prediction gain is less than a predetermined level; and

a fourth condition in which a voicing probability corresponding to the frequency band is less than a predetermined level.

8. The apparatus of claim 7, wherein the mode determination unit determines the encoding mode to be the time-based encoding mode when any of the first through fourth conditions are not satisfied and when any of following conditions are also not satisfied:

a fifth condition in which continuity of the periodicity of the input audio signal is continuously maintained for more than predetermined periods;

a sixth condition in which music continuity where the frequency spectral tilt is gentle and the frame energy is continuously maintained at a high level for more than a certain period, is more than a predetermined level, and

the mode determination unit determines the encoding mode to be the frequency-based encoding mode when any of the first through fourth conditions are not satisfied and at least one of the fifth and sixth conditions are satisfied.

9. The apparatus of claim 1, wherein the frequency domain feature extraction unit transforms the input audio signal of the time domain signal by one of a modulated lapped transform, a frequency-varying modulated lapped transform, and a fast Fourier transform and analyzes the frequency domain signal to generate a frequency domain feature corresponding to each frequency band.

10. The apparatus of claim 1, further comprising:

an encoding unit to encode with the determined encoding mode with respect to the each frequency band to generate an encoded data; and

a bit stream output unit to process a bit stream with respect to the encoded data and to output the processed bit stream.

11. The apparatus of claim 10, wherein, when the frequency domain feature extraction unit analyzes a frequency domain signal of a current frame of the input audio signal, the time domain feature extraction unit analyzes a time domain signal corresponding to the frequency domain signal of either the current frame or a next frame of the input audio signal.

12. The apparatus of claim 11, further comprising:

a long-term feature extraction unit generating a time domain long-term feature and a frequency domain long-term feature by analyzing the time domain feature and the frequency domain feature,

wherein;

the time domain feature is a time domain short-term feature of the input audio signal;

the frequency domain feature is a frequency domain short-term feature corresponding to the each frequency band; and

the mode determination unit determines the encoding mode according to the time domain long-term feature and the frequency domain long-term feature.

13. An adaptive time/frequency-based encoding mode determination method comprising:

generating a time domain feature by analyzing a time domain signal of an input audio signal;

generating a frequency domain feature corresponding to each frequency band generated by dividing a frequency domain corresponding to a frame of the input audio signal into a plurality of frequency domains, by analyzing a frequency domain signal of the input audio signal; and

determining one of a time-based encoding mode and a frequency-based encoding mode, with respect to the each frequency band, according to the time domain feature and the frequency domain feature.

14. The method of claim 13, wherein, when a frequency domain signal of a current frame of the input audio signal is analyzed in the generating a frequency domain feature, a time domain signal corresponding to a frequency domain signal of one of a current and a next frame of the input audio signal is analyzed in the generating the time domain feature.

15. The method of claim 14, further comprising:

generating a time domain long-term feature and a frequency domain long-term feature by analyzing the time domain feature and the frequency domain feature,

wherein;

the time domain feature is a time domain short-term feature of the input audio signal;

the frequency domain feature is a frequency domain short-term feature corresponding to the each frequency band; and

in the determining any one of a time-based encoding mode and a frequency-based encoding mode, the encoding mode is determined according to the time domain long-term feature and the frequency domain long-term feature.

16. The method of claim 15, wherein, in the determining one of a time-based encoding mode and a frequency-based encoding mode, when determining the encoding mode with respect to the current frame, a result of analyzing the time domain with respect to the next frame is used to calculate a short-term/long-term prediction gain with respect to a previous, the current, and the next frame via a frame feature buffer.

17. The method of claim 16, wherein the time domain short-term feature comprises a transition extent and a short-term/long-term prediction gain, and the frequency domain short-term feature comprises a voicing probability.

18. The method of claim 17, wherein the time domain long-term feature comprises a continuity of periodicity, a frequency spectral tilt, and/or a frame energy, and the frequency domain long-term feature comprises a correlation between channels.

19. The method of claim 18, wherein, in the determining one of a time-based encoding mode and a frequency-based encoding mode, the encoding mode is determined to be the frequency-based encoding mode when a stereo extent of the input audio signal is more than a predetermined level;

a transition extent is less than a predetermined level;

the short-term/long-term prediction gain is less than a predetermined level;

or a voicing probability corresponding to a frequency band is less than a predetermined level.

20. The method of claim 19, wherein, in the determining one of a time-based encoding mode and a frequency-based encoding mode, the encoding mode is determined to be the time-based encoding mode when continuity of the periodicity of the input audio signal is not continuously maintained for more than predetermined periods at a same time as the frequency spectral tilt is more than a predetermined level or the frame energy at a predetermined level is not continuously maintained for more than a certain period.

21. A computer readable recording medium in which a program to execute an adaptive time/frequency-based encoding mode determination method is recorded, the method comprising:

generating a time domain feature by analyzing a time domain signal of an input audio signal;

generating a frequency domain feature corresponding to each frequency band generated by dividing a frequency domain corresponding to a frame of the input audio signal into a plurality of frequency domains, by analyzing a frequency domain signal of the input audio signal; and

determining any one of a time-based encoding mode and a frequency-based encoding mode, with respect to the each frequency band, according to the time domain feature and the frequency domain feature.

22. An adaptive time and/or frequency-based encoding apparatus, comprising:

a mode determination unit to determine a time-based encoding mode and a frequency-based encoding mode as an encoding mode according to a frequency domain feature and a time domain feature with respect to respective frequency bands of a frame of an audio signal; and

an encoder to encode respective frequency bands according to corresponding ones of the time-based encoding mode and the frequency-based encoding mode.

23. The apparatus of claim 22, further comprising:

a domain feature extracting unit to generate a frequency domain feature corresponding to each frequency band generated by division of a frequency domain corresponding to the frame of the input audio signal into a plurality of frequency domains, by analysis of the frequency domain signal of the input audio signal.

24. The apparatus of claim 23, wherein the domain feature extraction unit comprises:

a frequency domain feature extraction unit to analyze a frequency domain signal of a current frame of the input audio signal; and

a time domain feature extraction unit to analyze a time domain signal corresponding to the frequency domain signal of either the current frame or a next frame of the input audio signal.

25. An adaptive time and/or frequency-based encoding apparatus, comprising:

a domain feature extraction unit to extract a time domain feature and a frequency domain feature with respect to a first frequency band and a second frequency band of an input audio signal, respectively;

a mode determination unit to determine a time-based encoding mode and a frequency-based encoding mode according to the time domain feature and the frequency domain feature; and

an encoder to encode the first frequency band according to the time-based encoding mode and the second frequency band according to the frequency-based encoding mode.

26. The apparatus of claim 25, wherein the mode determination unit generates first information on division of the first frequency band and the second frequency band and second information on the time-based encoding mode of the first frequency band and the frequency-based encoding mode of the second frequency band.

27. The apparatus of claim 26, further comprising:

an output unit to output a bit stream including the time-based encoded first frequency band, the frequency-based encoded second frequency band, the first information, and the second information.

28. An encoding and/or decoding system, comprising:

a mode determination unit to determine a time-based encoding mode and a frequency-based encoding mode as an encoding mode according to a frequency domain feature and a time domain feature with respect to respective frequency bands of a frame of an audio signal; and

an encoder to encode respective frequency bands according to corresponding ones of the time-based encoding mode and the frequency-based encoding mode and to generate a bit stream; and

a decoder to receive the bit stream and to decode the respective frequency bands according to corresponding ones of a time decoding mode corresponding to the time encoding mode and a frequency decoding mode corresponding to the frequency encoding mode.

29. An adaptive time and/or frequency-based decoding apparatus, comprising:

a bit stream input unit to receive a processed bit stream, the processed bit stream comprising;

time-based encoded data;

frequency-based encoded data;

information associated with a division of a frequency spectrum of a frequency domain signal into individual frequency bands; and

encoding mode information corresponding to a mode determination of the individual frequency bands; and

a decoding unit to decode the time-based encoded data and the frequency-based encoded data with respect to the individual frequency bands to generate decoded data representing an output audio signal.