Apparatus and method for converting an audio signal into a parameterized representation using band pass filters, apparatus and method for modifying a parameterized representation using band pass filter, apparatus and method for synthesizing a parameterized of an audio signal using band pass filters

• US 8,793,123 B2
• Filed: 03/10/2009
• Issued: 07/29/2014
• Est. Priority Date: 03/20/2008
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Disch, Sascha
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US12/922,823
• Publication Number:
  US 20110106529A1
• Time in Patent Office: 1,967 Days
• Field of Search: 704/205, 704500-504, 704/200.1, 704/209, 704/207, 704/220, 341/143, 455/265, 455/180.1, 455/190.1, 455/189.1, 455/209, 381/3, 375/240.01, 375/327, 375/376
• US Class Current: 704/205
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 19/09 Long term prediction, i.e. removing periodical redundancies, e.g. by using adaptive codebook or pitch predictor
• CPC Class Codes: G10L 19/16 Vocoder architecture
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

Apparatus for converting an audio signal into a parameterized representation, has a signal analyzer for analyzing a portion of the audio signal to obtain an analysis result; a band pass estimator for estimating information of a plurality of band pass filters based on the analysis result, wherein the information on the plurality of band pass filters has information on a filter shape for the portion of the audio signal, wherein the band width of a band pass filter is different over an audio spectrum and depends on the center frequency of the band pass filter; a modulation estimator for estimating an amplitude modulation or a frequency modulation or a phase modulation for each band of the plurality of band pass filters for the portion of the audio signal using the information on the plurality of band pass filters; and an output interface for transmitting, storing or modifying information on the amplitude modulation, information on the frequency modulation or phase modulation or the information on the plurality of band pass filters for the portion of the audio signal.

Claims

1. Apparatus for converting an audio signal into a parameterized representation, comprising:

a signal analyzer for analyzing a portion of the audio signal to acquire an analysis result, wherein the signal analyzer is operative to calculate a center of gravity position function for a spectral representation of the portion of the audio signal, wherein predetermined events in the center of gravity position function indicate candidate values for center frequencies of the plurality of band pass filters;

a band pass estimator for estimating information of a plurality of band pass filters based on the analysis result, wherein the information on the plurality of band pass filters comprises information on a filter shape for the portion of the audio signal, wherein the band width of a band pass filter is different over an audio spectrum and depends on the center frequency of the band pass filter, wherein the band pass estimator is operative to determine the center frequencies based on the candidate values;

a modulation estimator for estimating an amplitude modulation or a frequency modulation or a phase modulation for each band of the plurality of band pass filters for the portion of the audio signal using the information on the plurality of band pass filters; and

an output interface for transmitting, storing or modifying information on the amplitude modulation, information on the frequency modulation or phase modulation or the information on the plurality of band pass filters for the portion of the audio signal.

2. Apparatus in accordance with claim 1, in which the signal analyzer is operative to calculate a center of gravity position value for a band.

3. Apparatus in accordance with claim 1, in which the signal analyzer is operative to add negative power values of a first half of a band and adding positive power values of a second half of a band to acquire a center of gravity position candidate value, wherein the center of gravity position candidate values are smoothed over time to acquire smoothed center of gravity position values, and

wherein the band pass filter estimator is operative to determine the frequencies of zero crossings of the smoothed center of gravity position values over time.

4. Apparatus in accordance with claim 1, in which the band pass estimator is operative to determine the information of the center frequency or the band width of the band pass filters so that a spectrum from a lower start value to a higher end value is covered without a spectral hole, where the lower start value and the higher end value comprises at least five band pass filter bandwidths.

5. Apparatus in accordance with claim 1, in which the band pass estimator is operative to determine the information such that the frequency of zero crossings are modified in such a way that an approximately equal band pass center frequency spacing with respect to a perceptual scale results, where a distance between the band pass center frequencies and frequencies of zero crossings in a center of gravity position function is minimized.

6. Apparatus in accordance with claim 1, in which the modulation estimator is operative to form an analytical signal of a band pass signal for the band pass and to calculate a magnitude of the analytical signal to acquire information on the amplitude modulation of the audio signal in the band of the band pass filter.

7. Method of converting an audio signal into a parameterized representation, comprising:

analyzing a portion of the audio signal to acquire an analysis result, wherein a center of gravity position function for a spectral representation of the portion of the audio signal is calculated, wherein predetermined events in the center of gravity position function indicate candidate values for center frequencies of the plurality of band pass filters;

estimating information of a plurality of band pass filters based on the analysis result, wherein the information on the plurality of band pass filters comprises information on a filter shape for the portion of the audio signal, wherein the band width of a band pass filter is different over an audio spectrum and depends on the center frequency of the band pass filter, wherein the step of estimating determines the center frequencies based on the candidate values;

estimating an amplitude modulation or a frequency modulation or a phase modulation for each band of the plurality of band pass filters for the portion of the audio signal using the information on the plurality of band pass filters; and

transmitting, storing or modifying information on the amplitude modulation, information on the frequency modulation or phase modulation or the information on the plurality of band pass filters for the portion of the audio signal.

8. Apparatus for modifying a parameterized representation comprising, for a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising band widths, which depend on a band pass filter center frequency of the corresponding band pass filters, and amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, the modulation information being related to the center frequencies of the band pass filters, the apparatus comprising:

a modifier for modifying the time varying center frequencies and for generating a modified parameterized representation, in which the band widths of the band pass filters depend on the band pass filter center frequencies of the corresponding band pass filters.

9. Apparatus in accordance with claim 8, in which the modifier is operative to modify all center frequencies by multiplication with a constant factor or by only changing selected center frequencies in order to change the key mode of a piece of music from e.g. major to minor or vice versa.

10. Method of modifying a parameterized representation comprising, for a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising band widths, which depend on a band pass filter center frequency of the corresponding band pass filters, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, the modulation information being related to the center frequencies of the band pass filters, the method comprising:

modifying the time varying center frequencies and generating a modified parameterized representation, in which the band widths of the band pass filters depend on the band pass filter center frequencies of the corresponding band pass filters.

16. A non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method in accordance with claim 7, 10 or 15.

11. Apparatus for synthesizing a parameterized representation of an audio signal comprising a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising varying band widths, which depend on a band pass filter center frequency of the corresponding band pass filter, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, comprising:

an amplitude modulation synthesizer for synthesizing an amplitude modulation component based on the amplitude modulation information;

a frequency modulation or phase modulation synthesizer for synthesizing instantaneous frequency of phase information based on the information on a carrier frequency and a frequency modulation information for a respective band width,

wherein distances in frequency between adjacent carrier frequencies are different over a frequency spectrum,

an oscillator for generating an output signal representing an instantaneously amplitude modulated, frequency modulated or phase modulated oscillation signal for each band pass filter channel; and

a combiner for combining signals from the band pass filter channels and for generating an audio output signal based on the signals from the band pass filter channels,

wherein the amplitude modulation synthesizer comprises an overlap adder for overlapping and weighted adding subsequent blocks of amplitude modulation information to acquire the amplitude modulation component;

or

wherein the frequency modulation or phase modulation synthesizer comprises and overlap-adder for weighted adding two subsequent blocks of frequency modulation or phase modulation information or a combined representation of the frequency modulation information and the carrier frequency for a band pass signal to acquire the synthesized frequency information.

12. Apparatus in accordance with claim 11, in which the frequency modulation or phase modulation synthesizer comprises an integrator for integrating the synthesized frequency information and for adding, to the synthesized frequency information, a phase term derived from a phase of a component in spectral vicinity from a previous block of an output signal of the oscillator.

13. Apparatus in accordance with claim 12, in which the oscillator is a sinusoidal oscillator fed by a phase signal acquired by the adding operation.

14. Apparatus in accordance with claim 13, in which the oscillator comprises a modulator for modulating an output signal of the sinusoidal oscillator using the amplitude modulation component for the band.

15. Method of synthesizing a parameterized representation of an audio signal comprising a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising varying band widths, which depend on a band pass filter center frequency of the corresponding band pass filter, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, comprising:

synthesizing an amplitude modulation component based on the amplitude modulation information;

synthesizing instantaneous frequency or phase information based on the information on a carrier frequency and a frequency modulation information for a respective band width,

wherein distances in frequency between adjacent carrier frequencies are different over a frequency spectrum,

generating an output signal representing an instantaneously amplitude modulated, frequency modulated or phase modulated oscillation signal for each band pass filter channel; and

combining signals from the band pass filter channels and generating an audio output signal based on the signals from the band pass filter channels,

wherein the step of synthesizing an amplitude modulation component comprises a step of overlapping and weighted adding subsequent blocks of amplitude modulation information to acquire the amplitude modulation component;

or

wherein the step of synthesizing instantaneous frequency or phase information comprises a step of weighted adding two subsequent blocks of frequency modulation or phase modulation information or a combined representation of the frequency modulation information and the carrier frequency for a band pass signal to acquire the synthesized frequency information.

17. Apparatus for converting an audio signal into a parameterized representation, comprising:

a signal analyzer for analyzing a portion of the audio signal to acquire an analysis result;

a band pass estimator for estimating information of a plurality of band pass filters based on the analysis result, wherein the information on the plurality of band pass filters comprises information on a filter shape for the portion of the audio signal, wherein the band width of a band pass filter is different over an audio spectrum and depends on the center frequency of the band pass filter;

a modulation estimator for estimating an amplitude modulation or a frequency modulation or a phase modulation for each band of the plurality of band pass filters for the portion of the audio signal using the information on the plurality of band pass filters, wherein the modulation estimator is operative to downmix a band pass signal with a carrier comprising the center frequency of the respective band pass to acquire information on the frequency modulation or phase modulation in the band of the band pass filter; and

an output interface for transmitting, storing or modifying information on the amplitude modulation, information on the frequency modulation or phase modulation or the information on the plurality of band pass filters for the portion of the audio signal.

18. Method of converting an audio signal into a parameterized representation, comprising:

analyzing a portion of the audio signal to acquire an analysis result;

estimating information of a plurality of band pass filters based on the analysis result, wherein the information on the plurality of band pass filters comprises information on a filter shape for the portion of the audio signal, wherein the band width of a band pass filter is different over an audio spectrum and depends on the center frequency of the band pass filter;

estimating an amplitude modulation or a frequency modulation or a phase modulation for each band of the plurality of band pass filters for the portion of the audio signal using the information on the plurality of band pass filters, wherein a band pass signal is downmixed with a carrier comprising the center frequency of the respective band pass to acquire information on the frequency modulation or phase modulation in the band of the band pass filter; and

transmitting, storing or modifying information on the amplitude modulation, information on the frequency modulation or phase modulation or the information on the plurality of band pass filters for the portion of the audio signal.

23. A non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method in accordance with claim 18, 20 or 22.

19. Apparatus for modifying a parameterized representation comprising, for a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising band widths, which depend on a band pass filter center frequency of the corresponding band pass filters, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, the modulation information being related to the center frequencies of the band pass filters, the apparatus comprising:

a modifier for modifying the time varying center frequencies or for modifying the amplitude modulation or phase modulation or frequency modulation information and for generating a modified parameterized representation, in which the band widths of the band pass filters depend on the band pass filter center frequencies of the corresponding band pass filters,

wherein the modifier is operative to modify the amplitude modulation information or the phase modulation information or the frequency modulation information by a non-linear decomposition into a coarse structure and a fine structure and by only modifying either the coarse structure or the fine structure.

20. Method of modifying a parameterized representation comprising, for a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising band widths, which depend on a band pass filter center frequency of the corresponding band pass filters, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, the modulation information being related to the center frequencies of the band pass filters, the apparatus comprising:

modifying the time varying center frequencies or modifying the amplitude modulation or phase modulation or frequency modulation information and generating a modified parameterized representation, in which the band widths of the band pass filters depend on the band pass filter center frequencies of the corresponding band pass filters,

wherein the modifying modifies the amplitude modulation information or the phase modulation information or the frequency modulation information by a non-linear decomposition into a coarse structure and a fine structure and by only modifying either the coarse structure or the fine structure.

21. Apparatus for synthesizing a parameterized representation of an audio signal comprising a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising varying band widths, which depend on a band pass filter center frequency of the corresponding band pass filter, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, comprising:

an amplitude modulation synthesizer for synthesizing an amplitude modulation component based on the amplitude modulation information, wherein the amplitude modulation synthesizer comprises a noise adder for adding noise, the noise adder being controlled via transmitted side information, being fixedly set or being controlled by a local analysis;

a frequency modulation or phase modulation synthesizer for synthesizing instantaneous frequency of phase information based on the information on a carrier frequency and a frequency modulation information for a respective band width,

wherein distances in frequency between adjacent carrier frequencies are different over a frequency spectrum,

an oscillator for generating an output signal representing an instantaneously amplitude modulated, frequency modulated or phase modulated oscillation signal for each band pass filter channel; and

a combiner for combining signals from the band pass filter channels and for generating an audio output signal based on the signals from the band pass filter channels.

22. Method of synthesizing a parameterized representation of an audio signal comprising a time portion of an audio signal, band pass filter information for a plurality of band pass filters, the band pass filter information indicating time-varying band pass filter center frequencies of band pass filters comprising varying band widths, which depend on a band pass filter center frequency of the corresponding band pass filter, and comprising amplitude modulation or phase modulation or frequency modulation information for each band pass filter for the time portion of the audio signal, comprising:

synthesizing an amplitude modulation component based on the amplitude modulation information, the step of synthesizing comprising a step of adding noise controlled via transmitted side information, the side information being fixedly set or being controlled by a local analysis;

synthesizing instantaneous frequency or phase information based on the information on a carrier frequency and a frequency modulation information for a respective band width,

wherein distances in frequency between adjacent carrier frequencies are different over a frequency spectrum,

generating an output signal representing an instantaneously amplitude modulated, frequency modulated or phase modulated oscillation signal for each band pass filter channel; and

combining signals from the band pass filter channels and for generating an audio output signal based on the signals from the band pass filter channels.