Apparatus and method for modifying an audio signal using harmonic locking

• US 9,203,367 B2
• Filed: 08/16/2012
• Issued: 12/01/2015
• Est. Priority Date: 02/26/2010
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Disch, Sascha
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): ODUNUKWE, UBACHUKWU A
• Application Number:
  US13/587,143
• Publication Number:
  US 20130182862A1
• Time in Patent Office: 1,202 Days
• Field of Search: 381/61, 381/119, 381/98, 381/71.14, 704/500, 704/200, 704/205, 704/233, 700/94, 846/09
• US Class Current: 1/1
• CPC Class Codes: G10H 1/08 by combining tones G10H1/14, G10H1/16 take precedence; chord G10H1/38
• CPC Class Codes: G10H 1/20 for transposition
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2250/031 Spectrum envelope processing
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H03G 3/00 Gain control in amplifiers or frequency changers
• CPC Class Codes: H03G 5/00 Tone control or bandwidth control in amplifiers

Abstract

An apparatus for modifying an audio signal includes a filterbank processor, a fundamental determiner, an overtone determiner, a signal processor and a combiner. The filterbank processor generates a plurality of bandpass signals based on an audio signal and the fundamental determiner selects a bandpass signal of the plurality of bandpass signals to obtain a fundamental bandpass signal. The overtone determiner identifies a bandpass signal of the plurality of bandpass signals fulfilling an overtone criterion regarding the selected fundamental bandpass signal to obtain an overtone bandpass signal associated to the selected fundamental bandpass signal. The signal processor modifies the selected fundamental bandpass signal based on a predefined modification target. Additionally, the signal processor modifies an identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal. Further, the combiner combines the plurality of bandpass signals to obtain a modified audio signal.

Claims

1. An apparatus for modifying an audio signal, comprising:

a filterbank processor configured to generate a plurality of bandpass signals based on an audio signal;

a fundamental determiner configured to select a bandpass signal of the plurality of bandpass signals to acquire a fundamental bandpass signal;

an overtone determiner configured to identify a bandpass signal of the plurality of bandpass signals fulfilling an overtone criterion regarding the selected fundamental bandpass signal to acquire an overtone bandpass signal associated to the selected fundamental bandpass signal;

a signal processor configured to modify the selected fundamental bandpass signal based on a predefined modification target and configured to modify an identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal,

wherein the signal processor is configured to generate an amplitude modulated signal and a frequency modulated signal for each bandpass signal of the plurality of bandpass signals, wherein the signal processor is configured to modify the frequency modulated signal of the selected fundamental bandpass signal based on the predefined modification target, and wherein the signal processor is configured to modify the frequency modulated signal of the identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal; and

a combiner configured to combine the modified fundamental bandpass signal, the modified overtone bandpass signal and non selected bandpass signals of the plurality of bandpass signals to acquire a modified audio signal;

wherein

at least one of the filterbank processor, fundamental determiner, the overtone determiner, the signal processor, and the combiner comprises a hardware implementation.

2. The apparatus according to claim 1, wherein each bandpass signal of the plurality of bandpass signals comprises a carrier frequency, wherein the overtone determiner is configured to compare the carrier frequency of a bandpass signal of the plurality of bandpass signals with the carrier frequency of the selected fundamental bandpass signal, wherein an overtone criterion is fulfilled, if the carrier frequency of the bandpass is a multiple of the carrier frequency of the selected fundamental bandpass signal with a predefined carrier frequency tolerance.

3. The apparatus according to claim 1, wherein the overtone determinator is configured to compare an energy content of a bandpass signal of the plurality of bandpass signals with an energy content of the selected fundamental bandpass signal, wherein an overtone criterion is fulfilled, if a ratio of the energy content of the bandpass signal and the energy content of the selected fundamental bandpass signal is within a predefined energy tolerance range.

4. The apparatus according to claim 1, wherein the overtone determinator is configured to calculate a correlation value indicating a correlation of a temporal envelope of a bandpass signal of the plurality of bandpass signals with a temporal envelope of the selected fundamental bandpass signal, wherein an overtone criterion is fulfilled, if the correlation value is higher than a predefined correlation threshold.

5. The apparatus according to claim 1, wherein the fundamental determiner is configured to select a further bandpass signal of the plurality of bandpass signals without considering all already selected fundamental bandpass signals and all already identified overtone bandpass signals to acquire a further fundamental bandpass signal.

6. The apparatus according to claim 5, wherein the overtone determiner is configured to identify a bandpass signal of the plurality of bandpass signals fulfilling an overtone criterion regarding the further selected fundamental bandpass signal without considering all already identified overtone bandpass signals to acquire an overtone bandpass signal associated to the further selected fundamental bandpass signal.

7. The apparatus according to claim 5, wherein the signal processor is configured to modify the further selected fundamental bandpass signal based on a further predefined modification target.

8. The apparatus according to claim 1, wherein the fundamental determiner is configured to select the bandpass signal based on an energy criterion.

9. The apparatus according to claim 1, wherein the fundamental determiner is configured to determine an a-weighted energy content of each bandpass signal of the plurality of bandpass signals and configured to select a bandpass signal comprising the highest a-weighted energy content to acquire the fundamental bandpass signal.

10. The apparatus according to claim 1, comprising a carrier frequency determiner, wherein the filterbank processor comprises a filterbank and a signal converter, wherein the filterbank is configured to generate bandpass signals based on the audio signal, wherein the signal converter is configured to convert the generated bandpass signals to a subband domain to acquire the plurality of bandpass signals, wherein the carrier frequency determiner is configured to determine a plurality of carrier frequencies based on the audio signal, wherein the filterbank of the filterbank processor is configured to generate the bandpass signals, so that each bandpass signal comprises a frequency range comprising a different carrier frequency of the plurality of carrier frequencies to acquire a bandpass signal associated to each carrier frequency of the plurality of carrier frequencies.

11. The apparatus according to claim 1, comprising an envelope shape determiner and an envelope shaper, wherein the envelope shape determiner is configured to determine envelope shape coefficients based on the audio signal, wherein the audio signal is a frequency domain audio signal representing a time domain input audio signal, wherein the filterbank processor is configured to generate the plurality of bandpass signals in a subband domain based on the frequency domain audio signal, wherein the combiner is configured to combine at least a subset of the plurality of bandpass signals to acquire the modified audio signal representing a time domain audio signal, wherein the envelope shaper is configured to shape an envelope of the time domain audio signal based on the envelope shape coefficients, to shape an envelope of the plurality of subband domain bandpass signals comprising the modified subband domain bandpass signal based on the envelope shape coefficients or to shape an envelope of the plurality of subband domain bandpass signals based on the envelope shape coefficients before a subband domain bandpass signal is modified by the signal processor to acquire a shaped audio signal.

12. A method for modifying an audio signal, comprising:

generating a plurality of bandpass signals based on an audio signal;

selecting a bandpass signal of the plurality of bandpass signals to acquire a fundamental bandpass signal;

identifying a bandpass signal of the plurality of bandpass signals fulfilling an overtone criterion regarding the selected fundamental bandpass signal to acquire an overtone bandpass signal associated to the selected fundamental bandpass signal;

modifying the selected fundamental bandpass signal based on a predefined modification target by generating an amplitude modulated signal and a frequency modulated signal for each bandpass signal of the plurality of bandpass signals and by modifying the frequency modulated signal of the selected fundamental bandpass signal based on the predefined modification target,

modifying an identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal by modifying the frequency modulated signal of the identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal; and

combining the modified fundamental bandpass signal, the modified overtone bandpass signal and non selected bandpass signals of the plurality of bandpass signals to acquire a modified audio signal.

13. A non-transitory computer-readable medium including a computer program with a program code for performing, when the computer program runs on a digital signal processor, a computer or a micro controller, a method for modifying an audio signal, said method comprising:

generating a plurality of bandpass signals based on an audio signal;

selecting a bandpass signal of the plurality of bandpass signals to acquire a fundamental bandpass signal;

identifying a bandpass signal of the plurality of bandpass signals fulfilling an overtone criterion regarding the selected fundamental bandpass signal to acquire an overtone bandpass signal associated to the selected fundamental bandpass signal;

modifying the selected fundamental bandpass signal based on a predefined modification target by generating an amplitude modulated signal and a frequency modulated signal for each bandpass signal of the plurality of bandpass signals and by modifying the frequency modulated signal of the selected fundamental bandpass signal based on the predefined modification target,

modifying an identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal by modifying the frequency modulated signal of the identified overtone bandpass signal associated to the selected fundamental bandpass signal depending on the modification of the selected fundamental bandpass signal; and

combining the modified fundamental bandpass signal, the modified overtone bandpass signal and non selected bandpass signals of the plurality of bandpass signals to acquire a modified audio signal.