Audio signal transforming by utilizing a computational cost function

• US 8,706,496 B2
• Filed: 09/13/2007
• Issued: 04/22/2014
• Est. Priority Date: 09/13/2007
• Status: Active Grant

• Current Assignee: Universitat Pompeu Fabra
• Original Assignee: Universitat Pompeu Fabra
• Inventors: Sanjaume, Jordi Bonada
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Kazeminezhad, Farzad
• Application Number:
  US11/900,902
• Publication Number:
  US 20090076822A1
• Time in Patent Office: 2,413 Days
• Field of Search: 704/272, 704/270, 704/268
• US Class Current: 704/268
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 19/093 using sinusoidal excitation models

Abstract

A sequence is received of time domain digital audio samples representing sound (e.g., a sound generated by a human voice or a musical instrument). The time domain digital audio samples are processed to derive a corresponding sequence of audio pulses in the time domain. Each of the audio pulses is associated with a characteristic frequency. Frequency domain information is derived about each of at least some of the audio pulses. The sound represented by the time domain digital audio samples is transformed by processing the audio pulses using the frequency domain information. The sound transformation utilizes overlapping windows and a computational cost function which depends on a product of the number of the pitch periods and the inverse of the minimum fundamental frequency within the window is determined.

Claims

1. A method of audio signal transforming, the method comprising the steps of:

receiving a sequence of time domain multiple digital audio samples representing sound;

computer-implemented processing the time domain multiple digital audio samples to derive a corresponding sequence of multiple audio pulses in the time domain, each said audio pulse being associated with a characteristic frequency of a single pulse corresponding to a pitch associated with the audio pulse, each said audio pulse having a length corresponding to one pitch period of the respective audio pulse;

generating for each pulse a sequence of repetitions of said audio pulse, said audio pulse being repeated according to its own characteristic frequency;

deriving frequency domain information associated with at least some of the sequences of repetitions of said audio pulses, each said sequences of repetitions of said audio pulse being represented as a vector of sinusoids based on the derived frequency, said vector of sinusoids corresponds to a sinusoidal series expansion of the specific audio pulse;

computer-implemented transforming the sound represented by the time domain digital audio samples by processing the audio pulses using the frequency domain information;

overlapping the audio pulses in a pitch-synchronous manner;

transforming each input pulse according to transformation parameters including timbre modifications and estimating a timbre envelope defined by harmonics;

transforming the timbre envelope using a mapping function; and

obtaining the transformed timbre envelope at each frequency by computing values of the input timbre envelope at the mapped frequency;

wherein in the speech transformation an overlapping window is used, a computational cost function which depends on a product of the number of pitch periods and the inverse of the minimum fundamental frequency within the window is determined.

2. The method of claim 1, wherein the processing includes determining the characteristic frequency of each of the pulses and determining starting times of each of the audio pulses.

3. The method of claim 1, wherein the transforming is based on specified transformation parameters.

4. The method of claim 1, further comprising a step of presenting audio represented by the processed audio pulses.

5. The method of claim 1, wherein the sound comprises a monophonic harmonic sound.

6. The method of claim 1, wherein receiving the sound comprises reading from a memory.

7. The method of claim 1, wherein receiving the sound comprises receiving the sound in real time.

8. The method of claim 1, further comprising a step of delivering the transformed sound to a memory.

9. The method of claim 1, wherein at least one of the processing, deriving, and transforming is performed by software.

10. The method of claim 1, wherein at least one of the processing, deriving, and transforming is performed by hardware.

11. The method of claim 1, further comprising a step of using the transformed sound for voice synthesis.

12. The method of claim 1, further comprising a step of using the transformed sound for voice broadcasting.

13. The method of claim 1, further comprising a step of using the transformed sound for voice character transformation.

14. The method of claim 1, further comprising a step of using the transformed sound for forming additional parts for music production.

15. The method of claim 1, further comprising a step of using the transformed sound for voice enhancement.

16. The method of claim 1, further comprising a step of using the transformed sound in Internet protocol voice packets.

17. The method of claim 1, further comprising the step of computing the parameters including amplitude and frequency of each sinusoidal component or harmonics to be synthesized;

said estimation is carried out by interpolating the harmonic amplitudes, and

each input pulse according to the transformation parameters, including timbre modification.

18. The method of claim 1, wherein said interpolating uses linear or spline interpolation,

wherein frequencies are set to be multiples of the synthesis fundamental frequency;

the amplitudes are set by computing the timbre envelope values at the harmonic frequencies; and

the phases are computed by interpolating the pulse phase alignment or choosing the phase of one analyzed harmonic.

19. The method of claim 1, wherein each generated sequence is processed in the frequency domain to obtain a vector of sinusoidal components for each pulse;

a vector of sinusoidal components corresponding to each pulse is modified according to predetermined desired transformation parameters; and

rendering back each transformed synthesis pulse sequence to the time domain;

wherein each said audio pulse is repeated at a detected pitch rate and the repeated pulses are overlapped using an overlapping function that sums to a constant.

20. The method of claim 1, further comprising a computer-implemented step of synthesizing wherein a synthesis pulse sequence resulted from the transformation phase is synthesized, the step of synthesizing is initially conducted by computing the frequency domain content of each pulse, then by computing its time domain content, and by overlapping the audio pulses to obtain a resulting audio signal;

and

setting length of overlapping sections as a ratio of a synthesis pitch period length, wherein in speech transformation an overlapping window is used that is exactly constant in non-overlapping sections.

21. A computer program product tangibly embodied in one or more non-transitory machine-readable media for audio signal transforming, the computer program product comprising instructions that are executable by one or more processing devices to:

receive a sequence of time domain digital audio samples representing sound;

process the time domain digital audio samples to derive a corresponding sequence of audio pulses in the time domain, each of the audio pulses being associated with a characteristic frequency;

derive frequency domain information about each of at least some of the audio pulses;

and transform the sound represented by the time domain digital audio samples by processing the audio pulses using the frequency domain information;

wherein in the sound transformation an overlapping window is used, a computational cost function which depends on a product of the number of pitch periods and the inverse of the minimum fundamental frequency within the window is determined.

22. The computer program product of claim 21, wherein the instructions comprise a plug-in component for a software application.

23. The computer program product of claim 21, wherein the instructions comprise a software library for a digital signal processor.

24. The computer program product of claim 21, wherein the instructions comprise a downloadable application.

25. A system for audio signal transforming, the system comprising:

a receiver to receive a sequence of time domain digital audio samples representing sound;

and a processor to (a) process the time domain digital audio samples to derive a corresponding sequence of audio pulses in the time domain, each of the audio pulses being associated with a characteristic frequency;

(b) derive frequency domain information about each of at least some of the audio pulses;

and (c) transform the sound represented by the time domain digital audio samples by processing the audio pulses using the frequency domain information;

wherein in the sound transformation an overlapping window is used, a computational cost function which depends on a product of the number of pitch periods and the inverse of the minimum fundamental frequency within the window is determined.