System and method for voice transformation

• US 8,744,854 B1
• Filed: 09/24/2012
• Issued: 06/03/2014
• Est. Priority Date: 09/24/2012
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Chengjun Julian Chen
• Inventors: Chen, Chengjun Julian
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/625,317
• Publication Number:
  US 20140142946A1
• Time in Patent Office: 617 Days
• Field of Search: 704/266
• US Class Current: 704/269
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection

Abstract

The present invention is a method and system to convert speech signal into a parametric representation in terms of timbre vectors, and to recover the speech signal thereof. The speech signal is first segmented into non-overlapping frames using the glottal closure instant information, each frame is converted into an amplitude spectrum using a Fourier analyzer, and then using Laguerre functions to generate a set of coefficients which constitute a timbre vector. A sequence of timbre vectors can be subject to a variety of manipulations. The new timbre vectors are converted back into voice signals by first transforming into amplitude spectra using Laguerre functions, then generating phase spectra from the amplitude spectra using Kramers-Knonig relations. A Fourier transformer converts the amplitude spectra and phase spectra into elementary waveforms, then superposed to become the output voice. The method and system can be used for voice transformation, speech synthesis, and automatic speech recognition.

Claims

1. A method of voice transformation using one or more processors comprising:

segmenting the voice-signal into non-overlapping frames, wherein for voiced sections the frames are pitch periods;

generating amplitude spectra of the said frames using Fourier analysis;

transforming the said amplitude spectra into timbre vectors using orthogonal functions;

manipulating the voice parameters of the said timbre vectors to generate a set of new timbre vectors according to the specifications of voice transformation;

inverse transforming the new timbre vectors into new amplitude spectra using orthogonal functions;

generating new phase spectra from the new amplitude spectra using Kramers-Kronig relations;

generating elementary acoustic waves from the new amplitude spectra and the new phase spectra using Fourier transform;

producing a new voice waveform by superposing the said elementary acoustic waves according to the timing data given by the new timbre vectors.

2. The method of claim 1, wherein segmenting of the voice-signal is based on the glottal closure signals from an electroglottograph and by analyzing the sections of the voice-signal where glottal closure signals do not exist.

3. The method of claim 1, wherein segmenting of the voice-signal is based on analyzing the entirety of the voice-signal by a software comprising the capability of pitch period detection.

4. The method of claim 1, wherein the orthogonal functions are Laguerre functions.

5. The method of claim 1, wherein the voice parameter manipulating comprises executing local duration alternation using timbre interpolation.

6. The method of claim 1, wherein the voice parameter manipulating comprises executing overall speed alternation using timbre interpolation.

7. The method of claim 1, wherein the voice parameter manipulating comprises executing pitch profile alternation by changing the frame duration parameters of the timbre vectors.

8. The method of claim 1, wherein the voice parameter manipulating comprises executing intensity profile alternation by changing the intensity parameters of the timbre vectors.

9. The method of claim 1, wherein the voice parameter manipulating comprises executing speaker identity alternation by changing the head size and the spectral profiles.

10. The method of claim 1, wherein the voice parameter manipulating comprises implementing melody information through the frame duration parameters in the timbre vectors for singing synthesis.

11. A system of voice transformation comprising:

one or more data processing apparatus; and

a computer-readable medium coupled to the one or more data processing apparatus having instructions stored thereon which, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform a method comprising;

segmenting the voice-signal into non-overlapping frames, wherein for voiced sections the frames are pitch periods;

generating amplitude spectra of the said frames using Fourier analysis;

transforming the said amplitude spectra into timbre vectors using orthogonal functions;

manipulating the voice parameters of the said timbre vectors to generate a set of new timbre vectors according to the specifications of voice transformation;

inverse transforming the new timbre vectors into new amplitude spectra using orthogonal functions;

generating new phase spectra from the new amplitude spectra using Kramers-Kronig relations;

generating elementary acoustic waves from the new amplitude spectra and the new phase spectra using Fourier transform;

producing a new voice waveform by superposing the said elementary acoustic waves according to the timing data given by the new timbre vectors.

12. The system of claim 11, wherein segmenting of the voice-signal is based on the glottal closure signals from an electroglottograph and by analyzing the sections of the voice-signal where glottal closure signals do not exist.

13. The system of claim 11, wherein segmenting of the voice-signal is based on analyzing the entirety of the voice-signal by a software comprising the capability of pitch period detection.

14. The system of claim 11, wherein the orthogonal functions are Laguerre functions.

15. The system of claim 11, wherein the voice parameter manipulating comprises executing local duration alternation using timbre interpolation.

16. The system of claim 11, wherein the voice parameter manipulating comprises executing overall speed alternation using timbre interpolation.

17. The system of claim 11, wherein the voice parameter manipulating comprises executing pitch profile alternation by changing the frame duration parameters of the timbre vectors.

18. The system of claim 11, wherein the voice parameter manipulating comprises executing intensity profile alternation by changing the intensity parameters of the timbre vectors.

19. The system of claim 11, wherein the voice parameter manipulating comprises executing speaker identity alternation by changing the head size and the spectral profiles.

20. The system of claim 11, wherein the voice parameter manipulating comprises implementing melody information through the frame duration parameters in the timbre vectors for singing synthesis.