Envelope-invariant analytical speech resynthesis using periodic signals derived from reharmonized frame spectrum

• US 5,987,413 A
• Filed: 06/05/1997
• Issued: 11/16/1999
• Est. Priority Date: 06/10/1996
• Status: Expired due to Term

• Current Assignee: Dutoit, Thierry, Nicolas Pierret, Vincent Pagel
• Original Assignee: Dutoit, Thierry, Nicolas Pierret, Vincent Pagel
• Inventors: Pagel, Vincent, Pierret, Nicolas, Dutoit, Thierry
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US08/869,368
• Time in Patent Office: 894 Days
• Field of Search: 704/260, 704/267, 704/268, 704/278
• US Class Current: 704/267
• CPC Class Codes: G10L 13/07 Concatenation rules
• CPC Class Codes: G10L 21/04 Time compression or expansion

Abstract

Method envelope-invariant for audio signal synthesis from elementary audio waveforms stored in a dictionary wherein:

the waveforms are perfectly periodic, and stored as one of their period,

synthesis is obtained by overlap-adding of the waveforms obtained from time-domain repetition of the periodic waveforms with a weighting window whose size is approximately two times the period of the signals to weight, and whose relative position inside of the period is fixed to any value identical for all the periods, each extracted from a reharmonized and thus periodic waveform, obtained by modifying, without changing the spectral envelope, the frequencies and amplitudes of harmonics in the spectrum of a frame of the original continuous speech waveform,

whereby the time shift between two successive waveforms obtained by weighting the original signals is set to the imposed fundamental frequency of the signal to synthesize.

Claims

1. A method for audio synthesis from waveforms stored in a dictionary, comprising the steps of:

the waveforms are infinite and perfectly periodic, and are stored as one of their periods, itself represented as a sequence of sound samples of a priori any length;

a synthesis is carried out by overlapping and adding the waveforms multiplied by a weighting window whose length is approximately two times the period of the original waveform, and whose position relatively to the waveform can be set to any fixed value;

whereby the time shift between two successive weighted signals obtained by weighting the original waveforms is equal to the fundamental period requested for the synthetic signal, whose value is imposed.

2. The method for audio synthesis according to claim 1, wherein the fundamental period of the synthetic signal is greater or lower than the original period in the dictionary.

3. The method for audio synthesis according to claim 2, wherein the lengths of the periods stored in the dictionary are all identical.

4. The method for audio synthesis according to claim 3, wherein the phases of the lower-frequency harmonics (typically from 0 to 3 kHz) of the stored periodic waveforms have a fixed value per harmonic throughout the dictionary.

5. The method for audio synthesis according to claim 4, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

6. The method for audio synthesis according to claim 3, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

7. The method for audio synthesis according to claim 2, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

8. The method for audio synthesis according to claim 1, wherein the lengths of the periods stored in the dictionary are all identical.

9. The method for audio synthesis according to claim 8, wherein the phases of the lower-frequency harmonics (typically from 0 to 3 kHz) of the stored periodic waveforms have a fixed value per harmonic throughout the dictionary.

10. The method for audio synthesis according to claim 9, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

11. The method for audio synthesis according to claim 8, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

12. The method for audio synthesis according to claim 1, wherein the stored waveforms are obtained from the spectral analysis of a dictionary of audio signal segments such as diphones in the case of speech synthesis whereby a spectral analysis provides at regular time intervals an estimate of the instantaneous spectral envelope in each segment from which the waveforms are computed.

13. The method for audio synthesis according to any one of the preceding claims, wherein when concatenating two segments, the last periods of the first segment and the first period of the second segment are modified to smooth out the time-domain difference measured between the last period of the first segment and the first period of the second segment, this time-domain difference being added to each modified period with a weighting coefficient varying between -0.5 and 0.5 depending on the position of the modified period with respect to the concatenation point.

14. A method for audio synthesis from waveforms stored in a dictionary, comprising;

the waveforms are infinite and perfectly periodic and are obtained from the spectral analysis of a dictionary of audio signal segments, and are stored as one of their periods, itself represented as a sequence of sound samples of a priori any length;

a synthesis is carried out by overlapping and adding the waveforms multiplied by awaiting window whose length is approximately two times the period of the original waveform, and whose position relative to the waveform can be set at any fixed value;

whereby the time shift between two successive weighted signals obtained by weighting the original waveforms is equal to the fundamental period requested for the synthetic signal whose value is imposed;

wherein when concatenating two segments, the last period of the first segment and the first period of the second segment are modified to smooth out the time-domain difference measured between the last period of the first segment and the first period of the second segment, this time-domain difference being added to each modified period with a weighting coefficient varying between -0.5 and 0.5 depending on the position of the modified period with respect to the concatenation point, and for each base segment, replacement segments are stored whereby at synthesis time when two segments are about to be concatenated, the periods of the first base segment are modified so as to propagate, on the last periods of this segment, the difference between the last period of the base segment and the last period of one of its replacement segments and whereby the periods of the second base segment are modified so as to propagate, on the first periods of this segment, the difference between the first period of the base segment and the first period of one of its replacement segments, the propagation of these differences being performed by multiplying the measured differences by a weighting coefficient continuously varying from one to zero (from period to period) and adding the weighted differences to the periods of the base segments.