Speech synthesis method and apparatus, and dictionary generation method and apparatus

• US 20030229496A1
• Filed: 06/02/2003
• Published: 12/11/2003
• Est. Priority Date: 06/05/2002
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Yamada, Masayuki, Komori, Yasuhiro, Fukada, Toshiaki
• Granted Patent: US 7,546,241 B2
• Time in Patent Office: Days
• US Class Current: 704/258
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules

Abstract

In a speech synthesis process, micro-segments are cut from acquired waveform data and a window function. The obtained micro-segments are re-arranged to implement a desired prosody, and superposed data is generated by superposing the re-arranged micro-segments, so as to obtain synthetic speech waveform data. A spectrum correction filter is formed based on the acquired waveform data. At least one of the waveform data, micro-segments, and superposed data is corrected using the spectrum correction filter. In this way, “blur” of a speech spectrum due to the window function applied to obtain micro-segments is reduced, and speech synthesis with high sound quality is realized.

Claims

1. A speech synthesis method comprising:

an acquisition step of acquiring micro-segments from speech waveform data and a window function;

a re-arrangement step of re-arranging the micro-segments acquired in the acquisition step to change prosody upon synthesis;

a synthesis step of outputting synthetic speech waveform data on the basis of superposed waveform data obtained by superposing the micro-segments re-arranged in the re-arrangement step; and

a correction step of correcting at least one of the speech waveform data, the micro-segments, and the superposed waveform data using a spectrum correction filter formed based on the speech waveform data to be processed in the acquisition step.

2. The method according to claim 1, further comprising:

a formation step of forming the spectrum correction filter formed on the basis of the speech waveform data to be processed in the acquisition step.

3. The method according to claim 1, further comprising:

a speech synthesis dictionary which registers formation information for a spectrum correction filter in correspondence with each speech waveform data, and

wherein the correction step includes a step of forming the spectrum correction filter by acquiring formation information corresponding to the speech waveform data to be processed in the acquisition step from the speech synthesis dictionary.

4. The method according to claim 1, further comprising:

a speech synthesis dictionary which registers spectrum-corrected speech waveform data obtained by applying a spectrum correction filter formed based on each speech waveform data to that speech waveform data, and

wherein the acquisition step includes a step of acquiring the micro-segments from the spectrum-corrected speech waveform data obtained from the speech synthesis dictionary, and the window function.

5. The method according to claim 1, further comprising:

a formation step of forming the spectrum correction filter formed on the basis of the speech waveform data to be processed in the acquisition step, and separating the spectrum correction filter into a plurality of element filters, and

wherein the correction step includes a step of applying each of the plurality of element filters obtained in the formation step to not less than two out of the speech waveform data, the micro-segments, and the superposed waveform data.

6. The method according to claim 5, wherein the formation step includes a step of forming three element filters from the spectrum correction filter, and

the correction step includes a step of applying the three element filters to the speech waveform data, the micro-segments, and the superposed waveform data, respectively.

7. The method according to claim 5, wherein the formation step includes a step of obtaining the plurality of element filters by factorizing a characteristic polynomial that represents the spectrum correction filter to convert the spectrum correction filter into a product of a plurality of element filters.

8. The method according to claim 5, wherein the formation step includes a step of obtaining the plurality of element filters by approximating the spectrum correction filter by a filter expressed by a polynomial, and factorizing the polynomial to convert the spectrum correction filter into a product of element filters.

9. The method according to claim 1, further comprising:

a step of providing a speech synthesis dictionary which registers a plurality of element filters obtained by separating the spectrum correction filters formed based on the speech waveform data, and

wherein the correction step includes a step of acquiring the plurality of element filters corresponding to the speech waveform data to be processed in the acquisition step, and applying each of the plurality of obtained element filters to not less than two out of the speech waveform data, the micro-segments, and the superposed waveform data.

10. The method according to claim 1, wherein the re-arrangement step of re-arranging the micro-segments cut using the window function includes at least one of steps of changing an interval of the micro-segments, repeating a given micro-segment, and skipping the micro-segments.

11. The method according to claim 1, wherein the re-arrangement step includes a step of skipping the micro-segments, and

the correction step includes a step of applying the correction filter to remaining micro-segments after the skipping process.

12. The method according to claim 1, wherein the correction step includes a step of using a replacement filter that replaces the spectrum correction filter, and

said method further comprises a providing step of providing, to the acquisition step, modified speech waveform data which is generated by processing the speech waveform data to correct an influence of use of the replacement filter.

13. The method according to claim 12, wherein the replacement filter is obtained by approximating a correction filter formed based on the speech waveform data.

14. The method according to claim 13, wherein the providing step includes a step of obtaining the modified speech waveform data by applying a synthetic filter of an inverse filter of the replacement filter and the correction filter to the speech waveform data.

15. The method according to claim 13, wherein the replacement filter is obtained by generating an FIR filter based on an impulse response of the correction filter, and windowing the FIR filter at a low order.

16. The method according to claim 13, wherein an order of the replacement filter is lower than the correction filter.

17. The method according to claim 12, further comprising:

a speech synthesis dictionary which registers the modified speech waveform data generated in the providing step, and formation information of an alternative correction filter to be applied to each speech waveform data, and

wherein the acquisition step includes a step of acquiring micro-segments from speech waveform data read out from the speech synthesis dictionary, and

the correction step includes a step of forming a replacement filter to be used by reading out the formation information of the alternative correction filter corresponding to the speech waveform data from the speech synthesis dictionary.

18. The method according to claim 17, wherein the speech synthesis dictionary stores clustering information that registers formation information of representative correction filters for respective classes, which are obtained by clustering a plurality of alternative correction filters, and

each of the speech waveform data is modified according to a representative filter to be applied, and is stored together with information indicating a class to which that representative filter belongs.

29. A computer readable memory storing a control program for making a computer execute a speech synthesis dictionary generation method of claim 1.

19. A method of generating a dictionary used in a speech synthesis process, comprising:

a generation step of generating a spectrum correction filter on the basis of speech waveform data to be stored in a speech synthesis dictionary; and

a storage step of storing the spectrum correction filter generated in the generation step in correspondence with the speech waveform data.

20. The method according to claim 19, further comprising a separation step of separating the spectrum correction filter generated in the generation step into a plurality of element filters, and

the storage step includes a step of storing the plurality of element filters in correspondence with the speech waveform data.

21. A method of generating a dictionary used in a speech synthesis process, comprising:

a first generation step of generating a spectrum correction filter on the basis of each of speech waveform data to be stored in a speech synthesis dictionary;

a second generation step of generating spectrum-corrected speech waveform data by applying the spectrum correction filter to the corresponding speech waveform data; and

a storage step of storing the spectrum-corrected speech waveform data generated in the second generation step in the speech synthesis dictionary.

22. A method of generating a dictionary used in a speech synthesis process, comprising:

a first generation step of generating a replacement filter which replaces a spectrum correction filter obtained based on speech waveform data;

a second generation step of generating modified waveform data by processing the speech waveform data to correct an influence of use of the replacement filter; and

a storage step of storing the replacement filter generated in the first generation step in correspondence with the modified waveform data generated in the second generation step.

23. The method according to claim 22, wherein the generation step includes a step of generating, as the replacement filter, a filter which approximates the spectrum correction filter obtained based on the speech waveform data.

24. The method according to claim 22, wherein the first generation step includes a step of clustering a plurality of correction filters into a predetermined number of classes, and generating clustering information that assigns representative correction filters to the respective classes as replacement filters, and

the storage step includes a step of storing the clustering information, and storing class identification information which specifies one of the predetermined number of classes and is assigned to the modified waveform data generated in the second generation step.

25. A speech synthesis apparatus comprising:

acquisition means for acquiring micro-segments from speech waveform data and a window function;

re-arrangement means for re-arranging the micro-segments acquired by said acquisition means to change prosody upon synthesis;

synthesis means for outputting synthetic speech waveform data on the basis of superposed waveform data obtained by superposing the micro-segments re-arranged by said re-arrangement means; and

correction means for correcting at least one of the speech waveform data, the micro-segments, and the superposed waveform data using a spectrum correction filter formed based on the speech waveform data to be processed by said acquisition means.

26. A generation apparatus of a dictionary used in a speech synthesis process, comprising:

generation means for generating a spectrum correction filter on the basis of speech waveform data to be stored in a speech synthesis dictionary; and

storage means for storing the spectrum correction filter generated by said generation means in correspondence with the speech waveform data.

27. A generation apparatus of a dictionary used in a speech synthesis process, comprising:

first generation means for generating a spectrum correction filter on the basis of each of speech waveform data to be stored in a speech synthesis dictionary;

second generation means for generating spectrum-corrected speech waveform data by applying the spectrum correction filter to the corresponding speech waveform data; and

storage means for storing the spectrum-corrected speech waveform data generated by said second generation means in the speech synthesis dictionary.

28. A generation apparatus of a dictionary used in a speech synthesis process, comprising:

first generation means for generating a replacement filter which replaces a spectrum correction filter obtained based on speech waveform data;

second generation means for generating modified waveform data by processing the speech waveform data to correct an influence of use of the replacement filter; and

storage means for storing the replacement filter generated by said first generation means in correspondence with the modified waveform data generated by said second generation means.