Text-to-speech method and system, computer program product therefor

• US 8,321,224 B2
• Filed: 01/10/2012
• Issued: 11/27/2012
• Est. Priority Date: 12/16/2003
• Status: Expired due to Term

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Loquendo SpA (Microsoft Corporation)
• Inventors: Badino, Leonardo, Barolo, Claudia, Quazza, Silvia
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/347,353
• Publication Number:
  US 20120109630A1
• Time in Patent Office: 322 Days
• Field of Search: 704/260, 704/258, 704/269
• US Class Current: 704/260
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

A text-to-speech system adapted to operate on text in a first language including sections in a second language, includes a grapheme/phoneme transcriptor for converting the sections in the second language into phonemes of the second language; a mapping module configured for mapping at least part of the phonemes of the second language onto sets of phonemes of the first language; and a speech-synthesis module adapted to be fed with a resulting stream of phonemes including the sets of phonemes of the first language resulting from mapping and the stream of phonemes of the first language representative of the text, and to generate a speech signal from the resulting stream of phonemes.

Claims

1. A method of facilitating text-to-speech conversion of a text in a first language having sections in at least one second language, the method comprising:

converting the second language sections of text into phonemes of the second language; and

processing similarity tests configured to perform category-to-category comparisons of respective vector representatives of phonetic categories of a set of phonemes of the second language and respective vector representatives of phonetic categories of a set of candidate mapping phonemes of the first language, the similarity tests being independent of the first and second languages.

2. The method of claim 1, further including:

using results of the similarity tests to map at least part of the second language phonemes to sets of phonemes of the first language by;

assigning respective scores to results of the similarity tests; and

mapping one or more of the second language phonemes to a set of mapping phonemes of the first language, the set of mapping phonemes being selected from the candidate mapping phonemes as a function of the scores; and

including the first language sets of phonemes resulting from the mapping in a stream of phonemes of the first language representative of the text to produce a resulting stream of phonemes that are used to generate a speech signal.

3. The method of claim 2, wherein the phoneme of the second language, which is mapped to a set of mapping phonemes of the first language, is selected from:

a set of phonemes of the first language including three, two, or one phonemes of the first language, or an empty set, whereby no phoneme is included in the resulting stream for the phoneme in the second language.

4. The method of claim 2 a wherein the mapping comprises:

defining a threshold value for the results of the similarity tests; and

mapping onto the empty set of phonemes of the first language any phoneme of the second language for which any of the scores fails to reach the threshold value.

5. The method of claim 2, further including representing the phonemes of the second language and the candidate mapping phonemes of the first language as phonetic category vectors, whereby a vector representative of phonetic categories of each phoneme of the second language is subject to comparison with a set of phonetic category vectors representative of the phonetic categories of the candidate mapping phonemes in the first language.

6. The method of claim 5, wherein the comparison is carried out on a category-to-category basis by allotting respective score values to the category-by-category comparisons, the respective score values being aggregated to generate the scores.

7. The method of claim 6, further including allotting differentiated weights to the score values in aggregating the respective score values to generate the scores.

8. The method of claim 5, comprising selecting the phonetic categories from one or more of:

(a) two basic categories of vowel and consonant;

(b) a category diphthong;

(c) vowel characteristics unstressed/stressed, non-syllabic, long, nasalized, rhoticized, or rounded;

(d) vowel categories front, central, or back;

(e) vowel categories close, close-close-mid, close-mid, mid, open-mid, open open-mid, or open;

(f) consonant mode categories plosive, nasal, trill, tapflap, fricative, lateral-fricative, approximant, lateral, or affricate;

(g) consonant place categories bilabial, labiodental, dental, alveolar, postalveolar, retroflex, palatal, velar, uvular, pharyngeal, or glottal;

or

(h) other consonant categories voiced, long, syllabic, aspirated, unreleased, voiceless, or semiconsonant.

9. The method of claim 2, further including pronouncing the resulting stream of phonemes by means of a speaker voice of the first language.

10. A system for text-to-speech conversion of a text in a first language having sections in at least one second language, comprising:

a grapheme/phoneme transcriptor configured to convert sections in the second language into phonemes of the second language; and

a mapping module configured to process similarity tests configured to perform a category-to-category comparison between a vector representative of phonetic categories of each of the phonemes of the second language and a vector representative of phonetic categories of each of the set of candidate mapping phonemes, the similarity tests being independent of the first language and the second language.

11. The system of claim 10, wherein the mapping module is further configured to:

use results of the similarity tests to map at least part of the second language phonemes to sets of phonemes of the first language by;

assigning respective scores to results of the similarity tests;

mapping one or more of the second language phonemes to a set of mapping phonemes of the first language, the set of mapping phonemes being selected from the candidate mapping phonemes as a function of the scores; and

include the first language sets of phonemes resulting from the mapping in a stream of phonemes of the first language representative of the text to produce a resulting stream of phonemes that are used to generate a speech signal.

12. The system of claim 11, wherein the mapping module is configured to map the phoneme of the second language into a set of mapping phonemes of the first language selected from:

a set of phonemes of the first language including three, two or one phonemes of the first language, or an empty set, whereby no phoneme is included in the resulting stream for the phoneme in the second language.

13. The system of claim 12, wherein the mapping module is configured to:

define a threshold value for the results of the tests; and

map the empty set of phonemes of the first language to any phoneme of the second language for which any of the scores fails to reach the threshold value.

14. The system of claim 11, wherein the phonemes of the second language and the candidate mapping phonemes of the first language are represented as phonetic category vectors, whereby the mapping module is configured to subject respective vectors representative of phonetic categories of each the phoneme of the second language is subject to comparison with a set of phonetic category vectors representative of the phonetic categories of the candidate mapping phonemes in the first language.

17. The system of claim 14, wherein the mapping module is configured to operate based on phonetic categories including one or more of:

(a) two basic categories of vowel and consonant;

(b) the category diphthong;

(c) vowel characteristics unstressed/stressed, non-syllabic, long, nasalized, rhoticized, or rounded;

(d) vowel categories front, central, or back;

(e) vowel categories close, close-close-mid, close-mid, mid, open-mid, open-open-mid, or open;

(f) consonant mode categories plosive, nasal, trill, tapflap, fricative, lateral-fricative, approximant, lateral, or affricate;

(g) consonant place categories bilabial, labiodental, dental, alveolar, postalveolar, retroflex, palatal, velar, uvular, pharyngeal, or glottal;

or

(h) other consonant categories voiced, long, syllabic, aspirated, unreleased, voiceless, or semiconsonant.

18. The system of claim 11, wherein the speech-synthesis module is configured to pronounce the resulting stream of phonemes by means of a speaker voice of the first language.

19. The system of claim 14, wherein the mapping module is configured to carry out the comparison on a category-to-category basis by allotting respective score values to the category-by-category comparisons, the respective score values being aggregated to generate the scores.

15. A computer program product comprising computer readable instructions embodied on a non-transitory computer readable medium and configured, when executed on one or more computer processors, to facilitate text-to-speech conversion of a text in a first language having sections in at least one second language by:

converting the second language sections into phonemes; and

processing similarity tests configured to perform category-to-category comparisons of respective vector representatives of phonetic categories of a set of phonemes of the second language and respective vector representatives of phonetic categories of a set of candidate mapping phonemes of the first language, the similarity tests being independent of the first and second languages.

16. The system of claim 15, wherein the mapping module is configured to allot differentiated weights to the score values in aggregating the respective score values to generate the scores.

20. The computer program product of claim 15, wherein the computer readable instructions are further configured, when executed on one or more processors, to:

use results of the similarity tests to map at least part of the second language phonemes to sets of phonemes of the first language by;

assigning respective scores to results of the similarity tests; and

mapping one or more of the second language phonemes to a set of mapping phonemes of the first language, the set of mapping phonemes being selected from the candidate mapping phonemes as a function of the scores; and

include the first language sets of phonemes resulting from the mapping in a stream of phonemes of the first language representative of the text to produce a resulting stream of phonemes that are used to generate a speech signal.