Text-to-speech conversion system

• US 7,260,533 B2
• Filed: 07/19/2001
• Issued: 08/21/2007
• Est. Priority Date: 01/25/2001
• Status: Active Grant

• Current Assignee: LAPIS Semiconductor Co., Ltd. (ROHM Co., Ltd.)
• Original Assignee: OKI Electric Industry Company Limited
• Inventors: Kamanaka, Hiroki
• Primary Examiner(s): Armstrong; Angela
• Application Number:
  US09/907,660
• Publication Number:
  US 20030074196A1
• Time in Patent Office: 2,224 Days
• Field of Search: 704/200, 704/244, 704/258, 704/260, 704/267, 704/270, 700/83
• US Class Current: 704/260
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/07 Concatenation rules

Abstract

The system according to the invention comprises a text-to-speech conversion processing unit, and a phrase dictionary as well as a waveform dictionary, connected independently from each other to the conversion processing unit. The conversion processing unit is for converting any Japanese text inputted from outside into speech. In the phrase dictionary, voice-related terms representing the reproduced sounds of actually recorded sounds, for example, notations of terms such as onomatopoeic words, background sounds, lyrics, music titles, and so forth, are previously registered. Further, in the waveform dictionary, waveform data obtained from the actually recorded sounds, corresponding to the voice-related terms, are previously registered. Furthermore, the conversion processing unit is constituted such that as for a term in the text matching the voice-related term registered in the phrase dictionary upon correlation of the former with the latter, actually recorded speech waveform data corresponding to the relevant voice-related term matching the term in the text, registered in the waveform dictionary, is outputted as a speech waveform of the term.

Claims

1. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text into a synthesized speech waveform;

a phrase dictionary containing a plurality of sound-related terms that correspond to a plurality of waveform data generated from recorded sounds; and

a waveform dictionary containing the waveform data generated from the sound-related terms,

wherein said conversion system outputs just the speech waveform synthesized in the conversion processing unit from the inputted text, except in a case where a term in the inputted text matches one of the terms registered in said phrase dictionary, whereupon said conversion system substitutes the waveform from the waveform dictionary based on the waveform data corresponding to the one matching sound-related term, and outputs just the waveform without any overlap with the synthesized speech waveform.

2. A text-to-speech conversion system according to claim 1, further comprising an application determination unit for determining whether or not the term in the inputted text satisfies application conditions for correlation thereof with said phrase dictionary, and reading out only the sound-related term matching the term in the inputted text satisfying the application conditions from said phrase dictionary to said conversion processing unit.

3. A text-to-speech conversion system according to claim 2, wherein said application conditions include a condition that the term in the text is surrounded by quotation marks.

4. A text-to-speech conversion system according to claim 2, wherein said application conditions include a condition that a specific symbol is provided at least one of before and after the term in the text.

5. A text-to-speech conversion system according to claim 2, wherein said application conditions include a condition such that in the case where the sound related terms together with information on the subject thereof are registered in said phrase dictionary, there is a match between the information on the subject and the grammatical subject of the text.

6. A text-to-speech conversion system according to claim 2, further comprising application conditions change means capable of changing said application conditions.

7. A text-to-speech conversion system according to claim 2, wherein said application determination unit comprises a rules dictionary for storing the application conditions, and a condition determination unit for determining whether or not said phrase dictionary is to be applied, interconnecting said conversion processing unit and said phrase dictionary.

8. A text-to-speech conversion system according to claim 1, further comprising a controller for editing the registered contents of the sound-related terms registered in said phrase dictionary, and the corresponding waveform data registered in said waveform dictionary.

9. A text-to-speech conversion system according to claim 1, wherein said phrase dictionary is an onomatopoeic word dictionary for registering onomatopoeic words.

10. A text-to-speech conversion system according to claim 1, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files.

11. A text-to-speech conversion system according to claim 1, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files, said conversion processing unit comprising;

an input unit to which the text is inputted;

a pronunciation dictionary for registering pronunciation of respective words;

a text analyzer connected to said input unit, said pronunciation dictionary, and said phrase dictionary, for generating a phonetic/prosodic symbol string of the text by using the waveform file name of the sound-related term registered in said phrase dictionary against a term registered in both said pronunciation dictionary and said phrase dictionary among terms in the text inputted from said input unit, and by using the pronunciation of the respective words registered in said pronunciation dictionary against other terms;

a speech waveform memory for storing speech element data; and

a rule-based speech synthesizer connected to said speech waveform memory, said waveform dictionary, and said text analyzer, for converting respective symbols except said waveform file name, in said phonetic/prosodic symbol string, into a speech waveform with the use of said speech element data while reading out waveform data corresponding to said waveform file name from said waveform dictionary, thereby outputting a synthesized waveform consisting of the speech waveform and the waveform data.

12. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text into a synthesized speech waveform;

a phrase dictionary containing a plurality of sound-related terms that correspond to a plurality of waveform data generated from recorded sounds; and

a waveform dictionary containing the waveform data generated from the sound-related terms,

wherein said conversion system outputs just the speech waveform synthesized in the conversion processing unit from the inputted text, except in the case where there is a match between a term in the inputted text and one of the sound-related terms registered in said phrase dictionary, whereupon said conversion system overlaps the waveform based on the recorded waveform data corresponding to the one matching sound-related term and the speech waveform synthesized from the inputted text.

13. A text-to-speech conversion system according to claim 12, further comprising an application determination unit for determining whether or not the term in the inputted text satisfies application conditions for correlation thereof with said phrase dictionary, and reading out only the sound-related term matching the term in the inputted text satisfying the application conditions from said phrase dictionary to said conversion processing unit.

14. A text-to-speech conversion system according to claim 13, wherein said application conditions include a condition that the term in the text is surrounded by quotation marks.

15. A text-to-speech conversion system according to claim 13, wherein said application conditions include a condition that a specific symbol is provided at least one of before and after the term in the text.

16. A text-to-speech conversion system according to claim 13, wherein said application conditions include a condition that in the case where the sound-related terms together with information on the subject thereof are registered in said phrase dictionary, there is a match between the information on the subject and the grammatical subject of the inputted text.

17. A text-to-speech conversion system according to claim 13, further comprising application conditions change means capable of changing said application conditions.

18. A text-to-speech conversion system according to claim 13, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files.

19. A text-to-speech conversion system according to claim 13, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files, said conversion processing unit comprising;

an input unit to which the text is inputted;

a pronunciation dictionary for registering pronunciation of respective words;

a text analyzer connected to said input unit, said pronunciation dictionary, and said phrase dictionary, for generating a phonetic/prosodic symbol string of the text by using the waveform file name of the relevant sound-related term registered in said phrase dictionary against a term registered in both said pronunciation dictionary and said phrase dictionary among terms in the text inputted from said input unit, and by using the pronunciation of the respective words registered in said pronunciation dictionary against other terms;

a speech waveform memory for storing speech element data; and

a rule-based speech synthesizer connected to said speech waveform memory, said waveform dictionary, and said text analyzer, for converting respective symbols except said waveform file name, in said phonetic/prosodic symbol string, into a speech waveform with the use of said speech element data while reading out waveform data corresponding to said waveform file name from said waveform dictionary, thereby outputting the speech waveform and the waveform data concurrently.

20. A text-to-speech conversion system according to claim 13, wherein said application determination unit comprises a rules dictionary for storing the application conditions, and a condition determination unit for determining whether or not said phrase dictionary is to be applied, interconnecting said conversion processing unit and said phrase dictionary.

21. A text-to-speech conversion system according to claim 13, wherein said phrase dictionary is a background sound dictionary for registering notations of respective background sounds, with a waveform file name corresponding to each of the registered notations.

22. A text-to-speech conversion system according to claim 12, wherein said conversion processing unit has a function of adjusting the time length of the waveform data read out from said waveform dictionary.

23. A text-to-speech conversion system according to claim 12, further comprising a controller for editing the registered contents of the sound-related terms registered in said phrase dictionary, and the corresponding waveform data registered in said waveform dictionary.

24. A text-to-speech conversion system according to claim 12, wherein said phrase dictionary is a background sound dictionary for registering background sounds.

25. A text-to-speech conversion system according to claim 12, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files.

26. A text-to-speech conversion system according to claim 12, wherein the sound-related terms registered in said phrase dictionary include a notation of the relevant sound-related term, and a waveform file name corresponding to the notation, while the waveform data registered in said waveform dictionary are natural sound data of recorded sounds, and stored as waveform files, said conversion processing unit comprising;

an input unit to which the text is inputted;

a pronunciation dictionary for registering pronunciation of respective words;

a text analyzer connected to said input unit, said pronunciation dictionary, and said phrase dictionary, for generating a phonetic/prosodic symbol string of the text by using the waveform file name of the relevant sound-related term registered in said phrase dictionary against a term registered in both said pronunciation dictionary and said phrase dictionary among terms in the text inputted from said input unit, and by using the pronunciation of the respective words registered in said pronunciation dictionary against other terms;

a speech waveform memory for storing speech element data; and

a rule-based speech synthesizer connected to said speech waveform memory, said waveform dictionary, and said text analyzer, for converting respective symbols except said waveform file name, in said phonetic/prosodic symbol string, into a speech waveform with the use of said speech element data while reading out waveform data corresponding to said waveform file name from said waveform dictionary, thereby outputting the speech waveform and the waveform data concurrently.

27. A text-to-speech conversion system according to claim 12, wherein said phrase dictionary is a background sound dictionary for registering notations of respective background sounds, with a waveform file name corresponding to each of the registered notations.

28. A text-to-speech conversion system comprising:

a conversion processing unit for converting a text inputted into a speech waveform;

a phrase dictionary for registering a plurality of voice-related terms that correspond to a plurality of actual waveform data generated from actually recorded voices; and

a waveform dictionary for registering the actual waveform data corresponding to the voice-related terms,

wherein said conversion processing unit has a function such that in the case where there is a match between a term in the inputted text and one of the voice-related terms registered in said phrase dictionary, said conversion processing unit outputs an overlapped speech waveform including the speech waveform based on the actual waveform data corresponding to the one matching voice-related term and the speech waveform synthesized therein from the inputted text, and otherwise said conversion processing unit outputs the speech waveform synthesized therein from the inputted text;

wherein said conversion processing unit has a function of adjusting the time length of the waveform data read out from said waveform dictionary; and

wherein in case the time length of the read-out waveform data is longer than that of the speech waveform synthesized from the inputted text, the time length of the read-out waveform data is adjusted by truncating said waveform data at a time when said speech waveform comes to an end.

29. A text-to-speech conversion system comprising:

a conversion processing unit for converting a text inputted into a speech waveform;

a phrase dictionary for registering a plurality of voice-related terms that correspond to a plurality of actual waveform data generated from actually recorded voices; and

a waveform dictionary for registering the actual waveform data corresponding to the voice-related terms,

wherein said conversion processing unit has a function such that in the case where there is a match between a term in the inputted text and one of the voice-related terms registered in said phrase dictionary, said conversion processing unit outputs an overlapped speech waveform including the speech waveform based on the actual waveform data corresponding to the one matching voice-related term and the speech waveform synthesized therein from the inputted text, and otherwise said conversion processing unit outputs the speech waveform synthesized therein from the inputted text;

wherein said conversion processing unit has a function of adjusting the time length of the waveform data read out from said waveform dictionary; and

wherein in case the time length of the read-out waveform data is longer than that of the speech waveform synthesized from the inputted text, said time length is adjusted by gradually attenuating the sound volume of said waveform data so as to become zero at a time when said speech waveform comes to an end.

30. A text-to-speech conversion system comprising:

a conversion processing unit for converting a text inputted into a speech waveform;

a phrase dictionary for registering a plurality of voice-related terms that correspond to a plurality of actual waveform data generated from actually recorded voices; and

a waveform dictionary for registering the actual waveform data corresponding to the voice-related terms,

wherein said conversion processing unit has a function such that in the case where there is a match between a term in the inputted text and one of the voice-related terms registered in said phrase dictionary, said conversion processing unit outputs an overlapped speech waveform including the speech waveform based on the actual waveform data corresponding to the one matching voice-related term and the speech waveform synthesized therein from the inputted text, and otherwise said conversion processing unit outputs the speech waveform synthesized therein from the inputted text;

wherein said conversion processing unit has a function of adjusting the time length of the waveform data read out from said waveform dictionary; and

wherein in case the time length of the read-out waveform data is shorter than that of the speech waveform synthesized from the inputted text, said time length is adjusted by coupling together successive repetitions of said waveform data.

31. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text, containing lyrics, into a synthesized speech and song waveform;

a song phrase dictionary containing a plurality of pairs of lyrics or lyric phrases and song phoneme rhythm symbol strings corresponding thereto; and

a song phoneme rhythm symbol string processing unit for analyzing the song phoneme rhythm symbol strings in order to convert said song phoneme rhythm symbol strings into a plurality of synthesized song/speech waveforms,

wherein said conversion processing unit outputs just the speech waveform synthesized therein from the inputted text, except in a case where one of the lyrics in the inputted text matches with one of the lyrics registered in said song phrase dictionary, whereupon said conversion processing unit outputs just the synthesized song/speech waveforms, without overlapping said speech waveform.

32. A text-to-speech conversion system according to claim 31, further comprising an application determination unit for determining whether or not the lyric phrases in the inputted text satisfy application conditions for the correlation thereof with said song phrase dictionary, and reading out the song phoneme rhythm symbol string paired off with the registered lyrics matching the inputted lyrics satisfying the application conditions from said song phrase dictionary to said conversion processing unit.

33. A text-to-speech conversion system according to claim 32, wherein said application conditions include a condition that the lyrics in the inputted text are surrounded by quotation marks.

34. A text-to-speech conversion system according to claim 32, wherein said application conditions include a condition that a specific symbol is provided at least one of before and after the lyrics in the inputted text.

35. A text-to-speech conversion system according to claim 32, further comprising application conditions change means capable of changing said application conditions.

36. A text-to-speech conversion system according to claim 32, wherein said application determination unit comprises a rules dictionary for storing the application conditions, and a condition determination unit for determining whether or not said song phrase dictionary is to be applied, interconnecting said conversion processing unit and said song phrase dictionary.

37. A text-to-speech conversion system according to claim 31, further comprising a controller for editing the registered contents of the lyrics, and the song phoneme rhythm symbol string, paired off with the registered lyrics, respectively.

38. A text-to-speech conversion system according to claim 31, wherein said conversion processing unit comprises:

an input unit to which the text is inputted;

a pronunciation dictionary for registering pronunciation of respective words;

a text analyzer connected to said input unit, said pronunciation dictionary, and said phrase dictionary, for generating a phonetic/prosodic symbol string of the text by using said song phoneme rhythm symbol string registered in said song phrase dictionary against the lyrics among terms in the text inputted from said input unit, and by using the pronunciation of the respective words registered in said pronunciation dictionary against other terms;

a speech waveform memory for storing speech element data; and

a rule-based speech synthesizer connected to said speech waveform memory, said song phoneme rhythm symbol string processing unit, and said text analyzer, for converting respective symbols except said song phoneme rhythm symbol string, in the phonetic/prosodic symbol string, into a speech waveform with the use of said speech element data while collaborating with said song phoneme rhythm symbol string processing unit and said speech waveform memory for causing said song phoneme rhythm symbol string processing unit to generate waveform data corresponding to said song phoneme rhythm symbol string, thereby outputting a synthesized waveform consisting of the speech waveform and the waveform data.

39. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text containing a music title into a synthesized speech waveform;

a music title dictionary containing a plurality of music titles; and

a musical sound waveform generator for generating a musical sound waveform corresponding to one of the music titles, said musical sound waveform generator including a music dictionary for registering music data corresponding to the music titles, and a musical sound synthesizer for converting one of the music data into a musical sound waveform,

wherein said conversion processing unit outputs just the speech waveform synthesized therein from the inputted text, except in a case where the music title in the inputted text matches one of the registered music titles, whereupon the musical sound waveform corresponding to the one matching registered music title is superimposed on the speech waveform of the text before being outputted.

40. A text-to-speech conversion system according to claim 39, further comprising an application determination unit for determining whether or not the music title in the inputted text satisfies application conditions for the correlation thereof with said music title dictionary, and reading out only the registered music title matching the inputted music title satisfying the application conditions from said music title dictionary to said conversion processing unit.

41. A text-to-speech conversion system according to claim 40, wherein said application conditions include a condition that the music title in the inputted text is surrounded by quotation marks.

42. A text-to-speech conversion system according to claim 40, wherein said application conditions include a condition that a specific symbol is provided at least one of before and after the music title in the text.

43. A text-to-speech conversion system according to claim 40, further comprising application conditions change means capable of changing said application conditions.

44. A text-to-speech conversion system according to claim 40, wherein said application determination unit comprises a rules dictionary for storing the application conditions, and a condition determination unit for determining whether or not said music title dictionary is to be applied, interconnecting said conversion processing unit and said music title dictionary.

45. A text-to-speech conversion system according to claim 39, wherein said conversion processing unit has a function of adjusting the time length of the musical sound waveform sent from said musical sound synthesizer.

46. A text-to-speech conversion system according to claim 39, further comprising a controller for editing the contents of music titles registered in said music title dictionary, and the corresponding music data registered in said music dictionary.

47. A text-to-speech conversion system according to claim 39, wherein the music titles registered in said music title dictionary include the notation of the relevant music title, and the music file name corresponding to the notation, while the music data registered in said music dictionary, are stored as waveform files, said conversion processing unit comprising;

an input unit to which the text is inputted;

a pronunciation dictionary for registering pronunciation of respective words;

a text analyzer connected to said input unit, said pronunciation dictionary, and said phrase dictionary, for generating a phonetic/prosodic symbol string of the text by using the music file name against the relevant music title among terms in the text inputted from said input unit, and by using the pronunciation of the respective words registered in said pronunciation dictionary against all other terms;

a speech waveform memory for storing speech element data; and

a rule-based speech synthesizer connected to said speech waveform memory, said musical sound waveform generator, and said text analyzer, for converting respective symbols of the phonetic/prosodic symbol string into a speech waveform with the use of said speech element data while reading out the music data corresponding to said music file name from said musical sound waveform generator, thereby concurrently outputting the speech waveform and the music data.

48. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text containing a music title into a speech waveform;

a music title dictionary for registering a plurality of music titles; and

a musical sound waveform generator for generating a musical sound waveform corresponding to one of the music titles, said musical sound waveform generator including a music dictionary for registering music data corresponding to the music titles, and a musical sound synthesizer for converting one of the music data into a musical sound waveform,

wherein said conversion processing unit has a function such that in a case where the music title in the inputted text matches one of the registered music titles, the musical sound waveform corresponding to the one matching registered music title is superimposed on the speech waveform of the text before being outputted;

wherein said conversion processing unit has a function of adjusting the time length of the musical sound waveform sent from said musical sound synthesizer; and

wherein in case the time length of the musical sound waveform differs from the time length of the speech waveform of the text, the time length of the superimposed output is adjusted to be the longer of both the waveform time lengths.

49. A text-to-speech conversion system comprising:

a conversion processing unit for converting inputted text containing a music title into a speech waveform;

a music title dictionary for registering a plurality of music titles; and

a musical sound waveform generator for generating a musical sound waveform corresponding to one of the music titles, said musical sound waveform generator including a music dictionary for registering music data corresponding to the music titles, and a musical sound synthesizer for converting one of the music data into a musical sound waveform,

wherein said conversion processing unit has a function such that in a case where the music title in the inputted text matches one of the registered music titles, the musical sound waveform corresponding to the one matching registered music title is superimposed on the speech waveform of the text before being outputted;

wherein said conversion processing unit has a function of adjusting the time length of the musical sound waveform sent from said musical sound synthesizer; and

wherein in case the time length of the musical sound waveform is shorter than that of the speech waveform of the inputted text, said time length of the musical sound waveform is adjusted by coupling together successive repetitions of said musical sound waveform data.