Apparatus, medium, and method for generating record sentence for corpus and apparatus, medium, and method for building corpus using the same

• US 8,635,071 B2
• Filed: 02/17/2005
• Issued: 01/21/2014
• Est. Priority Date: 03/04/2004
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chung, Jihye, Cho, Jeongmi, Choo, Kihyun, Kim, Jeongsu
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US11/059,601
• Publication Number:
  US 20050197839A1
• Time in Patent Office: 3,260 Days
• Field of Search: 704/10, 704/7, 704/258, 704/260, 704/266, 704/270
• US Class Current: 704/260
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules

Abstract

A method, medium, and apparatus for generating a record sentence to establish a speech corpus, including generating a synthesized sentence of speech and synthesis information related to speech synthesis by performing speech synthesis for a predetermined sentence of text, selecting an unseen sentence including an unseen unit according to the synthesis information, generating a weight indicating a recording priority of the unseen unit included in the selected unseen sentence, and generating a record sentence by combining the unseen unit with the speech synthesis information according to the generated weight.

Claims

1. A method for generating a record sentence to establish a speech corpus, comprising:

generating a synthesized sentence of speech and synthesis information related to speech synthesis by performing speech synthesis for a predetermined sentence of text using candidate synthesis units transmitted from synthesis database;

selecting an unseen sentence including an unseen unit according to the synthesis information;

generating a weight indicating a recording priority of the unseen unit included in the selected unseen sentence;

generating a record sentence by combining the unseen unit with the speech synthesis information according to the generated weight; and

updating the speech corpus by storing the record sentence including the unseen unit,

wherein the synthesis database is updated based on the updated speech corpus,

wherein the unseen unit is selected as a synthesis unit when a speech unit of satisfactory quality cannot be obtained from candidate synthesis units extracted from the synthesis database, and is updated based on the updated synthesis database.

2. The method of generating the record sentence of claim 1, wherein the synthesis information comprises:

text information that is syntactic interpretation information regarding a synthesis unit and a text unit related to the speech synthesis.

3. The method of generating the record sentence of claim 1, wherein the synthesis information comprises:

synthesis unit information that is phonetic interpretation information regarding a synthesis unit and a text unit related to the speech synthesis.

4. The method of generating the record sentence of claim 2, wherein the text information comprises:

linguistic interpretation information regarding the sentence of text.

5. The method of generating the record sentence of claim 3, wherein the synthesis unit information comprises:

phonetic interpretation information regarding the sentence of speech.

6. The method of generating the record sentence of claim 4, wherein the text information comprises:

at least one of a type of sentence, part of speech, information on whether a word of the sentence is an unseen unit, word information, parsing information of the sentence, and/or pause information of the sentence.

7. The method of generating the record sentence of claim 5, wherein the synthesis unit information comprises:

at least one of a prosody matching rate when a synthesis unit is synthesized and/or a distortion rate of a signal waveform of the synthesis unit.

8. The method of generating the record sentence of claim 1, wherein the selecting of the unseen sentence including an unseen unit is performed according to a number of candidate synthesis units extracted from a synthesis database when speech synthesis is performed.

9. The method of generating the record sentence of claim 1, wherein the selecting of the unseen sentence including an unseen unit is performed according to a replacement satisfaction degree of a replacement unit selected when speech synthesis is performed.

10. The method of generating the record sentence of claim 1, wherein the selecting of the unseen sentence including an unseen unit is performed according to a phonetic quality level of the sentence of speech.

11. The method of generating the record sentence of claim 1, wherein selecting of the unseen sentence including an unseen unit is performed according to a prosody matching rate when the synthesis unit is synthesized, or according to a distortion rate of a signal waveform of the synthesis unit.

12. The method of generating the record sentence of claim 1, wherein the generating of the weight comprises:

extracting the unseen unit included in the selected unseen sentence; and

generating the weight for the extracted unseen unit,

wherein the weight for the unseen unit is determined according to a linguistic criterion and/or a phonetic criterion for the unseen unit.

13. The method of generating the record sentence of claim 12, wherein the weight for the unseen unit is determined according to at least one of the frequency of occurrence of the unseen unit, a type of a word having the unseen unit, a part of speech of the unseen unit, a matching rate of the unseen unit, and/or a distortion rate of the unseen unit.

14. The method of generating the record sentence of claim 12, further comprising:

generating a weight for a word having the unseen unit, wherein the weight for the word is determined according to a linguistic criterion for the word and/or a phonetic criterion for the word.

15. The method of generating the record sentence of claim 14, wherein the weight for the word is determined according to at least one of the weight of the unseen unit, a type of the word, a location of the word, a matching rate of the word and/or the distortion rate of the word.

16. The method of generating the record sentence of claim 14, further comprising:

generating a weight for the sentence having the unseen unit, wherein the weight for the sentence is determined according to a linguistic criterion for the unseen unit and/or a phonetic criterion for the unseen unit.

17. The method of generating the record sentence of claim 16, wherein the weight for the sentence is determined according to at least one of the weight of the unseen unit included in the sentence, the weight of the word included in the sentence, and a type of the sentence.

18. The method of generating the record sentence of claim 1, wherein the generating of the record sentence further comprises:

selecting the unseen unit according to the unseen unit weight; and

generating a record sentence by combining the selected unseen unit with the speech synthesis information.

19. The method of generating the record sentence of claim 18, wherein the generating of the record sentence by combining the selected unseen unit with the speech synthesis information comprises:

generating a first candidate record sentence by combining the selected unseen unit with the speech synthesis information; and

generating a second candidate record sentence by performing at least one of word replacement, word addition, content word replacement, content word addition, and/or sentence structure modification.

20. The method of generating the record sentence of claim 19, wherein the generating of the second candidate record sentence is performed according to at least one of morpheme analysis, syntax analysis, dependent structure analysis, case structure analysis, and/or semantic analysis.

21. The method of generating the record sentence of claim 19, wherein the generating of the record sentence by combining the selected unseen unit with the speech synthesis information comprises:

generating a weight for the generated second candidate record sentence; and

generating a new second candidate record sentence by performing word replacement when the generated sentence weight of the second candidate record sentence is less than a predetermined threshold.

22. A non-transitory medium comprising a computer readable code for performing the method of generating the record sentence of claim 1.

23. A method of establishing a speech corpus, comprising:

performing speech synthesis for a predetermined sentence of text using candidate synthesis units transmitted from a synthesis database;

extracting an unseen unit from an unseen sentence by using synthesis information related to the speech synthesis;

generating a record sentence according to the extracted unseen unit;

converting the record sentence including the unseen unit into a speech signal; and

updating by storing the record sentence converted into the speech signal in the speech corpus,

wherein the synthesis database is updated based on the updated speech corpus,

wherein the unseen unit is selected as a synthesis unit when a speech unit of satisfactory quality cannot be obtained from candidate synthesis units extracted from the synthesis database, and is updated based on the updated synthesis database,

the generating of the record sentence is performed by combining the selected unseen unit with the speech synthesis information, and

the combining of the selected unseen unit with the speech synthesis information comprises generating a weight according to a linguistic criterion for the unseen unit and extracting the unseen unit in order according to the generated weight.

24. The speech corpus establishing method of claim 23, wherein the combining of the selected unseen unit with the speech synthesis information further comprises generating a weight according to a phonetic criterion for the unseen unit.

25. The speech corpus establishing method of claim 23, wherein the generating of the record sentence comprises:

generating a first candidate record sentence by combining the extracted unseen unit with the speech synthesis information; and

generating a second candidate record sentence by performing word replacement for the generated first candidate record sentence.

26. The speech corpus establishing method of claim 25, wherein the generating of the record sentence comprises:

generating a sentence weight for the generated second candidate record sentence; and

generating a new second candidate record sentence by again performing word replacement when the sentence weight of the generated second candidate record sentence is less than a predetermined threshold.

27. A non-transitory medium comprising a computer readable code for performing the method of generating the record sentence of claim 23.

28. An apparatus for generating a record sentence for establishing a speech corpus, the apparatus comprising:

a speech synthesis unit that generates a synthesized sentence of speech and synthesis information indicating information related to speech synthesis by performing speech synthesis for a predetermined sentence of text using candidate synthesis units transmitted from a synthesis database;

an unseen sentence selection unit that selects an unseen sentence including an unseen unit according to the generated synthesis information;

a generation unit extraction unit that generates a weight indicating a recording priority of an unseen unit included in the selected unseen sentence; and

a record sentence generation unit that generates a record sentence by combining an unseen unit with the speech synthesis information according to the generated weight and automatically updating the speech corpus by storing the record sentence including the unseen unit,

wherein the synthesis database is updated based on the updated speech corpus,

wherein the unseen unit is selected as a synthesis unit when a speech unit of satisfactory quality cannot be obtained from candidate synthesis units extracted from the synthesis database, and is updated based on the updated synthesis database.

29. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the record sentence generation unit selects the unseen unit according to the unseen unit weight, generates a first candidate record sentence by combining the selected unseen unit with the speech synthesis information by performing at least one of a word replacement, a word addition, content word replacement, content word addition, and/or sentence structure modification, and generates a second candidate record sentence.

30. The apparatus for generating the record sentence for establishing the speech corpus of claim 29, wherein the generation of the second candidate record sentence is performed according to at least one of morpheme analysis, syntax analysis, dependent structure analysis, case structure analysis, and/or semantic analysis.

31. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the synthesis information comprises:

synthesis unit information that is phonetic interpretation information regarding a synthesis unit and a text unit related to speech synthesis.

32. The apparatus for generating the record sentence for establishing the speech corpus of claim 31, wherein the synthesis unit information comprises:

phonetic interpretation information regarding a sentence of speech.

33. The apparatus for generating the record sentence for establishing the speech corpus of claim 32, wherein the text information comprises:

at least one of a type of the sentence, parts of speech, information on whether a word is an unseen unit, word information, parsing information of the sentence, and/or pause information.

34. The apparatus for generating the record sentence for establishing the speech corpus of claim 33, wherein the synthesis unit information comprises:

at least one of a prosody matching rate when a synthesis unit is synthesized and/or a distortion rate of a signal waveform of a synthesis unit.

35. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the unseen sentence selection unit selects the unseen sentence according to at least one of the number of candidate synthesis units extracted from a synthesis database when speech synthesis is performed, and/or a replacement satisfaction degree of a replacement unit selected when speech synthesis is performed.

36. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the unseen sentence selection unit selects the unseen sentence according to a phonetic quality level of the unseen sentence of speech.

37. The apparatus for generating the record sentence for establishing the speech corpus of claim 36, wherein the unseen sentence selection unit selects the unseen sentence according to a prosody matching rate when the synthesis unit is synthesized and/or according to a distortion rate of a signal waveform of the synthesis unit.

38. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the generation unit extraction unit extracts the unseen unit included in the selected unseen sentence, and generates a weight for the extracted unseen unit that is calculated according to a linguistic criterion and/or a phonetic criterion of the unseen unit.

39. The apparatus for generating the record sentence for establishing the speech corpus of claim 38, wherein the weight for the unseen unit is generated according to at least one of a frequency of occurrence of the unseen unit, a type of word having the unseen unit, a part of speech of the unseen unit, a matching rate of the unseen unit, and/or a distortion rate of the unseen unit.

40. The apparatus for generating the record sentence for establishing the speech corpus of claim 38, wherein the generation unit extraction unit generates a weight for a word having the unseen unit according to the weight of the unseen unit, and the weight for the word is calculated according to a linguistic criterion for the word and/or a phonetic criterion for the word.

41. The apparatus for generating the record sentence for establishing the speech corpus of claim 40, wherein the weight for the word is generated according to at least one of the weight of the unseen unit, a type of the word, a location of the word, a matching rate of the word, and/or a distortion rate of the word.

42. The apparatus for generating the record sentence for establishing the speech corpus of claim 40, wherein the generation unit extraction unit generates a weight for the sentence having the unseen unit according to the word weight, and the weight for the sentence is calculated according to a linguistic criterion for the unseen unit and/or a phonetic criterion for the unseen unit.

43. The apparatus for generating the record sentence for establishing the speech corpus of claim 42, wherein the weight for the sentence is generated according to at least one of the weight of the unseen unit included in the sentence, the weight of the word included in the sentence, and/or a type of the sentence.

44. The apparatus for generating the record sentence for establishing the speech corpus of claim 28, wherein the synthesis information comprises:

text information that is syntactic interpretation information regarding a synthesis unit and a text unit related to speech synthesis.