Prosody editing apparatus and method

• US 9,601,106 B2
• Filed: 08/15/2013
• Issued: 03/21/2017
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Mori, Kouichirou, Kagoshima, Takehiko, Morita, Masahiro
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/968,154
• Publication Number:
  US 20140052446A1
• Time in Patent Office: 1,314 Days
• Field of Search: 704258-269
• US Class Current: 1/1
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

According to one embodiment, a prosody editing apparatus includes a storage, a first selection unit, a search unit, a normalization unit, a mapping unit, a display, a second selection unit, a restoring unit and a replacing unit. The search unit searches the storage for one or more second prosodic patterns corresponding to attribute information that matches attribute information of the selected phrase. The mapping maps each of the normalized second prosodic patterns on a low-dimensional space. The restoring unit restores a restored prosodic pattern according to the selected coordinates. The replacing unit replaces prosody of synthetic speech generated based on the selected phrase by the restored prosodic pattern.

Claims

1. A prosody editing apparatus comprising:

a storage configured to store attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases;

a search unit configured to search the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of a predetermined phrase, the second prosodic patterns being included in the first prosodic patterns;

a mapping unit configured to map each of the second prosodic patterns on a low dimensional space to generate mapping coordinates, the mapping coordinates being used to suppress a first prosodic pattern which is not assumed normally, wherein a first distance between coordinates of the first prosodic pattern and coordinates of a target prosodic pattern is not within a first threshold;

a selection unit configured to obtain coordinates selected from the mapping coordinates as selected coordinates;

a restoring unit configured to restore a second prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

a replacing unit configured to replace prosody of synthetic speech generated based on the predetermined phrase by the restored prosodic pattern.

2. The apparatus of claim 1, further comprising a generation unit configured to generate a third prosodic pattern associated with the predetermined phrase using a statistical model, and to add the third prosodic pattern to a prosodic pattern set.

3. The apparatus of claim 1, further comprising a speech synthesis unit configured to apply speech synthesis to the text based on the restored prosodic pattern to generate synthetic speech.

4. The apparatus of claim 1, wherein the attribute information items each includes a surface expression which indicates a character string of the phrase, and

the search unit searches for whether or not a surface expression of the predetermined phrase matches a surface expression of the phrase.

5. The apparatus of claim 1, wherein the attribute information items each includes a phoneme sequence which indicates a character string of the phoneme of the phrase, and

the search unit searches for whether or not a phoneme sequence of the predetermined phrase matches a phoneme sequence of the phrase.

6. The apparatus of claim 1, wherein the attribute information items each includes a mora count of the phrase and an accent type of the phrase, and

the search unit searches for whether or not a mora count of the predetermined phrase and an accent type of the predetermined phrase match a mora count of the phrase and an accent type of the phrase.

7. The apparatus of claim 1, wherein parameters of the first prosodic patterns each includes fundamental frequency of a phoneme, duration of the phoneme, and power of the phoneme, and

the mapping unit independently maps one or more parameters of the fundamental frequency, the duration, and the power.

8. The apparatus of claim 1, wherein the first prosodic patterns are expressed by fundamental frequency of a phoneme, duration of the phoneme, and power of the phoneme, and

the mapping unit couples and maps two or more parameters of the fundamental frequency, the duration, and the power.

9. The apparatus of claim 1, wherein if a second distance between the selected coordinates and the mapping coordinates is not more than a second threshold, the restoring unit obtains a fourth prosodic pattern before mapping the mapping coordinates as the restored prosodic pattern.

10. The apparatus of claim 1, further comprising a display configured to display the mapping coordinates.

11. The apparatus of claim 10, wherein the mapping unit clusters the mapping coordinates based on distances between the mapping coordinates, and determines representative points from each of clustered mapping coordinates, and

the display displays the representative points.

12. The apparatus of claim 1, further comprising a second selection unit configured to select the phrase from a text.

13. The apparatus of claim 1, further comprising a normalization unit configured to normalize the second prosodic patterns respectively.

14. The apparatus according to claim 1, wherein the low-dimensional space is represented by few coordinates.

15. The apparatus according to claim 1, wherein the low-dimensional space is represented by one or more coordinates that is smaller than elements no less than the number of phonemes of the phrase.

16. A prosody editing method comprising:

storing, in a storage, attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases;

searching the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of a predetermined phrase, the second prosodic patterns being included in the first prosodic patterns;

mapping each of the second prosodic patterns on a low-dimensional space to generate mapping coordinates, the mapping coordinates being used to suppress to suppress a first prosodic pattern which is not assumed normally, wherein a first distance between coordinates of the first prosodic pattern and coordinates of a target prosodic pattern is not within a first threshold;

obtaining coordinates selected from the mapping coordinates as selected coordinates;

restoring a second prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

replacing prosody of synthetic speech generated based on the predetermined phrase by the restored prosodic pattern.

17. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method comprising:

storing, in a storage, attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases;

searching the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of a predetermined phrase, the second prosodic patterns being included in the first prosodic patterns;

mapping each of the second prosodic patterns on a low-dimensional space to generate mapping coordinates, the mapping coordinates being used to suppress a first prosodic pattern which is not assumed normally, wherein a first distance between coordinates of the first prosodic pattern being suppressed and coordinates of a target prosodic pattern is not within a first threshold;

obtaining coordinates selected from the mapping coordinates as selected coordinates;

restoring a prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

replacing prosody of synthetic speech generated based on the predetermined phrase by the restored prosodic pattern.