PROSODY EDITING APPARATUS AND METHOD

• US 20140052446A1
• Filed: 08/15/2013
• Published: 02/20/2014
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: MORI, Kouichirou, MORITA, Masahiro, KAGOSHIMA, Takehiko
• Granted Patent: US 9,601,106 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

According to one embodiment, a prosody editing apparatus includes a storage, a first selection unit, a search unit, a normalization unit, a mapping unit, a display, a second selection unit, a restoring unit and a replacing unit. The search unit searches the storage for one or more second prosodic patterns corresponding to attribute information that matches attribute information of the selected phrase. The mapping maps each of the normalized second prosodic patterns on a low-dimensional space. The restoring unit restores a restored prosodic pattern according to the selected coordinates. The replacing unit replaces prosody of synthetic speech generated based on the selected phrase by the restored prosodic pattern.

Claims

1. A prosody editing apparatus comprising:

a storage configured to store attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases, the attribute information items each indicating an attribute associated with a phrase, the first prosodic patterns each including parameters which indicate a prosody type of the phrase and expresses prosody of the phrase, the parameters each including elements not less than the number of phonemes of the phrase;

a first selection unit configured to select a phrase including phonemes from text to obtain a selected phrase;

a search unit configured to search the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of the selected phrase to obtain as a prosodic pattern set, the second prosodic patterns being included in the first prosodic patterns;

a normalization unit configured to normalize the second prosodic patterns respectively;

a mapping unit configured to map each of the normalized second prosodic patterns on a low-dimensional space represented by one or more coordinates smaller than the number of the elements to generate mapping coordinates;

a display configured to display the mapping coordinates;

a second selection unit configured to obtain coordinates selected from the mapping coordinates as selected coordinates;

a restoring unit configured to restore a prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

a replacing unit configured to replace prosody of synthetic speech generated based on the selected phrase by the restored prosodic pattern.

2. The apparatus of claim 1, further comprising a generation unit configured to generate a third prosodic pattern associated with the selected phrase using a statistical model, and to add the third prosodic pattern to the prosodic pattern set.

3. The apparatus of claim 1, further comprising a speech synthesis unit configured to apply speech synthesis to the text based on the restored prosodic pattern to generate synthetic speech.

4. The apparatus of claim 1, wherein the attribute information items each includes a surface expression which indicates a character string of the phrase, and

the search unit searches for whether or not a surface expression of the selected phrase matches a surface expression of the phrase.

5. The apparatus of claim 1, wherein the attribute information items each includes a phoneme sequence which indicates a character string of the phoneme of the phrase, and

the search unit searches for whether or not a phoneme sequence of the selected phrase matches a phoneme sequence of the phrase.

6. The apparatus of claim 1, wherein the attribute information items each includes a mora count of the phrase and an accent type of the phrase, and

the search unit searches for whether or not a mora count of the selected phrase and an accent type of the selected phrase match a mora count of the phrase and an accent type of the phrase.

7. The apparatus of claim 1, wherein the parameters of the first prosodic patterns each includes fundamental frequency of a phoneme, duration of the phoneme, and power of the phoneme, and

the mapping unit independently maps one or more parameters of the fundamental frequency, the duration, and the power.

8. The apparatus of claim 1, wherein the first prosodic pattern is expressed by fundamental frequency of a phoneme, duration of the phoneme, and power of the phoneme, and

the mapping unit couples and maps two or more parameters of the fundamental frequency, the duration, and the power.

9. The apparatus of claim 1, wherein the mapping unit clusters the mapping coordinates based on distances between the mapping coordinates, and determines representative points from each of clustered mapping coordinates, and

the display displays the representative points.

10. The apparatus of claim 1, wherein if a distance between the selected coordinates and the mapping coordinates is not more than a threshold, the restoring unit obtains a prosodic pattern before mapping the mapping coordinates as the restored prosodic pattern.

11. A prosody editing method comprising:

storing, in a storage, attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases, the attribute information items each indicating an attribute associated with a phrase, the first prosodic patterns each including parameters which indicate a prosody type of the phrase and expresses prosody of the phrase, the parameters each including elements not less than the number of phonemes of the phrase;

selecting a phrase including phonemes from text to obtain a selected phrase;

searching the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of the selected phrase to obtain as a prosodic pattern set, the second prosodic patterns being included in the first prosodic patterns;

normalizing the second prosodic patterns respectively;

mapping each of the normalized second prosodic patterns on a low-dimensional space represented by one or more coordinates smaller than the number of the elements to generate mapping coordinates;

displaying the mapping coordinates;

obtaining coordinates selected from the mapping coordinates as selected coordinates;

restoring a prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

replacing prosody of synthetic speech generated based on the selected phrase by the restored prosodic pattern.

12. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method comprising:

storing, in a storage, attribute information items of phrases and one or more first prosodic patterns corresponding to each of the attribute information items of the phrases, the attribute information items each indicating an attribute associated with a phrase, the first prosodic patterns each including parameters which indicate a prosody type of the phrase and expresses prosody of the phrase, the parameters each including elements not less than the number of phonemes of the phrase;

selecting a phrase including phonemes from text to obtain a selected phrase;

searching the storage for one or more second prosodic patterns corresponding to an attribute information item that matches an attribute information item of the selected phrase to obtain as a prosodic pattern set, the second prosodic patterns being included in the first prosodic patterns;

normalizing the second prosodic patterns respectively;

mapping each of the normalized second prosodic patterns on a low-dimensional space represented by one or more coordinates smaller than the number of the elements to generate mapping coordinates;

displaying the mapping coordinates;

obtaining coordinates selected from the mapping coordinates as selected coordinates;

restoring a prosodic pattern according to the selected coordinates to obtain a restored prosodic pattern; and

replacing prosody of synthetic speech generated based on the selected phrase by the restored prosodic pattern.