Methods and apparatus for rapid acoustic unit selection from a large speech corpus

• US 7,761,299 B1
• Filed: 03/27/2008
• Issued: 07/20/2010
• Est. Priority Date: 04/30/1999
• Status: Expired due to Fees

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Beutnagel, Mark Charles, Mohri, Mehryar, Riley, Michael Dennis
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US12/057,020
• Time in Patent Office: 845 Days
• Field of Search: 704/258, 704/260, 704/263, 704/266
• US Class Current: 704/258
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/07 Concatenation rules
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

A speech synthesis system can select recorded speech fragments, or acoustic units, from a large database of acoustic units to produce artificial speech. The selected acoustic units are chosen to minimize a combination of target and concatenation costs for a given sentence. Concatenation costs are expensive to compute. Processing is reduced by pre-computing and caching the concatenation costs. The number of possible sequential pairs of acoustic units makes such caching prohibitive. A method for constructing an efficient concatenation cost database is provided by synthesizing a large body of speech, identifying the acoustic unit sequential pairs generated and their respective concatenation costs, and storing those concatenation costs likely to occur.

Claims

1. A concatenation cost database stored in a computer-readable medium, the concatenation cost database generated according to a method comprising:

synthesizing a body of speech;

identifying acoustic unit sequential pairs generated in the body of speech and their respective concatenation costs; and

storing the respective concatenation costs in a concatenation cost database.

2. The concatenation cost database of claim 1, wherein the identified acoustic unit sequential pairs are used to prune an acoustic unit database.

3. The concatenation cost database of claim 1, wherein the stored concatenation costs are derived using statistical techniques.

4. The concatenation cost database of claim 1, wherein the body of speech is synthesized using text-to-speech synthesis.

5. The concatenation cost database of claim 1, wherein the identified acoustic unit sequential pairs are a subset of all acoustic unit sequential pairs generated from the synthesized speech.

6. The concatenation cost database of claim 5, wherein the identified subset represents each unique acoustic sequential pair.

7. The concatenation cost database of claim 5, wherein the identified subset represents acoustic unit sequential pairs that are relatively inexpensive to concatenate.

8. The concatenation cost database of claim 1, wherein the identified acoustic unit sequential pairs represent acoustic unit sequential pairs unlikely to occur naturally.

9. A concatenation cost database stored in a computer-readable medium, the concatenation cost database generated according to a method comprising:

synthesizing a test body of text associated with an acoustic unit database;

pruning acoustic units from the acoustic unit database that are not used in the synthesis of the test body of text; and

storing, in a concatenation cost database, the respectable concatenation costs for sequential acoustic units in the pruned acoustic unit database.

10. A concatenation cost database stored in a computer-readable medium, the concatenation cost database generated according to a method comprising:

synthesizing a body of text;

logging a concatenation cost for each synthesized acoustic unit sequential pair; and

selecting, for entry into a concatenation cost database, a set of acoustic unit sequential pairs and their associated concatenation costs.

11. The concatenation cost database of claim 10, wherein the selecting occurs based on whether each acoustic unit sequential pair is unique.

12. The concatenation cost database of claim 10, wherein the selecting occurs based on whether each acoustic unit sequential pair has a relatively inexpensive concatenation cost.

13. A method comprising:

selecting a pair of acoustic units from an acoustic unit database;

identifying a concatenation cost between the pair of acoustic units based on communication with a concatenation cost database; and

synthesizing a speech signal using the concatenation cost for the selected pair of acoustic units.

14. The method of claim 13, wherein the concatenation cost is a measure of the mismatch between the pair of acoustic units.

15. The method of claim 13, wherein the concatenation cost database contains a subset of all possible acoustic unit sequential pairs.

16. The method of claim 13, wherein the communication with the concatenation cost database comprises:

extracting a concatenation cost of the pair of acoustic units from the concatenation cost database if the concatenation cost database contains the concatenation cost of the pair of acoustic units; and

determining a value of the concatenation cost of the pair of acoustic units if the concatenation cost data base does not contain the concatenation cost of the pair of acoustic units.

17. The method of claim 13, wherein the concatenation cost database is derived at least in part using statistical techniques which predict acoustic unit sequential pairs likely to occur in speech.

18. The method of claim 13, wherein the concatenation cost database is derived at least in part by assigning costs to acoustic unit sequential pairs.

19. The method of claim 13, wherein selecting at least one acoustic unit from the acoustic unit database further uses at least one target cost of an acoustic unit, the target cost being a measure of the mismatch between an acoustic unit and a phoneme.

20. The method of claim 16, wherein determining a value of the concatenation cost of the pair of acoustic units comprises computing the concatenation cost of the pair of acoustic units.