Methods and apparatus for text to speech processing using language independent prosody markup

• US 6,856,958 B2
• Filed: 04/30/2001
• Issued: 02/15/2005
• Est. Priority Date: 09/05/2000
• Status: Expired due to Fees

• Current Assignee: Alcatel-Lucent USA, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Kochanski, Gregory P., Shih, Chi-Lin
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/845,561
• Publication Number:
  US 20030009338A1
• Time in Patent Office: 1,387 Days
• Field of Search: 704/260, 704/258, 704/270, 704/275, 704/278, 704/207
• US Class Current: 704/260
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

Techniques are described for employing a set of tags to model phenomena which are smooth and subject to constraints. Tags may be used to model, for example, muscular movement producing speech. In one advantageous application, a set of tags defining prosodic characteristics is developed, and selected tags are placed in appropriate locations of a body of text. Each tag defines a constraint on the prosodic characteristics of speech produced by processing the text. Processing of the body of speech and the tags produces a set of equations which are solved to produce a curve defining prosodic characteristics over the scope of a phrase, and a further set of equations which are solved to produce a curve defining prosodic characteristics of individual words within a phrase. The data defined by the curves is used with the text to produce speech having the prosodic characteristics defined by the tags. A set of tags may be produced by reading of a training text by a target speaker to produce a training corpus reflecting the prosodic characteristics of the target speaker, and then analyzing the training corpus to generate tags modeling the prosodic characteristics of the training corpus.

Claims

1. A method of modeling phenomena comprising the steps of:

analyzing one or more instances of actual phenomena to identify characteristics of the instances of the actual phenomena;

creating a set of tags defining the identified characteristics of the one or more instances of the actual phenomena each tag controlling one or more aspects of one or more molded phenomena to be produced in response to the tags, the tags controlling the aspects of the modeled phenomena so as to create characteristics in the modeled phenomena similar to those exhibited by the one or more instances of the actual phenomena;

arranging selected members of the set of tags in a desired sequence to produce phenomena as defined by the sequence of tags; and

processing the tags in order to produce phenomena having the characteristics defined by the tags.

2. The method of claim 1 wherein the phenomena controlled by the tags are characteristics of speech, wherein the step of arranging selected members of the tags in a desired sequence comprises placing the selected members of the set of tags into a body of text and wherein the step of processing the tags comprises processing the body of text and the tags to produce speech having characteristics defined by the tags.

3. The method of claim 2 wherein the characteristics of speech arc prosodic characteristics of speech.

4. The method of claim 3 wherein each tag imposes a constraint on the prosodic characteristics of speech affected by the tag.

5. The method of claim 4 wherein each of the tags specifies an action to be taken and includes parameters defining attributes and associated values providing information about the action to be taken.

6. The method of claim 5 wherein each of the tags may include a parameter specifying the location at which the tag takes effect.

7. The method of claim 6 wherein the set of tags includes tags which establish settings which remain unchanged until altered by a subsequent tag.

8. The method of claim 7 wherein the set of tags includes members which define the pitch behavior of speech over the course of a phrase.

9. The method of claim 8 wherein the set of tags includes tags defining accents which define the pitch behavior of local influences within a phrase.

10. The method of claim 9 wherein each of the tags may include values defining type and strength in order to define interaction of the tag with other tags.

11. The method of claim 10 wherein a tag may compromise its shape, average pitch or both depending on the value defining type.

12. The method of claim 9 wherein the step of processing the tags includes establishing a pitch curve by creating and solving equations defined by tags which specify accents.

13. The method of claim 12 wherein the body of text and the tags are processed one minor phrase at a time.

14. The method of claim 13 wherein processing of a phrase includes using values describing properties prevailing near the end of an immediately preceding phrase.

15. The method of claim 9 wherein one or more tags are placed within a proper noun comprising two or more words, each such tag producing prosody indicating to a listener that the proper noun is to be interpreted as a single entity rather than as more than one entity.

16. The method of claim 15 wherein the tag produces an increase in the pitch and speed of speech over the speech affected by the tag.

17. The method of claim 9 wherein one or more tags are placed to produce a word having prosody indicating that the word requires confirmation.

18. The method of claim 17 wherein the prosody indicating that the word requires confirmation is characterized by a relatively high and increasing pitch across the word requiring confirmation.

19. The method of claim 6 wherein the set of tags includes tags defining phrase boundaries which mark boundaries between regions at which tags have effect.

20. The method of claim 19 wherein a tag which defines a phrase boundary prevents tags following the tag which marks the boundary from influencing speech components preceding the tag which marks the boundary.

21. The method of claim 8 wherein the step of processing the tags includes establishing a phrase curve by creating and solving equations defined by tags which specifying changes in pitch and tags which specifying rates of changes in pitch.

22. The method of claim 21 wherein the body of text and the tags are processed one minor phrase at a time.

23. The method of claim 22 wherein processing of a phrase includes using values describing properties prevailing near the end of an immediately preceding phrase.

24. The method of claim 2 wherein each tag imposes a constraint on motion of an articulator used to produce speech.

25. The method of claim 1 wherein each tag imposes a constraint on modeled muscular motions used to simulate gestures or facial expression.

26. A method of processing a body of text including tags defining prosodic characteristics of speech to be produced by processing the texts comprising the steps of:

extracting the tags from the text;

creating a set of equations defining a phrase curve;

solving the set of equations to produce the phrase curve;

creating a set of equations defining a pitch curve;

solving the set of equations to produce the pitch curve;

mapping linguistic concepts represented by the phrase curve and the pitch curve to acoustical observables; and

performing a nonlinear transformation to adjust the prosodic characteristics defined by tags to human perceptions and expectations.

27. A method of defining a set of tags specifying prosodic characteristics of speech of a target speaker, comprising the steps of:

selecting a body of training text;

receiving speech representing reading of the training text by the target speaker to form a training corpus, the training corpus representing actual sounds produced by the reading of the training text by the target speaker and exhibiting prosodic characteristics of actual speech of the target speaker;

analyzing the training corpus to identify prosodic characteristics of the training corpus; and

creating a set of tags defining the identified prosodic characteristics of the training corpus.

28. A method of placing tags in text for text to speech processing comprising the steps of:

placing tags in a body of training text to model prosodic characteristics of a training corpus produced by reading of the training text;

analyzing the placement of the tags in the training text to develop a set of rules for placement of tags in text; and

applying the rules to text for which text to speech processing is desired to place tags in the text in order to produce speech having desired prosodic characteristics.

29. A text to speech system for receiving text inputs comprising text to be processed to generate speech and tags defining prosodic characteristics of the speech to be generated, comprising:

a prosody tag generation component to analyze a training corpus to identify characteristics exhibited by one or more readings of text by one or more target speakers and to generate a set of tags defining the identified characteristics;

a text input interface for receiving the text input;

a speech modeler operative to process the text inputs to produce speech having the prosodic characteristics specified by the tags, such that the speech produced by the speech modeler is similar to that of the one or more target speakers; and

a speech output interface for producing the speech output.

30. The system of claim 29 wherein the speech modeler is further operative to process a training corpus representing a reading of text by a target speaker to produce tags defining prosodic characteristics of the training corpus and use the tags to produce speech having prosodic characteristics typical of the target speaker.