Optimization of an objective measure for estimating mean opinion score of synthesized speech

• US 7,386,451 B2
• Filed: 09/11/2003
• Issued: 06/10/2008
• Est. Priority Date: 09/11/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Peng, Hu, Zhao, Yong, Chu, Min
• Primary Examiner(s): Hudspeth; David
• Assistant Examiner(s): Sked; Matthew J
• Application Number:
  US10/660,388
• Publication Number:
  US 20050060155A1
• Time in Patent Office: 1,734 Days
• Field of Search: None
• US Class Current: 704/260
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 25/69 for evaluating synthetic or decoded voice signals

Abstract

A method is provided for optimizing an objective measure used to estimate mean opinion score or naturalness of synthesized speech from a speech synthesizer. The method includes using an objective measure that has components derived directly from textual information used to form synthesized utterances. The objective measure has a high correlation with mean opinion score such that a relationship can be formed between the objective measure and corresponding mean opinion score. The objective measure is altered to provide a different function of textual information derived from the utterances so as to improve the relationship between the scores of the objective measure and subjective ratings of the synthesized utterances.

Claims

1. A method for optimizing an objective measure, from which naturalness of synthesized speech can be estimated, wherein naturalness is a subjective quality of synthesized speech, the method comprising:

generating a set of synthesized utterances;

subjectively rating each of the synthesized utterances;

calculating a score for each of the synthesized utterances using an objective measure, the objective measure being a function of textual information derived from the utterances;

ascertaining a relationship between the scores of the objective measure and subjective ratings of the synthesized utterances; and

altering the objective measure in a manner beyond only changing one or more weighting factors in the objective measure to provide a different function of textual information derived from the utterances so as to improve the relationship between the scores of the objective measure and subjective ratings of the synthesized utterances.

2. The method of claim 1 wherein the step of altering is repeated, and wherein each repetition includes using the same subjective ratings of the synthesized utterances and textual information of the synthesized utterances.

3. The method of claim 1 wherein the objective measure includes components having categorical values, and wherein a distance between categories are empirically defined as values in distance tables, and wherein altering includes altering the values in the distance tables.

4. The method of claim 1 wherein the objective measure comprises one or more first order components from a set of factors and/or one or more higher order components being combinations of at least two factors from the set of factors, wherein the set of factors include:

an indication of a position of a speech unit in a phrase;

an indication of a position of a speech unit in a word;

an indication of a category for a phoneme preceding a speech unit;

an indication of a category for a phoneme following a speech unit;

an indication of a category for tonal identity of the current speech unit;

an indication of a category for tonal identity of a preceding speech unit;

an indication of a category for tonal identity of a following speech unit; and

an indication of a level of stress of a speech unit;

an indication of a coupling degree of pitch, duration and/or energy with a neighboring unit; and

an indication of a degree of spectral mismatch with a neighboring speech unit.

5. The method of claim 4 wherein the components of the objective measure include categorical values, and wherein a distance between categories are empirically defined as values in distance tables, and wherein altering includes altering the values in the distance tables.

6. The method of claim 4 wherein components of the objective measure each include a weighting value, and wherein altering includes altering the weighting values.

7. The method of claim 6 wherein altering the objective measure comprises selecting components of the objective measure as a function of the weighting factor of each component.

8. The method of claim 4 wherein altering the objective measure comprises selecting components of the objective measure as a function of its respective correlation to the subjective ratings of the synthesized utterances.

9. The method of claim 1 wherein the objective measure comprises an indication of a position of a speech unit in a phrase.

10. The method of claim 1 wherein the objective measure comprises an indication of a position of a speech unit in a word.

11. The method of claim 1 wherein the objective measure comprises an indication of a category for a phoneme preceding a speech unit.

12. The method of claim 1 wherein the objective measure comprises an indication of a category for a phoneme following a speech unit.

13. The method of claim 1 wherein the objective measure comprises an indication of a category for the tone of a preceding speech unit.

14. The method of claim 1 wherein the objective measure comprises an indication of a category for the tone of a following speech unit.

15. The method of claim 1 wherein the objective measure comprises an indication of a spectral mismatch between successive speech units.

16. The method of claim 1 wherein the objective measure comprises an indication of a category for tonal identity of the current speech unit.

17. The method of claim 1 wherein the objective measure comprises an indication of a coupling degree of pitch, duration and/or energy with a neighboring unit.

18. The method of claim 1 wherein the objective measure comprises an indication of level of stress of a speech unit.

19. The method of claim 1 wherein the objective measure score for each synthesized utterance is a function of a length of said each synthesized utterance.

20. The method of claim 19 wherein the length comprises a number of speech units in an utterance.

21. A method for optimizing an objective measure, from which naturalness of synthesized speech can be estimated, wherein naturalness is a subjective quality of synthesized speech, the method comprising:

generating a set of synthesized utterances;

subjectively rating each of the synthesized utterances;

calculating a score for each of the synthesized utterances using an objective measure, the objective measure being a function of textual information derived from speech units used in the utterances and the objective measure comprising components being based on single-order textual features or a combination of at least two single-order textual features, the components having categorical values, wherein a distance between categories are empirically defined as values in distance tables, the components each further having a weighting value;

ascertaining a relationship between the scores of the objective measure and subjective ratings of the synthesized utterances; and

altering the objective measure in a manner beyond only changing one or more weighting factors in the objective measure to provide a different function of textual information derived from the utterances so as to improve the relationship between the scores of the objective measure and subjective ratings of the synthesized utterances, wherein altering comprises altering the values in the distance tables followed by altering the weighting values.

22. The method of claim 21 and further comprising removing components of the objective measure as a function of the weighting values, and adjusting the weighting values of remaining components.

23. The method of claim 22 wherein altering the objective measure comprises selecting components of the objective measure as a function of the weighting factor of each component.

24. The method of claim 21 wherein altering the objective measure comprises selecting components of the objective measure as a function of its respective correlation to the subjective ratings of the synthesized utterances.

25. The method of claim 21 wherein the objective measure comprises at least one component being a combination of at least two factors from a set including:

an indication of a position of a speech unit in a phrase;

an indication of a position of a speech unit in a word;

an indication of a category for a phoneme preceding a speech unit;

an indication of a category for a phoneme following a speech unit;

an indication of a category for tonal identity of the current speech unit;

an indication of a category for tonal identity of a preceding speech unit;

an indication of a category for tonal identity of a following speech unit; and

an indication of a level of stress of a speech unit;

an indication of a coupling degree of pitch, duration and/or energy with a neighboring unit; and

an indication of a degree of spectral mismatch with a neighboring speech unit.