Synthesis by Generation and Concatenation of Multi-Form Segments

• US 20090048841A1
• Filed: 08/14/2007
• Published: 02/19/2009
• Est. Priority Date: 08/14/2007
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Pollet, Vincent, Breen, Andrew
• Granted Patent: US 8,321,222 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G10L 13/07 Concatenation rules
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]

Abstract

A speech synthesis system and method is described. A speech segment database references speech segments having various different speech representational structures. A speech segment selector selects from the speech segment database a sequence of speech segment candidates corresponding to a target text. A speech segment sequencer generates from the speech segment candidates sequenced speech segments corresponding to the target text. A speech segment synthesizer combines the selected sequenced speech segments to produce a synthesized speech signal output corresponding to the target text.

Claims

1. A speech synthesis system comprising:

a speech segment database referencing speech segments having a plurality of different speech representational structures;

a speech segment selector for selecting from the speech segment database a sequence of speech segment candidates corresponding to a target text;

a speech segment sequencer for generating from the speech segment candidates a sequenced speech segments corresponding to the target text; and

a speech segment synthesizer for combining the sequenced speech segments to produce a synthesized speech signal output corresponding to the target text.

2. A speech synthesis system according to claim 1, wherein the different speech representational structures include model-based speech signals and template-based speech signals.

3. A speech synthesis system according to claim 2, wherein the different speech representational structures include model-based speech signals augmented with template information.

4. A speech synthesis system according to claim 2, wherein the model-based speech representational structures are based on a statistical speech model.

5. A speech synthesis system according to claim 1, wherein the speech segment selector uses a statistical speech model for selecting the speech segment candidates.

6. A speech synthesis system according to claim 5, wherein the speech segment selector uses template information to augment the statistical model.

7. A speech synthesis system according to claim 1, wherein the different speech representational structures share at least one parameter component.

8. A speech synthesis system according to claim 7, wherein the shared parameter component is encoded differently in different speech representational structures.

9. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses static observations in generating the sequenced speech segments.

10. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses observations from the speech segment selector in generating the sequenced speech segments.

11. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses static observations and observations from the speech segment selector in generating the sequenced speech segments.

12. A speech synthesis system according to claim 1, wherein the speech segment selector uses statistically derived cost-functions for selecting the speech segment candidates.

13. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses statistically derived cost-functions for generating the sequenced speech segments.

14. A speech synthesis system according to claim 1, wherein the speech segment selector uses empirical rules for selecting the speech segment candidates.

15. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses empirical rules for generating the sequenced speech segments.

16. A speech synthesis system according to claim 1, wherein the speech segment selector uses psycho-acoustic rules for selecting the speech segment candidates.

17. A speech synthesis system according to claim 1, wherein the speech segment sequencer uses psycho-acoustic rules for generating the sequenced speech segments.

35. A speech synthesis system according to claim 12, wherein the statistically derived cost-functions are based on sequences of speech segment observations.

36. A speech synthesis system according to claim 35, wherein the sequences of speech segment observations are described by a Markov process.

37. A speech synthesis system according to claim 13, wherein the statistically derived cost-functions are based on sequences of speech segment observations.

38. A speech synthesis system according to claim 37, wherein the sequences of speech segment observations are described by a Markov process.

18. A method of speech synthesis comprising:

referencing in a speech segment database speech segments having a plurality of different speech representational structures;

selecting from the speech segment database a sequence of speech segment candidates corresponding to a target text;

generating from the speech segment candidates a sequenced speech segments corresponding to the target text; and

combining the sequenced speech segments to produce a synthesized speech signal output corresponding to the target text.

19. A method according to claim 18, wherein the different speech representational structures include model-based speech signals and template-based speech signals.

20. A method according to claim 19, wherein the different speech representational structures include model-based speech signals augmented with template information.

21. A method according to claim 19, wherein the model-based speech representational structures are based on a statistical speech model.

22. A method according to claim 18, wherein a statistical speech model is used in selecting the sequence of speech segment candidates.

23. A method according to claim 22, wherein the statistical speech model is augmented by template information.

24. A method according to claim 18, wherein the different speech representational structures share at least one parameter component.

25. A method according to claim 24, wherein the shared parameter component is encoded differently in different speech representational structures.

26. A method according to claim 18, wherein static observations are used in generating the sequenced speech segments.

27. A method according to claim 18, wherein observations from the selecting the sequence of speech segment candidates are used in the generating the sequenced speech segments.

28. A method according to claim 18, wherein static observations and observations from the selecting the sequence of speech segment candidates are used in the generating the sequenced speech segments.

29. A method according to claim 18, wherein statistically derived cost-functions are used for selecting the speech segment candidates.

30. A method according to claim 18, wherein statistically derived cost-functions are used for generating the sequenced speech segments.

31. A method according to claim 18, wherein empirical rules are used for selecting the speech segment candidates.

32. A method according to claim 18, wherein empirical rules are used for generating the sequenced speech segments.

33. A method according to claim 18, wherein psycho-acoustic rules are used for selecting the speech segment candidates.

34. A method according to claim 18, wherein psycho-acoustic rules are used for generating the sequenced speech segments.

39. A method according to claim 29, wherein the statistically derived cost-functions are based on sequences of speech segment observations.

40. A method according to claim 39, wherein the sequences of speech segment observations are described by a Markov process.

41. A method according to claim 30, wherein the statistically derived cost-functions are based on sequences of speech segment observations.

42. A method according to claim 41, wherein the sequences of speech segment observations are described by a Markov process.