Reduced unit database generation based on cost information

• US 6,988,069 B2
• Filed: 01/31/2003
• Issued: 01/17/2006
• Est. Priority Date: 01/31/2003
• Status: Expired due to Term

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: SpeechWorks International, Inc. (Microsoft Corporation)
• Inventors: Phillips, Michael Stuart
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Harper, V. Paul
• Application Number:
  US10/355,143
• Publication Number:
  US 20040153324A1
• Time in Patent Office: 1,082 Days
• Field of Search: 704/277, 704/270, 704/260, 704/259, 704/258, 704/257, 704/254, 704/241, 704/220, 704/211
• US Class Current: 704/258
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management

Abstract

An arrangement is provided for generating a reduced unit database of a desired size to be used in text to speech operations. A reduced unit database with a desired size is generated based on a full unit database. The reduction is carried out with respect to a text database with a plurality of sentences. Units from the full database are pruned to minimize an overall cost associated with using alternative units other than the units in the reduced unit database.

Claims

1. A method comprising:

determining a desired size of a reduced unit database for text to speech operations;

generating the reduced unit database of the desired size based on a full unit data base in order to minimize an overall cost in using the units in the reduced unit database to accomplish the text to speech operations; and

performing the text to speech operations using the reduced unit database with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that a cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database; and

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database, wherein said pruning comprises;

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an a cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

2. The method according to claim 1 wherein the text to speech operations are performed by any one of:

an application software;

a firmware; and

a hardware.

3. The method according to claim 1, wherein the text to speech operations are performed on a device that includes any one of:

a computer;

a personal data assistant;

a cellular phone; and

a dedicated device deployed for an application.

4. The method according to claim 3, wherein the computer includes any one of:

a personal computer;

a laptop;

a special purpose computer; and

a general purpose computer.

5. The method according to claim 3, wherein the desired size of the reduced unit database is determined according to at least some features of the device.

6. The method according to claim 5, wherein the features of the device include any one of:

the amount of memory available on the device; and

the computation capability of the device.

7. The method according to claim 1, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

8. The method according to claim 7, further comprising:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

9. The method according to claim 1, wherein said determining the cost increase comprises:

determining an original overall cost across all relevant sentences for which the next unit is selected during the text to speech operations;

performing text to speech operations on the relevant sentences,

wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

10. The method according to claim 1, further comprising:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are stored in a compressed form.

11. The method according to claim 1, further comprising:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

12. A method to generate a reduced unit database based on a full unit database, comprising:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database; and

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database;

wherein said pruning comprises;

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining a cost increase induced when a next unit in the reduced unit database Is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

13. The method according to claim 12, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies a desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

14. The method according to claim 13, further comprising:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

15. The method according to claim 12, wherein said

determining the cost increase comprises:

determining an original overall cost across all relevant sentences for which the next unit is selected during the text to speech operations;

performing text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations, and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

16. The method according to claim 15, wherein the overall cost across the relevant sentences is computed as a summation of the costs associated with individual relevant sentences.

17. The method according to claim 12, wherein the cost of using selected units to achieve text to speech with respect to a sentence includes at least one of:

context cost; and

concatenation cost.

18. The method according to claim 12, further comprising:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are in a compressed form.

19. The method according to claim 12, further comprising:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

20. A system, comprising:

a unit database reduction mechanism capable of generating a reduced unit database of a desired size from a full unit database based on cost information; and

a text to speech mechanism capable of performing text to speech operations using the reduced unit database;

wherein the unit database reduction mechanism comprises;

a text database including a plurality of sentences; and

a cost-based subset unit generation mechanism capable of pruning the full unit database to generate the reduced unit database using cost information associated with unit selection in carrying out text to speech operations with respect to the plurality of sentences in the text database using a unit pruning mechanism capable of pruning the units selected from the full unit database to produce the reduced unit database according to the cost associated with each of the sentences and at least one pruning criterion, wherein the unit pruning mechanism further comprises;

a cost increase estimation mechanism capable of estimating a cost increase related to a pruned unit, the cost increase being induced when the pruned unit is made unavailable for unit selection during text to speech operations; and

a cost increase based pruning mechanism capable of determining whether the pruned unit is to be removed according to the cost increase and the at least one pruning criterion.

21. The system according to claim 20, wherein the cost based subset unit generation mechanism comprises:

a unit selection based text to speech mechanism capable of selecting units from the full unit database with respect to the sentences in the text database and producing a cost associated with each of the sentences.

22. The system according to claim 21, further comprising a pruning criteria determination mechanism capable of adjusting the at least one pruning criterion when the reduced unit database after said pruning exceeds the desired size.

23. The system according to claim 20, wherein the cost increase estimation mechanism comprises:

an original overall cost computation mechanism capable of estimating an original overall cost associated with the pruned unit across relevant sentences for which the pruned unit is selected;

an alternative unit selection mechanism capable of performing text to speech operations an the relevant sentences, wherein the pruned unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the pruned unit;

an alternative overall cost determination mechanism capable of estimating an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected in place of the pruned unit; and

a cost increase determiner capable of estimating the cost increase based on the original overall cost and the alternative overall cost associated the pruned unit.

24. The system according to claim 20, further comprising a unit compression mechanism capable of compressing the units in the reduced unit database after the unit

pruning mechanism generates the reduced unit database to provide the reduced unit database in a compressed form.

25. The system according to claim 20, further comprising a unit compression mechanism capable of compressing the units in the full unit database to provide the full unit database in a compressed form prior to the unit selection based text to speech mechanism performs text to speech operations.

26. A unit database reduction mechanisms, comprising:

a text database including a plurality of sentences;

a full unit database; and

a cost based subset unit generation mechanism capable of pruning the full unit database to produce a reduced unit database using cost information related to unit selection in carrying out text to speech operations with respect to the plurality of sentences in the text database wherein the cost based subset unit generation mechanism comprises;

a unit selection based text to speech mechanism capable of selecting units from the full unit database with respect to the sentences in the text database and producing a cost associated with each of the sentences; and

a unit pruning mechanism capable of pruning the units selected from the full unit database to produce the reduced unit database, wherein the unit pruning mechanism comprises;

a cost increase estimation mechanism capable of estimating a cost increase related to a pruned unit, the cost increase being induced when the pruned unit is made unavailable for unit selection during text to speech operations; and

a cost increase based pruning mechanism capable of determining whether the pruned unit is to be removed according to the cost increase and the at least one pruning criterion.

27. The system according to claim 26, further comprising a pruning criteria determination mechanism capable of adjusting the at least one pruning criterion when the reduced unit database after said pruning exceeds a desired size.

28. The system according to claim 26, wherein the cost increase estimation mechanism comprises:

an original overall cost computation mechanism capable of estimating an original overall cost associated with the pruned unit across relevant sentences for which the pruned unit is selected;

an alternative unit selection mechanism capable of performing text to speech operations on the relevant sentences, wherein the pruned unit is made unavailable for unit selection so that at least one alternative unit is selected in place of the pruned unit;

an alternative overall cost determination mechanism capable of estimating an alternative overall cost across the relevant sentences for which the at least one alternative unit is selected in place of the pruned unit; and

a cost increase determiner capable of estimating the cost increase based on the original overall cost and the alternative overall cost associated the pruned unit.

29. The system according to claim 26, further comprising a unit compression mechanism capable of compressing the units in the reduced unit database after the unit pruning mechanism generates the reduced unit database to provide the reduced unit database in a compressed form.

30. The system according to claim 26, further comprising a unit compression mechanism capable of compression the units in the full unit database to provide the full unit database in a compressed form prior to the unit selection based text to speech mechanism performs text to speech operations.

31. An article comprising a storage medium having stored thereon instructions that, when executed by a machine, result in the following:

determining a desired size of a reduced unit database for text to speech operations;

generating the reduced unit database of the desired size based on a full unit database, wherein the reduced unit database is generated to minimize an overall cost in using the units in the reduced unit database to accomplish the text to speech operations; and

performing the text to speech operations using the reduced unit database, wherein said generating the reduced unit database comprises;

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database;

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database;

wherein said pruning comprises;

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining a cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

32. The article according to claim 31, wherein the desired size of the reduced unit database is determined according to at least some features of a device.

33. The article according to claim 32, wherein the features of the device include any one of:

the amount of memory available on the device;

the computation capability of the device.

34. The article according to claim 31, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

36. The article according to claim 31, wherein said determining the cost increase comprises:

determining an original overall cost across all relevant sentences for which the next unit is selected during the text to speech operations;

performing text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

37. The article according to claim 31, the instructions, when executed by a machine, further result in:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are stored in a compressed form.

38. The article according to claim 31, the instructions, when executed by a machine, further result in:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

39. An article comprising a storage medium having stored thereon instructions for generating a reduced unit database based on a full unit database that, when executed result in:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that a cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database; and

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database;

wherein said pruning comprises;

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining a cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

35. The article according to claim 43, the instructions, when executed by a machine, further result in:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

40. The article according to claim 39, wherein said pruning comprises:

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

41. The article according to claim 40, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies a desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

42. The article according to claim 40, wherein the instructions, when executed by a machine, further result in:

if the number of units in the reduced unit database exceeds a desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

43. The article according to claim 40, wherein said determining the cost increase comprises:

determining an original overall cost across all relevant sentences for which the next unit is selected during the text to speech operations;

performing text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit is selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit is selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

44. The article according to claim 43, wherein the overall cost across the relevant sentences is computed as a summation of the costs associated with individual relevant sentences.

45. The article according to claim 39, wherein the cost of using selected units to achieve text to speech with respect to a sentence includes at least one of:

a context cost; and

a concatenation cost.

46. The article according to claim 39, the instructions, when executed by a machine, further result in:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are in a compressed form.

47. The article according to claim 39, the instructions, when executed by a machine, further result in:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.