Reduced unit database generation based on cost information

• US 20040153324A1
• Filed: 01/31/2003
• Published: 08/05/2004
• Est. Priority Date: 01/31/2003
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Phillips, Michael S.
• Granted Patent: US 6,988,069 B2
• Time in Patent Office: Days
• US Class Current: 704/277
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management

Abstract

An arrangement is provided for generating a reduced unit database of a desired size to be used in text to speech operations. A reduced unit database with a desired size is generated based on a full unit database. The reduction is carried out with respect to a text database with a plurality of sentences. Units from the full database are pruned to minimize an overall cost associated with using alternative units other than the units in the reduced unit database.

Claims

1. A method, comprising:

determining a desired size of a reduced unit database for text to speech operations;

generating the reduced unit database of the desired size based on a full unit database, wherein the reduced unit database is generated to minimize an overall cost in using the units in the reduced unit database to accomplish the text to speech operations; and

performing the text to speech operations using the reduced unit database.

2. The method according to claim 1, wherein the text to speech operations are performed by any one of:

an application software;

a firmware; and

a hardware.

3. The method according to claim 1, wherein the text to speech operations are performed on a device that includes any one of:

a computer;

a personal data assistant;

a cellular phone; and

a dedicated device deployed for an application.

4. The method according to claim 3, wherein the computer includes any one of:

a personal computer;

a laptop;

a special purpose computer; and

a general purpose computer.

5. The method according to claim 3, wherein the desired size of the reduced unit database is determined according to at least some features of the device.

6. The method according to claim 5, wherein the features of the device include any one of:

the amount of memory available on the device;

the computation capability of the device.

7. The method according to claim 1, wherein said generating the reduced unit database comprises:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database;

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database.

8. The method according to claim 7, wherein said pruning comprises:

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

9. The method according to claim 8, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size;

nd

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

10. The method according to claim 9, further comprising:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

11. The method according to claim 8, wherein said determining the cost increase comprises:

determining an original overall cost across all relevent sentences for which the next, unit is selected during the text to speech operations;

perfroming text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

12. The method according to claim 7, further comprising:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are stored in a compressed form.

13. The method according to claim 7, further comprising:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

14. A method to generate a reduced unit database based on a full unit database, comprising:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database;

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database.

15. The method according to claim 14, wherein said pruning comprises:

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

16. The method according to claim 15, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

17. The method according to claim 16, further comprising:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

18. The method according to claim 15, wherein said determining the cost increase comprises:

determining an original overall cost across all relevent sentences for which the next unit is selected during the text to speech operations;

perfroming text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

19. The method according to claim 18, wherein the overall cost across the relevant sentences is computed as a summation of the costs associated with individual relevant sentences.

20. The method according to claim 14, wherein the cost of using selected units to achieve text to speech with respect to a sentence includes at least one of:

context cost; and

concatenation cost.

21. The method according to claim 14, further comprising:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are in a compressed form.

22. The method according to claim 14, further comprising:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

23. A system, comprising:

a unit database reduction mechanism capable of generating a reduced unit database of a desired size from a full unit database based on cost information; and

a text to speech mechanism capable of performing text to speech operations using the reduced unit database.

24. The system according to claim 23, wherein the unit database reduction mechanism comprises:

a text database including a plurality of sentences; and

a cost based subset unit generation mechanism capable of pruning the full unit database to generate the reduced unit database using cost information associated with unit selection in carrying out text to speech operations with respect to the plurality of sentences in the text database.

25. The system according to claim 24, wherein the cost based subset unit generation mechanism comprises:

a unit selection based text to speech mechanism capable of selecting units from the full unit database with respect to the sentences in the text database and producing a cost associated with each of the sentences;

a unit pruning mechanism capable of pruning the units selected from the full unit database to produce the reduced unit database, wherein said pruning is performed according to the cost associated with each of the sentences and at least one pruning criterion.

26. The system according to claim 25, further comprising a pruning criteria determination mechanism capable of adjusting the at least one pruning criterion when the reduced unit database after said pruning exceeds the desired size.

27. The system according to claim 25, wherein the unit pruning mechanism comprises:

a cost increase estimation mechanism capable of estimating a cost increase related to a pruning unit, the cost increase being induced when the pruning unit is made unavailable for unit selection during text to speech operations; and

a cost increase based pruning mechanism capable of determining whether the pruning unit is to be removed according to the cost increase and the at least one pruning criterion.

28. The system according to claim 27, wherein the cost increase estimation mechanism comprises:

an original overall cost computation mechanism capable of estimating an original overall cost associated with the pruning unit across relevant sentences for which the pruning unit is selected;

an alternative unit selection mechanism capable of performing text to speech operations on the relevant sentences, wherein the pruning unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the pruning unit;

an alternative overall cost determination mechanism capable of estimating an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected in place of the pruning unit; and

a cost increase determiner capable of estimating the cost increase based on the original overall cost and the alternative overall cost associated the pruning unit.

29. The system according to claim 25, further comprising a unit compression mechanism capable of compressing the units in the reduced unit database after the unit pruning mechanism generates the reduced unit database to provide the reduced unit database in a compressed form.

30. The system according to claim 25, further comprising a unit compression mechanism capable of compressing the units in the full unit database to provide the full unit database in a compressed form prior to the unit selection based text to speech mechanism performs text to speech operations.

31. A unit database reduction mechanism, comprising:

a text database including a plurality of sentences;

a full unit database; and

a cost based subset unit generation mechanism capable of pruning the full unit database to produce a reduced unit database using cost information related to unit selection in carrying out text to speech operations with respect to the plurality of sentences in the text database.

32. The mechanism according to claim 31, wherein the cost based subset unit generation mechanism comprises:

a unit selection based text to speech mechanism capable of selecting units from the full unit database with respect to the sentences in the text database and producing a cost associated with each of the sentences;

a unit pruning mechanism capable of pruning the units selected from the full unit database to produce the reduced unit database, wherein said pruning is performed according to the cost associated with each of the sentences and at least one pruning criterion.

33. The system according to claim 32, further comprising a pruning criteria determination mechanism capable of adjusting the at least one pruning criterion when the reduced unit database after said pruning exceeds the desired size.

34. The system according to claim 32, wherein the unit pruning mechanism comprises:

a cost increase estimation mechanism capable of estimating a cost increase related to a pruning unit, the cost increase being induced when the pruning unit is made unavailable for unit selection during text to speech operations; and

a cost increase based pruning mechanism capable of determining whether the pruning unit is to be removed according to the cost increase and the at least one pruning criterion.

35. The system according to claim 34, wherein the cost increase estimation mechanism comprises:

an original overall cost computation mechanism capable of estimating an original overall cost associated with the pruning unit across relevant sentences for which the pruning unit is selected;

an alternative unit selection mechanism capable of performing text to speech operations on the relevant sentences, wherein the pruning unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the pruning unit;

an alternative overall cost determination mechanism capable of estimating an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected in place of the pruning unit; and

a cost increase determiner capable of estimating the cost increase based on the original overall cost and the alternative overall cost associated the pruning unit.

36. The system according to claim 32, further comprising a unit compression mechanism capable of compressing the units in the reduced unit database after the unit pruning mechanism generates the reduced unit database to provide the reduced unit database in a compressed form.

37. The system according to claim 32, further comprising a unit compression mechanism capable of compression the units in the full unit database to provide the full unit database in a compressed form prior to the unit selection based text to speech mechanism performs text to speech operations.

38. An article comprising a storage medium having stored thereon instructions that, when executed by a machine, result in the following:

determining a desired size of a reduced unit database for text to speech operations;

generating the reduced unit database of the desired size based on a full unit database, wherein the reduced unit database is generated to minimize an overall cost in using the units in the reduced unit database to accomplish the text to speech operations; and

performing the text to speech operations using the reduced unit database.

39. The article according to claim 38, wherein the desired size of the reduced unit database is determined according to at least some features of a device.

40. The article according to claim 39, wherein the features of the device include any one of:

the amount of memory available on the device;

the computation capability of the device.

41. The article according to claim 38, wherein said generating the reduced unit database comprises:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database;

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database.

42. The article according to claim 38, wherein said pruning comprises:

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

43. The article according to claim 42, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size;

nd

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

44. The article according to claim 43, the instructions, when executed by a machine, further result in:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

45. The article according to claim 42, wherein said determining the cost increase comprises:

determining an original overall cost across all relevent sentences for which the next unit is selected during the text to speech operations;

perfroming text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

46. The article according to claim 38, the instructions, when executed by a machine, further result in:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are stored in a compressed form.

47. The article according to claim 38, the instructions, when executed by a machine, further result in:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.

48. An article comprising a storage medium having stored thereon instructions for generating a reduced unit database based on a full unit database that, when executed, result in:

performing text to speech operations with respect to every sentence in a text database using units selected from the full unit database, wherein units are selected so that the cost of using the selected units to achieve text to speech is minimized;

computing a unit selection cost associated with each of the sentences in the text database;

pruning the units that are selected during the text to speech operations based on the unit selection costs to produce the reduced unit database.

49. The article according to claim 48, wherein said pruning comprises:

initializing the reduced unit database using the units selected during the text to speech operations performed with respect to the sentences in the text database;

determining an cost increase induced when a next unit in the reduced unit database is made unavailable for unit selection based text to speech operations;

retaining the next unit in the reduced unit database if the cost increase satisfies at least one pruning criterion; and

repeating said determining and said removing until at least one condition is satisfied.

50. The article according to claim 49, wherein the at least one condition includes at least one of:

the number of retained units in the reduced unit database satisfies the desired size; and

the number of retained units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion.

51. The article according to claim 49, the instructions, when executed by a machine, further result in:

if the number of units in the reduced unit database exceeds the desired size after all the units in the reduced unit database have been processed with respect to the at least one pruning criterion, adjusting the at least one pruning criterion to create updated at least one pruning criterion; and

performing operations between said determining and said repeating using the updated at least one pruning criterion in place of the at least one pruning criterion.

52. The article according to claim 49, wherein said determining the cost increase comprises:

determining an original overall cost across all relevent sentences for which the next unit is selected during the text to speech operations;

perfroming text to speech operations on the relevant sentences, wherein the next unit is made unavailable for unit selection so that at least one alternative unit are selected in place of the next unit;

computing an alternative overall cost across the relevant sentences for which the at least one alternative unit are selected during the text to speech operations; and

estimating the cost increase associated with the next unit based on the original overall cost and the alternative overall cost.

53. The article according to claim 52, wherein the overall cost across the relevant sentences is computed as a summation of the costs associated with individual relevant sentences.

54. The article according to claim 48, wherein the cost of using selected units to achieve text to speech with respect to a sentence includes at least one of:

context cost; and

concatenation cost.

55. The article according to claim 48, the instructions, when executed by a machine, further result in:

compressing the units in the reduced unit database after said pruning so that the units in the reduced unit database are in a compressed form.

56. The article according to claim 48, the instructions, when executed by a machine, further result in:

compressing the full unit database prior to said performing text to speech operations so that the unit selection during said performing is based on a compressed full unit database.