Method, device and article of manufacture for neural-network based orthography-phonetics transformation

• US 5,930,754 A
• Filed: 06/13/1997
• Issued: 07/27/1999
• Est. Priority Date: 06/13/1997
• Status: Expired due to Fees

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Karaali, Orhan, Miller, Corey Andrew
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US08/874,900
• Time in Patent Office: 774 Days
• Field of Search: 704/259, 704/258, 704/232
• US Class Current: 704/259
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

A method (2000), device (2200) and article of manufacture (2300) provide, in response to orthographic information, efficient generation of a phonetic representation. The method provides for, in response to orthographic information, efficient generation of a phonetic representation, using the steps of: inputting an orthography of a word and a predetermined set of input letter features; utilizing a neural network that has been trained using automatic letter phone alignment and predetermined letter features to provide a neural network hypothesis of a word pronunciation.

Claims

1. A method for providing, in response to orthographic information, efficient generation of a phonetic representation, comprising the steps of:

a) inputting an orthography of a word and a predetermined set of input letter features;

b) utilizing a neural network that has been trained using automatic letter phone alignment and predetermined letter features to provide a neural network hypothesis of a word pronunciation.

2. The method of claim 1 wherein the predetermined letter features for a letter represent a union of features of predetermined phones representing the letter.

3. The method of claim 1 wherein the pretrained neural network has been trained using the steps of:

a) providing a predetermined number of letters of an associated orthography consisting of letters for the word and a phonetic representation consisting of phones for a target pronunciation of the associated orthography;

b) aligning the associated orthography and phonetic representation using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information corresponding to the letters, based on a union of features of predetermined phones representing each letter;

d) providing a predetermined amount of context information; and

e) training the neural network to associate the input orthography with a phonetic representation.

4. The method of claim 3, step (a), wherein the predetermined number of letters is equivalent to the number of letters in the word.

5. The method of claim 1 where a pronunciation lexicon is reduced in size by using neural network word pronunciation hypotheses which match target pronunciations.

6. The method of claim 3 further including providing a predetermined number of layers of output reprocessing in which phones, neighboring phones, phone features and neighboring phone features are passed to succeeding layers.

7. The method of claim 3 further including, during training, employing a feature-based error function to characterize a distance between target and hypothesized pronunciations during training.

8. The method of claim 1, step (b) wherein the neural network is a feed-forward neural network.

9. The method of claim 1, step (b) wherein the neural network uses backpropagation of errors.

10. The method of claim 1, step (b) wherein the neural network has a recurrent input structure.

11. The method of claim 1, wherein the predetermined letter features include articulatory features.

12. The method of claim 1, wherein the predetermined letter features include acoustic features.

13. The method of claim 1, wherein the predetermined letter features include a geometry of articulatory features.

14. The method of claim 1, wherein the predetermined letter features include a geometry of acoustic features.

15. The method of claim 1, step (b), wherein the automatic letter phone alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

16. The method of claim 3, step (a), wherein the letters and phones are contained in a sliding window.

17. The method of claim 1, wherein the orthography is described using a feature vector.

18. The method of claim 1, wherein the pronunciation is described using a feature vector.

19. The method of claim 6, wherein the number of layers of output reprocessing is 2.

20. The method of claim 3, step (b), where the featurally-based substitution cost function uses predetermined substitution, insertion and deletion costs and a predetermined substitution table.

21. A device for providing, in response to orthographic information, efficient generation of a phonetic representation, comprising:

a) an encoder, coupled to receive an orthography of a word and a predetermined set of input letter features, for providing digital input to a pretrained orthography-pronunciation neural network, wherein the pretrained neural network has been trained using automatic letter phone alignment and predetermined letter features;

b) the pretrained orthography-pronunciation neural network, coupled to the encoder, for providing a neural network hypothesis of a word pronunciation.

22. The device of claim 21 wherein the pretrained neural network is trained using feature-based error backpropagation.

23. The device of claim 21 wherein the predetermined letter features for a letter represent a union of features of predetermined phones representing the letter.

24. The device of claim 21 wherein the device includes at least one of:

a) a microprocessor;

b) application specific integrated circuit; and

c) a combination of a) and b).

25. The device of claim 21 wherein the pretrained neural network has been trained in accordance with the following scheme:

a) providing a predetermined number of letters of an associated orthography consisting of letters for the word and a phonetic representation consisting of phones for a target pronunciation of the associated orthography;

b) aligning the associated orthography and phonetic representation using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information corresponding to the letters, based on a union of features of predetermined phones representing each letter;

d) providing a predetermined amount of context information; and

e) training the neural network to associate the input orthography with a phonetic representation.

26. The device of claim 25, step (a) wherein the predetermined number of letters is equivalent to the number of letters in the word.

27. The device of claim 21, where a pronunciation lexicon is reduced in size by using neural network word pronunciation hypotheses which match target pronunciations.

28. The device of claim 21 further including providing a predetermined number of layers of output reprocessing in which phones, neighboring phones, phone features and neighboring phone features are passed to succeeding layers.

29. The device of claim 21 further including, during training, employing a feature-based error function to characterize the distance between target and hypothesized pronunciations during training.

30. The device of claim 21, wherein the neural network is a feed-forward neural network.

31. The device of claim 21, wherein the neural network uses backpropagation of errors.

32. The device of claim 21, wherein the neural network has a recurrent input structure.

33. The device of claim 21, wherein the predetermined letter features include articulatory features.

34. The device of claim 21, wherein the predetermined letter features include acoustic features.

35. The device of claim 21, wherein the predetermined letter features include a geometry of articulatory features.

36. The device of claim 21, wherein the predetermined letter features include a geometry of acoustic features.

37. The device of claim 21, step (b), wherein the automatic letter phone alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

38. The device of claim 25, step (a), wherein the letters and phones are contained in a sliding window.

39. The device of claim 21, wherein the orthography is described using a feature vector.

40. The device of claim 21, wherein the pronunciation is described using a feature vector.

41. The device of claim 28, wherein the number of layers of output reprocessing is 2.

42. The device of claim 25, step (b), where the featurally-based substitution cost function uses predetermined substitution, insertion and deletion costs and a predetermined substitution table.

43. An article of manufacture for converting orthographies into phonetic representations, comprising a computer usable medium having computer readable program code means thereon comprising:

a) inputting means for inputting an orthography of a word and a predetermined set of input letter features;

b) neural network utilization means for utilizing a neural network that has been trained using automatic letter phone alignment and predetermined letter features to provide a neural network hypothesis of a word pronunciation.

44. The article of manufacture of claim 43 wherein the predetermined letter features for a letter represent a union of features of predetermined phones representing the letter.

45. The article of manufacture of claim 43 wherein the pretrained neural network has been trained in accordance with the following scheme:

a) providing a predetermined number of letters of an associated orthography consisting of letters for the word and a phonetic representation consisting of phones for a target pronunciation of the associated orthography;

b) aligning the associated orthography and phonetic representation using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information corresponding to the letters, based on a union of features of predetermined phones representing each letter;

d) providing a predetermined amount of context information; and

e) training the neural network to associate the input orthography with a phonetic representation.

46. The article of manufacture of claim 45, step (a), wherein the predetermined number of letters is equivalent to the number of letters in the word.

47. The article of manufacture of claim 43 where a pronunciation lexicon is reduced in size by using neural network word pronunciation hypotheses which match target pronunciations.

48. The article of manufacture of claim 43 further including providing a predetermined number of layers of output reprocessing in which phones, neighboring phones, phone features and neighboring phone features are passed to succeeding layers.

49. The article of manufacture of claim 43 further including, during training, employing a feature-based error function to characterize the distance between target and hypothesized pronunciations during training.

50. The article of manufacture of claim 43, wherein the neural network is a feed-forward neural network.

51. The article of manufacture of claim 43, wherein the neural network uses backpropagation of errors.

52. The article of manufacture of claim 43, wherein the neural network has a recurrent input structure.

53. The article of manufacture of claim 43, wherein the predetermined letter features include articulatory features.

54. The article of manufacture of claim 43, wherein the predetermined letter features include acoustic features.

55. The article of manufacture of claim 43, wherein the predetermined letter features include a geometry of articulatory features.

56. The article of manufacture of claim 43, step (b), wherein the automatic letter phone alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

57. The article of manufacture of claim 45, step (a), wherein the letters and phones are contained in a sliding window.

58. The article of manufacture of claim 43, wherein the orthography is described using a feature vector.

59. The article of manufacture of claim 43, wherein the pronunciation is described using a feature vector.

60. The article of manufacture of claim 47, wherein the number of layers of output reprocessing is 2.

61. The article of manufacture of claim 45, step (b), where the featurally-based substitution cost function uses predetermined substitution, insertion and deletion costs and a predetermined substitution table.