Method device and article of manufacture for neural-network based generation of postlexical pronunciations from lexical pronunciations

• US 6,134,528 A
• Filed: 06/13/1997
• Issued: 10/17/2000
• Est. Priority Date: 06/13/1997
• Status: Expired due to Fees

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Massey, Noel, Miller, Corey Andrew, Karaali, Orhan
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Lerner, Martin
• Application Number:
  US08/874,834
• Time in Patent Office: 1,222 Days
• Field of Search: 704/232, 704/255, 704/256, 704/257, 704/259, 704/251, 704/252, 704/253, 704/254, 704/260, 704/266, 704/269, 704/258
• US Class Current: 704/258
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

A method (2000), device (2200) and article of manufacture (2300) provide, in response to lexical pronunciation information, efficient generation of postlexical pronunciation information. A method is presented for providing, in response to a lexical pronunciation, efficient generation of a postlexical pronunciation, including the steps of: determining lexical phones, lexical features, and boundary information for a predetermined portion of text; and utilizing a pretrained neural network that was pretrained using lexical phones, postlexical phones, lexical features, and boundary information to generate a neural network hypothesis for a postlexical pronunciation of the predetermined portion of text.

Claims

1. A method for providing, in response to a lexical pronunciation, efficient generation of a postlexical pronunciation, comprising the steps of:

a) determining lexical phones, lexical features, and boundary information for a predetermined portion of text; and

b) utilizing a pretrained neural network that was pretrained using lexical phones, postlexical phones, lexical features, and boundary information to generate a neural network hypothesis for a postlexical pronunciation of the predetermined portion of text, the pretrained neural network having been trained using at least the steps of;

providing a predetermined length context window of the lexical phones;

providing a predetermined length context window of lexical phone features.

2. The method of claim 1 wherein the pretrained neural network has been trained using the steps of:

a) providing associated lexical pronunciations consisting of lexical phones for the predetermined portion of text and postlexical pronunciations consisting of postlexical phones for a target utterance;

b) aligning the associated lexical and postlexical representations using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information for the lexical phones;

d) providing information with respect to the distance between each lexical phone and each of syllable, word, phrase, clause and sentence boundaries;

e) providing information with respect to the adjacency between each lexical phone and each of the syllable, the word, the phrase, the clause and the sentence boundaries;

f) providing the predetermined length context window of the lexical phones;

g) providing the predetermined length context window of lexical phone features; and

h) training a neural network to associate the input lexical phones with the postlexical phones.

3. The method of claim 2 further including, in step (h), employing a feature-based error function to characterize the distance between target and hypothesized postlexical pronunciations.

4. The method of claim 2, step (b), wherein the alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

5. The method of claim 2, step (b), wherein the featurally-based substitution cost function uses predetermined substitution, insertion and deletion costs and a predetermined substitution table.

6. The method of claim 1, step (b), wherein the neural network is a feed-forward neural network.

7. The method of claim 1, step (b), wherein the neural network uses backpropagation of errors.

8. The method of claim 1, wherein the lexical features include articulatory features.

9. The method of claim 1, wherein the lexical features include acoustic features.

10. The method of claim 1, wherein the lexical features include a geometry of articulatory features.

11. The method of claim 1, wherein the lexical features include a geometry of acoustic features.

12. The method of claim 1, wherein the lexical phones are described using a feature vector.

13. The method of claim 1, wherein the postlexical phones are described using a feature vector.

14. The method of claim 1, step (b), wherein the neural network is trained using intonational information.

15. The method of claim 1, step (b), wherein the neural network is trained using prosodic information.

16. The method of claim 1, step (b), comprising generating a single neural network hypothesis for a postlexical pronunciation of the predetermined portion of text.

17. A device for providing, in response to a lexical pronunciation, efficient generation of a postlexical pronunciation, comprising:

a) an encoder, coupled to receive lexical phones, lexical features, boundary distance information and boundary adjacency infomation for a predetermined portion of text, for providing digital input to a pretrained postlexical neural network, wherein the pretrained postlexical neural network has been trained using lexical phones, postlexical phones, lexical features, boundary distance information and boundary adjacency information and where the lexical and postlexical phones have been aligned using an automatic feature-based dynamic programming alignment; and

b) a pretrained postlexical neural network, coupled to the encoder, for generating a neural network hypothesis for a postlexical pronunciation of the predetermined portion of text, the pretrained neural network having been trained using at least the steps of;

providing a predetermined length context window of the lexical phones;

providing a predetermined length context window of lexical phone features.

18. The device of claim 17 wherein the pretrained neural network has been trained in accordance with the following scheme:

a) providing associated lexical pronunciations consisting of lexical phones for the predetermined portion of text and postlexical pronunciations consisting of postlexical phones for a target utterance;

b) aligning the associated lexical and postlexical representations using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information for the lexical phones;

d) providing information with respect to the distance between each lexical phone and each of syllable, word, phrase, clause and sentence boundaries;

e) providing information with respect to the adjacency between each lexical phone and each of the syllable, the word, the phrase, the clause and the sentence boundaries;

f) providing the predetermined length context window of the lexical phones;

g) providing the predetermined length context window of lexical phone features; and

h) training a neural network to associate the input lexical phones with the postlexical phones.

19. The device of claim 18, step (b), wherein the alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

20. the device of claim 18, step (b), wherein the featurally-based substitution cost function uses predetermined substitution, insertikon and deletion costs and a predetermined substitution table.

21. The device of claim 17 further including, during training, employing a feature-based error function to characterize the distance between target and hypothesized postlexical pronunciations.

22. The device of claim 17 wherein the device includes at least one of:

a) a microprocessor;

b) application specific integrated circuit; and

c) a combination of (a) and (b).

23. The device of claim 17, step (b), wherein the neural network is a feed-forward neural network.

24. The device of claim 17, step (b), wherein the neural network uses backpropagation of errors.

25. The device of claim 17, wherein the lexical features include articulatory features.

26. The device of claim 17, wherein the lexical features include acoustic features.

27. The device of claim 17, wherein the lexical features include a geometry of articulatory features.

28. The device of claim 17, wherein the lexical features include a geometry of acoustic features.

29. The device of claim 17, wherein the lexical phones are described using a feature vector.

30. The device of claim 17, wherein the postlexical phones are described using a feature vector.

31. The device of claim 17, step (b), wherein the neural network is trained using intonational information.

32. The device of claim 17, step (b), wherein the neural network is trained using prosodic information.

33. The device of claim 17 being joperable to generate a single neural network hypothesis for postlexical pronunciation of the predetermined portion of text.

34. An article of manufacture for generating postlexical pronunciations from lexical pronunciations, comprising a computer usable medium having computer readable program code means thereon comprising:

a) lexical information determination means for determining lexical phones, lexical features, boundary distance information and boundary adjacency information for a predetermined portion of text; and

b) postlexical neural network utilization means for utilizing a pretrained neural network that was pretrained using lexical phones, postlexical phones, lexical features, boundary distance information and boundary adjacency information to generate a neural network hypothesis for a postlexical pronunciation of the predetermined portion of text, the pretrained neural network having been trained using at least the steps of;

providing a predetermined length context window of the lexical phones;

providing a predetermined length context window of lexical phone features.

35. The article of manufacture of claim 34 wherein the pretrained neural network has been trained in accordance with the following scheme:

a) providing associated lexical pronunciations consisting of lexical phones for the predetermined portion of text and postlexical pronunciations consisting of postlexical phones for a target utterance;

b) aligning the associated lexical and postlexical representations using a dynamic programming alignment enhanced with a featurally-based substitution cost function;

c) providing acoustic and articulatory information for the lexical phones;

d) providing information with respect to the distance between each lexical phone and each of syllable, word, phrase, clause and sentence boundaries;

e) providing information with respect to the adjacency between each lexical phone and each of the syllable, the word, the phrase, the clause and the sentence boundaries;

f) providing the predetermined length context window of the lexical phones;

g) providing the predetermined length context window of lexical phone features; and

h) training a neural network to associate the input lexical phones with the postlexical phones.

36. The article of manufacture of claim 35 further including, in step (h), employing a feature-based error function to characterize the distance between target and hypothesized postlexical pronunciations.

37. The article of manufacture of claim 35, step (b), wherein the alignment is based on consonant and vowel locations in the orthography and associated phonetic representation.

38. The article of manufacture of claim 35, step (b), wherein the featurally-based substitution cost function uses predetermined substitution, insertion and deletion costs and a predetermined substitution table.

39. The article of manufacture of claim 34, step (b), wherein the neural network is a feed-forward neural network.

40. The article of manufacture of claim 34, step (b), wherein the neural network uses backpropagation of errors.

41. The article of manufacture of claim 34, wherein the lexical features include articulatory features.

42. The article of manufacture of claim 34, wherein the lexical features include acoustic features.

43. The article of manufacture of claim 34, wherein the lexical features include a geometry of articulatory features.

44. The article of manufacture of claim 34, wherein the lexical features include a geometry of acoustic features.

45. The article of manufacture of claim 34, wherein the lexical phones are described using a feature vector.

46. The article of manufacture of claim 34, wherein the postlexical phones are described using a feature vector.

47. The article of manufacture of claim 34, step (b), wherein the neural network is trained using intonational information.

48. The article of manufacture of claim 34, step (b), wherein the neural network is trained using prosodic information.

49. The article of manufacture of claim 34, being operable to generate a single neural network hypothesis for a postlexical pronunciation of the predetermined portion of text.