System and method for spelling recognition using speech and non-speech input

• US 7,574,356 B2
• Filed: 07/19/2004
• Issued: 08/11/2009
• Est. Priority Date: 07/19/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Parthasarathy, Sarangarajan
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US10/894,201
• Publication Number:
  US 20060015336A1
• Time in Patent Office: 1,849 Days
• Field of Search: 704/231, 704/235, 704/251, 704/240, 704/252, 704253-255, 704/270, 704/275, 379 8801- 8803, 379/67, 379/77, 379/89
• US Class Current: 704/231
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/086 Recognition of spelled words

Abstract

A system and method for non-speech input or keypad-aided word and spelling recognition is disclosed. The method comprises performing spelling recognition via automatic speech recognition (ASR) on received speech from a user, the ASR being performed using a statistical letter model trained on domain data and producing a letter lattice R_LN. If an ASR confidence is below a predetermined level, then the method comprises receiving non-speech input from the user, generating a keypad constraint grammar K and generating a letter string based on a composition of finite state transducers R_LN and K. Other variations of the invention include recognizing input by first receiving non-speech input, dynamically generating an unweighted grammar, generating a weighted grammar using domain data, and then performing speech, and thus spelling, recognition on input speech using the weighted grammar.

Claims

1. A method for recognizing a combination of speech and alternate input, the method comprising:

receiving a non-speech input from a user;

dynamically constructing an unweighted grammar permitting all letter sequences that map to the received non-speech input;

constructing a weighted grammar using the unweighted grammar and a statistical letter model trained on domain data;

receiving speech from the user associated with the non-speech input, wherein receiving the speech occurs after receiving the non-speech input and after constructing the weighted grammar; and

recognizing the received speech and non-speech input using the constructed weighted grammar.

2. The method of claim 1, wherein the statistical letter model is an N-gram letter model.

3. The method of claim 2, wherein the N-gram letter model is unsmoothed.

4. The method of claim 1, wherein the non-speech input is keypad input.

5. The method of claim 1, wherein the non-speech input is received via a touch sensitive screen.

6. The method of claim 5, wherein the non-speech input is received via a stylus on a touch-sensitive screen.

7. The method of claim 1, further comprising generating a final letter string based on a database lookup.

16. The method of claim 5, wherein the database lookup is a database of valid names.

8. A method of recognizing input from a user, the method comprising:

performing spelling recognition via automatic speech recognition (ASR) on received speech from a user, the ASR being performed using a statistical letter model trained on domain data and producing a letter lattice R_LN;

if an ASR confidence is below a predetermined level, then;

receiving non-speech input from the user;

generating a non-speech constraint grammar K; and

generating a letter string based on a composition of R_LN and K.

9. The method of claim 8, wherein the statistical letter model is an N-gram letter model.

10. The method of claim 8, wherein the statistical letter model is unsmoothed.

11. The method of claim 8, wherein the non-speech input is keypad input.

12. The method of claim 8, wherein R_LN and K are finite state transducers.

13. The method of claim 8, wherein generating a letter string further comprises generating a letter string based on a composition of finite state transducers R_LN, K and a transducer that eliminates silence and other filler words.

14. The method of claim 8, further comprising generating a final letter string based on a database lookup.

15. The method of claim 14, wherein generating the final letter string based on a database lookup further comprises using a finite state network that accepts only valid letter strings.

17. The method of claim 8, wherein the received non-speech input only comprises a portion of a word or name.

18. The method of claim 8, further comprising, if an ASR confidence is below a predetermined level, prompting the user to enter the first three or less letters of the input by using a keypad.

19. A system for recognizing a combination of speech and alternate input, the system comprising:

means for receiving a non-speech input from a user;

means for dynamically constructing an unweighted grammar permitting all letter sequences that map to the received non-speech input;

means for constructing a weighted grammar using the unweighted grammar and a statistical letter model trained on domain data;

means for receiving speech from the user associated with the non-speech input, wherein receiving the speech occurs after receiving the non-speech input and after constructing the weighted grammar; and

means for recognizing the received speech and non-speech input using the constructed weighted grammar.

20. A system for recognizing input from a user, the system comprising:

means for performing spelling recognition via automatic speech recognition (ASR) on received speech from a user, the ASR being performed using a statistical letter model trained on domain data and producing a letter lattice R_LN;

if an ASR confidence is below a predetermined level, then the means for performing spelling recognition further;

receives non-speech input from the user;

generates a non-speech constraint grammar K; and

generates a letter string based on a composition of R_LN and K.

21. The system of claim 20, wherein the statistical letter model is an N-gram letter model.

22. A computer-readable medium storing instructions for controlling a computing device to recognize a combination of speech and non-speech input, the instructions comprising:

receiving a non-speech input from a user;

dynamically constructing an unweighted grammar permitting all letter sequences that map to the received non-speech input;

constructing a weighted grammar using the unweighted grammar and a statistical letter model trained on domain data;

receiving speech from the user associated with the non-speech input, wherein receiving the speech occurs after receiving the non-speech input and after constructing the weighted grammar; and

recognizing the received speech and non-speech input using the constructed weighted grammar.

23. A computer-readable medium storing instructions for controlling a computing device to recognize input from a user, the instructions comprising:

performing spelling recognition via automatic speech recognition (ASR) on received speech from a user, the ASR being performed using a statistical letter model trained on domain data and producing a letter lattice R_LN;

if an ASR confidence is below a predetermined level, then;

receiving non-speech input from the user;

generating a non-speech constraint grammar K; and

generating a letter string based on a composition of R_LN and K.