System and method for spelling recognition using speech and non-speech input

• US 8,229,747 B2
• Filed: 05/17/2011
• Issued: 07/24/2012
• Est. Priority Date: 07/19/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Parthasarathy, Sarangarajan
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US13/109,293
• Publication Number:
  US 20110218808A1
• Time in Patent Office: 434 Days
• Field of Search: 704/231, 704/251, 704/235, 704/240, 704/252, 704253-255, 704/270, 704/275, 379 8801- 8803, 379/77
• US Class Current: 704/257
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/086 Recognition of spelled words

Abstract

A system and method for non-speech input or keypad-aided word and spelling recognition is disclosed. The method includes generating an unweighted grammar, selecting a database of words, generating a weighted grammar using the unweighted grammar and a statistical letter model trained on the database of words, receiving speech from a user after receiving the non-speech input and after generating the weighted grammar, and performing automatic speech recognition on the speech and non-speech input using the weighted grammar. If a confidence is below a predetermined level, then the method includes receiving non-speech input from the user, disambiguating possible spellings by generating a letter lattice based on a user input modality, and constraining the letter lattice and generating a new letter string of possible word spellings until a letter string is correctly recognized.

Claims

1. A method comprising:

receiving, via a processor, a speech input;

constructing an unweighted grammar permitting all letter sequences that map to the speech input;

generating keypad constraints using the unweighted grammar and a weighted statistical letter model trained on a database of words;

receiving non-speech input constrained by the keypad constraints; and

recognizing the speech input and the non-speech input using the unweighted grammar and the weighted statistical letter model.

2. The method of claim 1, wherein the database of words is a domain of words related to the non-speech input.

3. The method of claim 1, wherein the weighted statistical letter model is an N-gram letter model.

4. The method of claim 3, wherein the N-gram letter model is unsmoothed.

5. The method of claim 1, further comprising:

generating a final letter string based on the database of words.

6. The method of claim 5, wherein generating the final letter string based on the database of words further comprises using a finite state network that accepts only valid letter strings.

7. The method of claim 1, wherein the non-speech input comprises a portion of a word.

8. The method of claim 1, further comprising:

if an automated speech recognition confidence is below a predetermined level, prompting the user to enter a first three or less letters of the speech input by using a keypad to yield the non-speech input.

9. A system comprising:

a processor;

a non-transitory computer-readable storage medium having stored therein instructions which, when executed by the processor, cause the processor to perform a method comprising;

constructing an unweighted grammar permitting all letter sequences that map to speech input;

generating keypad constraints using the unweighted grammar and a weighted statistical letter model trained on a database of words;

receiving non-speech input constrained by the keypad constraints; and

recognizing the speech input and the non-speech input using the unweighted grammar and the weighted statistical letter model.

10. The system of claim 9, wherein the database of words is a domain of words related to the non-speech input.

11. The system of claim 9, wherein the weighted statistical letter model is an N-gram letter model.

12. The system of claim 11, wherein the N-gram letter model is unsmoothed.

13. The system of claim 9, the non-transitory computer-readable storage medium having stored therein further instructions which, when executed by the processor, cause the processor to perform a method further comprising:

generating a final letter string based on the database of words.

14. The system of claim 13, wherein the final letter string is generated based on the database of words via a finite state network that accepts only valid letter strings.

15. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform a method comprising:

receiving a speech input;

constructing an unweighted grammar permitting all letter sequences that map to the speech input;

generating keypad constraints using the unweighted grammar and a weighted statistical letter model trained on a database of words;

receiving non-speech input constrained by the keypad constraints; and

recognizing the speech input and the non-speech input using the unweighted grammar and the weighted statistical letter model.

16. The non-transitory computer-readable storage medium of claim 15, wherein the database of words is a domain of words related to the non-speech input.

17. The non-transitory computer-readable storage medium of claim 15, wherein the weighted statistical letter model is an N-gram letter model.

18. The non-transitory computer-readable storage medium of claim 17, wherein the N-gram letter model is unsmoothed.

19. The non-transitory computer-readable storage medium of claim 15, the instructions further comprising:

controlling the processor to generate a final letter string based on the database of words.

20. The non-transitory computer-readable storage medium of claim 19, wherein the final letter string based on the database of words further comprises using a finite state network that accepts only valid letter strings.