System and method for spelling recognition using speech and non-speech input

• US 7,949,528 B2
• Filed: 07/22/2009
• Issued: 05/24/2011
• Est. Priority Date: 07/19/2004
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Parthasarathy, Sarangarajan
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US12/507,388
• Publication Number:
  US 20090281806A1
• Time in Patent Office: 671 Days
• Field of Search: 704/257, 704/255, 704/9, 704/243, 704/260, 704/275, 704/8, 704/256
• US Class Current: 704/257
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/086 Recognition of spelled words

Abstract

A system and method for non-speech input or keypad-aided word and spelling recognition is disclosed. The method includes generating an unweighted grammar, selecting a database of words, generating a weighted grammar using the unweighted grammar and a statistical letter model trained on the database of words, receiving speech from a user after receiving the non-speech input and after generating the weighted grammar, and performing automatic speech recognition on the speech and non-speech input using the weighted grammar. If a confidence is below a predetermined level, then the method includes receiving non-speech input from the user, disambiguating possible spellings by generating a letter lattice based on a user input modality, and constraining the letter lattice and generating a new letter string of possible word spellings until a letter string is correctly recognized.

Claims

1. A system for recognizing a combination of speech and alternate input, the method comprising:

a processor;

a first module configured to control the processor to generate an unweighted grammar permitting all letter sequences that map to a received non-speech input;

a second module configured to control the processor to select a database of words;

a third module configured to control the processor to generate a weighted grammar using the unweighted grammar and a statistical letter model trained on the database of words;

a fourth module configured to control the processor to receive speech from a user associated with the non-speech input after receiving the non-speech input and after generating the weighted grammar; and

a fifth module configured to control the processor to process the received speech and non-speech input using the weighted grammar.

2. The system of claim 1, wherein the database of words is a domain of words related to the non-speech input.

3. The system of claim 1, further comprising a sixth module configured to control the processor to perform speech recognition based on the received speech and non-speech input using the weighted grammar.

4. The system of claim 1, wherein the statistical letter model is an N-gram letter model.

5. The system of claim 4, wherein the N-gram letter model is unsmoothed.

6. The system of claim 1, further comprising a sixth module configured to control the processor to generate a final letter string based on a database lookup.

7. The system of claim 1, wherein the non-speech input comprises a portion of a word.

8. A method of recognizing input from a user, the method comprising:

receiving a first input from a user;

performing spelling recognition via an automatic speech recognition system on the first input, the speech recognition being performed using a statistical letter model trained on a database of words;

generating a letter lattice based on the first input; and

performing, with each second input received from the user after the first input, until a letter string is correctly recognized;

constraining the letter lattice based on the each sound input to yield a constrained letter lattice; and

generating a new letter string of possible word spellings based on the constrained letter lattice.

9. The method of claim 8, wherein constraining the letter lattice further comprises locating the most probably path through the lattice.

10. The method of claim 8, wherein the first input comprises input from one of a speech input device and a non-speech input device.

11. The method of claim 8, wherein the statistical letter model is an N-gram letter model.

12. The method of claim 11, wherein the statistical letter model is unsmoothed.

13. The method of claim 8, further comprising generating a final letter string based on a database lookup.

14. The method of claim 13, wherein generating the final letter string based on a database lookup further comprises using a finite state network that accepts only valid letter strings.

15. The method of claim 13, wherein receiving input comprises receiving a portion of a word.

16. The method of claim 8, further comprising, if an automated speech recognition confidence is below a predetermined level, prompting the user to enter the first three or less letters of the input by using a keypad.

17. A computer-readable storage medium storing instructions for controlling a computing device having a processor to recognize input from a user, the instructions comprising controlling the processor to perform steps comprising:

generating an unweighted grammar permitting all letter sequences that map to a received non-speech input;

selecting a database of words;

generating a weighted grammar using the unweighted grammar and a statistical letter model trained on the database of words;

receiving speech from a user associated with the non-speech input after receiving the non-speech input and after generating the weighted grammar;

performing recognition via automatic speech recognition (ASR) on the received speech and non-speech input using the weighted grammar; and

if an automated speech recognition confidence is below a predetermined level;

disambiguating possible spellings by generating a letter lattice based on a user input modality; and

constraining the letter lattice and generating a new letter string of possible word spellings, with each portion of the speech received from the user, until a letter string is correctly recognized.

18. The computer-readable storage medium of claim 17, wherein the user input modality comprises speech input devices and non-speech input devices.

19. The computer-readable storage medium of claim 17, wherein the statistical letter model is an N-gram letter model.

20. The computer-readable storage medium of claim 19, wherein the statistical letter model is unsmoothed.