HYBRID SPEECH RECOGNITION

• US 20130151250A1
• Filed: 12/08/2011
• Published: 06/13/2013
• Est. Priority Date: 12/08/2011
• Status: Active Grant

• Current Assignee: Lenovo PC International Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: VanBlon, Russell Speight
• Granted Patent: US 9,620,122 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

Described is a technology by which speech is locally and remotely recognized in a hybrid way. Speech is input and recognized locally, with remote recognition invoked if locally recognized speech data was not confidently recognized. The part of the speech that was not confidently recognized is sent to the remote recognizer, along with any confidently recognized text, which the remote recognizer may use as context data in interpreting the part of the speech data that was sent. Alternative text candidates may be sent instead of corresponding speech to the remote recognizer.

Claims

1. A method comprising:

inputting speech data;

recognizing the speech data on a first recognizer into a first recognition result comprising recognized text;

evaluating confidence with respect to the first recognition result; and

if not confident as to at least part of the first recognition result, sending non-confident data to a second recognizer in conjunction with any confidently recognized text of the first recognition result, receiving a second recognition result from the second recognizer, and outputting text corresponding to the speech data based on the first recognition result and the second recognition result.

2. The method of claim 1 wherein recognizing the speech data on a first recognizer comprises providing the speech data to a local device recognizer.

3. The method of claim 1 wherein sending the non-confident data to the second recognizer comprises transmitting the non-confident data to a remote recognition service.

4. The method of claim 1 wherein sending the non-confident data to the second recognizer comprises transmitting speech data corresponding to a low confidence word set of one or more words to a remote recognition service.

5. The method of claim 4 wherein evaluating the confidence comprises determining that at least one word set has a recognition confidence below a threshold level.

6. The method of claim 1 further comprising, at the second recognizer, receiving the speech data and the confidently recognized text, and using the confidently recognized text as context in processing the speech data.

7. The method of claim 1 wherein sending the non-confident data to the second recognizer comprises transmitting ambiguous text corresponding to a low confidence word set of one or more words to a remote recognition service.

8. The method of claim 7 wherein evaluating the confidence comprises determining that at least one word has ambiguous candidates recognized for that word.

9. The method of claim 7 wherein evaluating the confidence comprises determining that at least two word sets are based upon ambiguous segmentation resulting in ambiguous candidate word sets.

10. The method of claim 1 wherein outputting the text corresponding to the speech data based on the first recognition result and the second recognition result comprises merging results to include at least one word set from the first recognition result and at least one word set from the second recognition result.

11. The method of claim 1 wherein outputting the text corresponding to the speech data based on the first recognition result and the second recognition result comprises comparing a confidence score associated with at least one word set of the first recognition result with another confidence score associated with a corresponding word set of the second recognition result.

12. The method of claim 1 further comprising, at the second recognizer, receiving the non-confident data in conjunction with any confidently recognized text, and using the confidently recognized text as context in processing the non-confident data.

13. A system comprising:

a memory; and

a processor which is operatively coupled to the memory and which executes code stored in the memory, the processor, in response to execution of the code, being configured to;

receive input data and to provide the input data for recognition by a local recognizer into local recognition results,

process the local recognition results to determine whether the local recognition results meet a confidence criterion, and if not, transmit a combination of confident data and non-confident data corresponding to the local recognition results to a remote recognizer to obtain remote recognition results, and to use the remote recognition results to output a final recognition result corresponding to the input data.

14. The system of claim 13 wherein the input data comprises speech data, wherein the confident data comprises text data, and wherein the non-confident data comprises a portion of the speech data.

15. The system of claim 13 wherein the confident data corresponds to information that provides context relative to the non-confident data.

16. The system of claim 13 wherein the input data comprises speech data, wherein the confident data comprises confidently recognized text data, and wherein the non-confident data comprises ambiguously recognized text data.

17. The system of claim 13 wherein the local recognizer and the code are incorporated into a mobile device.

18. The system of claim 13 wherein the input data comprises speech data or hand written data.

19. One or more processor-readable media having processor-executable instructions, which when executed perform steps, comprising:

receiving local recognition data from a local speech recognizer based upon input speech, the local recognition data including word sets, each word set comprising one or more words and having an associated confidence score;

processing the local recognition data into one or more confidently-recognized word sets and one or more non-confidently-recognized word sets;

sending text corresponding to the one or more confidently-recognized word sets, and speech data corresponding to the one or more non-confidently-recognized word sets, to a remote recognizer;

receiving remote recognition data from the remote recognizer; and

outputting a final recognition result based at least in part on the remote recognition data.

20. The processor-readable media of claim 19 wherein the remote recognition data includes at least one word set, each word set comprising one or more words and having an associated confidence score, and wherein outputting the final recognition result comprises merging one or more word sets from the local recognition data with one more word sets from the remote recognition data based upon confidences score comparisons.