Hybrid speech recognition

• US 9,620,122 B2
• Filed: 12/08/2011
• Issued: 04/11/2017
• Est. Priority Date: 12/08/2011
• Status: Active Grant

• Current Assignee: Lenovo PC International Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: VanBlon, Russell Speight
• Primary Examiner(s): Godbold, Douglas
• Assistant Examiner(s): Villena, Mark
• Application Number:
  US13/314,533
• Publication Number:
  US 20130151250A1
• Time in Patent Office: 1,951 Days
• Field of Search: 704 10, 704201, 704231, 704235, 704257, 704275
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

Described is a technology by which speech is locally and remotely recognized in a hybrid way. Speech is input and recognized locally, with remote recognition invoked if locally recognized speech data was not confidently recognized. The part of the speech that was not confidently recognized is sent to the remote recognizer, along with any confidently recognized text, which the remote recognizer may use as context data in interpreting the part of the speech data that was sent. Alternative text candidates may be sent instead of corresponding speech to the remote recognizer.

Claims

1. A method, comprising:

receiving handwriting input at a first device;

executing, at the first device, handwriting recognition on the handwriting input to render a first recognition result comprising first text derived from the handwriting input;

evaluating confidence in at least a portion of the first recognition result;

in response to identification of a confidence evaluation for at least a portion of the first recognition result as being below a threshold level, sending data pertaining to at least a first portion of the handwriting input to a second device different from the first device and sending the first text to the second device, wherein the threshold level varies based on a speed of communication of the first device with the second device; and

receiving from the second device a second recognition result comprising second text derived from the data.

2. The method of claim 1 wherein sending the data pertaining to the first portion of the handwriting input to the second device comprises transmitting to the second device data corresponding to the portion of the first recognition result evaluated as being below the threshold level, the data corresponding to the portion of the first recognition result evaluated as being below the threshold level comprising at least one character derived from the first portion of the handwriting input.

3. The method of claim 1 wherein sending the data pertaining to at least the first portion of the handwriting input to the second device comprises transmitting to the second device ambiguous handwriting data corresponding to at least one low confidence character recognized from the received handwriting input.

4. The method of claim 3 wherein evaluating confidence comprises determining that at least one character of the received handwriting input implicates ambiguous candidates.

5. The method of claim 1 wherein outputting data based on the first text and the second text comprises comparing a confidence score associated with at least one character of the first recognition result with another confidence score associated with at least one character of the second recognition result.

6. The method of claim 1 further comprising encoding the first text and the data pertaining to at least the first portion of the handwriting input for sending the data pertaining to at least the first portion of the handwriting input.

7. The method of claim 1, further comprising:

outputting data at the first device based on the first text and the second text.

8. The method of claim 7 wherein outputting data based on the first text and the second text comprises merging the first text and the second text and presenting the merged first text and second text at the first device.

9. A method, comprising:

receiving speech input at a first device;

converting the speech input to first text, wherein convening of the first portion comprises identifying at least a first potential candidate corresponding to the first portion and a second potential candidate corresponding to the first portion;

determining that at least a first portion of the speech input cannot be identified with at least a threshold level of confidence;

transmitting the first text to a second device; and

receiving, from the second device, second text comprising at least a portion of the first text and comprising one of the first potential candidate and the second potential candidate;

wherein the threshold level of confidence varies based on a speed of communication of the first device with the second device.

10. The method of claim 9, comprising:

outputting the second text at the first device.

11. The method of claim 9, wherein the first text is transmitted to the second device in an order in which respective portions of the speech input are received that correspond to respective portions of the first text.

12. The method of claim 9, comprising:

encoding the first text for transmission to the second device.

13. A first device, comprising:

a processor; and

storage accessible to the processor, the storage bearing instructions executable by the processor to;

receive speech input;

determine that at least a first portion of the speech input cannot be identified with at least a threshold level of confidence;

convert the speech input to first text, wherein conversion of the first portion comprises identifying at least a first potential candidate corresponding to the first portion and a second potential candidate corresponding to the first portion;

transmit the first text to a second device; and

receive, from the second device, second text comprising at least a portion of the first text and comprising one of the first potential candidate and the second potential candidate;

wherein the threshold level of confidence varies based on a speed of communication of the first device with the second device.

14. The first device of claim 13, wherein the instructions are executable to:

output the second text at the first device.

15. The first device of claim 13, wherein the first text is transmitted to the second device in an order in which respective portions of the speech input are received that correspond to respective portions of the first text.