HYBRIDIZED CLIENT-SERVER SPEECH RECOGNITION

• US 20120215539A1
• Filed: 02/22/2012
• Published: 08/23/2012
• Est. Priority Date: 02/22/2011
• Status: Active Grant

• Current Assignee: Speak With Me, Inc.
• Original Assignee: Speak With Me, Inc.
• Inventors: Juneja, Ajay
• Granted Patent: US 9,674,328 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 2250/14 including a card reading device
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

A recipient computing device can receive a speech utterance to be processed by speech recognition and segment the speech utterance into two or more speech utterance segments, each of which can be to one of a plurality of available speech recognizers. A first one of the plurality of available speech recognizers can be implemented on a separate computing device accessible via a data network. A first segment can be processed by the first recognizer and the results of the processing returned to the recipient computing device, and a second segment can be processed by a second recognizer implemented at the recipient computing device.

Claims

1. A computer program product comprising a computer-readable storage medium storing instructions that, when executed by a computing system comprising at least one programmable processor, cause the computing system to perform operations comprising:

receiving, at a recipient computing device, a speech utterance to be processed by speech recognition;

segmenting the speech utterance into two or more speech utterance segments;

assigning each of the two or more speech utterance segments to one of a plurality of available speech recognizers, the assigning comprising designating a first segment of the two or more speech utterance segments for processing by a first speech recognizer of the plurality of available speech recognizers that is implemented on a separate computing device than the recipient computing device and a second segment of the two or more speech utterance segments for processing by a second speech recognizer of the plurality of available speech recognizers that is implemented on the recipient computing device;

sending the first segment from the recipient computing device to the separate computing device for processing;

receiving first segment processing results back from the separate computing device, the sending and the receiving occurring via a data network;

processing the second segment at the recipient computing device to generate second segment processing results; and

returning a completed speech recognition result, the completed speech recognition result comprising the first segment processing results and the second processing results.

2. A computer program product as in claim 1, wherein the segmenting comprises initially analyzing the speech utterance using an acoustic model implemented on one or more processors at the recipient computing device, the initially analyzing identifying features of the received speech utterance that can be more efficiently processed by the separate computing device than the recipient computing device.

3. A computer program product as in claim 2, wherein the initially analyzing comprises determining a confidence score and an audio quality metric for at least one of the plurality of the two or more speech utterance segments.

4. A computer program product as in claim 3, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when at least one of the confidence score and the audio quality metric are below respective threshold values, the respective threshold values being determined based on one or more criteria defined at design time or dynamically evaluated at run time.

5. A computer program product as in claim 1, wherein the recipient computing device comprises a thin client computing device or terminal and the separate computing device comprises at least one server accessible over the data network from the thin client computing device or terminal.

6. A computer program product as in claim 1, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when the first segment is determined by the initially analyzing to include one or more words that relate to data that are more readily accessible at the separate computing device than at the recipient computing device.

7. A computer program product as in claim 6, wherein the first segment processing results comprise analysis of the first segment performed by the separate computing device using an adapted language model, the adapted language model being adapted based on the data that are more readily accessible at the separate computing device than at the recipient computing device.

8. A recipient computing device comprising:

a computing system comprising at least one programmable processor;

a computer-readable storage medium storing instructions that, when executed by the computing system, cause the at least one programmable processor of the computing system to perform operations comprising;

receiving, at the recipient computing device, a speech utterance to be processed by speech recognition;

segmenting the speech utterance into two or more speech utterance segments;

assigning each of the two or more speech utterance segments to one of a plurality of available speech recognizers, the assigning comprising designating a first segment of the two or more speech utterance segments for processing by a first speech recognizer of the plurality of available speech recognizers that is implemented on a separate computing device than the recipient computing device and a second segment of the two or more speech utterance segments for processing by a second speech recognizer of the plurality of available speech recognizers that is implemented on the recipient computing device;

sending the first segment from the recipient computing device to the separate computing device for processing;

receiving first segment processing results back from the separate computing device, the sending and the receiving occurring via a data network;

processing the second segment at the recipient computing device to generate second segment processing results; and

returning a completed speech recognition result, the completed speech recognition result comprising the first segment processing results and the second processing results.

9. A recipient computing device as in claim 8, wherein the segmenting comprises initially analyzing the speech utterance using an acoustic model implemented on one or more processors at the recipient computing device, the initially analyzing identifying features of the received speech utterance that can be more efficiently processed by the separate computing device than the recipient computing device.

10. A recipient computing device as in claim 9, wherein the initially analyzing comprises determining a confidence score and an audio quality metric for at least one of the plurality of the two or more speech utterance segments.

11. A recipient computing device as in claim 10, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when at least one of the confidence score and the audio quality metric are below respective threshold values, the respective threshold values being determined based on one or more criteria defined at design time or dynamically evaluated at run time.

12. A recipient computing device as in claim 8, wherein the recipient computing device comprises a thin client computing device or terminal and the separate computing device comprises at least one server accessible over the data network from the thin client computing device or terminal.

13. A recipient computing device as in claim 8, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when the first segment is determined by the initially analyzing to include one or more words that relate to data that are more readily accessible at the separate computing device than at the recipient computing device.

14. A recipient computing device as in claim 13, wherein the first segment processing results comprise analysis of the first segment performed by the separate computing device using an adapted language model, the adapted language model being adapted based on the data that are more readily accessible at the separate computing device than at the recipient computing device.

15. A method comprising:

receiving, at a recipient computing device comprising a computing system comprising at least one programmable processor, a speech utterance to be processed by speech recognition;

segmenting, by the computing system, the speech utterance into two or more speech utterance segments;

assigning, by the computing system, each of the two or more speech utterance segments to one of a plurality of available speech recognizers, the assigning comprising designating a first segment of the two or more speech utterance segments for processing by a first speech recognizer of the plurality of available speech recognizers that is implemented on a separate computing device than the recipient computing device and a second segment of the two or more speech utterance segments for processing by a second speech recognizer of the plurality of available speech recognizers that is implemented on the recipient computing device;

sending, by the computing system, the first segment from the recipient computing device to the separate computing device for processing;

receiving, by the computing system, first segment processing results back from the separate computing device, the sending and the receiving occurring via a data network;

processing, by the computing system, the second segment at the recipient computing device to generate second segment processing results; and

returning, by the computing system, a completed speech recognition result, the completed speech recognition result comprising the first segment processing results and the second processing results.

16. A method as in claim 15, wherein the segmenting comprises initially analyzing the speech utterance using an acoustic model implemented on one or more processors at the recipient computing device, the initially analyzing identifying features of the received speech utterance that can be more efficiently processed by the separate computing device than the recipient computing device.

17. A method as in claim 16, wherein the initially analyzing comprises determining a confidence score and an audio quality metric for at least one of the plurality of the two or more speech utterance segments.

18. A method as in claim 17, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when at least one of the confidence score and the audio quality metric are below respective threshold values, the respective threshold values being determined based on one or more criteria defined at design time or dynamically evaluated at run time.

19. A method as in claim 15, wherein the recipient computing device comprises a thin client computing device or terminal and the separate computing device comprises at least one server accessible over the data network from the thin client computing device or terminal.

20. A method as in claim 15, wherein the assigning comprises designating the first segment for processing by the first speech recognizer implemented on the separate computing device when the first segment is determined by the initially analyzing to include one or more words that relate to data that are more readily accessible at the separate computing device than at the recipient computing device.