Allocation of speech recognition tasks and combination of results thereof

• US 20060009980A1
• Filed: 07/12/2004
• Published: 01/12/2006
• Est. Priority Date: 07/12/2004
• Status: Active Grant

• Current Assignee: Valtrus Innovations Limited (f/k/a Dolya Holdco 9 Limited) (Key Patent Innovations Limited)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Yacoub, Sherif, Burke, Paul M.
• Granted Patent: US 8,589,156 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

A system, method, computer-readable medium, and computer-implemented system for optimizing allocation of speech recognition tasks among multiple speech recognizers and combining recognizer results is described. An allocation determination is performed to allocate speech recognition among multiple speech recognizers using at least one of an accuracy-based allocation mechanism, a complexity-based allocation mechanism, and an availability-based allocation mechanism. The speech recognition is allocated among the speech recognizers based on the determined allocation. Recognizer results received from multiple speech recognizers in accordance with the speech recognition task allocation are combined.

Claims

1. A system for using multiple speech recognizers, the system comprising:

an allocation determination mechanism for determining an allocation of speech recognition tasks among multiple speech recognizers, wherein the multiple speech recognizers are executed on more than one device at least one of which is a mobile device;

a task allocation mechanism for allocating speech recognition tasks among the multiple speech recognizers based on an allocation determined by the allocation determination mechanism; and

a combination mechanism for combining recognizer results received from the multiple speech recognizers in accordance with the speech recognition task allocation.

2. The system of claim 1, wherein the allocation determination mechanism includes at least one of an accuracy-based allocation mechanism, a complexity-based allocation mechanism, and an availability-based allocation mechanism.

3. The system of claim 2, wherein the allocation determination mechanism allocates between a mobile-device-based speech recognizer and at least one server-based speech recognizer.

4. The system of claim 2, wherein the accuracy-based allocation mechanism includes using a confusion matrix for each of the multiple speech recognizers.

5. The system of claim 4, wherein the confusion matrix is generated based on a predetermined audio environment type.

6. The system of claim 5, wherein the predetermined audio environment type includes at least one of a noisy environment and a normal environment.

7. The system of claim 5, further comprising:

an audio environment determination mechanism for determining an audio environment type.

8. The system of claim 7, wherein the audio environment determination mechanism determines an appropriate confusion matrix for use with the determined audio environment type by the multiple speech recognizers.

9. A method for using multiple speech recognizers, comprising the steps of:

determining an allocation of speech recognition tasks among the multiple speech recognizers, wherein the multiple speech recognizers execute on more than one device at least one of which is a mobile device;

allocating speech recognition tasks among the multiple speech recognizers based on the determined allocation; and

combining recognizer results received from the multiple speech recognizers in accordance with the speech recognition task allocation.

10. The method of claim 9, wherein the determining step further includes determining the allocation based on at least one of a required accuracy of speech recognition output, a complexity level of speech recognition input, and an availability of the speech recognizers.

11. The method of claim 10, wherein the accuracy-based allocation mechanism includes a confusion matrix for each of the multiple speech recognizers.

12. The method of claim 10, further comprising:

generating a confusion matrix based on a predetermined audio environment type.

13. The method of claim 12, wherein the predetermined audio environment type includes at least one of a noisy environment and a normal environment.

14. The method of claim 12, further comprising the steps of:

determining an audio environment type at the mobile device; and

determining an appropriate confusion matrix for use by the multiple speech recognizers based on the determined audio environment type.

15. A computer-readable medium comprising:

at least one sequence of machine executable instructions; and

the medium bearing the executable instructions in machine form, wherein execution of the instructions by a processor causes the processor to;

determine an allocation of speech recognition tasks among multiple speech recognizers, allocate speech recognition tasks among multiple speech recognizers based on the determined allocation, and combine recognition results received from the multiple speech recognizers in accordance with the speech recognition task allocation, wherein the multiple speech recognizers execute on more than one device at least one of which is a mobile device.

17. The computer-readable medium of claim 15, wherein the allocation determination is performed using at least one of an accuracy-based allocation mechanism, a complexity-based allocation mechanism, and an availability-based allocation mechanism.

18. The computer-readable medium of claim 17, further comprising instructions which, when executed by the processor, cause the processor to generate a confusion matrix based on a predetermined audio environment type.

19. The computer-readable medium of claim 18, further comprising instructions which, when executed by the processor, cause the processor to determine an audio environment type at the mobile device and determine an appropriate confusion matrix for use by the multiple speech recognizers based on the determined audio environment type.

20. A computer-implemented system for allocating speech recognition tasks among multiple speech recognizers, the system comprising:

a processor for receiving and transmitting data; and

a memory coupled to the processor, the memory having stored therein instructions causing the processor to determine an allocation of speech recognition tasks among multiple speech recognizers, allocate speech recognition tasks among multiple speech recognizers based on the determined allocation, and combine recognition results received from the multiple speech recognizers in accordance with the speech recognition task allocation, wherein the multiple speech recognizers execute on more than one device at least one of which is a mobile device.

21. The system of claim 20, wherein the allocation determination is performed using at least one of an accuracy-based allocation mechanism, a complexity-based allocation mechanism, and an availability-based allocation mechanism.

22. The system of claim 21, further comprising instructions which, when executed by the processor, cause the processor to generate a confusion matrix based on a predetermined audio environment type.

23. The system of claim 21, further comprising instructions which, when executed by the processor, cause the processor to determine an audio environment type at the mobile device and determine an appropriate confusion matrix for use by the multiple speech recognizers based on the determined audio environment type.