Multipass ASR controlling multiple applications

• US 9,093,076 B2
• Filed: 04/30/2012
• Issued: 07/28/2015
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: 2236008 Ontario Inc. (Blackberry Limited)
• Inventors: Fry, Darrin Kenneth
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/460,443
• Publication Number:
  US 20130289996A1
• Time in Patent Office: 1,184 Days
• Field of Search: 704/231, 704/235, 704/252
• US Class Current: 1/1
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A multipass processing system includes a first grammar-based speech recognition system that compares a spoken utterance to a sub-grammar. The sub-grammar includes keywords or key phrases from active grammars that each uniquely identifies one of many application engines. The first grammar-based speech recognition system generates a first grammar-based speech recognition result and a first grammar-based confidence score. A demultiplexer receives the spoken utterance through an input. The demultiplexer transmits the spoken utterance to one of many other grammar-based speech recognition systems based on the first grammar-based speech recognition-result.

Claims

1. A multipass processing system comprising:

a first grammar-based speech recognition system that compares a spoken utterance to a sub-grammar comprising keywords or key phrases from active grammars of a plurality of other grammar-based speech recognition systems and each of the keywords or key phrases uniquely identifies one of a plurality of application engines and an associated one of the plurality of other grammar-based speech recognition systems, to generate a first grammar-based speech recognition result and a first grammar-based confidence score;

a demultiplexer that receives the spoken utterance through an input and transmits the spoken utterance to a selected one of the plurality of other grammar-based speech recognition systems wherein the selection of the other grammar-based speech recognition system is based on the first grammar-based speech recognition result, and wherein the selected one of the plurality of other grammar-based speech recognition systems generates a second grammar-based speech recognition result and a second confidence score; and

a system management controller communicatively coupled to the plurality of other grammar-based speech recognition systems, which receives the second grammar-based speech recognition result and the second confidence score and enables forwarding of the spoken utterance from the demultiplexer to a second one of the plurality of the other grammar-based speech recognition systems in instances when the second confidence score does not exceed a predetermined threshold.

2. The multipass processing system of claim 1 where the plurality of other grammar based speech recognition systems are configured to compare the spoken utterance to a vocabulary that includes at least one keyword or key phrase of the sub-grammar.

3. The multipass processing system of claim 1 where the first grammar-based speech recognition-result comprises text.

4. The multipass processing system of claim 1 further comprising a second sub-grammar comprising other keywords or other key phrases from the active grammars where the first grammar-based speech recognition system compares the spoken utterance to the second sub-grammar when the first-grammar based confidence score does not exceed a predetermined threshold.

5. The multipass processing system of claim 4 where the demultiplexer executes a comparison between the first grammar-based confidence score and the predetermined threshold and prompts the first grammar-based speech recognition system to compare the spoken utterance to the second sub-grammar.

6. The multipass processing system of claim 1 wherein the system management controller communicates and arbitrates with a plurality of the plurality of other grammar-based speech recognition systems based on results received from the plurality of the plurality of other grammar-based speech recognition systems to determine which of the plurality of the plurality of other grammar-based speech recognition systems publishes its speech recognition result to a respective application engine.

7. The multipass processing system of claim 1 further comprising a plurality of application control systems that directly control the plurality of application engines by transmitting electronic signals to the signal inputs of the plurality of application engines.

8. The multipass processing system of claim 1 where the first grammar-based speech recognition result comprises a language agnostic output.

9. The multipass processing system of claim 1 where the spoken utterance comprises a converted frame of data.

10. The multipass processing system of claim 1 where the first grammar-based speech recognition system, the demultiplexer, and the plurality of other grammar-based speech recognition systems comprise a plurality of stateless devices or stateless systems.

11. The multipass processing system of claim 1 where the first grammar-based speech recognition system, the demultiplexer, and one of the plurality of other grammar-based speech recognition systems process the same spoken utterance.

12. The multipass processing system of claim 1 where the first grammar-based speech recognition system, the demultiplexer, and one of the plurality of other grammar-based speech recognition systems process the same spoken utterance in a sequential order.

13. The multipass processing system of claim 1 where the demultiplexer comprises a demultiplexer controller that transmits control signals and a demultiplexing logic remote from the demultiplexer controller that transmits the spoken utterance to one of the plurality of other grammar-based speech recognition systems.

14. The multipass processing system of claim 1 where each of the first grammar-based speech recognition systems, the demultiplexer, and the plurality of other grammar-based speech recognition systems comprise a separate computing thread executed by one processor.

15. The multipass processing system of claim 1 where each of the first grammar-based speech recognition system, the demultiplexer, and the plurality of other grammar-based speech recognition systems are executed by a plurality of parallel processors.

16. The multipass processing system of claim 1 where the grammar-based speech recognition system comprises

a natural language-based speech recognition system that compares the spoken utterance to a natural language vocabulary to generate a natural language speech recognition result and a natural language confidence score;

and

a master conversation module engine that compares the natural language speech recognition result generated by the natural language-based speech processing system to a plurality of selected words that are each mapped to one of a plurality of domain specific words and phrases to generate the first grammar-based speech recognition result and the first grammar-based confidence score.

17. A computer implemented method of automatically recognizing speech comprising:

capturing a speech utterance by converting frames of spoken utterances into electronic signals;

recognizing the speech utterance by comparing the frames of speech to a sub-grammar comprising keywords or key phrases from active grammars of a plurality of other grammar-based speech recognition systems and each of the keywords or key phrases uniquely identifies one of a plurality of application engines and an associated one of the plurality of other grammar-based speech recognition systems;

generating a first grammar-based speech recognition result and a first grammar-based confidence score;

comparing the first grammar-based confidence score to a predetermined threshold;

forwarding the frames of speech to a selected one of the plurality of other grammar-based speech recognition systems wherein the selection of the other grammar-based speech recognition system is based on the results of the comparison, and wherein the selected one of the plurality of other grammar-based speech recognition systems generates a second grammar-based speech recognition result and a second confidence score; and

forwarding of the spoken utterance to a second one of the plurality of the other grammar-based speech recognition systems when the second confidence score does not exceed a predetermined threshold.

18. The computer implemented method of claim 17 further comprising comparing the spoken utterance to a second sub-grammar comprising other keywords or other key phrases from the active grammars.

19. The computer implemented method of claim 17 where the method of automatically recognizing speech comprises a stateless process.

20. A computer implemented method comprising:

capturing speech utterance by converting frames of spoken utterances into electronic signals;

recognizing the speech utterance by comparing the frames of speech to a sub-grammar comprising keywords or key phrases from active grammars of a plurality of other grammar-based speech recognition systems and each of the keywords or key phrases uniquely identifies one of a plurality of application engines and an associated one of the plurality of other grammar-based speech recognition systems;

generating a first grammar-based speech recognition result and a first grammar-based confidence score;

comparing the first grammar-based confidence score to a predetermined threshold; and

forwarding the frames of speech to a selected one of the plurality of other grammar-based speech recognition systems wherein the selection of the other grammar-based speech recognition system is based on the results of the comparison, and wherein the selected one of the plurality of other grammar-based speech recognition systems generates a second grammar-based speech recognition result and a second confidence score; and

forwarding of the frames of speech to a second one of the plurality of the other grammar-based speech recognition systems when the second confidence score does not exceed a predetermined threshold.

21. The method of claim 20 where the acts of capturing the speech utterance, recognizing the speech utterance, generating the first grammar-based speech recognition result, comparing the first grammar-based confidence score, and forwarding the frames of speech occur in a sequential order in real-time.