POST PROCESSING OF NATURAL LANGUAGE ASR

• US 20130289988A1
• Filed: 04/30/2012
• Published: 10/31/2013
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: Blackberry Limited
• Original Assignee: QNX Software Systems Limited (Canada) (Blackberry Limited)
• Inventors: Fry, Darrin Kenneth
• Granted Patent: US 9,431,012 B2
• Time in Patent Office: Days
• US Class Current: 704/239
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A post-processing speech system includes a natural language-based speech recognition system that compares a spoken utterance to a natural language vocabulary that includes words used to generate a natural language speech recognition result. A master conversation module engine compares the natural language speech recognition result to domain specific words and phrases. A voting engine selects a word or a phrase from the domain specific words and phrases that is transmitted to an application control system. The application control system transmits one or more control signals that are used to control an internal or an external device or an internal or an external process.

Claims

1. A post-processing automated speech recognition system comprising:

a natural language-based speech recognition system that compares a spoken utterance to a natural language vocabulary comprising words in active grammars to generate a natural language speech recognition result and a natural language confidence score;

a master conversation module engine that compares the natural language speech recognition result generated by the natural language-based speech processing system to a plurality of active sub-grammars that are mapped to a plurality of domain specific words and phrases;

a voting module engine interfaced to the master conversation module engine that selects a word or a phrase from the plurality of active sub-grammars to be transmitted to an application control system that transmits control signals used to control an internal or an external device or an internal or an external process;

where the natural language-based speech recognition system, the master conversation module engine, and the voting module engine are executed by one or more processors.

2. The post-processing automated speech recognition system of claim 1 where the results generated by the natural language-based speech recognition system and the result selected by the voting module engine comprise language agnostic symbols.

3. The post-processing automated speech recognition system of claim 1 where a portion of the natural language-based speech recognition system is remote from the master conversation module engine and the plurality of active sub-grammars are remote from a portion of the natural language-based speech processing system.

4. The post-processing automated speech recognition system of claim 1 further comprising means to resolve an ambiguity between the comparisons between the natural language speech recognition result generated by the natural language-based speech recognition system and the plurality of active sub-grammars.

5. The post-processing automated speech recognition system of claim 1 further comprising a grammar-based speech recognition system that compares spoken utterance to a grammar-based vocabulary comprising the active sub-grammars that are mapped to the plurality of domain specific words and phrases.

6. The post-processing automated speech recognition system of claim 5 further comprising an alignment engine that selects between the word or the phrase selected by the voting module engine and recognition results generated by the grammar-based speech recognition system, where the alignment engine transmits the selection to one of a plurality of application control systems.

7. The post-processing automated speech recognition system of claim 6 where the voting module engine comprises a portion of the master conversation module engine and the word or the phrase selected by the voting engine comprises the recognition result of the master conversation module engine, and where the alignment engine selects between the recognition results of the master conversation module engine and the recognition result generated by the grammar-based speech recognition system based on the natural language confidence score and a grammar-based confidence score.

8. The post-processing automated speech recognition system of claim 7 where the alignment engine selects between the recognition results of the master conversation module engine and the recognition result generated by the grammar-based speech recognition system based on an order the recognition results are received at the alignment engine.

9. The post-processing automated speech recognition system of claim 6 further comprising an application engine that controls an internal or external device or process in response to the alignment engine'"'"'s transmission of a selection result.

10. The post-processing automated speech recognition system of claim 1 further comprising a grammar-based speech processing system that generates a grammar-based speech recognition and grammar-based confidence score synchronously with the natural language-based speech recognition system.

11. The post-processing automated speech recognition system of claim 10 where the grammar-based speech processing system and the natural language-based speech processing system process the same spoken utterance.

12. The post-processing automated speech recognition system of claim 10 where the grammar-based speech processing system and the natural language-based speech recognition system are executed by a plurality of processors that run simultaneously.

13. The post-processing automated speech recognition system of claim 10 where the grammar-based speech processing system and the natural language-based speech recognition system, and the voting module engine are executed by a plurality of parallel processors.

14. The post-processing automated speech recognition system of claim 10 where the grammar-based speech processing system and the natural language-based speech recognition system, and the voting module engine each comprise a computing thread executed by a multitasking processor.

15. The post-processing automated speech recognition system of claim 1 further comprising a grammar-based speech recognition system that generates grammar-based speech recognition results synchronously with a plurality of natural language-based speech recognition systems interfaced to the master conversation module engine.

16. A computer implemented method of automatically recognizing speech comprising:

capturing speech utterance by converting frames of spoken utterances into electronic signals;

recognizing speech utterances by comparing the frames of speech to a list of words in active grammars;

generating a natural language-based speech recognition result and a natural language confidence score;

comparing the natural language-based speech recognition result to a domain specific vocabulary that comprise sub-grammars that are mapped to domain specific words or phrases; and

selecting a speech recognition result based on a voting scheme.

17. The computer implemented method of claim 16 further comprising generating multiple confidence scores that reflect a likelihood that each of a plurality of words and a plurality of phrases that comprise the sub-grammars match the speech recognition result.

18. The computer implemented method of claim 17 where the natural language-based speech recognition result and sub-grammars comprise indicia representing a plurality of words and a plurality of phrases.

19. A method of automatically recognizing speech comprising:

capturing speech utterance by converting frames of speech into analog signals;

recognizing speech utterances by comparing the frames of speech to a list of words in active grammars;

generating automated speech recognition results from a natural language-based speech recognition system;

comparing speech recognition results to a domain specific vocabulary that comprise sub-grammars mapped to one or more application control signals that control an internal or external device or process through an application engine; and

selecting a speech recognition result based on a voting scheme.

20. The method of claim 19 further comprising executing a second selection by selecting between the speech recognition result and a grammar-based recognition result that is generated by a grammar-based speech recognition system that processes the same analog speech signals.