System and methods for improving accuracy of speech recognition utilizing concept to keyword mapping

• US 8,352,266 B2
• Filed: 03/08/2011
• Issued: 01/08/2013
• Est. Priority Date: 10/05/2004
• Status: Active Grant

• Current Assignee: iNAGO Corporation
• Original Assignee: iNAGO Corporation
• Inventors: Farmaner, Gary, Dicarlantonio, Ron, Leonard, Huw
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US13/043,282
• Publication Number:
  US 20110191099A1
• Time in Patent Office: 672 Days
• Field of Search: 704/231, 704/235, 704/251, 704/236, 704/257, 704/9
• US Class Current: 704/257
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks

Abstract

The invention provides a system and method for improving speech recognition. A computer software system is provided for implementing the system and method. A user of the computer software system may speak to the system directly and the system may respond, in spoken language, with an appropriate response. Grammar rules may be generated automatically from sample utterances when implementing the system for a particular application. Dynamic grammar rules may also be generated during interaction between the user and the system. In addition to arranging searching order of grammar files based on a predetermined hierarchy, a dynamically generated searching order based on history of contexts of a single conversation may be provided for further improved speech recognition. Dialogue between the system and the user of the system may be recorded and extracted for use by a speech recognition engine to refine or create language models so that accuracy of speech recognition relevant to a particular knowledge area may be improved.

Claims

1. A speech recognition method for resolving unrecognized sound groups in a partially recognized speech using concept to keywords mappings and sentence types, each sentence type having a plurality of associated grammar rules, the grammar rules being expressed in concepts, said method comprising the steps of:

parsing the partially recognized speech using a natural language processor into a pre-determined sentence type and an associated ordered list of recognized words and place holders corresponding to the unrecognized sound groups;

selecting a list of grammar rules associated with the sentence type from a plurality of grammar rules, each grammar rule of said list having a plurality of constituent concepts, each of said constituent concepts corresponding to one of the recognized words and the unrecognized sound groups;

for each said unrecognized sound group, merging said corresponding constituent concepts in all said selected grammar rules into a list of concepts;

expanding said list of merged concepts using the concept to keywords mappings to produce a list of candidate words wherein each of the merged concepts are expandable to a plurality of words conceptually related to each of the expanded concepts and wherein the plurality of words are the list of candidate words; and

providing the list of candidate words to a speech recognition engine to enable the speech recognition engine to use the list of candidate words to resolve each said unrecognized sound groups in the partially recognized speech corresponding to the place holders.

2. The speech recognition method of claim 1, further comprising the step of preparing a plurality of concept to keywords mappings prior to the step of expansion.

3. The speech recognition method of claim 1, wherein the step of selecting said list of grammar rules includes the steps of comparing the partially recognized speech with each of the plurality of grammar rules and discarding any grammar rules that do not match the partially recognized speech.

4. The speech recognition method of claim 3, wherein the step of comparing includes comparing sentence types and the step of discarding includes discarding grammar rules that do not have the same sentence type as the partially recognized speech.

5. The speech recognition method of claim 3, wherein the step of comparing includes comparing the partially recognized speech with said constituent concepts of each of the plurality of grammar rules and the step of discarding includes discarding grammar rules that do not match any recognized words in the partially recognized speech.

6. The speech recognition method of claim 3, further comprising the step of determining a conversation context of the partially recognized speech, wherein each of said selected grammar rules further has a context designation and the step of comparing including comparing the context designation with the conversation context and the step of discarding includes discarding grammar rules that do not have the conversation context matching the context designation.

7. A method for generating a dynamic grammar file for use by a speech recognition engine to resolve unrecognized sound groups in a speech using concept to keywords mappings and sentence types, each sentence type having a plurality of associated grammar rules, the grammar rules being expressed in concepts, said method comprising the steps of:

parsing the partially recognized speech using a natural language processor into a pre-determined sentence type and an associated ordered list of recognized words and place holders corresponding to the unrecognized sound groups;

selecting a list of grammar rules associated with the sentence type from a plurality of grammar rules stored by a storage unit on a storage medium, each grammar rule of said list having a plurality of constituent concepts, each of said constituent concepts corresponding to one of the recognized words and the unrecognized sound groups;

for each place holder corresponding to said unrecognized sound group, merging said corresponding constituent concepts in all said selected grammar rules into a list of concepts;

generating the dynamic grammar rule from said ordered list by replacing each concept of said list of merged concepts with a list of keywords a list of candidate words using the concept to keywords mappings stored by the storage unit wherein each of the keywords mapped from the merged concept are expandable to a plurality of words conceptually related to each of the merged concepts and wherein the plurality of words are the list of candidate words; and

providing the dynamic grammar rule to the speech recognition engine to enable the speech recognition engine to use the dynamic grammar rule to resolve the unrecognized sound groups in the partially recognized speech corresponding to the place holders.

8. The method according to claim 7, further comprising the step of assigning a context designation to said dynamic grammar rule.

9. The method according to claim 7, wherein said concept to keywords mapping has a context attribute and the context designation assigned to said dynamic grammar rule corresponds to said context attribute.

10. The method according to claim 7, further comprising the step of preparing a plurality of concept to keywords mappings.

11. The method according to claim 7, wherein the step of selecting said list of grammar rules includes the steps of comparing the partially recognized speech with each of the plurality of grammar rules and discarding any grammar rules that do not match the partially recognized speech.

12. The method according to claim 7, wherein the step of comparing includes comparing sentence types and the step of discarding includes discarding grammar rules that do not have the same sentence type as the partially recognized speech.

13. The method according to claim 7, wherein the step of comparing includes comparing the partially recognized speech with discarding includes discarding grammar rules that do not match any recognized words in the partially recognized speech.

14. The method according to claim 7, further comprising the step of determining a conversation context of the partially recognized speech, wherein each of said selected grammar rules further has a context designation and the step of comparing including comparing the context designation with the conversation context and the step of discarding includes discarding grammar rules that do not have the conversation context matching the context designation.

15. A grammar rule generator for use in a speech recognition system, said speech recognition system converting an audible signal representative of spoken words to a textual output from, said speech recognition system comprising a natural language processor for parsing a partially recognized sentence, said grammar rule generator, and a speech recognition engine, said grammar rule generator being configured to generate a dynamic grammar rule from the partially recognized sentence and provide the dynamic grammar rule to the speech recognition engine by:

utilizing the natural language processor to parse the partially recognized speech into a pre-determined sentence type and an ordered list of recognized words and place holders corresponding to unrecognized sound groups;

selecting a list of grammar rules associated with the pre-determined sentence type from a plurality of grammar rules stored by a storage unit on a storage medium, each grammar rule in said list having a plurality of constituent concepts, each of said constituent concepts corresponding to one of the recognized words and the unrecognized sound groups by position in said order list;

for each place holder corresponding to said unrecognized sound group, merging said corresponding constituent concepts in all said selected grammar rules into a list of concepts;

generating the dynamic grammar rule from said ordered list by replacing each concept in said list of merged concepts with a list of keywords a list of candidate words using pre-determined concept to keywords mappings stored by the storage unit wherein each of the keywords mapped from the merged concept are expandable to a plurality of words conceptually related to each of the merged concepts and wherein the plurality of words are the list of candidate words; and

providing the dynamic grammar rule to the speech recognition engine to enable the speech recognition engine to use the dynamic grammar rule to resolve the unrecognized sound groups in the partially recognized speech corresponding to the place holders and to convert the audible signal to the textual output.

16. The grammar rule generator of claim 15, the grammar rule generator being further configured to assign a context designation to said dynamic grammar rule.

17. The grammar rule generator of claim 16, wherein said pre-determined concept to keywords mappings have a context attribute and the context designation assigned to said dynamic grammar rule corresponds to said context attribute.

18. The grammar rule generator of claim 15, the grammar rule generator being further configured to compare the partially recognized speech with each of the plurality of grammar rules when selecting the list of the grammar rules and discard any grammar rules from the selected grammar rules that do not match the partially recognized speech.

19. A method of speech recognition for use by a speech recognition engine to resolve unrecognized sound groups in a speech, said method comprising the steps of:

preparing a plurality of grammar rules storing said plurality of grammar rules on a storage medium, each of said plurality of grammar rules having a context designation assigned thereto and comprising an ordered list of concepts, wherein each of the concepts are expandable to a plurality of words conceptually related to each of the expanded concepts;

determining a conversation context of a speech being recognized by a speech recognition engine and recording said conversation context in a context history;

if said conversation context corresponds to one of said context designations, assigning a ranking order to said context designation in a search sequence as a function of said context history; and

directing said speech recognition engine to search said plurality of grammar rules following said search sequence and utilize grammar rules identified in said search to resolve unrecognized sound groupings in a speech, wherein each of said identified grammar rules are provided to the speech recognition engine with the concepts in the ordered list of concepts expanded to the plurality of words for use by the speech recognition engine to resolve the unrecognized sound groupings in the speech corresponding to the expanded concepts in the ordered list.

20. The method according to claim 19, wherein said ranking order correlates to how recent said conversation context appears in said context history.

21. The method according to claim 19, wherein said ranking order correlates to how frequent said conversation context appears in said context history.

22. The method according to claim 19, wherein said ranking order correlates to total length of time said conversation context represents in said context history.

23. A computer readable storage medium containing computer program instructions stored thereon, when executed by a processor, causing the processor to perform steps of the method of claim 19.

24. A method for improving recognition accuracy of a speech recognition system, the speech recognition system having a speech recognition engine for converting audible signal representative of spoken words into a textual output and a storage unit, the method comprising the steps of:

selecting a first plurality of pre-defined concepts stored on a storage medium of the storage unit;

selecting a second plurality of pre-defined concept to keywords mappings stored on the storage medium, each concept of said first plurality of concepts having at least one concept to keywords mapping;

selecting a third plurality of pre-defined sentence types stored on the storage medium, each pre-defined sentence type being associated with a pre-arranged ordered list of concepts, said pre-arranged ordered list of concepts being formed from said first plurality of pre-defined concepts, each of the concepts in the pre-arranged ordered list of concepts being expandable to a plurality of words that is conceptually related thereto;

providing said first plurality of pre-defined concepts, said second plurality of pre-defined concept to keywords mappings and said third plurality of pre-defined sentence types, together with said associated pre-arranged ordered lists of concepts, to the speech recognition engine of the speech recognition system to enable the speech recognition engine to use said first plurality of concepts, said second plurality of pre-defined concept to keywords mappings, said third plurality of sentence types, and said associated pre-arranged ordered lists of concepts to resolve unrecognized sound groupings in the audible signal, wherein each of the concepts in the ordered lists of concepts are expanded to the plurality of conceptually related words for use by the speech recognition engine to resolve the unrecognized sound groupings in the speech corresponding to the expanded concepts in the ordered list.

25. The method of claim 24, further comprising the steps of:

entering a sample utterance;

parsing said sample utterance into a sentence type and an associated ordered list of concepts using a natural language processor;

generating a grammar rule from said sentence type and said associated ordered list of concepts using a grammar rule generator; and

providing said grammar rule to the speech recognition engine to enable the speech recognition engine to use said grammar rule to resolve unrecognized sound groupings in the audible signal.

26. The method of claim 24, further comprising the steps of:

entering a plurality of sample utterances;

parsing each of said sample utterances into a sentence type and an associated second ordered list of concepts using a natural language processor;

generating a plurality of grammar rules each from a respective one of said plurality of sentence types and a respective one of said plurality of associated second ordered lists of concepts using a grammar rule generator; and

providing said plurality of grammar rules to the speech recognition engine to enable the speech recognition engine to use said plurality of grammar rules to resolve unrecognized sound groupings in the audible signal.

27. The method of claim 26, further comprising the steps of:

providing a text input corresponding to a partially recognized audible signal to a natural language processor;

parsing said text input into a second sentence type and an associated ordered list of recognized words and unrecognized sound groupings using the natural language processor;

selecting a list of grammar rules associated with the second sentence type from said plurality of grammar rules, each grammar rule of said list having a plurality of constituent concepts;

expanding each of said constituent concepts at a location corresponding to one of said unrecognized sound groupings into a plurality of related words; and

providing said plurality of related words to the speech recognition engine to enable the speech recognition engine to use said plurality of related words to resolve the one of said unrecognized sound groupings.

28. A method of generating a grammar rule for use by a speech recognition engine, said method comprising the steps of:

parsing a sample sentence using a natural language processor into an ordered list of keywords that matches a pre-arranged ordered list of keywords stored by a storage unit on a storage medium, the pre-arranged ordered list being associated with a pre-defined sentence type;

matching each keyword of said ordered list to a concept using a concept to keywords mapping, wherein each of a respective matched concept is expandable to a plurality of words conceptually related thereto; and

producing the grammar rule from said ordered list by replacing each said concept with a list of keywords using the concept to keywords mapping each keyword of the ordered list with the plurality of words that is conceptually related to the respective matched concept,

wherein said grammar rule is to be provided to the speech recognition engine for the speech recognition engine to use the plurality of words in said grammar rule to resolve unrecognized sound groupings in a speech corresponding to the replaced keywords in the ordered list of keywords.

29. The method according to claim 28, further comprising the step of assigning a context designation to said grammar rule.

30. The method according to claim 28, wherein said concept to keywords mapping has a context attribute and the context designation assigned to said grammar rule corresponds to said context attribute.

31. The method according to claim 28, further comprising the step of preparing a plurality of concept to keywords mappings.