Auto-generation of parsing grammars from a concept ontology

• US 10,811,004 B2
• Filed: 03/28/2013
• Issued: 10/20/2020
• Est. Priority Date: 03/28/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Peters, Stephen Douglas, Tremblay, Real
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US13/852,165
• Publication Number:
  US 20140297282A1
• Time in Patent Office: 2,763 Days
• Field of Search: 704254, 704 9, 704275, 7042701, 704257, 704244, 704 2, 3254, 700 49, 379 52
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules

Abstract

An ontology stores information about a domain of an automatic speech recognition (ASR) application program. The ontology is augmented with information that enables subsequent automatic generation of a speech understanding grammar for use by the ASR application program. The information includes hints about how a human might talk about objects in the domain, such as preludes (phrases that introduce an identification of the object) and postludes (phrases that follow an identification of the object).

Claims

1. A method comprising:

receiving, by a computing device, an audio input based on an utterance by a user, wherein the audio input is received through an automatic speech recognition (ASR) application;

converting, by the computing device, the audio input into a string of words;

determining, by the computing device, a concept from the string of words by;

querying a predefined ontology for one or more ludes that comprise at least one word of the string of words, wherein the predefined ontology describes a plurality of hierarchically organized concepts related to a domain of the ASR application, and wherein the predefined ontology stores instance information comprising respective one or more ludes corresponding to each of the plurality of hierarchically organized concepts;

determining that one of the respective one or more ludes comprises at least one word of the string of words, wherein the one of the respective one or more ludes corresponds to the concept; and

identifying a prelude-postlude pair for the respective one or more ludes comprising the at least one word of the string of words;

generating, by the computing device and based on the predefined ontology, a speech understanding grammar associated with the concept by;

receiving a user selection of the concept determined from the string of words;

generating a grammar skeleton with one or more rule groups, wherein each rule group includes one or more rules with one or more placeholders for a prelude-postlude pair; and

automatically populating at least one placeholder of the one or more placeholders of the one or more rules with the identified prelude-postlude pair from the predefined ontology for the respective one or more ludes comprising the at least one word of the string of words;

sending, by the computing device, the speech understanding grammar to the ASR application.

2. The method of claim 1, wherein the sub-concept is associated with a semantic attachment that is configured to identify the super-concept.

3. The method of claim 1, wherein each lude comprises the prelude and the postlude.

4. The method of claim 3, wherein the prelude and the postlude pair are associated with a time order.

5. The method of claim 4, further comprising:

determining a first word from the string of words that is bound by the prelude and the postlude; and

determining the concept based on the first word.

6. The method of claim 5 further comprising:

determining, by the computing device and based on the concept, external grammars stored in an external computerized database;

generating, by the computing device, a pointer to the external grammars; and

sending, by the computing device, the pointer to the ASR application.

7. The method of claim 1, wherein the querying the predefined ontology further comprises:

comparing the instance information to at least one word from the string of words.

8. The method of claim 7, further comprising:

determining the concept based on determining that the instance information is configured to identify the concept.

9. The method of claim 1 further comprising:

determining, by the computing device and based on the plurality of hierarchically organized concepts, external grammars stored in an external computerized database;

generating, by the computing device, a pointer to the external grammars; and

sending, by the computing device, the pointer to the ASR application.

10. The method of claim 1, further comprising retrieving, from the predefined ontology, a first hint and a second hint, wherein the first and second hints relate to the concept, wherein the generating comprises concatenating the retrieved first and second hints, and wherein at least one of the first and second hints is used to determine one or more of the sub-concept or the super-concept.

11. The method of claim 10, wherein the first hint and second hints comprise the prelude or the postlude.

12. A method comprising:

receiving, by a computing device, an audio input associated with a user utterance;

converting, by the computing device, the audio input into a string of words;

performing, by the computing device, natural language processing on the string of words by;

comparing at least one word from the string of words to a prelude and a postlude, wherein the prelude and the postlude are a pair configured to determine one or more concepts from a predefined ontology, wherein the prelude is a phrase introducing identification of an instance of the concept and the postlude is a phrase that follows an identification of the instance of the concept;

determining a plurality of grammars associated with the plurality of prelude and postlude pairs by;

causing display of the one or more concepts based on determining that the at least one word corresponds to the prelude or the postlude pair;

receiving a user selection of a concept;

in response to the receiving the user selection, determining, at least one of a sub-concept and a super-concept corresponding to the concept, wherein the at least one of the sub-concept and the super-concept have a hierarchical relationship to the concept;

determining, based on the concept and the at least one of the sub-concept and the super-concept, a plurality of preludes and postludes, wherein each of the plurality of prelude and postlude are pairs configured to identify one of the concept and the at least one of the sub-concept and the super-concept; and

generating a grammar skeleton with one or more rule groups, wherein each rule group includes one or more rules with one or more placeholders for a prelude and postlude pair; and

automatically populating at least one placeholder of the one or more placeholders of the one or more rules with the prelude and postlude pair from the concept of the predefined ontology and the at least one of the sub-concept and the super-concept; and

transmitting, by the computing device, the plurality of grammars to an application program of a user device.

13. The method of claim 12, wherein the at least one of the sub-concept and super-concept and the concept are part of a hierarchically organized group of concepts stored in a computerized database.

14. The method of claim 12, wherein the determining the super-concept is based on determining a semantic attachment associated with the sub-concept, and wherein the semantic attachment is configured to identify the super-concept.

15. The method of claim 12, further comprising:

determining, by the computing device and based on at least one of the sub-concept, the concept and the super-concept, external grammars stored in an external computerized database;

generating, by the computing device, a pointer to the external grammars; and

transmitting, by the computing device, the pointer to the application program of the user device.

16. A method comprising:

receiving, by a computing device, an audio input associated with a user utterance;

converting, by the computing device, the audio input into a string of words;

performing, by the computing device, natural language processing on the string of words comprising;

determining that the string of words comprises a stored prelude and postlude pair of a predefined ontology based on comparing each word from the string of words to the stored prelude and postlude pair of the predefined ontology, wherein the prelude is a phrase introducing identification of an instance of the concept and the postlude is a phrase that follows an identification of the instance of the concept;

determining, based on the stored prelude and postlude pair, a concept;

determining, based on the stored prelude and postlude pair, a rule configured to parse at least one word from the string of words as a sub-concept;

determining, based on the rule, the sub-concept;

determining a super-concept associated with the concept and the sub-concept;

determining a plurality of grammars by;

receiving a user selection of the concept;

generating a grammar skeleton with one or more rule groups, wherein each rule group includes one or more rules with one or more placeholders for a prelude and postlude pair, wherein each rule refers to at least one of the concept, the sub-concept, the super-concept, and the stored prelude and postlude pair; and

automatically populating at least one placeholder of the one or more placeholders of the one or more rules with the stored prelude and postlude pair of the predefined ontology based upon, at least in part, the user selection of the concept; and

transmitting, by the computing device, the plurality of grammars to an application program of a user device.

17. The method of claim 16, wherein the sub-concept, the concept and the super-concept are part of a hierarchically organized group of concepts stored in a computerized database.

18. The method of claim 17, wherein the determining the plurality of grammars is further based on determining one or more preludes and postludes for each of the hierarchically organized group of concepts.

19. The method of claim 16, wherein the determining the super-concept is based on determining a semantic attachment associated with the sub-concept, wherein the semantic attachment is configured to identify the super-concept.

20. The method of claim 16, further comprising:

determining, by the computing device and based on at least one of the sub-concept, the concept and the super-concept, external grammars stored in an external computerized database;

generating, by the computing device, a pointer to the external grammars; and

transmitting, by the computing device, the pointer to the application program of the user device.

21. The method of claim 16, wherein the determining the concept further comprises:

wherein the stored prelude and postlude pair are associated with a time order; and

identifying, by the computing device, a first word from the string of words, that is bound by the stored prelude and postlude pair.