System and method of spoken language understanding in human computer dialogs

• US 8,190,436 B2
• Filed: 12/05/2002
• Issued: 05/29/2012
• Est. Priority Date: 12/07/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Rahim, Mazin G, Bangalore, Srinivas, Gupta, Narendra K.
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US10/310,596
• Publication Number:
  US 20030130841A1
• Time in Patent Office: 3,463 Days
• Field of Search: None
• US Class Current: 704/270.1
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A system and method are disclosed that improve automatic speech recognition in a spoken dialog system. The method comprises partitioning speech recognizer output into self-contained clauses, identifying a dialog act in each of the self-contained clauses, qualifying dialog acts by identifying a current domain object and/or a current domain action, and determining whether further qualification is possible for the current domain object and/or current domain action. If further qualification is possible, then the method comprises identifying another domain action and/or another domain object associated with the current domain object and/or current domain action, reassigning the another domain action and/or another domain object as the current domain action and/or current domain object and then recursively qualifying the new current domain action and/or current object. This process continues until nothing is left to qualify.

Claims

1. A method comprising:

partitioning, via a processor, a recognizer output associated with a spoken utterance into independent clauses;

identifying a dialog act from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

identifying at least one of an object and an action within each of the independent clauses, wherein the object is a domain-dependent object and the action is a domain-dependent action;

qualifying the dialog act with a corresponding one of the object and the action within a respective independent clause to generate a semantic representation, wherein qualifying the dialog act comprises recursively extending the semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in each of the independent clauses until nothing is left to qualify; and

transmitting the semantic representation to a dialog manager for use in a dialog management application.

2. The method of claim 1, wherein the dialog manager is configured to use the semantic representation to determine a response to a user input.

3. A method comprising:

applying, via a processor, a first domain-independent module to partition a speech recognizer output associated with a spoken utterance into independent clauses;

applying a second domain-independent module to identify dialog acts from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

applying a first domain-dependent module to identify at least one of an object and an action within the independent clauses, wherein the object is a domain-dependent object and the action is a domain-dependent action; and

applying a second domain-dependent module to qualify the dialog acts with a corresponding one of the object and the action within a respective independent clause, wherein qualifying the dialog act comprises recursively extending a semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in the independent clauses until nothing is left to qualify.

4. The method of claim 3, wherein the semantic representation is used by a dialog manager in a spoken dialog system to determine a response to a user input.

5. A method comprising:

partitioning, via a processor, a speech recognizer output into independent clauses;

identifying a dialog act from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

identifying at least one of a current domain object and a current domain action within each of the independent clauses, wherein the current domain object is a domain-dependent object and the current domain action is a domain-dependent action;

qualifying the dialog act with a corresponding one of the current domain object and the current domain action in a respective independent clause, wherein qualifying the dialog act comprises recursively extending a semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in each of the independent clauses until nothing is left to qualify;

determining whether further qualification is possible for one of the current domain object and the current domain action; and

if further qualification is possible;

identifying at least one of another domain action and another domain object associated with at least one of the current domain object and the current domain action;

reassigning at least one of the another domain action and the another domain object as at least one of the current domain action and the current domain object; and

returning to qualifying the dialog act.

6. The method of claim 5, wherein the semantic representation is used by a dialog manager in a spoken dialog system to determine a response to a user input.

7. A spoken dialog system comprising:

a classifier that identifies, via a processor, independent clauses within the data from the speech recognizer;

a dialog act identifier that identifies a dialog act from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act, and wherein the object is a domain-dependent object and the action is a domain-dependent action; and

a dialog act qualifier that identifies domain-dependent actions and domain-dependent objects in each of the independent clauses, the dialog act qualifier qualifying the dialog act with the domain-dependent actions and domain-dependent objects, wherein qualifying the dialog act comprises recursively extending a semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in each of the independent clauses until nothing is left to qualify; and

an output module configured to output the semantic representation.

8. The system of claim 7, wherein the semantic representation is used by a dialog manager in a spoken dialog system to determine a response to a user input.

9. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform a method comprising:

partitioning a recognizer output associated with a spoken utterance into independent clauses;

identifying a dialog act from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

identifying at least one of an object and an action within each of the independent clauses, wherein the object is a domain-dependent object and the action is a domain-dependent action;

qualifying the dialog act with a corresponding one of the object and the action within a respective independent clause to generate a semantic representation, wherein qualifying the dialog act comprises recursively extending the semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in each of the independent clauses until nothing is left to qualify; and

transmitting the semantic representation to a dialog manager for use in dialog management.

10. The non-transitory computer-readable storage medium of claim 9, wherein the dialog manager is configured to use the semantic representation in a spoken dialog system to determine a response to a user input.

11. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform a method comprising:

applying a first domain-independent module to partition a speech recognizer output associated with a spoken utterance into independent clauses;

applying a second domain-independent module to identify dialog acts from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

applying a first domain-dependent module to identify at least one of an object and an action within the independent clauses, wherein the object is a domain-dependent object and the action is a domain-dependent action; and

applying a second domain-dependent module to qualify the dialog acts with a corresponding one of the object and the action within a respective independent clause, wherein applying the second domain-dependent module further qualifies the dialog acts by recursively extending a semantic representation by qualifying at least one of the domain-dependent object and the domain-dependent action in each of the independent clauses until nothing is left to qualify.

12. The non-transitory computer-readable storage medium of claim 11, wherein the semantic representation is used by a dialog manager in a spoken dialog system to determine a response to a user input.

13. A system comprising:

a processor; and

a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform a method comprising;

partitioning a speech recognizer output into independent clauses;

identifying a dialog act from a hierarchical dialog act taxonomy for each of the independent clauses, wherein the hierarchical dialog act taxonomy comprises a tree structure, wherein the dialog act is a domain-independent dialog act;

identifying at least one of an object and an action within each of the independent clauses, wherein the object is a domain-dependent object and the action is a domain-dependent action; and

qualifying the dialog act with a corresponding one of the object and the action within a respective independent clause to generate a semantic representation, wherein qualifying the dialog act comprises recursively extending the semantic representation by qualifying corresponding ones of the object and the action in each of the independent clauses until nothing is left to qualify.

14. The system of claim 13, wherein the semantic representation is used by a dialog manager in a spoken dialog system to determine a response to a user input.