Systems and methods for building asset based natural language call routing application with limited resources

• US 8,370,127 B2
• Filed: 06/16/2006
• Issued: 02/05/2013
• Est. Priority Date: 06/16/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Jan, Ea-Ee, Li, Xiang, Lubensky, David, Stewart, Osamuyimen Thompson
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): ADESANYA, OLUJIMI A
• Application Number:
  US11/454,694
• Publication Number:
  US 20080010280A1
• Time in Patent Office: 2,426 Days
• Field of Search: 704/2, 704/5, 704/239, 704/257
• US Class Current: 704/9
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A method of processing limited natural language data to automatically develop an optimal feature set, bypassing the standard Wizard of OZ (WOZ) approach. Natural language understanding models process existing data from other domains, such as the Internet, for domain-specific adaptation through the use of an optimal feature set. When the optimal feature set is passed on to any engine, the optimal feature set produces robust models that can be used for natural language call routing.

Claims

1. A computer-implemented method for providing a statistical initial topic classification model for use in a natural language call routing application to map a natural language user statement to at least one of a plurality of topics in the natural language call routing application, the computer-implemented method comprising acts of:

accessing a voice user interface (VUI) specification that specifies a VUI for allowing users to interact with the natural language call routing application, the VUI specification comprising a plurality of expressions of possible user intentions composed by at least one VUI designer, the VUI specification further comprising, for each one of the plurality of expressions of possible user intentions, at least one system response or action defined by the at least one VUI designer as corresponding to the one of the plurality of expressions of possible user intentions;

analyzing the VUI specification to determine a plurality of topic descriptions, each one of the plurality of topic descriptions describing a meaning of a corresponding one of the plurality of topics; and

using, as training data, the plurality of topic descriptions determined by analyzing the VUI specification to build the statistical initial topic classification model, comprising;

identifying keywords in the plurality of topic descriptions used as training data to build the statistical initial topic classification model;

providing an optimal feature set comprising at least one combination of at least some of the keywords; and

using at least one computer to build the statistical initial topic classification model based at least partially on the optimal feature set.

2. The method of claim 1, wherein the act of identifying keywords in the training data comprises removing filler words from the training data, the filler words being provided in a filler word set.

3. The method of claim 2, further comprising an act of continuously updating the filler word set.

4. The method of claim 1, wherein, for at least one topic in the plurality of topics, a corresponding topic description comprises no more than two natural language sentences describing the at least one topic.

5. The method of claim 1, wherein accessing the VUI specification comprises accessing at least one call flow document defining how a call from a user is to be handled, and wherein analyzing the VUI specification comprises parsing the call flow document to extract the plurality of topic descriptions.

6. The method of claim 1, wherein each topic description describes a relationship between an intent of a user and a corresponding system response.

7. The method of claim 1, further comprising:

filtering text data using the statistical initial topic classification model to produce filter output, the text data from a source other than the natural language call routing application;

providing additional training data based at least in part on the filter output; and

refining the statistical initial topic classification model using the additional training data.

21. The method of claim 1, wherein accessing the VUI specification comprises obtaining the VUI specification from the at least one VUI designer.

8. At least one tangible non-transitory computer-readable storage medium having encoded thereon computer instructions that, when executed, perform a method for providing a statistical initial topic classification model for use in a natural language call routing application to map a natural language user statement to at least one of a plurality of topics in the natural language call routing application, the method comprising acts of:

accessing a voice user interface (VUI) specification that specifies a VUI for allowing users to interact with the natural language call routing application, the VUI specification comprising a plurality of expressions of possible user intentions composed by at least one VUI designer, the VUI specification further comprising, for each one of the plurality of expressions of possible user intentions, at least one system response or action defined by the at least one VUI designer as corresponding to the one of the plurality of expressions of possible user intentions;

analyzing the VUI specification to determine a plurality of topic descriptions, each one of the plurality of topic descriptions describing a meaning of a corresponding one of the plurality of topics; and

using, as training data, the plurality of topic descriptions determined by analyzing the VUI specification to build the statistical initial topic classification model, comprising;

identifying keywords in the plurality of topic descriptions used as training data to build the statistical initial topic classification model;

providing an optimal feature set comprising at least one combination of at least some of the keywords; and

using at least one computer to build the statistical initial topic classification model based at least partially on the optimal feature set.

9. The at least one tangible non-transitory computer-readable storage medium of claim 8, wherein the act of identifying keywords in the training data comprises removing filler words from the training data, the filler words being provided in a filler word set.

10. The at least one tangible non-transitory computer-readable storage medium of claim 9, wherein the method further comprises an act of continuously updating the filler word set.

11. The at least one tangible non-transitory computer-readable storage medium of claim 8, wherein, for at least one topic in the plurality of topics, a corresponding topic description comprises no more than two natural language sentences describing the at least one topic.

12. The at least one tangible non-transitory computer-readable storage medium of claim 8, wherein accessing the VUI specification comprises accessing at least one call flow document defining how a call from a user is to be handled, and wherein analyzing the VUI specification comprises parsing the at least one call flow document to extract the plurality of topic descriptions.

13. The at least one tangible non-transitory computer-readable storage medium of claim 8, wherein each topic description describes a relationship between an intent of a user and a corresponding system response.

14. The at least one tangible non-transitory computer-readable storage medium of claim 8, further comprising:

filtering text data using the statistical initial topic classification model to produce filter output, the text data from a source other than the natural language call routing application;

providing additional training data based at least in part on the filter output; and

refining the statistical initial topic classification model using the additional training data.

22. The at least one tangible non-transitory computer-readable storage medium of claim 8, wherein accessing the VUI specification comprises obtaining the VUI specification from the at least one VUI designer.

15. A system for providing a statistical initial topic classification model for use in a natural language call routing application to map a natural language user statement to at least one of a plurality of topics in the natural language call routing application, comprising:

at least one processor programmed to;

access a voice user interface (VUI) specification that specifies a VUI for allowing users to interact with the natural language call routing application, the VUI specification comprising a plurality of expressions of possible user intentions composed by at least one VUI designer, the VUI specification further comprising, for each one of the plurality of expressions of possible user intentions, at least one system response or action defined by the at least one VUI designer as corresponding to the one of the plurality of expressions of possible user intentions;

analyze the VUI specification to determine a plurality of topic descriptions, each one of the plurality of topic descriptions describing a meaning of a corresponding one of the plurality of topics; and

use, as training data, the plurality of topic descriptions determined by analyzing the VUI specification to build the statistical initial topic classification model, comprising;

identifying keywords in the plurality of topic descriptions used as training data to build the statistical initial topic classification model;

providing an optimal feature set comprising at least one combination of at least some of the keywords; and

building the statistical initial topic classification model based at least partially on the optimal feature set.

16. The system of claim 15, wherein the at least one processor is further programmed to:

identify keywords in the training data by removing filler words from the training data, the filler words being provided in a filler word set; and

continuously update the filler word set.

17. The system of claim 15, wherein, for at least one topic in the plurality of topics, a corresponding topic description comprises no more than two natural language sentences describing the at least one topic.

18. The system of claim 15, wherein accessing the VUI specification comprises accessing at least one call flow document defining how a call from a user is to be handled, and wherein analyzing the VUI specification comprises parsing the call flow document to extract the plurality of topic descriptions.

19. The system of claim 15, wherein each topic description describes a relationship between an intent of a user and a corresponding system response.

20. The system of claim 15, wherein the at least one processor is further programmed to:

filter text data using the statistical initial topic classification model to produce filter output, the text data from a source other than the natural language call routing application;

provide additional training data based at least in part on the filter output; and

refine the statistical initial topic classification model using the additional training data.

23. The system of claim 15, wherein accessing the VUI specification comprises obtaining the VUI specification from the at least one VUI designer.