Facilitating development of a spoken natural language interface

• US 9,330,659 B2
• Filed: 02/25/2013
• Issued: 05/03/2016
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ju, Yun-Cheng, Philipose, Matthai, Han, Seungyeop
• Primary Examiner(s): Hang, Vu B
• Application Number:
  US13/775,643
• Publication Number:
  US 20140244254A1
• Time in Patent Office: 1,163 Days
• Field of Search: 704/9, 704/10, 704/231, 704/243, 704/244, 704/246, 704/251, 704/257
• US Class Current: 1/1
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

A development system is described for facilitating the development of a spoken natural language (SNL) interface. The development system receives seed templates from a developer, each of which provides a command phrasing that can be used to invoke a function, when spoken by an end user. The development system then uses one or more development resources, such as a crowdsourcing system and a paraphrasing system, to provide additional templates. This yields an extended set of templates. A generation system then generates one or more models based on the extended set of templates. A user device may install the model(s) for use in interpreting commands spoken by an end user. When the user device recognizes a command, it may automatically invoke a function associated with that command. Overall, the development system provides an easy-to-use tool for producing an SNL interface.

Claims

1. A computer-readable storage device encoded with computer executable instructions, said instructions causing a computing device to execute a development framework, comprising:

a developer interface module configured to provide a development interface, the developer interface module comprising;

logic configured to receive a set of seed templates, each seed template identifying a command phrasing for use in invoking a particular function performed by a program, when spoken; and

logic configured to collect a set of added templates, each added template identifying another command phrasing for use in invoking the particular function,

the set of seed templates and the set of added templates forming an extended set of templates with all of the templates in the extended set providing a different command phrasing matched to a single user intent relative to the particular function;

a resource interface module configured to interact with one or more development resources to provide the set of added templates; and

a data store for storing the extended set of templates associated with the particular function,

the extended set of templates being for use in training one or more models for use on a user device, and said one or more models being applied to determine whether any phrase spoken by an end user is a member of the extended set so as to invoke the particular function.

2. The development framework of claim 1,

wherein the developer interface module further includes logic configured to receive description information, the description information providing a description of the particular function, and

wherein the data store also stores the description information.

3. The development framework of claim 2, wherein the description information includes one or more of:

a natural language description of the particular function; and

an identification of a handler function that is used to invoke the particular function.

4. The development framework of claim 1,

wherein, for one type of function, each command phrasing includes at least one slot for specifying variable slot information,

wherein the developer interface module further includes logic configured to identify at least one grammar, each grammar providing a mechanism for processing the variable slot information associated with a slot, and

wherein the data store also stores an identification of said at least one grammar.

5. The development framework of claim 1, wherein said at least one development resource is a crowdsourcing system, and wherein the crowdsourcing system is configured to:

present solicitation information to at least one crowdsourcing participant; and

receive one or more added templates from said at least one crowdsourcing participant, said at least one crowdsourcing participant providing said one or more added templates in response to the solicitation information.

6. The development framework of claim 1, wherein said at least one development resource is a paraphrasing system, and wherein the paraphrasing system is configured to:

receive at least one seed template; and

provide one or more added templates that are translations of said at least one seed template, said one or more added templates being expressed in a same language as said at least one seed template.

7. The development framework of claim 1, wherein the developer interface module further comprises logic configured to edit the extending set of templates.

8. The development framework of claim 1, further comprising a generation system comprising at least one training system configured to:

receive the extended set of templates; and

generate one or more models based on the extended set of templates.

9. The development framework of claim 8,

wherein the generation system is configured to combine the extended set of templates associated with the particular function with other extended sets of templates associated with other functions, to provide a master set of templates, and

wherein the generation system generates said one of more models based on the master set of templates.

10. The development framework of claim 8, wherein one training system comprises a statistical language model training system that is configured to provide a statistical language model for use by a speech recognition engine.

11. The development framework of claim 10, wherein the statistical language training system operates on the extended set of templates in a form in which slots are expressed as slot tokens.

12. The development framework of claim 8, wherein one training system comprises a vector space model training system that is configured to provide a vector space model for use by an intent determination engine.

13. The development framework of claim 12, wherein the vector space model training system determines a weight for each token in the extended set of templates by treating each template in the extended set as an expression of the single user intent.

14. The development framework of claim 8,

wherein the development framework is configured to update the extended set of templates by adding one or more user-provided templates provided by an end user, to provide an updated set of templates, and

wherein the generation system is configured to regenerate said one or more models based on the updated set of templates.

15. A method, implemented by one or more computer devices, comprising:

receiving description information, the description information providing a description of a particular function performed by a program resource;

receiving a set of seed templates, each seed template identifying a command phrasing for use in invoking the particular function, when the command phrasing is spoken;

providing a set of added templates using at least one development resource, each added template identifying another command phrasing for use in invoking the particular function,

the set of seed templates and the set of added templates comprising an extended set of templates with all of the templates in the extended set providing a different command phrasing matched to a single user intent relative to the particular function;

storing the description information and the extended set of templates in a data store,

said at least one development resource including at least a crowdsourcing system and a paraphrasing system; and

applying the extended set of templates to train one or more models for use in interpreting and responding to one or more spoken user commands when any phrase spoken by the user is a member of the extended set so as to invoke the particular function.

16. The method of claim 15, wherein each command phrasing includes at least one slot for specifying variable slot information, the method further comprising:

receiving an identification of at least one grammar, each grammar providing a mechanism for processing slot information associated with a slot; and

storing the identification of said at least one grammar.

17. A device, comprising:

a voice processing module for receiving a voice signal in response to a command spoken by a user, the voice processing module comprising;

a speech recognition engine configured to interpret the voice signal using a statistical language model, to provide a speech recognition result;

an intent determination engine configured to interpret the speech recognition result using an intent-matching model, to provide an intent determination result,

the intent determination result identifying a particular function to be performed by a program in response to the command spoken by the user;

logic configured to determine whether the speech recognition result and the intent determination result satisfy a prescribed test, to provide a determination result; and

a dispatch module configured to invoke a program resource that performs the particular function, upon an affirmative determination result,

the statistical language model and the intent-matching model being provided by a generation system based on a training corpus of templates, the templates comprising;

a set of seed templates with each seed template providing a different command phrasing matched to the intent determination result identifying the particular function; and

a set of added templates provided by one or more development systems, the set of added templates being related to the set of seed templates, such that each of the added templates provides a different command phrasing matched to the intent determination result identifying the particular function.

18. The device of claim 17, wherein the statistical language model is configured to generate a word sequence that represents a most likely interpretation of the voice signal.

19. The device of claim 18, wherein the statistical language model is further configured to:

identify at least one slot in the word sequence; and

apply a specified grammar to process slot information associated with said at least one slot.

20. The device of claim 18, wherein the intent matching model is configured to determine a template in the training corpus that most closely matches the word sequence identified by the statistical language model.