Using multiple modality input to feedback context for natural language understanding

• US 10,332,514 B2
• Filed: 02/17/2017
• Issued: 06/25/2019
• Est. Priority Date: 08/29/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bodell, Michael, Bain, John, Chambers, Robert, Cross, Karen M., Kim, Michael, Gedge, Nick, Penn, Daniel Frederick, Patel, Kunal, Tecot, Edward Mark, Waltmunson, Jeremy C.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US15/436,437
• Publication Number:
  US 20170169824A1
• Time in Patent Office: 858 Days
• Field of Search: 704 9, 704 10, 704255, 704257, 704235
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Input context for a statistical dialog manager may be provided. Upon receiving a spoken query from a user, the query may be categorized according to at least one context clue. The spoken query may then be converted to text according to a statistical dialog manager associated with the category of the query and a response to the spoken query may be provided to the user.

Claims

1. A computer-implemented method for recognizing speech, the method comprising:

receiving a spoken query for a first input field;

identifying a category for the spoken query based on at least a user-inputted value associated with a second input field;

converting the spoken query to text according to a statistical dialog manager of a spoken dialog system of a computerized natural language service, the statistical dialog manager associated with the identified category and utilized to statistically weight terms belonging to the identified category; and

providing a response to the spoken query.

2. The method of claim 1, wherein the first input field and the second input field are displayed in a web page and the spoken query is associated with the web page.

3. The method of claim 2, wherein providing the response to the spoken query comprises:

performing a function associated with the web page; and

returning at least one result of performing the function.

4. The method of claim 3, further comprising providing the response as a spoken response via a text to speech conversion.

5. The method of claim 1, further comprising identifying a subcategory of the category based on at least one of the following:

a previous query with respect to the spoken query, a domain name for a webpage containing the first input field and the second input field.

6. The method of claim 1, further comprising providing the converted spoken text query to a web browser.

7. The method of claim 6, wherein the statistical dialog manager executes on a server communicatively coupled to the web browser via a network.

8. The method of claim 1, wherein the identification of the category for the spoken query is based on both the name of a first input field and the user-inputted value associated with the second input field.

9. The method of claim 1, wherein the statistical dialog manager executes on the user device.

10. A system for providing input context to a speech recognizer, the system comprising:

at least one processor; and

a memory operatively connected to the at least one processor, the memory storing instructions that when executed by the at least one processor perform a set of operations comprising;

receiving a spoken query for a first input field;

identifying a category for the spoken query based on a user-inputted value entered into a second input field;

converting the spoken query to text according to a statistical dialog manager of a spoken dialog system of a computerized natural language service, the statistical dialog manager associated with the category and utilized to statistically weight terms belonging to the category; and

providing a response to the spoken query.

11. The system of claim 10, wherein the first input field is displayed in a web page and the spoken query is associated with the web page.

12. The system of claim 11, wherein providing the response to the spoken query comprises:

performing a function associated with the web page; and

returning at least one result of performing the function.

13. The system of claim 10, wherein the operations further comprise identifying a subcategory of the category based on a domain name for a webpage containing the first input field and the second input field.

14. The system of claim 10, wherein the identification of the category for the spoken query is further based on a previous spoken query.

15. The system of claim 10, wherein identifying the category is further based on user profile data for a user providing the spoken query.

16. The system of claim 15, wherein the user profile data includes a location of a user.

17. A storage device, having computer-executable instructions that, when executed by at least one processor, perform a method receiving spoken input, the method comprising:

analyzing contextual data associated with a web page, wherein the plurality of contextual data comprises a plurality of inputs and responses associated with an interactive form of the web page;

based on the analysis, determining a plurality of categories comprising a category and one or more subcategories of the category;

building a plurality of statistical dialog managers, wherein a first statistical dialog manager in the plurality of statistical dialog managers is associated with the category and at least one additional statistical dialog manager in the plurality of statistical dialog managers is associated with each of the one or more subcategories;

receiving a spoken user input via a web browser application in communication with the web page;

categorizing the spoken user input according to at least one of a name of a first input field and a user-inputted value associated with a second input field;

determining a subcategory for the spoken user input;

converting the categorized spoken user input to text via the first statistical dialog manager associated with the category of the spoken user input and the at least one additional statistical dialog manager associated with the one or more subcategories, wherein the subcategory is based on additional page elements within the web page; and

applying the converted spoken user input to a web page element.

18. The storage device of claim 17, wherein determining the subcategory is based on at least one of a previous query and a domain name for the web page.

19. The storage device of claim 18, wherein categorizing the spoken user input is further based on user profile data for a user providing the spoken user input.

20. The storage device of claim 17, wherein applying the converted spoken user input includes adding the spoken user input into the first input field.