Using multiple modality input to feedback context for natural language understanding

• US 9,576,573 B2
• Filed: 08/29/2011
• Issued: 02/21/2017
• Est. Priority Date: 08/29/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bodell, Michael, Bain, John, Chambers, Robert, Cross, Karen M., Kim, Michael, Gedge, Nick, Penn, Daniel Frederick, Patel, Kunal, Tecot, Edward Mark, Waltmunson, Jeremy C.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US13/219,891
• Publication Number:
  US 20130054238A1
• Time in Patent Office: 2,003 Days
• Field of Search: 704/9, 704/10, 704/255, 704/257, 704/235
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Input context for a statistical dialog manager may be provided. Upon receiving a spoken query from a user, the query may be categorized according to at least one context clue. The spoken query may then be converted to text according to a statistical dialog manager associated with the category of the query and a response to the spoken query may be provided to the user.

Claims

1. A computer-implemented method for recognizing speech, the method comprising:

receiving a spoken query during a web browsing session from a user having associated user profile data;

identifying a context of the web browsing session, wherein the context is based at least on a domain name of a web page viewed during the web browsing session;

identifying a category for the spoken query based on the context of the web browsing session and the user profile data;

identifying a subcategory of the category for the spoken query based on at least one or more page elements of the web page;

converting the spoken query to text according to a statistical dialog manager associated with the category and at least one additional statistical dialog manager associated with the subcategory, the statistical dialog manager and the at least one additional statistical dialog manager being utilized to statistically weight terms belonging to the category and the one or more subcategories more than terms in other categories and subcategories; and

providing a response to the spoken query.

2. The method of claim 1, wherein the spoken query is associated with the web page.

3. The method of claim 2, wherein providing the response to the spoken query comprises:

performing a function associated with the web page; and

returning at least one result of performing the function.

4. The method of claim 3, further comprising providing the response as a spoken response via a text to speech conversion.

5. The method of claim 1, wherein the context of the web browsing session additionally comprises at least one of the following:

an input field name, a previous query, and a user-inputted value associated with a second input field.

6. The method of claim 1, wherein the query is received via a web browser.

7. The method of claim 6, wherein the statistical dialog manager executes on a server communicatively coupled to the web browser via a network.

8. The method of claim 1, wherein the query is received via a user device.

9. The method of claim 8, wherein the statistical dialog manager executes on the user device.

10. The method of claim 1, wherein the user profile data comprises a location of a user.

11. A system for providing input context to a speech recognizer, the system comprising:

a memory storage; and

a processing unit coupled to the memory storage, wherein the processing unit is operable to;

receive a plurality of operation data,

identify a plurality of categories associated with the plurality of operation data, the plurality of categories comprising a category and one or more subcategories of the category,

build a plurality of statistical dialog managers, wherein a first statistical dialog manager in the plurality of statistical dialog managers is associated with the category and at least one additional statistical dialog manager in the plurality of statistical dialog managers is associated with each of the one or more subcategories,

receive a spoken user query during a web browsing session,

categorize the spoken user query based on user profile data and a context of the web browsing session, wherein the context of the web browsing session is based on at least a domain name of a web page viewed during the web browsing session,

identify a subcategory based on additional page elements in the web page; and

provide a response to the spoken user query via the first statistical dialog manager associated with the category of the spoken user query and the at least one additional statistical dialog manager associated with each of the one or more subcategories, the first statistical dialog manger and the at least one additional statistical dialog manager being utilized to statistically weight terms belonging to the category and the one or more subcategories more than terms in other categories and subcategories.

12. The system of claim 11, wherein the plurality of operation data comprises a plurality of messages exchanged between users.

13. The system of claim 11, wherein the plurality of operation data comprises a plurality of session logs.

14. The system of claim 13, wherein the plurality of session logs are associated with a web page comprising an interactive form.

15. The system of claim 11, wherein the plurality of operation data is associated with a single user.

16. The system of claim 11, wherein the plurality of operation data is aggregated from a plurality of users.

17. The system of claim 16, wherein the plurality of aggregated operation data is anonymized.

18. The system of claim 11, wherein the processing unit is operative to categorize the spoken user query according to at least one of the following:

an input field name, a previous query, and a user-inputted value associated with a second input field.

19. The system of claim 11, wherein the user profile data comprises a location of a user.

20. A computer-readable storage device which stores a set of instructions which when executed by a computing device, causes the computing device to perform a method for providing input context to a speech recognizer, the method comprising:

collecting a plurality of contextual data associated with a web page, wherein the plurality of contextual data comprises a plurality of inputs and responses associated with an interactive form of the web page;

determining whether at least one element of the plurality of contextual data comprises a private data element;

in response to determining that the at least one element of the plurality of contextual data comprises the private data element, discarding the private data element;

identifying a plurality of categories associated with the plurality of contextual data from which the private data element has been discarded, the plurality of categories comprising a category and one or more subcategories of the category;

building a plurality of statistical dialog managers, wherein a first statistical dialog manager in the plurality of statistical dialog managers is associated with the category and at least one additional statistical dialog manager in the plurality of statistical dialog managers is associated with each of the one or more subcategories;

receiving a spoken user query via a web browser application in communication with a web page;

receiving user profile data comprising at least a location of a user;

categorizing the spoken user query according to the user profile data and at least one context clue provided by the web browser, wherein the at least one context clue comprises a domain name and at least one of the following;

an input field name, the user profile data, a previous query, and a user-inputted value associated with a second input field;

converting the categorized spoken user query to text via the first statistical dialog manager associated with the category of the spoken user query and the at least one additional statistical dialog manager associated with the one or more subcategories, wherein the subcategory is based on additional page elements within a web page;

applying the converted spoken user query to a search engine; and

providing a response to the spoken user query via the first statistical dialog manager associated with the category of the spoken user query and the at least one additional statistical dialog manager associated with each of the one or more subcategories, via the web browser application, the first statistical dialog manger and the at least one additional statistical dialog manager being utilized to statistically weight terms belonging to the category and the one or more subcategories more than terms in other categories and subcategories.