Context carryover in language understanding systems or methods

• US 9,747,279 B2
• Filed: 04/28/2015
• Issued: 08/29/2017
• Est. Priority Date: 04/17/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Boies, Daniel, Sarikaya, Ruhi, Feizollahi, Zhaleh, Xu, Puyang
• Primary Examiner(s): Hang, Vu B
• Application Number:
  US14/698,400
• Publication Number:
  US 20160307567A1
• Time in Patent Office: 854 Days
• Field of Search: 704 9, 704231, 704235, 704270, 704275, 704255, 704256, 704257
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Systems and methods for determining a user intent or goal for contextual language understanding by utilizing information from one or more previous user natural language inputs and one or more previous system generated responses to the user natural language inputs are provided. More specifically, the systems and methods utilize a common schema for determining features from the responses and natural language inputs and provide carryover tracking between responses and the natural language inputs. Accordingly, the systems and methods for contextual language understanding provide for a more accurate, a more reliable, and a more efficient context carryover and goal tracking system when compared to systems and methods that do not utilized the responses in determining the user goal/intent.

Claims

1. A system comprising:

at least one processor; and

a memory encoding computer executable instruction that, when executed by the at least one processor, cause the at least one processor to perform a method for contextual language understanding, the method comprising;

receiving a first natural language input based on input from a first user;

identifying a first set of entities in the first natural language input utilizing a schema;

receiving a first response to the first natural language input based on the first set of entities,

wherein the first response is generated by the system;

identifying a second set of entities in the first response utilizing the schema;

receiving a second natural language input;

identifying a third set of entities in the second natural language input utilizing the schema;

identifying a first set of carryover entities from any previous set of entities for carryover based on the third set of entities;

determining a first user intent based on the third set of entities and the first set of carryover entities; and

generating a second response based on the first user intent.

2. The system of claim 1, wherein the first natural language input is at least one of a spoken language input or a textual input.

3. The system of claim 1, wherein the system is a user device,

wherein the first response is a first action performed by the user device in order to answer the first natural language input, and

wherein the second response is a second action performed by the user device in order to answer the second natural language input,

wherein the first set of carryover entities does not contain any entities from the first set of entities.

4. The system of claim 1, wherein the system is a user device, and

wherein the second natural language input is received from additional input from the first user, and

wherein the second response is generated by the user device.

5. The system of claim 1, wherein the system is a user device, and

wherein the user device is at least one of:

a mobile telephone;

a smart phone;

a tablet;

a phablet;

a smart watch;

a wearable computer;

a personal computer;

a desktop computer;

a gaming system;

or

a laptop computer.

6. The system of claim 1, wherein the method further comprises:

sending the second response to a user device for performance of the second response,

wherein the system is a server in communication with the user device.

7. The system of claim 6, wherein the first natural language input and the second natural language input is received from first data from the user device, and

wherein the first response is received from second data from the user device.

8. The system of claim 6, the method further comprising:

determining a second user intent based on the first set of entities;

generating the first response based on the second user intent, and

wherein the first natural language input and the second natural language input is received from data from the user device, and

wherein the first response is received after the first response is generated by the server.

9. The system of claim 1, the method further comprising:

identifying a fourth set of entities in the second response utilizing the schema;

receiving a third natural language input,

identifying a fifth set of entities in the third natural language input utilizing the schema,

identifying a second set of carryover entities from any of the previous set of entities for, wherein the previous set of entities now include the third set of entities and the fourth set of entities, for carry over based on the fifth set of entities;

determining a second user intent based on the fifth set of entities and the second set of carryover entities; and

generating a third response based on the second user intent.

10. The system of claim 1, wherein the system is running a specific application, the specific application is at least one of:

a digital assistant application;

a voice recognition application;

an email application;

a social networking application;

a collaboration application;

an enterprise management application;

a messaging application;

a word processing application;

a spreadsheet application;

a database application;

a presentation application;

a contacts application;

a gaming application;

an e-commerce application;

an e-business application;

a transactional application;

an exchange application;

or

a calendaring application.

11. A system comprising:

a prediction system, the prediction system identifies entities in received data utilizing a common schema,

wherein the data include natural language inputs and responses, and

wherein the responses are generated by the system and the natural language inputs are based on input from a user;

a tracking system, the tracking system compares entities from a previously generated response and a previously received natural language input to the entities from the current turn and determines whether an entity from at least one of the previously generated response and the previously received natural language input should carry over to a current turn of a conversation based on this comparison to form a first set of carryover entities; and

an intent system, the intent system determines a user intent based on the first set of carryover entities and the entities from the current turn and generates a current response based on the user intent.

12. The system of claim 11, wherein the natural language inputs include at least one of a spoken language input or a textual input.

13. The system of claim 11, wherein the responses are actions performed by a user device in order to answer the natural language inputs.

14. The system of claim 11, wherein the system is a user device.

15. The system of claim 14, wherein the user device is at least one of:

a mobile telephone;

a smart phone;

a tablet;

a phablet;

a smart watch;

a wearable computer;

a personal computer;

a desktop computer;

a gaming system;

or

a laptop computer.

16. The system of claim 11, wherein the system is server in communication with a user device.

17. The system of claim 16, wherein the natural language inputs are received from first data from the user device, and

wherein the responses are received from both second data from the user device and from the server generation of responses.

18. The system of claim 11, wherein the prediction system utilizes a knowledge backend.

19. The system of claim 11, wherein each entity relates to a word, a term or a phrase from within the natural language inputs and the responses, and

wherein the first set of carryover entities does not include any entities from the natural language input and the previously generated response.

20. A method for contextual language understanding, the method comprising:

receiving a first natural language input as first data based on input from a first user from a user device;

determining, by the server, a first prediction utilizing a schema based on the first natural language input;

receiving a first response based on the first prediction as second data from the user device, wherein the first response is generated by one of the server or the user device;

determining, by the server, a second prediction utilizing the schema based on the first response;

receiving a second natural language input as third data from the user device;

determining, by the server, a third prediction utilizing the schema based on the second natural language input;

comparing the third prediction to any previous prediction to select carryover entities from at least the first and the second predictions;

combining the third prediction and the carryover entities to produce a final intent prediction of the second natural language input;

generating, by the server, a second response based on the final intent prediction of the second natural language input; and

sending, by the server, the second response to the user device for performance of the second response.