Ontology-Crowd-Relevance Deep Response Generation

• US 20160342685A1
• Filed: 05/22/2015
• Published: 11/24/2016
• Est. Priority Date: 05/22/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Labutov, Igor I., Basu, Sumit, Vanderwende, Lucretia
• Granted Patent: US 10,061,848 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/367 Ontology

Abstract

Generating responses to input utilizing an ontology-crowd-relevance methodology is described. The techniques described herein access a plurality of data items and determine an ontology associated with the plurality of data items. The ontology includes one or more ontological elements. Furthermore, the techniques describe sending, to a plurality of devices, a request to generate response templates based on the one or more ontological elements and receiving, from the plurality of devices, the response templates directed to the one or more ontological elements.

Claims

1. A system comprising:

one or more processors;

memory; and

one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;

accessing a plurality of data items;

determining an ontology for the plurality of data items, the ontology including one or more ontological elements;

sending, to a plurality of devices, a request to generate response templates based on the one or more ontological elements; and

receiving, from the plurality of devices, the response templates directed to the one or more ontological elements.

2. The system as claim 1 recites, the operations further comprising:

storing the response templates in a database;

receiving a new data item associated with the one or more ontological elements; and

based at least in part on receiving the new data item, accessing individual response templates of the response templates stored in the database, the individual response templates mapped to the one or more ontological elements.

3. The system as claim 1 recites, wherein the one or more ontological elements comprise:

a label element that is semantically associated with an individual data item of the individual data items; and

a sub-label element that is semantically associated with a portion of the individual data item.

4. The system as claim 1 recites, wherein the response templates are generally applicable data items that are associated with the one or more ontological elements.

5. The system as claim 1 recites, the operations further comprising generating a response based at least in part on an individual response template of the response templates directed to the one or more ontological elements, the generating comprising:

accessing a data item from the plurality of data items, the data item associated with the one or more ontological elements;

accessing, from a database storing the response templates, the individual response template; and

supplementing the individual response template with corresponding information specific to the data item to generate the response.

6. The system as claim 5 recites, the operations further comprising:

sending the response to a second plurality of devices; and

receiving, from the second plurality of devices, feedback including a relevance score associated with the individual response template, the relevance score based at least in part on determining a relatedness between the response and the data item.

7. The system as claim 6 recites, the operations further comprising training a relevance model based at least in part on the relevance score, the data item, and the response.

8. A computer-implemented method comprising:

accessing a plurality of data items, the plurality of data items being associated with one or more labels and one or more sub-labels;

sending, to a plurality of devices, a request to generate a response template associated with a label of the one or more labels and a sub-label of the one or more sub-labels;

receiving, from the plurality of devices, a plurality of response templates associated with the label and the sub-label, wherein the plurality of response templates are generally applicable to data items associated with the label and sub-label; and

storing at least some response templates of the plurality of response templates in a database.

9. A computer-implemented method as claim 8 recites, wherein the database maps each of the plurality of response templates to the label and the sub-label.

10. A computer-implemented method as claim 8 recites, wherein:

individual data items of the plurality of data items are text documents;

the label corresponds to a category that is semantically associated with a substantial majority of a text document of the text documents; and

the sub-label corresponds to a section that is semantically associated with a portion of the text document, the portion being less than the substantial majority of the text document.

11. A computer-implemented method as claim 8 recites, further comprising training a ontology model based at least in part on the plurality of data items and a corresponding label and sub-label, the ontology model being trained to infer the label and the sub-label from new data items.

12. A computer-implemented method as claim 8 recites, wherein individual response templates of the plurality of response templates include one or more slots configured to be filled with information specific to a data item of the plurality of data items to generate a response associated with the data item.

13. A computer-implemented method as claim 8 recites, further comprising:

accessing a data item of the plurality of data items;

generating a response associated with the data item based at least in part on supplementing an individual response template of the plurality of response templates with corresponding information specific to the data item;

sending the response to one or more devices; and

receiving, from the one or more devices, feedback including a relevance score associated with the response, the relevance score based at least in part on a relatedness between the response and the data item.

14. A computer-implemented method as claim 13 recites, further comprising training a relevance model based at least in part on the relevance score, the response, and the data item, the training including:

determining a first feature vector based at least in part on a distance between the data item and the response;

accessing a second data item with a second relevance score that is above a predetermined threshold, the second relevance score based at least in part on the relatedness between the second data item and the response;

determining a second feature vector based at least in part on a distance between the data item and the second data item;

combining the first feature vector and the second feature vector to determine a combined feature vector; and

training the relevance model based at least in part on the combined feature vector, the response, and the data item.

15. A computer-implemented method as claim 8 recites, wherein:

the plurality of devices are associated with a plurality of users; and

receiving the plurality of response templates comprises receiving the plurality of response templates authored by the plurality of users.

16. A system comprising:

one or more processors;

memory; and

one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;

receiving a data item from a user device;

extracting ontological elements from the data item;

accessing two or more response templates associated with the ontological elements;

creating two or more responses based at least in part on supplementing the two or more response templates with information that is specific to the data item;

ranking individual responses of the two or more responses based at least in part on a relevance between the individual responses and the data item; and

causing at least one of the ranked individual responses to be presented via the user device.

17. A system as claim 16 recites, wherein the operations further comprise extracting the ontological elements from the data item based at least in part on applying an ontology model to the data item and inferring the ontological elements from the data item.

18. A system as claim 16 recites, wherein the ranking the individual responses is based at least in part on applying a relevance model to the two or more responses to determine individual responses of the two or more responses that are most relevant to the data item.

19. A system as claim 16 recites, wherein:

the data item is a corpus of text;

the ontological elements include a label and a sub-label associated with the corpus of the text;

the two or more response templates comprise question templates associated with the label and the sub-label; and

the two or more responses comprise questions that are answerable based at least in part on the corpus of text.

20. A system as claim 19 recites, wherein the questions are answerable using more than a single sentence of text in the corpus of text.