Techniques for inferring the unknown intents of linguistic items

• US 9,870,356 B2
• Filed: 02/13/2014
• Issued: 01/16/2018
• Est. Priority Date: 02/13/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hakkani-Tr, Dilek, Celikyilmaz, Fethiye Asli, Heck, Larry P., Tur, Gokhan, Ji, Yangfeng
• Primary Examiner(s): Gonzales, Vincent
• Application Number:
  US14/180,335
• Publication Number:
  US 20150227845A1
• Time in Patent Office: 1,433 Days
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 5/00 Computing arrangements using knowledge-based models

Abstract

Functionality is described herein for determining the intents of linguistic items (such as queries), to produce intent output information. For some linguistic items, the functionality deterministically assigns intents to the linguistic items based on known intent labels, which, in turn, may be obtained or derived from a knowledge graph or other type of knowledge resource. For other linguistic items, the functionality infers the intents of the linguistic items based on selection log data (such as click log data provided by a search system). In some instances, the intent output information may reveal new intents that are not represented by the known intent labels. In one implementation, the functionality can use the intent output information to train a language understanding model.

Claims

1. A method implemented by one or more computing devices, the method comprising:

receiving input data, the input data comprising linguistic items including;

a first set of linguistic items with known intent labels, the known intent labels representing known relations between entities provided by a knowledge resource; and

a second set of linguistic items without known intent labels provided by the knowledge resource;

determining intents for the linguistic items in the input data to produce intent output information, the determining comprising;

when a respective linguistic item corresponds to a member of the first set, deterministically assigning a respective known intent to the respective linguistic item based at least on a respective known intent label associated with the respective linguistic item; and

when the respective linguistic item corresponds to a member of the second set, inferring the intent associated with the respective linguistic item based at least on selection log data; and

storing the intent output information in a data store,

the determining including discovering a new intent for an individual linguistic item of the second set that identifies an individual entity represented in the knowledge resource, the new intent identifying a new relation for the individual entity that is not included in the known relations provided by the knowledge resource,

the selection log data reflecting actions of users associated with using various linguistic items with the known intents and with the new intent.

2. The method of claim 1, wherein the linguistic items in the input data correspond to respective queries submitted to a search system, and the selection log data is made up of click selections made by users in response to submitting the queries.

3. The method of claim 1, wherein the individual entity is a person.

4. The method of claim 3, wherein the knowledge resource is a knowledge graph.

5. The method of claim 1, wherein the known intent labels correspond to informational relations between entities specified in the knowledge resource.

6. The method of claim 1, wherein the new intent discovered for the individual linguistic item corresponds to a navigational intent in which a request is made to navigate in a particular manner with respect to the individual entity identified by the individual linguistic item.

7. The method of claim 1, wherein the new intent discovered for the individual linguistic item corresponds to a transactional intent in which a request is made to perform some action on the individual entity identified by the individual linguistic item.

8. The method of claim 1, further comprising, prior to performing said determining, identifying, for at least some of the linguistic items in the input data, entity types associated with entities specified in the input data.

9. The method of claim 1, further comprising, prior to performing said determining, identifying, for at least some of the linguistic items in the input data, components of click selections specified in the input data.

10. The method of claim 1, wherein said inferring uses a Bayesian hierarchical graphical model, and wherein the model represents user actions based on a process which involves:

drawing an intent based on a distribution of intents, to provide a specified intent;

generating a linguistic item having one or more words, each word being drawn according to a distribution of words associated with the specified intent; and

generating a click selection according to a distribution of click selections associated with the specified intent.

11. The method of claim 1, wherein said inferring uses a Bayesian hierarchical graphical model, and where the model represents user actions based on a process which involves:

drawing an intent based on a distribution of intents, to provide a specified intent;

generating one or more entity types, each entity type being drawn according to a distribution of entity types associated with the specified intent;

generating one or more context words, each context word being drawn according to a distribution of context words associated with the specified intent; and

generating one or more click components, each click component being drawn according to a distribution of click components associated with the specified intent.

12. The method of claim 1, wherein said inferring employs a Markov chain Monte Carlo technique to produce the intent output information.

13. The method of claim 1, wherein said inferring employs a variational technique to produce the intent output information.

14. The method of claim 1, further comprising training a language understanding model based at least on the intent output information.

15. A computer system comprising:

a processing device; and

a computer readable storage medium storing computer readable instructions which, when executed by the processing device, cause the processing device to;

receive input data, the input data comprising a plurality of linguistic items, individual linguistic items corresponding to one or more words submitted by a user and being associated with a click selection made by the user in response to submitting the one or more words; and

use a generative model to determine intents for at least some of the linguistic items in the input data, the intents including at least one previously-known intent and at least one new intent discovered by the generative model,

the generative model representing user actions based on a process which involves drawing a specified intent for a specified linguistic item based at least on a distribution of intents, the distribution representing probabilities associated with the at least one previously-known intent and the at least one new intent discovered by the generative model.

16. The computer system of claim 15,

wherein the input data includes a first set of linguistic items with known intent labels, and a second set of linguistic items without known intent labels, and

wherein the computer readable instructions, when executed by the processing device, cause the processing device to determine the intents by:

when a respective linguistic item corresponds to a member of the first set, deterministically assigning an intent to the respective linguistic item based at least on a respective known intent label associated with the respective linguistic item; and

when the respective linguistic item corresponds to a member of the second set, inferring the intent associated with the respective linguistic item using the generative model in conjunction with click selection data.

17. The computer system of claim 15, wherein the computer readable instructions, when executed by the processing device, cause the processing device to:

generate the specified linguistic item having one or more words, each word being drawn according to a distribution of words associated with the specified intent; and

generate a click selection according to a distribution of click selections associated with the specified intent.

18. The computer system of claim 15, wherein the computer readable instructions, when executed by the processing device, cause the processing device to:

generate one or more entity types, each entity type being drawn according to a distribution of entity types associated with the specified intent;

generate one or more context words, each context word being drawn according to a distribution of context words associated with the specified intent; and

generate one or more click components, each click component being drawn according to a distribution of click components associated with the specified intent.

19. A computer system comprising:

a processing device; and

a computer readable storage medium storing computer readable instructions which, when executed by the processing device, cause the processing device to;

receive input data, the input data comprising a first set of linguistic items with known intent labels representing known intents and a second set of linguistic items without known intent labels, the known intent labels being obtained or derived from a knowledge resource that represents the known intents as known relations between entities;

determine intents for respective linguistic items in the input data to produce intent output information, the intents being determined based at least on the known intent labels and selection log data, the selection loci data reflecting selections made by users in response to submitting linguistic items with the known intents and with new intents that are not represented by the knowledge resource, the new intents representing new relations between individual entities that are not provided by the known relations of the knowledge resource;

store the intent output information in a data store, the intent output information including at least one of the new intents that is discovered using the selection log data; and

produce a language understanding model based at least on the intent output information.

20. The computer system of claim 19, wherein the computer readable instructions, when executed by the processing device, cause the processing device to:

produce the language understanding model based at least on the intent output information in combination with labels added by at least one human annotator.