Additive context model for entity resolution

• US 9,697,475 B1
• Filed: 12/23/2013
• Issued: 07/04/2017
• Est. Priority Date: 12/12/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ringgaard, Michael, Pereira, Fernando Carlos das Neves, Subramanya, Amarnag
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Kim, David H
• Application Number:
  US14/138,606
• Time in Patent Office: 1,289 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Systems and methods are disclosed for using an additive context model for entity disambiguation. An example method may include receiving a span of text from a document and a phrase vector for the span. The phrase vector may have a quantity of features and represent a context for the span. The method also includes determining a quantity of candidate entities from a knowledge base that have been referred to by the span. For each of the quantity of candidate entities, the method may include determining a support score for the candidate entity for each feature in the phrase vector, combining the support scores additively, and computing a probability that the span resolves to the candidate entity given the context. The method may also include resolving the span to a candidate entity with a highest probability.

Claims

1. A computer system comprising:

at least one processor; and

memory storing;

a graph-structured knowledge base of entities connected by relationships, and

instructions that, when executed by the at least one processor, causes the computer system to perform operations comprising;

receiving a span of text from a document and a quantity of phrases from the document for the span, the phrases representing a context for the span,

determining that the span refers to a quantity of candidate entities from the knowledge base,

for each of the quantity of candidate entities;

providing the entity and the phrases as input to an additive context model, the context model having been trained to provide a support score for an entity-phrase pair,

receiving one or more support scores from the additive context model for the entity,

computing a first probability for the entity by adding the support scores together and dividing by the quantity of phrases, the first probability representing a likelihood that the context resolves to the entity,

receiving a second probability representing a likelihood that the span resolves to the entity regardless of context, and

computing a third probability for the entity by combining the first probability with the second probability, and

resolving the span to an entity that has a highest third probability.

2. The system of claim 1, wherein the computer system comprises a plurality of entity servers and the context model is partitioned across the entity servers based on entity.

3. The system of claim 2, wherein the instructions include instructions that, when executed by the at least one processor, cause the computer system to further perform operations comprising:

receiving a plurality of spans from the document, each span being associated with a respective context;

determining entity servers associated with candidate entities for each of the spans;

sending respective requests to the determined entity servers, each request causing a recipient entity server to provide input to its portion of the additive context model and compute the first probability based on support scores provided by the model; and

receiving the first probabilities from the entity servers.

4. The system of claim 1, wherein a phrase in the quantity of phrases is a noun phrase from the document.

5. The system of claim 1, wherein the document is the text of a query.

6. The system of claim 1, wherein the instructions include instructions that, when executed by the at least one processor, cause the computer system to further perform operations comprising:

training the additive context model using labeled data;

using the trained additive context model on unlabeled data, resulting in labeling the unlabeled data, wherein each label assigned by the additive context model has an associated confidence score; and

using data associated with labels having confidence scores that meet a threshold to re-train the context model.

7. The system of claim 6, wherein training the context model includes repeating using the trained context model on unlabeled data and re-training the context model until convergence.

8. The system of claim 1, wherein when the highest third probability does not meet a confidence threshold, the instructions include instructions that, when executed by the at least one processor, cause the system to resolve the span to an entity representing entities unknown to the knowledge base.

9. A method comprising:

receiving a span of text from a document;

receiving a phrase vector for the span, the phrase vector having a quantity of features and representing a context for the span;

determining, using at least one silicon-based hardware processor, a quantity of candidate entities from a knowledge base for an ambiguous entity mention included in the span;

for each of the quantity of candidate entities;

determining, using the at least one silicon-based hardware processor, a support score for the candidate entity for each feature in the phrase vector,

combining, using the at least one silicon-based hardware processor, the support scores additively, and

computing, using the combined support scores, a probability that the span resolves to the candidate entity given the context; and

resolving, using the at least one silicon-based hardware processor, the span to a candidate entity with a highest probability.

10. The method of claim 9, wherein when the highest probability does not meet a confidence threshold, the method includes resolving the span to an entity representing entities unknown to the knowledge base.

11. The method of claim 9, wherein the features of the phrase vector include noun phrases from the document and phrases coreferential with the span of text.

12. The method of claim 9, wherein the resolving is performed without a full coherency model.

13. The method of claim 9, wherein the probability is a first probability and the method further comprises:

for each candidate entity, combining the first probability with a second probability, the second probability representing a prior belief that the span refers to the candidate entity; and

resolving the span to the candidate entity with a highest combined probability.

14. The method of claim 9, wherein combining the support scores additively comprises:

computing a sum of the support scores for the candidate entity; and

dividing the sum by the quantity of features.

15. The method of claim 9, wherein the support score is stored in a context model that is partitioned across a plurality of entity servers, the partitioning being based on entity.

16. The method of claim 9, wherein the combining and calculating is performed according to the equation

17. A computer system comprising:

at least one hardware processor; and

memory storing instructions that, when executed by the at least one processor, cause the computer system to;

provide labeled data to an additive context model for training, the additive context model inferring a most likely entity for a mention given a context of the mention, the additive context model storing, for each feature, at least one support score-entity pair,

generate labels for unlabeled data using the trained model, the unlabeled data comprising entity mentions with respective phrase vectors, and where each label generated by the additive context model was based on additively combining support scores, and

re-train the model using the generated labels for the unlabeled data and the labeled data.

18. The computer system of claim 17, wherein the instructions further include instructions that, when executed by the at least one processor, cause the computer system to:

re-estimate the support scores of the context model after generating the labels;

determine whether the re-estimated support scores converge with the support scores of the model; and

perform the re-training and repeat generating the labels when the re-estimated support scores do not converge with the support scores of the model.

19. The computer system of claim 18, wherein re-estimating the support scores of the context model after iteration u is performed according to the equation of

20. The computer system of claim 17, wherein each generated label has an associated confidence score and the instructions include instructions that, when executed by the at least one processor, cause the computer system to use generated labels that meet a confidence threshold to re-train the model.