Information Extraction from Question And Answer Websites

• US 20160283491A1
• Filed: 03/25/2015
• Published: 09/29/2016
• Est. Priority Date: 03/25/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lu, Wei Lwun, Savenkov, Denis, Dalton, Jeffrey, Agichtein, Eugene, Subramanya, Amarnag, Gabrilovich, Evgeniy
• Granted Patent: US 9,875,296 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Methods, systems, and apparatus for obtaining a resource, identifying a first portion of text of the resource that is characterized as a question, and a second part of text of the resource that is characterized as an answer to the question, identifying an entity that is referenced by one or more terms of the text that is characterized as the question, a relationship type that is referenced by one or more other terms of the text that is characterized as the question, and an entity that is referenced by the text that is characterized as the answer to the question, and adjusting a score for a relationship of the relationship type for the entity that is referenced by the one or more terms of the text that is characterized as the question and the entity that is referenced by the text that is characterized as the answer to the question.

Claims

1. A computer-implemented method comprising:

obtaining a resource;

identifying (i) a first portion of text of the resource that is characterized as a question, and (ii) a second portion of text of the resource that is characterized as an answer to the question;

identifying (i) an entity that is referenced by one or more terms of the first portion of text that is characterized as the question, (ii) a relationship type that is referenced by one or more other terms of the first portion of the text that is characterized as the question, and (iii) an entity that is referenced by the second portion of text that is characterized as the answer to the question; and

adjusting a score associated with a relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

2. The computer-implemented method of claim 1, wherein the resource is a question and answer (Q&

A) website resource.

3. The computer-implemented method of claim 1, wherein identifying the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question comprises:

comparing the first portion of the text that is characterized as the question to one or more templates, each template being associated with a particular relationship type;

determining that the first portion of the text that is characterized as the question matches a particular template from among the one or more templates; and

identifying, as the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question, a relationship type that is associated with the particular template.

4. The computer-implemented method of claim 3, wherein each of the one or more templates is one of a surface-based template or a parser-based template.

5. The computer-implemented method of claim 1, comprising:

aggregating the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question;

comparing the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and the one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question; and

establishing, at an entity relationship model and based at least on the comparison, a relationship of the relationship type between the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

6. The computer-implemented method of claim 1, wherein identifying the entity that is referenced by the second portion of text that is characterized as the answer to the question comprises:

determining an entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question;

determining a target entity class based on (i) the entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question, and (ii) the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question; and

identifying, as the entity that is referenced by the second portion of text that is characterized as the answer to the question, an entity that is (i) referenced by the second portion of text that is characterized as the question, and (ii) matches the target entity class.

7. The computer-implemented method of claim 1, wherein identifying (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question comprises:

submitting the resource to a machine-learnt classifier that is configured to identify portions of text that are characterized as questions in question and answer (Q&

A) website resource and portions of text that are characterized as answers in question and answer (Q&

A) website resources; and

receiving, from the machine-learnt classifier, information that identifies (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

obtaining a resource;

identifying (i) a first portion of text of the resource that is characterized as a question, and (ii) a second portion of text of the resource that is characterized as an answer to the question;

identifying (i) an entity that is referenced by one or more terms of the first portion of text that is characterized as the question, (ii) a relationship type that is referenced by one or more other terms of the first portion of the text that is characterized as the question, and (iii) an entity that is referenced by the second portion of text that is characterized as the answer to the question; and

adjusting a score associated with a relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

9. The system of claim 8, wherein the resource is a question and answer (Q&

A) website resource.

10. The system of claim 8, wherein identifying the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question comprises:

comparing the first portion of the text that is characterized as the question to one or more templates, each template being associated with a particular relationship type;

determining that the first portion of the text that is characterized as the question matches a particular template from among the one or more templates; and

identifying, as the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question, a relationship type that is associated with the particular template.

11. The system of claim 8, wherein each of the one or more templates is one of a surface-based template or a parser-based template.

12. The system of claim 8, wherein the operations comprise:

aggregating the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question;

comparing the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and the one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question; and

establishing, at an entity relationship model and based at least on the comparison, a relationship of the relationship type between the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

13. The system of claim 8, wherein identifying the entity that is referenced by the second portion of text that is characterized as the answer to the question comprises:

determining an entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question;

determining a target entity class based on (i) the entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question, and (ii) the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question; and

identifying, as the entity that is referenced by the second portion of text that is characterized as the answer to the question, an entity that is (i) referenced by the second portion of text that is characterized as the question, and (ii) matches the target entity class.

14. The system of claim 8, wherein identifying (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question comprises:

submitting the resource to a machine-learnt classifier that is configured to identify portions of text that are characterized as questions in question and answer (Q&

A) website resource and portions of text that are characterized as answers in question and answer (Q&

A) website resources; and

receiving, from the machine-learnt classifier, information that identifies (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question.

15. A non-transitory computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining a resource;

identifying (i) a first portion of text of the resource that is characterized as a question, and (ii) a second portion of text of the resource that is characterized as an answer to the question;

identifying (i) an entity that is referenced by one or more terms of the first portion of text that is characterized as the question, (ii) a relationship type that is referenced by one or more other terms of the first portion of the text that is characterized as the question, and (iii) an entity that is referenced by the second portion of text that is characterized as the answer to the question; and

adjusting a score associated with a relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

16. The computer-readable device of claim 15, wherein the resource is a question and answer (Q&

A) website resource.

17. The computer-readable device of claim 15, wherein identifying the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question comprises:

comparing the first portion of the text that is characterized as the question to one or more templates, each template being associated with a particular relationship type;

determining that the first portion of the text that is characterized as the question matches a particular template from among the one or more templates; and

identifying, as the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question, a relationship type that is associated with the particular template.

18. The computer-readable device of claim 15, wherein the operations comprise:

aggregating the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question;

comparing the score associated with the relationship of the relationship type for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question and the one or more other scores that are each associated with the relationship of the relationship type for the entity that is referenced by the one or more other terms of the first portion of text that is characterized as the question and an entity that is referenced by a portion of text that is characterized as an answer to the question; and

establishing, at an entity relationship model and based at least on the comparison, a relationship of the relationship type between the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question and the entity that is referenced by the second portion of text that is characterized as the answer to the question.

19. The computer-readable device of claim 15, wherein identifying the entity that is referenced by the second portion of text that is characterized as the answer to the question comprises:

determining an entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question;

determining a target entity class based on (i) the entity class for the entity that is referenced by the one or more terms of the first portion of text that is characterized as the question, and (ii) the relationship type that is referenced by the one or more other terms of the first portion of the text that is characterized as the question; and

identifying, as the entity that is referenced by the second portion of text that is characterized as the answer to the question, an entity that is (i) referenced by the second portion of text that is characterized as the question, and (ii) matches the target entity class.

20. The computer-readable device of claim 15, wherein identifying (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question comprises:

submitting the resource to a machine-learnt classifier that is configured to identify portions of text that are characterized as questions in question and answer (Q&

A) website resource and portions of text that are characterized as answers in question and answer (Q&

A) website resources; and

receiving, from the machine-learnt classifier, information that identifies (i) the first portion of text of the resource that is characterized as the question, and (ii) the second portion of text of the resource that is characterized as the answer to the question.