Selecting the best answer to a fact query from among a set of potential answers

• US 7,953,720 B1
• Filed: 03/31/2005
• Issued: 05/31/2011
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ritchford, Thomas W., Rohde, Douglas L. T.
• Primary Examiner(s): Hicks; Michael J
• Application Number:
  US11/097,690
• Time in Patent Office: 2,252 Days
• Field of Search: None
• US Class Current: 707/706
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/953 Querying, e.g. by the use of web search engines

Abstract

A method and system for selecting a best answer to a factual query. Possible answers to a factual query are identified. The possible answers are scored and the best scoring possible answers are compared to other possible answers to determine how well they are supported. The most supported answer is chosen to be presented to the user.

Claims

1. A computer-implemented method for selecting an answer to a factual query, comprising:

receiving a factual query;

searching a fact repository to identify a set of possible factual answers to the factual query, wherein the fact repository stores factual information extracted from a plurality of documents;

determining respective scores for at least a subset of the possible factual answers;

identifying a first answer of the subset of the set of possible factual answers to the factual query, the first answer having a best score;

identifying, as an answer that is contradictory to the first answer, a second answer of the subset, wherein the second answer is the best scoring answer that is contradictory to the first answer; and

generating a response including the first answer only if the score of the first answer exceeds a predefined threshold and exceeds the score of the second answer by at least a first predefined margin.

2. The method of claim 1, further comprising generating the response including the first answer only if the score of the first answer further satisfies a condition with respect to the score of a third answer of the subset.

3. The method of claim 2, wherein:

the third answer is one of the subset and is unrelated to the first answer; and

the condition is that the score of the first answer exceeds the score of the third answer by at least a second predefined margin.

4. The method of claim 1, wherein respective factual answers of the subset are associated with respective attributes and attribute-values in the fact repository;

and

identifying, as an answer that is contradictory to the first answer, a second answer of the subset includes identifying, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with respective attributes and attribute-values associated with the first answer and the second answer.

5. The method of claim 1, wherein identifying, as an answer that is contradictory to the first answer, a second answer of the subset includes identifying, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with a result of a pair-wise field comparison of the first answer and the second answer.

6. A computer-implemented method for selecting an answer to a factual query, comprising:

receiving a factual query;

identifying a set of possible factual answers to the factual query by searching a fact repository that stores factual information extracted from a plurality of documents;

determining for each of the possible factual answers a respective score;

identifying first answers of the set of possible factual answers to the factual query with best scores;

for each of the first answers;

identifying, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer; and

determining a supported score by mathematically combining the score of the respective first answer and the scores of the second answers;

identifying a third answer of the first answers with a best supported score; and

generating a response including the third answer only if the best supported score satisfies a first condition, wherein the first condition includes the best supported score being above a predefined threshold.

7. The method of claim 6, further comprising:

identifying a fourth answer of the possible answers as an answer that is contradictory to the third answer; and

determining a supported score for the fourth answer.

8. The method of claim 7, wherein the first condition further includes the best supported score exceeding the supported score of the fourth answer by at least a predefined margin.

9. The method of claim 7, wherein determining the supported score for the fourth answer includes:

identifying answers of the possible answers that support the fourth answer; and

determining the supported score for the fourth answer by mathematically combining the score of the fourth answer and the scores of the answers that support the fourth answer.

10. The method of claim 6, further comprising:

identifying a fifth answer of the possible answers that is unrelated to the third answer; and

determining a supported score for the fifth answer.

11. The method of claim 10, wherein the first condition further includes the best supported score exceeding the supported score of the fifth answer by at least a predefined margin.

12. The method of claim 10, wherein determining the supported score for the fifth answer includes:

identifying answers of the possible answers that support the fifth answer; and

determining the supported score for the fifth answer by mathematically combining the score of the fifth answer and the scores of the answers that support the fifth answer.

13. The method of claim 6, wherein respective factual answers of the set of possible factual answers are associated with respective attributes and attribute-values in the fact repository;

and

identifying, as answers that support the respective first answer, second answers of the set of possible factual answers includes identifying, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with respective attributes and attribute-values associated with the first answer and respective answers of the second answers.

14. The method of claim 6, wherein identifying, as answers that support the respective first answer, second answers of the set of possible factual answers includes identifying, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with results of pair-wise field comparisons of the first answer and respective answers of the second answers.

15. A system for selecting an answer to a factual query, comprising:

one or more processors;

memory;

a fact repository to store factual information extracted from a plurality of documents;

a query receiver to receive a factual query;

a possible answer identifier to search the fact repository to identify a set of possible factual answers to the factual query;

an answer scorer to determine respective scores for at least a subset of the set of possible factual answers to the factual query;

a best answer identifier to identify a first answer of the subset, the first answer having a best score, and to identify, as an answer that is contradictory to the first answer, a second answer of the subset, wherein the second answer is the best scoring answer that is contradictory to the first answer; and

a response generator to generate a response including the first answer only if the score of the first answer satisfies a first condition, wherein the first condition includes the score of the first answer exceeding a predefined threshold and exceeding the score of the second answer by at least a first predefined margin.

16. The system of claim 15, the response generator to generate the response including the first answer only if the best score further satisfies a second condition with respect to the score of a third answer of the possible answers.

17. The system of claim 16, wherein:

the third answer is one of the possible answers and is unrelated to the first answer; and

the second condition includes a condition that the best score exceeds the score of the third answer by at least a second predefined margin.

18. The system of claim 15, wherein respective factual answers of the subset are associated with respective attributes and attribute-values in the fact repository;

and

the best answer identifier is configured to identify, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with respective attributes and attribute-values associated with the first answer and the second answer.

19. The system of claim 15, wherein the best answer identifier is configured to identify, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with a result of a pair-wise field comparison of the first answer and the second answer.

20. A system for selecting an answer to a factual query, comprising:

one or more processors;

memory;

a query receiver to receive a factual query;

a possible answer identifier to identify a set of possible factual answers to the factual query by searching a fact repository that stores factual information extracted from a plurality of documents;

an answer scorer to determine for each of the possible factual answers a respective score;

a best answer identifier to identify first answers of the set of possible factual answers to the factual query with best scores;

an answer comparator to identify, for each of the first answers, as answers that support the respective first answer, second answers of the possible factual answers that support the respective first answer;

a supported scorer to determine for each of the first answers a supported score by mathematically combining the score of the respective first answer and the scores of the second answers;

an answer selector to identify a third answer of the first answers with a best supported score; and

a response generator to generate a response including the third answer if the best supported score satisfies a first condition, wherein the first condition includes the best supported score being above a predefined threshold.

21. The system of claim 20, wherein:

the answer comparator further includes instructions to identify a fourth answer of the possible answers as an answer that is contradictory to the third answer; and

the supported scorer further includes instructions to determine a supported score for the fourth answer.

22. The system of claim 21, wherein the first condition further includes the best supported score exceeding the supported score of the fourth answer by at least a predefined margin.

23. The system of claim 20, wherein:

the answer comparator further includes instructions to identify a fifth answer of the possible answers that is unrelated to the third answer; and

the supported scorer further includes instructions to determine a supported score for the fifth answer.

24. The system of claim 23, wherein the first condition includes the best supported score exceeding the supported score of the fifth answer by at least a predefined margin.

25. The system of claim 20, wherein respective factual answers of the subset are associated with respective attributes and attribute-values in the fact repository;

and

the best answer identifier is configured to identify, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with respective attributes and attribute-values associated with the first answer and respective answers of the second answers.

26. The system of claim 20, wherein the best answer identifier is configured to identify, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with results of pair-wise field comparisons of the first answer and respective answers of the second answers.

27. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of a computer system, the one or more programs comprising instructions for:

receiving a factual query;

searching a fact repository to identify a set of possible factual answers to the factual query, wherein the fact repository stores factual information extracted from a plurality of documents;

determining respective scores for at least a subset of the set of possible factual answers;

identifying a first answer of the subset of possible factual answers, the first answer having a best score;

identifying, as an answer that is contradictory to the first answer, a second answer of the subset, wherein the second answer is the best scoring answer that is contradictory to the first answer; and

generating a response including the first answer only if the score of the first answer exceeds a predefined threshold and exceeds the score of the second answer by at least a first predefined margin.

28. The computer readable storage medium of claim 27, wherein the one or more programs include instructions for generating the response including the first answer only if the score of the first answer further satisfies a condition with respect to the score of a third answer of the subset.

29. The computer readable storage medium of claim 28, wherein:

the third answer is one of the subset and is unrelated to the first answer; and

the condition is that the score of the first answer exceeds the score of the third answer by at least a second predefined margin.

30. The computer readable storage medium of claim 27, wherein respective factual answers of the subset are associated with respective attributes and attribute-values in the fact repository;

and

the instructions for identifying, as an answer that is contradictory to the first answer, a second answer of the subset include instructions for identifying, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with respective attributes and attribute-values associated with the first answer and the second answer.

31. The computer readable storage medium of claim 27, wherein the instructions for identifying, as an answer that is contradictory to the first answer, a second answer of the subset include instructions for identifying, as an answer that is contradictory to the first answer, the second answer of the subset in accordance with a result of a pair-wise field comparison of the first answer and the second answer.

32. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of a computer system, the one or more programs comprising instructions for:

receiving a factual query;

identifying a set of possible factual answers to the factual query by searching a fact repository that stores factual information extracted from a plurality of documents;

determining for each of the possible factual answers a respective score;

identifying first answers of the set of possible factual answers to the factual query with best scores;

for each of the first answers;

identifying, as answers that support the respective first answer, second answers of the possible factual answers that support the respective first answer; and

determining a supported score by mathematically combining the score of the respective first answer and the scores of the second answers;

identifying a third answer of the first answers with a best supported score; and

generating a response including the third answer only if the best supported score exceeds a predefined threshold.

33. The computer readable storage medium of claim 32, wherein the one or more programs include instructions for:

identifying a fourth answer of the possible answers as an answer that is contradictory to the third answer; and

determining a supported score for the fourth answer.

34. The computer readable storage medium of claim 33, wherein the instructions for generating the response including the third answer include instructions for generating the response including the third answer only if the best supported score exceeds the supported score of the fourth answer by at least a predefined margin.

35. The computer readable storage medium of claim 32, wherein respective factual answers of the subset are associated with respective attributes and attribute-values in the fact repository;

and

the instructions for identifying, as answers that support the respective first answer, second answers of the set of possible factual answers include instructions for identifying, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with respective attributes and attribute-values associated with the first answer and respective answers of the second answers.

36. The computer readable storage medium of claim 32, wherein the instructions for identifying, as answers that support the respective first answer, second answers of the set of possible factual answers include instructions for identifying, as answers that support the respective first answer, second answers of the set of possible factual answers that are distinct from the respective first answer and support the respective first answer in accordance with results of pair-wise field comparisons of the first answer and respective answers of the second answers.