PROVIDING ANSWERS TO QUESTIONS INCLUDING ASSEMBLING ANSWERS FROM MULTIPLE DOCUMENT SEGMENTS

• US 20120084293A1
• Filed: 09/24/2011
• Published: 04/05/2012
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Eric W., Chu-Carroll, Jennifer, Ferrucci, David A., Murdock, James W. IV
• Granted Patent: US 9,600,601 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06N 5/027 Frames
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/00 : ICT specially adapted for the handling or processing of medical references
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

A method, system and computer program product for generating answers to questions. In one embodiment, the method comprises receiving an input query, identifying a plurality of candidate answers to the query; and for at least one of these candidate answers, identifying at least one proof of the answer. This proof includes a series of premises, and a multitude of documents are identified that include references to the premises. A set of these documents is selected that include references to all of the premises. This set of documents is used to generate one or more scores for the one of the candidate answers. A defined procedure is applied to the candidate answers to determine a ranking for the answers, and this includes using the one or more scores for the at least one of the candidate answers in the defined procedure to determine the ranking for this one candidate answer.

Claims

1. A method of generating answers to questions, comprising:

receiving an input query;

conducting a search in a data source to identify a candidate answer to the input query;

identifying a proof of the candidate answer, said proof including a series of premises for establishing said proof;

identifying documents that include references to said premises, each of said documents including one or more references to one or more of said premises;

selecting a set of the documents that, as a set, include references to all of the premises; and

using the selected set of documents to generate a score for the candidate answer.

2. The method according to claim 1, wherein the selecting a set of documents includes using a defined algorithm to identify a quasi-minimal set of documents.

3. The method according to claim 1, wherein the selecting a set of documents includes using a defined algorithm to identify a set of documents having the minimum number of documents needed to include references to all of the premises.

4. The method according to claim 1, wherein:

each of the multitude of documents include references to a given number of the premises; and

the selecting a set of documents includes selecting the documents for the set in order of the number of the premises referenced in each of the documents.

5. The method according to claim 1, wherein the selecting a set of documents includes selecting a plurality of the documents for said set based on the number of the premises referenced in each of the documents.

6. A system for generating answers to questions, comprising:

a computer device comprising at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory; and

at least one processor coupled to the memory and operative for;

receiving an input query;

conducting a search in a data source to identify a candidate answer to the input query;

identifying a proof of the candidate answer, said proof including a series of premises for establishing said proof;

identifying documents that include references to said premises, each of said documents including one or more references to one or more of said premises;

selecting a set of the documents that, as a set, include references to all of the premises; and

using the selected set of documents to generate a score for the candidate answer.

7. The system according to claim 6, wherein the selecting a set of documents includes using a defined algorithm to the identified sets of documents to identify a quasi-minimal set of documents.

8. The system according to claim 6, wherein the selecting a set of documents includes using a defined algorithm to identify a set of documents having the minimum number of documents needed to include references to all of the premises.

9. The system according to claim 6, wherein:

each of the multitude of documents include references to a given number of the premises; and

the selecting a set of documents includes selecting the documents for the set in order of the number of the premises referenced in each of the documents.

10. The system according to claim 6, wherein the selecting a set of documents includes selecting a plurality of the documents for said set based on the number of the premises referenced in each of the documents.

11. An article of manufacture comprising:

at least one tangible computer readable medium having computer readable program code logic to generate answers to questions, the computer readable program code logic, when executing, performing the following;

receiving an input query;

conducting a search in a data source to identify a candidate answer to the input query;

identifying a proof of the candidate answer, said proof including a series of premises for establishing said proof;

identifying documents that include references to said premises, each of said documents including one or more references to one or more of said premises;

selecting a set of the documents that, as a set, include references to all of the premises; and

using the selected set of documents to generate a score for the candidate answer.

12. The article of manufacture according to claim 11, wherein the selecting a set of documents includes using a defined algorithm to identify a quasi-minimal set of documents.

13. The article of manufacture according to claim 11, wherein the selecting a set of documents includes using a defined algorithm to identify a set of documents having the minimum number of documents needed to include references to all of the premises.

14. The article of manufacture according to claim 11, wherein:

each of the multitude of documents include references to a given number of the premises; and

the selecting a set of documents includes selecting the documents for the set in order of the number of the premises referenced in each of the documents.

15. The article of manufacture according to claim 11, wherein the selecting a set of documents includes selecting a plurality of the documents for said set based on the number of the premises referenced in each of the documents.

16. A method of indexing documents, comprising:

for each of the documents, annotating spans of text in said each document that refer to entities with entity types to form entity annotations, and annotating spans of text in said each document that refer to facts with fact types to form relation annotations;

for each of the annotated spans of text that refers to one of the facts, linking said one of the facts to said each annotation; and

recording in an index the entities, the facts, the annotations that refer to said entities, and the annotations that refer to said facts.

17. The method according to claim 16, further comprising:

determining which of the entity annotations refer to the same fact; and

determining which of the relation annotations refer to the same fact.

18. The method according to claim 16, further comprising using the index to identify a candidate answer for an input query.

19. The method according to claim 18, further comprising using the fact and entities in the index to identify a proof for the candidate answer.

20. The method according to claim 19, further comprising:

identifying premises of the proof; and

using the index to identify one or more of the documents that refer to each of said premises.

21. A system for retrieving documents, comprising:

a computer device comprising at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory; and

at least one processor coupled to the memory and operative for;

searching through a given index to identify a candidate answer for an input query from a user;

identifying at least one proof for the candidate answer, said proof including a series of premises;

searching through the index to identify a multitude of documents that include references to the premises of said one proof;

selecting a set of the documents that, as a set, include references to all of the premises; and

returning the set of documents to the user.

22. The system according to claim 21, wherein the selecting a set of documents includes using a defined algorithm to identify a quasi-minimal set of documents.

23. The method according to claim 21, wherein the selecting a set of documents includes using a defined algorithm to identify a set of documents having the minimum number of documents needed to include references to all of the premises.

24. The method according to claim 21, wherein:

each of the multitude of documents include references to a given number of the premises; and

the selecting a set of documents includes selecting the documents for the set in order of the number of the premises referenced in each of the documents.

25. The system according to claim 24, wherein the forming said index includes:

for each of the documents, annotating spans of text in said each document that refer to entities with entity types to form entity annotations, and annotating spans of text in said each document that refer to facts with fact types to form relation annotations;

for each of the annotated spans of text that refers to one of the facts, linking said one of the facts to said each annotation; and

recording in the index the entities, the facts, the annotations that refer to said entities, and the annotations that refer to said facts.