System and method for providing answers to questions

• US 8,275,803 B2
• Filed: 05/14/2008
• Issued: 09/25/2012
• Est. Priority Date: 05/14/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Eric W., Ferrucci, David, Lally, Adam, Zadrozny, Wlodek W.
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US12/152,411
• Publication Number:
  US 20090287678A1
• Time in Patent Office: 1,595 Days
• Field of Search: None
• US Class Current: 707/802
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

A system, method and computer program product for providing answers to questions based on any corpus of data. The method facilitates generating a number of candidate passages from the corpus that answer an input query, and finds the correct resulting answer by collecting supporting evidence from the multiple passages. By analyzing all retrieved passages and that passage'"'"'s metadata in parallel, there is generated an output plurality of data structures including candidate answers based upon the analyzing. Then, by each of a plurality of parallel operating modules, supporting passage retrieval operations are performed upon the set of candidate answers, and for each candidate answer, the data corpus is traversed to find those passages having candidate answer in addition to query terms. All candidate answers are automatically scored causing the supporting passages by a plurality of scoring modules, each producing a module score. The modules scores are processed to determine one or more query answers; and, a query response is generated for delivery to a user based on the one or more query answers.

Claims

1. A computer-implemented method of generating answers to questions based on a corpus of data, said method comprising:

receiving an input query;

performing a query context analysis upon said input query to break down said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents potentially including candidate answers, wherein all documents potentially including candidate answers are stored in a data storage device;

analyzing all of said documents and each document'"'"'s metadata, in a candidate answer generation module, to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user,

wherein each said plurality of parallel implemented scoring modules for scoring all candidate answers using the said supporting passages automatically conducts, in parallel, one or more analyses each producing a candidate score, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer; and

,

wherein a further score comprises a textual alignment score obtained by said processor implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers; and

,

wherein a further score comprises a deeper analysis score obtained by said processor implementing executable instructions for determining the meaning of the supporting passages and input queries by analyzing lexical or semantic relations.

2. The computer-implemented method as claimed in claim 1, wherein said query context analysis includes determining, from said query, one or more predicate argument structures for each input query.

3. The computer-implemented method as claimed in claim 1, wherein said query context analysis includes determining, from said query, one or more lexical answer types for each input query.

4. The computer-implemented method as claimed in claim 1, further including extending said one or more searchable components using a functionality for term weighting and query expansion.

5. The computer-implemented method as claimed in claim 1, further comprising a candidate answer generation module operating in parallel on said passages retrieved by the said searchable components.

6. The computer-implemented method as claimed in claim 1, wherein said scoring of said candidate answers using said supporting passages by said parallel implemented scoring modules includes automatically conducting, in parallel, one or more analyses each producing said candidate score.

7. The computer-implemented method as claimed in claim 6, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer.

8. The computer-implemented method as claimed in claim 6, wherein a further produced candidate score comprises a textual alignment score obtained by implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers.

9. The computer-implemented method as claimed in claim 8, wherein said determining if placement of words in said supporting passages are in alignment includes determining whether said words in said supporting passages are one of:

a same order, a similar order, or with a similar distance between them.

10. The computer-implemented method as claimed in claim 6, wherein a further produced candidate score comprises a deeper analysis score obtained by implementing executable instructions for determining the meaning of the supporting passages and input queries and computing how well lexical and semantic relations in the candidate answer passages are satisfied.

11. The computer-implemented method as claimed in claim 1, wherein said one or more query answer includes one of:

a single query answer, or a ranked list of query answers.

12. The computer-implemented method as claimed in claim 11, wherein the generated query response is one of:

the selected query answer or an elaboration question, said elaboration question generated for delivery to a user, said elaboration question requiring user input information in response, and said user input response information being used in said query answer selecting.

13. The computer-implemented method as claimed in claim 11, further comprising, prior to generating said query response, conducting an interactive session with said user and generating, for delivery to said user, one or more elaboration questions, an elaboration question requiring user input information in response, and said input response information being used in answer determining.

14. The computer-implemented method as claimed in claim 11, further comprising:

determining if a query answer or ranked list of query answers is above a threshold rank level, and if below said threshold rank level, delivering a query response to a user comprising one or more clarification questions, each clarification question requiring user input information in response, said user input information being added to said query.

15. The computer-implemented method as claimed in claim 11, wherein an input query or said query response or both said input query and query response is provided in accordance with one or more of multiple modalities including text, audio, image, video, tactile or gesture.

16. The computer-implemented method as claimed in claim 11, further comprising:

providing a previously obtained candidate answer ranking function operating on a collection of correctly scored examples by applying machine learning technique to a corpus of scored question answer pairs.

17. The computer-implemented method as claimed in claim 1, wherein said determining a single answer further comprises:

collecting results across all data structures having said scored candidate answers,

normalizing and merging candidate answers produced by a same answer scorer across multiple instances of the candidate answer, and aggregating the results; and

,

implementing a candidate answer scoring function to produce a final candidate score.

18. The computer-implemented method as claimed in claim 1, wherein said scoring module implements a candidate answer scoring function performing one of:

context independent scoring where the candidate answer is scored independently of the passage; and

,

context dependent scoring where the candidate answer score depends on the passage content.

19. The computer-implemented method as claimed in claim 16, wherein said applying said final candidate answer scoring function to produce said final candidate answer comprises:

applying a logistic regression function or linear regression function to a complete feature set or subset.

20. A system for generating answers to questions based on a corpus of data comprising:

a memory storage device;

a processor in communication with said memory storage device and configured to perform a method comprising;

receiving an input query;

performing a query context analysis upon said input query to break down said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents potentially including candidate answers, wherein all documents potentially including candidate answers are stored in a data storage device;

analyzing all of said documents and each document'"'"'s metadata, in a candidate answer generation module, to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user,

wherein each said plurality of parallel implemented scoring modules for scoring all candidate answers using the said supporting passages automatically conducts, in parallel, one or more analyses each producing a candidate score, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer; and

,

wherein a further score comprises a textual alignment score obtained by said processor implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers; and

,

wherein a further score comprises a deeper analysis score obtained by said processor implementing executable instructions for determining the meaning of the supporting passages and input queries by analyzing lexical or semantic relations.

21. The system as claimed in claim 20, wherein an input query or said query response or both said input query and query response is provided in accordance with one or more of multiple modalities including text, audio, image, video, tactile or gesture.

22. The system as claimed in claim 20, wherein said performing query context analysis includes:

determining, from said query, one or more predicate argument structures for each input query; and

,

determining, from said query, one or more lexical answer types for each input query.

23. The system as claimed in claim 20, wherein said performing query context analysis includes generating a first plurality of data structures each comprising said query terms including said searchable components, said processor further configured to perform:

initiating a parallel search for said candidate answers by distributing said first plurality of data structures to enable concurrent search results processing providing said candidate answers.

24. The system as claimed in claim 20, wherein said processor is further configured to

generate a second plurality of data structures each comprising candidate answer sets, said processor further configured to perform:

splitting said candidate answer sets into separate data structures each including one or more candidate answers, and providing said data structures to said plurality of parallel operating means performing concurrent parallel supporting passage retrieval operations.

25. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for generating answers to questions based on a corpus of data, said method steps including the steps of:

receiving an input query;

performing a query context analysis upon said input query to break down said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents potentially including candidate answers, wherein all documents potentially including candidate answers are stored in a data storage device;

analyzing all of said documents and each document'"'"'s metadata, in a candidate answer generation module, to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user,

wherein each said plurality of parallel implemented scoring modules for scoring all candidate answers using the said supporting passages automatically conducts, in parallel, one or more analyses each producing a candidate score, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer; and

,

wherein a further score comprises a textual alignment score obtained by said processor implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers; and

,

wherein a further score comprises a deeper analysis score obtained by said processor implementing executable instructions for determining the meaning of the supporting passages and input queries by analyzing lexical or semantic relations.