System and method for providing answers to questions

• US 8,768,925 B2
• Filed: 09/12/2012
• Issued: 07/01/2014
• Est. Priority Date: 05/14/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Eric W., Ferrucci, David, Lally, Adam, Zadrozny, Wlodek W.
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US13/612,473
• Publication Number:
  US 20130007033A1
• Time in Patent Office: 657 Days
• Field of Search: None
• US Class Current: 707/736
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Providing answers to questions based on any corpus of data implements a method that generates a number of candidate passages from the corpus that answer an input query, and finds the correct resulting answer by collecting supporting evidence from the multiple passages. By analyzing all retrieved passages and that passage'"'"'s metadata in parallel, an output plurality of data structures is generated including candidate answers based upon the analyzing. Then, supporting passage retrieval operations are performed upon the set of candidate answers, and for each candidate answer, the data corpus is traversed to find those passages having candidate answer in addition to query terms. All candidate answers are automatically scored by a plurality of scoring modules, each producing a module score. The modules scores are processed to determine one or more query answers; and, a query response is generated based on the one or more query answers.

Claims

1. A computer-implemented method of generating answers to questions based on a corpus of data, said method comprising:

receiving an input query;

breaking said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents including candidate answers, wherein all documents including candidate answers are stored in a data storage device;

analyzing said obtained documents and each document'"'"'s metadata to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score component, said candidate score components comprising a term match score, a textual alignment score and semantic relation score;

combining each of said candidate score components to determine an overall candidate score for each of said candidate answers;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user.

2. The computer-implemented method as claimed in claim 1, wherein said breaking said input query into query terms comprises determining, from said query, one or more predicate argument structures for each input query.

3. The computer-implemented method as claimed in claim 1, wherein said breaking said input query into further query terms further comprises:

determining, from said query, one or more lexical answer types for each input query.

4. The computer-implemented method as claimed in claim 1, further comprising:

extending said one or more searchable components using a functionality for term weighting and query expansion.

5. The computer-implemented method as claimed in claim 1, wherein said scoring said candidate answers using said supporting passages includes implementing parallel scoring modules for automatically conducting, in parallel, one or more analyses, each producing said candidate score.

6. The computer-implemented method as claimed in claim 5, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer.

7. The computer-implemented method as claimed in claim 5, wherein a further produced candidate score comprises a textual alignment score obtained by implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers.

8. The computer-implemented method as claimed in claim 7, wherein said determining if placement of words in said supporting passages are in alignment includes determining whether said words in said supporting passages are one of:

a same order, a similar order, or with a similar distance between them.

9. The computer-implemented method as claimed in claim 5, wherein a further produced candidate score comprises a deeper analysis score obtained by implementing executable instructions for determining the meaning of the supporting passages and input queries and computing how well lexical and semantic relations in the candidate answer passages are satisfied.

10. The computer-implemented method as claimed in claim 1, wherein the generated query response is one of:

the selected query answer or an elaboration question, said elaboration question generated for delivery to a user, said elaboration question requiring user input information in response, and said user input response information being used in said query answer selecting.

11. The computer-implemented method as claimed in claim 10, further comprising, prior to generating said query response, conducting an interactive session with said user and generating, for delivery to said user, one or more elaboration questions, an elaboration question requiring user input information in response, and said input response information being used in answer determining.

12. The computer-implemented method as claimed in claim 10, further comprising:

determining if a query answer or ranked list of query answers is above a threshold rank level, and if below said threshold rank level, delivering a query response to a user comprising one or more clarification questions, each clarification question requiring user input information in response, said user input information being added to said query.

13. The computer-implemented method as claimed in claim 10, wherein an input query or said query response or both said input query and query response is provided in accordance with one or more of multiple modalities including text, audio, image, video, tactile or gesture.

14. The computer-implemented method as claimed in claim 10, further comprising:

providing a previously obtained candidate answer ranking function operating on a collection of correctly scored examples by applying machine learning technique to a corpus of scored question answer pairs.

15. A system for generating answers to questions based on any corpus of data comprising:

a memory storage device;

a processor in communication with said memory storage device and configured to perform a method comprising;

receiving an input query;

breaking said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents including candidate answers, wherein all documents including candidate answers are stored in a data storage device;

analyzing said obtained documents and each document'"'"'s metadata to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score component, said candidate score components comprising a term match score, a textual alignment score and semantic relation score;

combining each of said candidate score components to determine an overall candidate score for each of said candidate answers;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user.

16. The system as claimed in claim 15, wherein said breaking said input query into query terms comprises determining, from said query, one or more predicate argument structures for each input query.

17. The system as claimed in claim 15, wherein said breaking said input query into further query terms further comprises:

determining, from said query, one or more lexical answer types for each input query.

18. The system as claimed in claim 15, wherein said processor is configured to extend said one or more searchable components using a functionality for term weighting and query expansion.

19. The system as claimed in claim 15, wherein said processor is configured to implement parallel scoring modules for automatically conducting, in parallel, one or more analyses, each producing said candidate score using said supporting passages.

20. The system as claimed in claim 19, wherein one candidate score comprises a term match score obtained by said processor further implementing executable instructions for counting the number of terms in said supporting passage, and determining if said number matches a number of terms in a candidate answer.

21. The system as claimed in claim 19, wherein a further produced candidate score comprises a textual alignment score obtained by said processor further implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers.

22. The system as claimed in claim 21, wherein said determining if placement of words in said supporting passages are in alignment includes determining by said processor whether said words in said supporting passages are one of:

a same order, a similar order, or with a similar distance between them.

23. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for generating answers to questions based on any corpus of data, said method steps including the steps of:

receiving an input query;

breaking said input query into query terms, said query terms comprising searchable components;

conducting a first search in said corpus of data using one of more of said searchable components to obtain documents including candidate answers, wherein all documents including candidate answers are stored in a data storage device;

analyzing obtained documents and each document'"'"'s metadata to generate a set of candidate answers;

conducting a second search in said corpus of data using said candidate answers and said searchable components of said query terms to obtain one or more supporting passages, wherein said supporting passages have at least one of said candidate answers and at least one of said searchable components of said query terms;

scoring said candidate answers using said supporting passages, wherein said scoring is carried out by a plurality of parallel implemented scoring modules, each scoring module producing a candidate score component, said candidate score components comprising a term match score, a textual alignment score and semantic relation score;

combining each of said candidate score components to determine an overall candidate score for each of said candidate answers;

selecting one or more query answers based on said candidate score;

generating a query response based on said one or more query answers for delivery to a user.

24. The program storage device as claimed in claim 23, wherein said breaking said input query into query terms comprises determining, from said query, one or more predicate argument structures for each input query.

25. The program storage device as claimed in claim 23, wherein said breaking said input query into further query terms further comprises:

determining, from said query, one or more lexical answer types for each input query.

26. The program storage device as claimed in claim 23, further comprising:

extending said one or more searchable components using a functionality for term weighting and query expansion.

27. The program storage device as claimed in claim 23, wherein said scoring said candidate answers using said supporting passages includes implementing parallel scoring modules for automatically conducting, in parallel, one or more analyses, each producing said candidate score.

28. The program storage device as claimed in claim 27, wherein one candidate score comprises a term match score obtained by implementing executable instructions for counting the number of terms in said supporting passage and determining if said number matches a number of terms in a candidate answer.

29. The program storage device as claimed in claim 27, wherein a further produced candidate score comprises a textual alignment score obtained by implementing executable instructions for determining if placement of words in said supporting passages are in alignment with placement of words of said candidate answers.