System and method for question answering by reformulating word problems

• US 9,378,273 B2
• Filed: 03/13/2014
• Issued: 06/28/2016
• Est. Priority Date: 03/13/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lally, Adam P., Chu-Carroll, Jennifer, Mueller, Erik T.
• Primary Examiner(s): Truong, Dennis
• Application Number:
  US14/207,861
• Publication Number:
  US 20150261849A1
• Time in Patent Office: 838 Days
• Field of Search: 707/723, 707/708, 707/722, 707/769, 707/771, 704/257, 434/322
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

A computer-implemented method of answering questions. A word problem is received into a user interface of a question-answering (QA) system operating on a computerized device having a processor. The word problem comprises a question and contextual phrases providing context for the question. The question is parsed from the word problem, using a parsing function of the processor. The contextual phrases are parsed from the word problem, using the parsing function of the processor. A type is identified for the question, using a typing function of the processor. A phrase is identified in the question having the same type as the question. A reformulated question is produced by replacing the phrase in the question having the same type as the question with a contextual phrase. The reformulated question is input to the QA system. Answers to the reformulated question are received from the QA system.

Claims

1. A method comprising:

receiving a word problem comprising a question and contextual phrases providing context for said question;

identifying said question and said contextual phrases in said word problem using a parsing function;

identifying first types for each phrase of said contextual phrases;

identifying a referring phrase in said question;

identifying a second type for said referring phrase;

determining if any of said first types matches said second type;

creating a replacement phrase if any of said first types matches said second type, said replacement phrase comprising each phrase of said contextual phrases having a type matching said second type;

producing a reformulated question by replacing said referring phrase in said question with said replacement phrase;

inputting said reformulated question to a question-answering (QA) system; and

receiving answers to said reformulated question from said QA system.

2. The method of claim 1, further comprising:

if said first types do not match said second type, forming a query based on said contextual phrases;

inputting said query to said QA system;

receiving an answer from said QA system;

identifying a third type for said answer;

determining if said third type matches said second type; and

producing a reformulated question by replacing said referring phrase in said question with said answer if said third type matches said second type.

3. The method according to claim 1, said identifying said question and said contextual phrases in said word problem comprising:

dividing said word problem into sentences using sentence boundary detection; and

identifying said question as a sentence ending with a question mark.

4. The method according to claim 1, said identifying said question and said contextual phrases in said word problem comprising:

identifying said contextual phrases using named entity recognition.

5. The method of claim 1, further comprising:

dividing said word problem into sentences;

identifying said contextual phrases from said word problem, said contextual phrases comprising a portion of said sentences.

6. The method of claim 1, further comprising:

creating a verification question based on said answers to said reformulated question and said replacement phrase;

inputting said verification question to said QA system;

receiving from said QA system verification answers to said verification question; and

verifying said answers to said reformulated question based on said verification answers to said verification question.

7. A method comprising:

receiving a word problem comprising a first question and contextual phrases providing context for said first question;

parsing said first question from said word problem;

identifying a referring phrase in said first question;

parsing said contextual phrases from said word problem;

identifying types for each phrase of said contextual phrases;

identifying a type for said referring phrase;

producing a reformulated question by replacing, said referral phrase in said first question with a replacement phrase said replacement phrase comprising phrases of said contextual phrases having types for each phrase of said contextual phrases matching said type for said referring phrase;

inputting said reformulated question to a question-answering (QA) system;

receiving from said QA system first answers to said reformulated question, each of said first answers having a confidence score;

identifying a missing link in said first question, said missing link comprising at least one of said first answers being related to said replacement phrase and having a confidence score below a predetermined value;

creating a second question based on said missing link and said replacement phrase;

inputting said second question to said QA system; and

receiving from said QA system second answers to said second question.

8. The method of claim 7, further comprising:

identifying first types for said first answers; and

identifying a second type for said first question,

said missing link comprising at least one of said first answers being related to said replacement phrase and having a type different from said second type.

9. The method according to claim 7, said parsing said first question from said word problem comprising:

dividing said word problem into sentences using sentence boundary detection; and

identifying said first question as a sentence ending with a question mark.

10. The method according to claim 7, said parsing said contextual phrases from said word problem comprising:

identifying said contextual phrases using named entity recognition.

11. The method according to claim 7, said producing a reformulated question further comprising:

identifying third types for each phrase of said contextual phrases;

identifying a fourth type for said referring phrase;

creating said replacement phrase, based on said fourth type for said referring phrase and said third types for each phrase of said contextual phrases; and

producing a reformulated question by replacing said referring phrase in said first question with a replacement phrase of the same type.

12. The method of claim 7, further comprising:

dividing said word problem into sentences;

identifying said contextual phrases from said word problem, said contextual phrases comprising a portion of said sentences; and

producing said reformulated question based on said first question and said contextual phrases.

13. The method of claim 7, further comprising:

creating a verification question based on said second answers to said second question;

inputting said verification question to said QA system;

receiving from said QA system verification answers to said verification question; and

verifying said second answers to said second question based on said verification answers to said verification question.

14. A method comprising;

receiving a word problem;

using sentence boundary detection, automatically dividing said word problem into sentences;

identifying a first question from said word problem;

identifying contextual phrases from said word problem, said contextual phrases comprising a portion of said sentences and providing context for said question;

identifying first types for each phrase of said contextual phrases;

identifying a referring phrase in said first question;

identifying a second type for said referring phrase;

creating a replacement phrase comprising phrases of said contextual phrases having first type matching said second type;

producing a reformulated question by replacing said referring phrase in said first question with said replacement phrase;

inputting said reformulated question to a question-answering (QA) system;

receiving answers to said reformulated question from said QA system; and

creating a second question by incorporating one or more answers to said reformulated question into said first question, said second question comprising one of;

an interrogative word question, a yes-no question, and a multiple-choice question.

15. The method according to claim 14, said identifying a first question from said word problem comprising:

identifying said first question as a sentence ending with a question mark.

16. The method according to claim 14, said identifying contextual phrases from said word problem comprising:

identifying said contextual phrases using named entity recognition.

17. The method according to claim 14, further comprising:

receiving from said QA system first answers to said reformulated question, each of said first answers having a confidence score;

identifying a missing link in said first question, said missing link comprising at least one of said first answers being related to said replacement phrase and having a confidence score below a predetermined value;

creating a third question based on said missing link and said replacement phrase;

inputting said third question to said QA system; and

receiving from said QA system third answers to said third question, said second question incorporating one or more answers to said third question into said first question.

18. The method of claim 17, further comprising:

identifying first types for said first answers; and

identifying a second type for said first question,

said missing link comprising at least one of said first answers being related to said replacement phrase and having a type different from said second type.

19. The method of claim 14, said interrogative word question comprising one of:

a what question, a when question, a where question, a who question, a why question, and a how question.

20. The method of claim 14, said word problem comprising a natural language query.