Natural language question answering system and method, and paraphrase module

• US 9,613,025 B2
• Filed: 11/19/2015
• Issued: 04/04/2017
• Est. Priority Date: 11/19/2014
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Young Rae, Heo, Jeong, Kim, Hyun Ki, Ryu, Pum Mo, Bae, Yong Jin, Oh, Hyo Jung, Lee, Chung Hee, Lee, Hyung Jik, Lim, Soo Jong, Lim, Joon Ho, Jang, Myung Gil, Choi, Mi Ran
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US14/945,921
• Publication Number:
  US 20160140958A1
• Time in Patent Office: 502 Days
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A natural language question answering system and method, and a paraphrase module are provided. The natural language question answering system includes a conversion module configured to generate a plurality of modified questions by paraphrasing a user'"'"'s question; a plurality of question answering engines configured to receive each of the user'"'"'s question and the modified questions, and select candidate answers corresponding to each of the user'"'"'s question and the modified questions; and a detection module configured to detect at least one among the selected candidate answers as an answer.

Claims

1. A natural language question answering system, comprising:

one or more processors that process computer executable program code embodied in computer readable storage media, the computer executable program code comprising;

conversion program code that generates a plurality of modified questions by paraphrasing a user'"'"'s question;

question answering engine program code that receives each of the user'"'"'s question and the modified questions, and select candidate answers corresponding to each of the user'"'"'s question and the modified questions; and

detection program code that detects at least one among the searched candidate answers as an answer,

wherein the conversion program code comprises;

evaluation program code that confirms at least one among meaning preservation, grammar preservation, and an expression popularity on a plurality of paraphrase questions paraphrasing the user'"'"'s question, and calculates a reliability value of the plurality of paraphrase questions; and

prioritization program code that selects N modified questions in a sequence in which the reliability value is great among the plurality of paraphrase questions, and outputs the N modified question, and

wherein the conversion program code calculates the reliability value of the plurality of paraphrase questions by considering a weight value with respect to a possibility in which a specific postposition or affix is added to a word or a phrase included in the plurality of paraphrase questions from a language model which is previously learned.

2. The natural language question answering system of claim 1, wherein the conversion program code comprises:

analysis program code that performs at least one among a morpheme analysis, an object name recognition, and a sentence structure analysis on the user'"'"'s question, wherein the conversion program code paraphrases the user'"'"'s question using the analysis result of the analysis unit program code.

3. The natural language question answering system of claim 1, wherein the conversion program code performs at least one of an operation of substituting a word or a phrase of another language for a word or a phrase included in the user'"'"'s question, and an operation of changing a sentence structure of the user'"'"'s question and substituting a synonym for the word included in the user'"'"'s question, and generates the plurality of modified questions.

4. The natural language question answering system of claim 1, wherein the conversion program code comprises:

first conversion program code that confirms a pair of bilingual words or phrases with respect to a word or a phrase included in the user'"'"'s question based on a corpus and a language model, and performs a paraphrase of a word unit or a phrase unit with respect to the user'"'"'s question using the pair of the confirmed bilingual word or phrase.

5. The natural language question answering system of claim 1, wherein the conversion program code comprises:

a second conversion unit configured to perform a paraphrase of changing a sentence structure after substituting a synonym for at least one of a morpheme and an object name in the user'"'"'s question based on a synonym dictionary.

6. The natural language question answering system of claim 1, wherein the conversion program code generates the plurality of modified questions which are grammatically natural by determining a postposition or an affix which is grammatically proper after paraphrasing a word or a phrase included in the user'"'"'s question based on a language model which is previously learned.

7. The natural language question answering system of claim 1, wherein the question answering engine program code calculates a reliability value with respect to each candidate answer, and provides a predetermined number of candidate answer lists in a sequence in which the reliability value is great among the candidate answers to the detection module.

8. The natural language question answering system of claim 1, wherein the detection program code combines the candidate answers and the reliability value of each candidate answer transmitted from the question answering engine program code and detects the candidate answer in which the reliability value is greatest among the candidate answers as the answer.

9. A paraphrase module, comprising:

a processor that processes computer executable program code embodied in computer readable storage media, the computer executable program code comprising;

analysis program code that analyzes a user'"'"'s question; and

conversion program code that performs at least one of an operation of substituting a word or a phrase of another language for a word or a phrase included in the user'"'"'s question, and an operation of changing a sentence structure of the user'"'"'s question and substituting a synonym for the word included in the user'"'"'s question, and generate a plurality of modified questions in which the user'"'"'s question is paraphrased,

evaluation program code that confirms at least one among meaning preservation, grammar preservation, and an expression popularity on a plurality of paraphrase questions paraphrasing the user'"'"'s question, and calculates a reliability value of the plurality of paraphrase questions; and

prioritization program code that selects N modified questions which are previously set in a sequence in which the reliability value is great among the plurality of paraphrase questions, and outputs the N modified questions,

wherein the user'"'"'s question and the plurality of modified questions are provided to question answering engine program code, and

wherein the evaluation program code calculates the reliability value of the plurality of paraphrase questions by considering a weight value with respect to a possibility in which a specific postposition or affix is added to a word or a phrase included in the plurality of paraphrase questions from a language model which is previously learned.

10. The paraphrase module of claim 9, wherein the analysis program code performs at least one among a morpheme analysis, an object name recognition, and a sentence structure analysis on the user'"'"'s question.

11. The paraphrase module of claim 9, wherein the conversion program code comprises:

first conversion program code that substitutes a word or phrase of another language for a word or a phrase included in the user'"'"'s question based on a corpus including a pair of bilingual words or phrases.

12. The paraphrase module of claim 9, wherein the conversion program code comprises:

second conversion program code that converts a morpheme included in the user'"'"'s question into an object name-based synonym which is previously confirmed using a synonym dictionary, and changes a sentence structure with respect to the user'"'"'s question in which the conversion on the synonym is performed.

13. A natural language question answering method by a natural language question answering system, comprising:

processing by one or more processors computer executable program code embodied in computer readable storage, wherein the computer executable program code;

generates a plurality of modified questions by paraphrasing a user'"'"'s question;

receives each of the user'"'"'s question and the modified questions by a question answering engine program code,

selects candidate answers corresponding to each of the user'"'"'s question and the modified questions; and

detects at least one among the searched candidate answers as an answer,

wherein evaluation program code confirms at least one among meaning preservation, grammar preservation, and an expression popularity on a plurality of paraphrase questions paraphrasing the user'"'"'s question, and calculates a reliability value of the plurality of paraphrase questions;

wherein prioritization program code selects N modified questions in a sequence in which the reliability value is great among the plurality of paraphrase questions, and outputs the N modified question, and

wherein conversion program code calculates the reliability value of the plurality of paraphrase questions by considering a weight value with respect to a possibility in which a specific postposition or affix is added to a word or a phrase included in the plurality of paraphrase questions from a language model which is previously learned.

14. The natural language question answering method of claim 13, wherein the generating of the plurality of modified questions comprises at least one of:

substituting a word or a phrase of another language for a word or a phrase included in the user'"'"'s question; and

converting a sentence structure after substituting a synonym for the word included in the user'"'"'s question.

15. The natural language question answering method of claim 13, wherein the detecting of the at least one among the selected candidate answers as the answer comprises:

confirming the candidate answers and the reliability value of each candidate answer transmitted from the question answering engine program code;

calculating a combined reliability value by combining candidate answers repeated among the candidate answers; and

detecting the candidate answer in which the combined reliability value is the greatest as the answer.