Question pre-processing in a question and answer system

• US 10,489,400 B2
• Filed: 03/03/2017
• Issued: 11/26/2019
• Est. Priority Date: 03/03/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Abu-Saba, Maria, Curran, David M., Harpur, Liam, O'Doherty, Simon P.
• Primary Examiner(s): Nguyen, Merilyn P
• Application Number:
  US15/448,811
• Publication Number:
  US 20180253431A1
• Time in Patent Office: 998 Days
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems

Abstract

Mechanisms are provided for assembling natural language user inputs into questions for a question-and-answer (QA) system. The mechanisms initialize a timer to a threshold amount of time, receive a first input of content-bearing natural language from a user computing device, and initiate running of the timer in response. The mechanisms determine whether a second input of content-bearing natural language is received from the user computing device prior to the threshold amount of time being reached by the timer. If so, the second input is grouped with the first input to generate a question comprising content of both the first input and the second input, and the timer is reset. In response to the timer reaching the threshold without further input from the user computing device, the question is provided to the QA system as an input question for processing by the QA system.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

initialize a timer to a threshold amount of time;

receive a first input of content-bearing natural language from a user computing device;

initiate running of the timer in response to receiving the first input of content-bearing natural language;

determine whether or not a second input of content-bearing natural language is received from the user computing device prior to the threshold amount of time being reached by the timer;

in response to receiving the second input of content-bearing natural language prior to the threshold amount of time being reached by the timer, group the second input with the first input to generate a question comprising content of both the first input and the second input, and reset the running of the timer; and

provide, in response to the timer reaching the threshold amount of time without further input being received from the user computing device, the question to a Question-and-Answer (QA) system as an input question for processing by the QA system.

2. The computer program product of claim 1, further comprising:

in response to one or more further inputs of content-bearing natural language being received prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to each other or to the first and second inputs to determine if the one or more further inputs are semantically related to at least one of the first input or the second input; and

grouping together each of the one or more further inputs with at least one of the first input or the second input that are determined to be semantically related to each other, to generate sets of grouped inputs, wherein providing the question to the QA system comprises outputting a group of questions to the QA system comprising at least one question, wherein each question in the group of questions is formed by one set of grouped inputs in the sets of grouped inputs.

3. The computer program product of claim 2, wherein a determination of a semantic relationship between the one or more further inputs and at least one of the first input or the second input is based on a confidence score of similarity between contents of the one or more further inputs and contents of at least one of the first input or the second input.

4. The computer program product of claim 3, wherein the confidence score is compounded with elapsed time between inputs being compared, wherein larger elapsed times between inputs being compared reduce the confidence score.

5. The computer program product of claim 1, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of inputs determined to belong to a single same question.

6. The computer program product of claim 1, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of pairs of inputs determined to be a last input associated with one question and a first input associated with a next question.

7. The computer program product of claim 1, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of inputs determined to belong to a single same question, and an average time difference between reception times of pairs of inputs determined to be a last input associated with one question and a first input associated with a next question.

8. The computer program product of claim 1, wherein the threshold amount of time is determined based on at least one of a time of day, a user status, or a domain of questions.

9. The computer program product of claim 2, further comprising:

in response to one or more further inputs being received prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to previous inputs associated with previously generated questions in a same question-and-answer (QA) session between the user computing device and the QA system to determine if the one or more further inputs are semantically related to a previous input in the previous inputs; and

in response to the one or more further inputs being semantically related to the previous input, associating the previous input with a group corresponding to a further input with which the previous input is semantically related.

10. The computer program product of claim 2, further comprising:

in response to one or more further inputs being received by the question compiler prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to previous inputs from a same user submitted during a previous question-and-answer (QA) session between the user computing device and the QA system to determine if the one or more further inputs are semantically related to a previous input in the previous inputs; and

in response to the one or more further inputs being semantically related to the previous input, associating the previous input with a group corresponding to a further input with which the previous input is semantically related.

11. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

initialize a timer to a threshold amount of time;

receive a first input of content-bearing natural language from a user computing device;

initiate running of the timer in response to receiving the first input of content-bearing natural language;

determine whether or not a second input of content-bearing natural language is received from the user computing device prior to the threshold amount of time being reached by the timer;

in response to receiving the second input of content-bearing natural language prior to the threshold amount of time being reached by the timer, group the second input with the first input to generate a question comprising content of both the first input and the second input, and reset the running of the timer; and

provide, in response to the timer reaching the threshold amount of time without further input being received from the user computing device, the question to a Question-and-Answer (QA) system as an input question for processing by the QA system.

12. The apparatus of claim 11, wherein the instructions further cause the processor to:

in response to one or more further inputs of content-bearing natural language being received prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to each other or to the first and second inputs to determine if the one or more further inputs are semantically related to at least one of the first input or the second input; and

grouping together each of the one or more further inputs with at least one of the first input or the second input that are determined to be semantically related to each other, to generate sets of grouped inputs, wherein providing the question to the QA system comprises outputting a group of questions to the QA system comprising at least one question, wherein each question in the group of questions is formed by one set of grouped inputs in the sets of grouped inputs.

13. The apparatus of claim 12, wherein a determination of a semantic relationship between the one or more further inputs and at least one of the first input or the second input is based on a confidence score of similarity between contents of the one or more further inputs and contents of at least one of the first input or the second input.

14. The apparatus of claim 13, wherein the confidence score is compounded with elapsed time between inputs being compared, wherein larger elapsed times between inputs being compared reduce the confidence score.

15. The apparatus of claim 11, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of inputs determined to belong to a single same question.

16. The apparatus of claim 11, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of pairs of inputs determined to be a last input associated with one question and a first input associated with a next question.

17. The apparatus of claim 11, wherein the threshold amount of time is determined based on an analysis of an average time difference between reception times of inputs determined to belong to a single same question, and an average time difference between reception times of pairs of inputs determined to be a last input associated with one question and a first input associated with a next question.

18. The apparatus of claim 11, wherein the threshold amount of time is determined based on at least one of a time of day, a user status, or a domain of questions.

19. The apparatus of claim 12, wherein the instructions further cause the processor to:

in response to one or more further inputs being received prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to previous inputs associated with previously generated questions in a same question-and-answer (QA) session between the user computing device and the QA system to determine if the one or more further inputs are semantically related to a previous input in the previous inputs; and

in response to the one or more further inputs being semantically related to the previous input, associating the previous input with a group corresponding to a further input with which the previous input is semantically related.

20. The apparatus of claim 12, wherein the instructions further cause the processor to:

in response to one or more further inputs being received by the question compiler prior to the timer reaching the threshold amount of time, comparing the one or more further inputs to previous inputs from a same user submitted during a previous question-and-answer (QA) session between the user computing device and the QA system to determine if the one or more further inputs are semantically related to a previous input in the previous inputs; and

in response to the one or more further inputs being semantically related to the previous input, associating the previous input with a group corresponding to a further input with which the previous input is semantically related.