Mining forums for solutions to questions and scoring candidate answers

• US 9,471,874 B2
• Filed: 12/07/2013
• Issued: 10/18/2016
• Est. Priority Date: 12/07/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Byron, Donna K., LaVoie, Jason D.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Tran, Mai T
• Application Number:
  US14/099,926
• Publication Number:
  US 20150161512A1
• Time in Patent Office: 1,046 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

An approach is provided for mining threaded online discussions. In the approach, performed by an information handling system, a natural language processing (NLP) analysis is performed on threaded discussions pertaining to a given topic. The analysis is performed across multiple web sites with each of the web sites including one or more threaded discussions. The analysis results in harvested discussions pertaining to the topic. The harvested discussions are correlated and a question is identified from the harvested discussions. A set of candidate answers is also identified from the harvested discussions, with one of the candidate answers being selected as the most likely answer to the identified question.

Claims

1. A method, in an information handling system comprising a processor and a memory, of mining threaded online discussions, the method comprising:

performing, by the information handling system, a natural language processing (NLP) analysis of one or more threaded discussions pertaining to a given topic, wherein the analysis is performed across one or more web sites with each of the web sites including one or more of the threaded discussions, wherein the analysis results in a plurality of harvested discussions;

correlating the plurality of harvested discussions across a plurality of threads from the one or more web sites;

identifying a question from the harvested discussions;

identifying a plurality of candidate answers from the harvested discussions, wherein each of the plurality of candidate answers pertain to the identified question;

aggregating and merging a selected plurality of harvested discussions corresponding to each of the candidate answers, wherein the selected plurality of harvested discussions are supporting evidence corresponding to the respective candidate answer;

generating a supporting evidence score based on one or more factors of the supporting evidence for each of the candidate answers, wherein at least one of the factors is selected from the group consisting of a quality of the supporting evidence, and a quantity of the supporting evidence;

generating an answer post score for each of the candidate answers based on an identification of a rating within the threaded discussions pertaining to the respective candidate answer;

generating a post provider score for each of the candidate answers based on an identified expertise level that corresponds to a provider of the respective candidate answer;

generating a follow-up score for each of the candidate answers based on one or more follow-up comments from posters that indicate that the respective candidate answer was correct; and

scoring each of the plurality of candidate answers, wherein the scoring calculates an overall score corresponding to each of the candidate answers, wherein the overall score is based upon one or more component scores selected from the group consisting of the supporting evidence score, the answer post score, the post provider score, and the follow-up score, and wherein a selected answer has the highest overall score when compared to the other candidate answers.

2. The method of claim 1 further comprising:

comparing a plurality of questions found in the threaded discussions to a posed question, wherein the identified question matches the posed question; and

adding one or more of the correlated harvested discussions to a corpus that is utilized in a deep question answering system.

3. The method of claim 1 further comprising:

performing at least one sentiment analysis on each of the plurality of candidate answers, wherein the sentiment analyses result in a sentiment analysis score corresponding to each of the candidate answers.

4. The method of claim 1 further comprising:

identifying a plurality of follow-up postings corresponding to the identified question;

analyzing each of the follow-up postings to identify a conversational move corresponding to each of the follow up postings, wherein at least one of the conversational postings is selected from the group consisting of an answer, a clarification, a rejection, or a different conversational move; and

generating a contribution tree based on the follow-up postings and their identified conversational moves.

5. The method of claim 4 further comprising:

pruning one or more of the follow-up postings from the contribution tree based on a contribution analysis, wherein the pruned follow-up postings have a contribution analysis result selected from the group consisting of an answer leading to a new question, an overly deep follow-up posting, and another pruning criteria.

6. An information handling system comprising:

one or more processors;

a memory coupled to at least one of the processors;

a display; and

a set of instructions stored in the memory and executed by at least one of the processors to mine threaded online discussions, wherein the set of instructions perform actions of;

performing, by the information handling system, a natural language processing (NLP) analysis of one or more threaded discussions pertaining to a given topic, wherein the analysis is performed across one or more web sites with each of the web sites including one or more of the threaded discussions, wherein the analysis results in a plurality of harvested discussions;

correlating the plurality of harvested discussions across a plurality of threads from the one or more web sites;

identifying a question from the harvested discussions;

identifying a plurality of candidate answers from the harvested discussions, wherein each of the plurality of candidate answers pertain to the identified question;

aggregating and merging a selected plurality of harvested discussions corresponding to each of the candidate answers, wherein the selected plurality of harvested discussions are supporting evidence corresponding to the respective candidate answer;

generating a supporting evidence score based on one or more factors of the supporting evidence for each of the candidate answers, wherein at least one of the factors is selected from the group consisting of a quality of the supporting evidence, and a quantity of the supporting evidence;

generating an answer post score for each of the candidate answers based on an identification of a rating within the threaded discussions pertaining to the respective candidate answer;

generating a post provider score for each of the candidate answers based on an identified expertise level that corresponds to a provider of the respective candidate answer;

generating a follow-up score for each of the candidate answers based on one or more follow-up comments from posters that indicate that the respective candidate answer was correct; and

scoring each of the plurality of candidate answers, wherein the scoring calculates an overall score corresponding to each of the candidate answers, wherein the overall score is based upon one or more component scores selected from the group consisting of the supporting evidence score, the answer post score, the post provider score, and the follow-up score, and wherein a selected answer has the highest overall score when compared to the other candidate answers.

7. The information handling system of claim 6 wherein the actions further comprise:

comparing a plurality of questions found in the threaded discussions to a posed question, wherein the identified question matches the posed question; and

adding one or more of the correlated harvested discussions to a corpus that is utilized in a deep question answering system.

8. The information handling system of claim 6 wherein the actions further comprise:

identifying a plurality of follow-up postings corresponding to the identified question;

analyzing each of the follow-up postings to identify a conversational move corresponding to each of the follow up postings, wherein at least one of the conversational postings is selected from the group consisting of an answer, a clarification, a rejection, or a different conversational move; and

generating a contribution tree based on the follow-up postings and their identified conversational moves.

9. The information handling system of claim 8 wherein the actions further comprise:

pruning one or more of the follow-up postings from the contribution tree based on a contribution analysis, wherein the pruned follow-up postings have a contribution analysis result selected from the group consisting of an answer leading to a new question, an overly deep follow-up posting, and another pruning criteria.

10. A computer program product stored in a computer readable medium, comprising computer instructions that, when executed by an information handling system, causes the information handling system to mine threaded online discussions by performing actions comprising:

performing, by the information handling system, a natural language processing (NLP) analysis of one or more threaded discussions pertaining to a given topic, wherein the analysis is performed across one or more web sites with each of the web sites including one or more of the threaded discussions, wherein the analysis results in a plurality of harvested discussions;

correlating the plurality of harvested discussions across a plurality of threads from the one or more web sites;

identifying a question from the harvested discussions;

identifying a plurality of candidate answers from the harvested discussions, wherein each of the plurality of candidate answers pertain to the identified question;

aggregating and merging a selected plurality of harvested discussions corresponding to each of the candidate answers, wherein the selected plurality of harvested discussions are supporting evidence corresponding to the respective candidate answer;

generating a supporting evidence score based on one or more factors of the supporting evidence for each of the candidate answers, wherein at least one of the factors is selected from the group consisting of a quality of the supporting evidence, and a quantity of the supporting evidence;

generating an answer post score for each of the candidate answers based on an identification of a rating within the threaded discussions pertaining to the respective candidate answer;

generating a post provider score for each of the candidate answers based on an identified expertise level that corresponds to a provider of the respective candidate answer;

generating a follow-up score for each of the candidate answers based on one or more follow-up comments from posters that indicate that the respective candidate answer was correct; and

scoring each of the plurality of candidate answers, wherein the scoring calculates an overall score corresponding to each of the candidate answers, wherein the overall score is based upon one or more component scores selected from the group consisting of the supporting evidence score, the answer post score, the post provider score, and the follow-up score, and wherein a selected answer has the highest overall score when compared to the other candidate answers.

11. The computer program product of claim 10 wherein the actions further comprise:

comparing a plurality of questions found in the threaded discussions to a posed question, wherein the identified question matches the posed question; and

adding one or more of the correlated harvested discussions to a corpus that is utilized in a deep question answering system.

12. The computer program product of claim 10 wherein the actions further comprise:

performing at least one sentiment analysis on each of the plurality of candidate answers, wherein the sentiment analyses result in a sentiment analysis score corresponding to each of the candidate answers.

13. The computer program product of claim 10 wherein the actions further comprise:

identifying a plurality of follow-up postings corresponding to the identified question;

analyzing each of the follow-up postings to identify a conversational move corresponding to each of the follow up postings, wherein at least one of the conversational postings is selected from the group consisting of an answer, a clarification, a rejection, or a different conversational move; and

generating a contribution tree based on the follow-up postings and their identified conversational moves.

14. The computer program product of claim 13 wherein the actions further comprise:

pruning one or more of the follow-up postings from the contribution tree based on a contribution analysis, wherein the pruned follow-up postings have a contribution analysis result selected from the group consisting of an answer leading to a new question, an overly deep follow-up posting, and another pruning criteria.