Answer Ranking In Community Question-Answering Sites

• US 20100191686A1
• Filed: 01/23/2009
• Published: 07/29/2010
• Est. Priority Date: 01/23/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Wei-Ying, Wang, Xin-Jing, Zhang, Lei
• Granted Patent: US 8,346,701 B2
• Time in Patent Office: Days
• US Class Current: 706/46
• CPC Class Codes: G06F 16/33 Querying
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

In some implementations, a plurality of first questions and corresponding first answers are identified at a community question-answer (CQA) site as a plurality of first question-answer (q-a) pairs. A query thread comprised of a second question and a plurality of candidate second answers is selected for making a determination of answer quality. A set of the first questions that are similar to the second question are identified from the plurality of first questions. First linking features between the identified set of first questions and their corresponding first answers are used for determining an analogy with second linking features between the second question and candidate answers for ranking the candidate answers.

Claims

1. A method carried out by a first community question-answer (CQA) server comprising:

collecting a plurality of first questions and corresponding first answers as a plurality of first question-answer (q-a) pairs;

selecting a second question and a plurality of candidate second answers for making a determination of answer quality;

identifying a set of the first questions that are similar to the second question;

determining first linking features between the identified group of first questions and the corresponding first answers;

comparing the first linking features with second linking features determined between the second question and each of the candidate second answers for predicting analogous linking features; and

ranking the candidate second answers based on the comparing.

2. The method according to claim 1, further comprising:

creating a link prediction model from the plurality of first q-a pairs; and

using the link prediction model in combination with the first linking features to predict a likelihood of the second linking features being analogous to the first linking features.

3. The method according to claim 2, further comprising:

ranking the candidate second answers according to the predicted likelihood that a second q-a pair formed from the second question and a particular candidate second answer has second linking features analogous to the first linking features; and

selecting a best answer based on the ranking of the candidate answers.

4. The method according to claim 3, further comprising:

applying a set-based reasoning method using the learned link prediction model to determine a score for each candidate second answer, the score representing the predicted likelihood that a second q-a pair formed from the second question and the particular candidate second answer has second linking features analogous to the first linking features

5. The method according to claim 1, further comprising:

connecting said first CQA server with a second CQA server over a network, said second CQA server containing a question-answer database, wherein the collecting the first q-a pairs further comprises extracting at least some of the first q-a pairs from the second CQA server over the network.

6. The method according to claim 5, further comprising:

collecting the first q-a pairs from both the second CQA server and from the first CQA server to form a knowledge base at the first CQA server.

7. The method according to claim 1, wherein the collecting the first q-a pairs further comprises collecting both positive first q-a pairs containing a first answer previously identified as a high-quality answer, and negative first q-a pairs containing a first answer that is not a high-quality answer.

8. The method according to claim 7, wherein the identifying the plurality of the first questions that are similar to the second question further comprises considering only first questions that are part of the positive first q-a pairs.

9. The method according to claim 1, further comprising:

using textual features and non-textual features for determining linking features between the first q-a pairs and the second q-a pairs, said textual features being selected from a set comprising;

question and answer term frequency;

novel word term frequency;

or

a number of words that the question and answer have in common; and

said non-textual features being selected from a set comprising;

a raw length of the question and answer;

a ratio of question raw length to answer raw length;

a number of answers that the question received;

answer position in a q-a thread;

a number of votes marking an answer as good;

a number of votes marking an answer as bad;

a rating that the asker of the question assigned to the answer;

or

a time span between when the question was posted and the latest answer.

10. The method according to claim 1, further comprising:

receiving the second question from a first user of a first computer in communication with the first CQA server via a network;

receiving the candidate second answers from one or more second users of one or more second computers in communication with the first CQA server, wherein the second question and the candidate second answers comprise a query thread selected by the first CQA server for determining which answer among the candidate second answers is the best answer; and

presenting the selected best answer to users of the CQA site along with an indication that the selected second answer has been selected as the best answer.

11. One or more processor-readable storage mediums having processor-readable program code embodied therein, said processor-readable program code implemented at least in part by one or more processors for implementing the method of claim 1.

12. A system comprising:

a first community question-answer (CQA) server configured for communication with a plurality of client computers via a network,

wherein the first CQA server is configured to obtaining a plurality of first questions and corresponding first answers as a plurality of first question-answer (q-a) pairs,

wherein the first CQA is configured to select a second question and a plurality of candidate second answers received from the client computers for determination of a best answer to the second question, and

wherein the first CQA server is configured to identify a set of the first questions that are similar to the second question, determine first linking features between the identified set of first questions and the corresponding first answers, determine second linking features between the second question and each of the second candidate answers, and predict a likelihood of analogous linking features between the first linking features and the second linking features for selecting a best answer from the candidate second answers.

13. The system according to claim 12, further comprising:

a second CQA server connected for communication with said first CQA, said second CQA server containing a question-answer database, wherein said first CQA server is configured to extract at least some of the first q-a pairs from the second CQA server.

14. The system according to claim 12, further comprising:

a memory at the first CQA server, said memory storing a knowledge base comprised of the obtained first q-a pairs.

15. The system according to claim 12, wherein the first CQA server is further configured to create a link prediction model from the plurality of first q-a pairs and apply the link prediction model in combination with the first linking features to predict a likelihood of the second linking features being analogous to the first linking features.

16. The method according to claim 15, wherein the first CQA server is further configured to rank the candidate second answers according to the predicted likelihood that a second q-a pair formed from the second question and a particular candidate second answer has second linking features analogous to the first linking features.

17. The system according to claim 12, wherein the first CQA server is further configured to collect both positive first q-a pairs containing a first answer previously identified as a high-quality answer, and negative first q-a pairs containing a first answer that is not a high-quality answer when obtaining the first q-a pairs.

18. One or more processor-readable storage mediums containing processor-readable program code implemented at least in part by one or more processors for executing acts comprising:

providing a community question-answer (CQA) application for presenting a CQA website implemented at a first CQA server;

obtaining a plurality of positive first question-answer (q-a) pairs, each positive first q-a pair containing a first question and a first answer previously identified as a high-quality answer, and a plurality of negative first q-a pairs, each negative first q-a pair containing a first question and a first answer that is not a high-quality answer;

storing the plurality of positive and negative first q-a pairs in a knowledge base on a memory at the first CQA server;

creating a link prediction model from the plurality of positive and negative first q-a pairs;

receiving a second question from a first computer in communication with the first CQA server;

receiving a plurality of candidate second answers from one or more second computers in communication with the first CQA server in response to the second question;

identifying a set of the first questions that are similar to the second question by considering only the first questions that are part of the positive first q-a pairs;

determining first linking features between the identified set of first questions and the corresponding first answers making up each q-a pair in the identified group;

identifying a plurality of candidate second q-a pairs, each comprising the second question and one of the candidate second answers;

determining second linking features between the second question and each of the candidate second answers for each candidate second q-a pair;

using the link prediction model in combination with the first linking features for determining a likelihood of the second linking features of each candidate second q-a pair being analogous to the first linking features;

obtaining a score for each candidate second q-a pair based upon the determined likelihood;

ranking the candidate second q-a pairs according to the scores; and

selecting the candidate second q-a pair having a score indicating a highest probability of analogy to the first linking features as containing a best answer.

19. The one or more processor-accessible storage mediums according to claim 18, further comprising:

marking the selected second candidate answer determined to be the best answer and presenting the selected second candidate answer as the best answer to users of the CQA website in conjunction with the second question.

20. The one or more processor-accessible storage mediums according to claim 18, further comprising:

connecting said first CQA server with a second CQA server over a network, said second CQA server containing a question-answer database, wherein the obtaining the first q-a pairs comprises extracting at least some of the first q-a pairs from the second CQA server over the network.