USER EVALUATION IN A COLLABORATIVE ONLINE FORUM

• US 20100162135A1
• Filed: 12/24/2008
• Published: 06/24/2010
• Est. Priority Date: 12/24/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Morsy, Ahmed A., Wanas, Nayer M., Ammar, Waleed A., El-Saban, Motaz A., Ashour, Heba M., El Baradei, Mostafa M.
• Granted Patent: US 8,893,024 B2
• Time in Patent Office: Days
• US Class Current: 715/753
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104

Abstract

Users of a collaborative online forum may be evaluated automatically with the results made available to a supervisor. In an example embodiment, a method involves accessing multiple posts that are associated with multiple users, with each post of the multiple posts including content. Post scores for the multiple posts are ascertained based on one or more features. User interactions are analyzed with regard to the multiple posts to identify relationships among the multiple users and the content included in the multiple posts. The post scores and the user interactions are evaluated to determine respective collaboration scores for respective users of the multiple users. A user interface that provides access to the collaboration scores is presented to a supervisor.

Claims

1. One or more processor-accessible tangible media comprising processor-executable instructions for user evaluation in a collaborative online forum, wherein the processor-executable instructions, when executed, direct one or more devices to perform acts comprising:

accessing multiple posts that are associated with multiple users, each post of the multiple posts including content;

ascertaining post scores for the multiple posts based on one or more features;

analyzing user interactions with regard to the multiple posts to identify relationships among the multiple users and the content included in the multiple posts;

evaluating the post scores and the user interactions to determine respective collaboration scores for respective users of the multiple users responsive to multiple metrics; and

presenting a user interface that provides access to the collaboration scores via a supervisor dashboard that displays interactivity graphs reflecting interactions among at least selected ones of the multiple users.

2. The one or more processor-accessible tangible media as recited in claim 1, wherein the act of presenting comprises:

incorporating into the supervisor dashboard an interactivity graph that represents sizes of contributions from the selected ones of the multiple users and that represents strengths of interactions between and among the selected ones of the multiple users.

3. The one or more processor-accessible tangible media as recited in claim 1, wherein the act of presenting comprises:

incorporating into the supervisor dashboard an interactivity graph that represents magnitudes of popularity for multiple topics extracted from the content of the multiple posts and that represents strengths of interactions between and among the multiple topics.

4. The one or more processor-accessible tangible media as recited in claim 1, wherein the act of presenting comprises:

incorporating into the supervisor dashboard one or more user interface elements that enable a supervisor to explore the collaborative online forum to which the multiple posts have been submitted by selected topic and by constituent threads forming the selected topic.

5. A method, which is implemented by one or more devices, for user evaluation in a collaborative online forum, the method comprising acts of:

accessing multiple posts that are associated with multiple users, each post of the multiple posts including content;

ascertaining post scores for the multiple posts based on one or more features;

analyzing user interactions with regard to the multiple posts to identify relationships among the multiple users and the content included in the multiple posts;

evaluating the post scores and the user interactions to determine respective collaboration scores for respective users of the multiple users; and

presenting a user interface that provides access to the collaboration scores.

6. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features using a classifier, wherein the one or more features include five feature groups comprising a relevance feature group, an originality feature group, a forum-specific feature group, a surface feature group, and a posting component feature group.

7. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features, wherein the one or more features include a relevance feature group comprising at least;

(i) a first feature that measures whether a given post is on thread topic and (ii) a second feature that measures whether the given post is on sub-forum topic.

8. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features, wherein the one or more features include an originality feature group comprising at least;

(i) a first feature that measures a degree of overlap between a given post and leading posts in a particular thread and (ii) a second feature that measures a number of posts between the given post and a most-overlapping post as determined by the first feature.

9. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features, wherein the one or more features include a forum-specific feature group comprising at least;

(i) a first feature that measures a ratio of quoted text to a size of a given post as normalized by a size of an original post and (ii) a second feature that measures a number of replies that the given post generates.

10. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features, wherein the one or more features include a surface feature group comprising at least;

(i) a first feature that measures an inter-posting time for a given post as normalized by an average inter-posting time of a community of the collaborative online forum, (ii) a second feature that measures a word count of the given post as normalized by an average length for posts of a particular thread, and (ii) a third feature that measures use of creative punctuation in the given post as normalized by a length of the given post.

11. The method as recited in claim 5, wherein the act of ascertaining comprises:

ascertaining the post scores for the multiple posts based on the one or more features, wherein the one or more features include a posting component feature group comprising at least;

(i) a first feature that measures a value contributed to a given post by a web-link that is included in the given post and (ii) a second feature that measures questions contained within the given post with respect to other questions in the collaborative online forum.

12. The method as recited in claim 5, wherein the act of evaluating comprises:

evaluating the post scores and the user interactions to determine the collaboration scores for the multiple users responsive to at least four metrics, wherein the four metrics comprise a quality metric, a quantity metric, a consistency metric, and an interactivity metric.

13. The method as recited in claim 5, wherein the act of evaluating comprises:

evaluating the post scores to determine an individual collaboration score for an individual user of the multiple users responsive to a quality metric that reflects the post scores for the individual user on a thread level.

14. The method as recited in claim 5, wherein the act of evaluating comprises:

evaluating the user interactions and the content of the multiple posts to determine an individual collaboration score for an individual user of the multiple users responsive to a quantity metric that reflects a volume of contributions by the individual user relative to a volume of contributions from other users.

15. The method as recited in claim 5, wherein the act of evaluating comprises:

evaluating the user interactions to determine an individual collaboration score for an individual user of the multiple users responsive to a consistency metric that reflects whether the individual user contributes regularly over a period of time instead of in a burst-like pattern.

16. The method as recited in claim 5, wherein the act of evaluating comprises:

evaluating the user interactions to determine an individual collaboration score for an individual user of the multiple users responsive to an interactivity metric that reflects how many other users the individual user interacts with by considering a number of different threads to which the individual user posts and a number of participants contributing to the different threads.

17. The method as recited in claim 5, wherein the act of presenting comprises:

presenting the user interface that provides access to the collaboration scores, wherein the user interface includes a dashboard that displays one or more interactivity graphs representing at least selected ones of the multiple users.

18. A device that is adapted to evaluate users in a collaborative online forum, the device comprising:

a collaboration evaluator to access multiple posts that are associated with multiple users, each post of the multiple posts including content;

the collaboration evaluator including;

a post score ascertainer to ascertain post scores for the multiple posts based on one or more features;

a user interactions analyzer to analyze user interactions with regard to the multiple posts to identify relationships among the multiple users and the content included in the multiple posts; and

a collaboration score determiner to evaluate the post scores and the user interactions to determine respective collaboration scores for respective users of the multiple users; and

a dashboard creator to present a user interface that provides access to the collaboration scores.

19. The device as recited in claim 18, wherein the dashboard creator is to present the user interface that provides access to the collaboration scores with the user interface including an ability to display a listing of top collaborators for different interaction categories.

20. The device as recited in claim 18, wherein the collaboration score determiner is to determine the collaboration scores for the multiple users responsive to multiple ranking metrics;

and wherein the dashboard creator is to present the user interface that provides access to the collaboration scores with the user interface to enable a supervisor to adjust relative percentage values of the ranking metrics that are to be used by the collaboration score determiner.