Stimulating online discussion in interactive learning environments

• US 10,192,456 B2
• Filed: 11/30/2016
• Issued: 01/29/2019
• Est. Priority Date: 12/01/2015
• Status: Active Grant

• Current Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Original Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Inventors: King, Gary, Mazur, Eric, Miller, Kelly, Lukoff, Brian
• Primary Examiner(s): Shah, Milap
• Application Number:
  US15/365,004
• Publication Number:
  US 20170154541A1
• Time in Patent Office: 790 Days
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G09B 5/12 different stations being capable of presenting different information simultaneously G09B5/14 takes precedence
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

In various embodiments, online discussions in connection with an educational resource are improved by analyzing annotations made by students assigned to a discussion group to identify high-quality annotations likely to generate responses and stimulate discussion threads and by making the identified annotations visible to students not assigned to the discussion group.

Claims

1. A method of increasing productivity of a discussion server hosting online discussions in connection with an educational resource provided to students over network-connected devices, the method comprising the steps of:

(a) distributing an interactive educational resource over a network to a plurality of student devices, the student devices being associated with students enrolled in a class utilizing the educational resource;

(b) providing a discussion server configured to host a plurality of different online discussions;

(c) hosting, at a discussion server, an online discussion for receiving and making visible, to student devices assigned to a first discussion group, annotations concerning the educational resource and received by the discussion server from the student devices assigned to the first discussion group;

(d) computationally analyzing the annotations to identify high-quality annotations likely to generate responses and stimulate discussion threads based on at least one of historical performance, average word length, or word sophistication; and

(e) making the identified annotations visible to student devices associated with students assigned to one or more second discussion groups to improve quality of annotations received by the discussion server from student devices assigned to the one or more second discussion groups and increase a proportion of generative and argumentative discussion threads in online discussions associated with the one or more second discussion groups, thereby increasing productivity of the discussion server.

2. The method of claim 1, further comprising, prior to step (d):

receiving an initial set of annotations at the discussion server, each of the initial set of annotations having a discussion thread associated therewith, wherein at least a portion of the initial set of annotations constitutes a training set;

extracting portions of annotations within the training set, thereby producing a plurality of seed features; and

computationally deriving, from the seed features, one or more evaluation features predictive of thread lengths of discussion threads associated with annotations in the training set.

3. The method of claim 2, wherein step (d) comprises using a machine-learning model to predict a thread length associated with each annotation based on the one or more evaluation features, the model being predictive in accordance with a prediction algorithm and generated by steps comprising:

dividing the initial set of annotations into the training set and a testing set, each of the training set and the testing set comprising a plurality of annotations and thread lengths associated therewith; and

in accordance with the prediction algorithm, identifying the one or more evaluation features having a predictive reliability metric exceeding a threshold.

4. The method of claim 3, further comprising the steps of:

computationally predicting, based on the one or more evaluation features, thread lengths for one or more annotations within the testing set; and

adjusting parameters of the model based on the predictions prior to computationally analyzing annotations not within the testing set or training set to identify high-quality annotations.

5. The method of claim 3, wherein the prediction algorithm is a classification tree.

6. The method of claim 5, wherein the prediction algorithm is a random forest comprising a plurality of regression trees.

7. The method of claim 2, wherein producing the plurality of seed features comprises applying natural-language processing to annotations within the training set.

8. The method of claim 1, wherein the discussion server hosts a plurality of simultaneous discussions each visible only to a discussion group consisting of a subset of the students enrolled in the class.

9. The method of claim 1, wherein the first and second discussion groups each comprise a different subset of students simultaneously enrolled in the class.

10. The method of claim 1, wherein the first discussion group corresponds to a first session of the class and the second discussion group comprises students enrolled in a second, subsequent session of the class.

11. The method of claim 1, further comprising, after step (d):

computationally identifying clusters of high-quality annotations relating to a same portion or related portions of the educational resource;

for each cluster, extracting and summarizing text from the annotations indicative of a topic to which the annotations relate; and

combining, in an electronically represented document, the extracted and summarized text and (i) at least some of the annotations and the identified portion or portions of the educational resource or (ii) clickable links thereto.

12. The method of claim 11, wherein the text from each of the clusters is represented in the document in the form of a panel.

13. The method of claim 1, further comprising, after step (d), redefining the first discussion group to include one or more students not assigned to the first discussion group in step (c).

14. An educational system comprising:

a plurality of student devices for executing an interactive educational resource received over a network, the student devices being configured to receive student annotations associated with the educational resource and transmit at least some of the annotations to a discussion server;

a computer memory;

a student database;

a resource server in electronic communication with the student devices, the resource server comprising a communication module and being configured to make the resource available to student devices associated with students enrolled in a class;

a discussion server, in electronic communication with the student devices and configured to host a plurality of different online discussions, for receiving and making visible, to student devices assigned to a first discussion group in the student database, annotations concerning the educational resource and received from the student devices assigned to the first discussion group; and

stored in the computer memory, an analysis module for computationally analyzing the annotations to identify high-quality annotations likely to generate responses and stimulate discussion threads based on at least one of historical performance, average word length, or word sophistication,

wherein the discussion server is configured to make the identified annotations visible to student devices associated with students assigned to one or more second discussion groups to improve quality of annotations received by the discussion server from student devices assigned to the one or more second discussion groups and increase a proportion of generative and argumentative discussion threads in online discussions associated with the one or more second discussion groups, thereby increasing productivity of the discussion server.

15. The system of claim 14, wherein the analysis module is configured to:

extract portions of annotations within a training set of annotations, thereby producing a plurality of seed features; and

computationally derive, from the seed features, one or more evaluation features predictive of thread lengths of discussion threads associated with annotations in the training set.

16. The system of claim 15, wherein the analysis module uses a machine-learning model to predict a thread length associated with each annotation based on the one or more evaluation features, the model being predictive in accordance with a prediction algorithm and generated by steps comprising:

dividing an initial set of annotations into the training set and a testing set, each of the training set and the testing set comprising a plurality of annotations and thread lengths associated therewith; and

in accordance with the prediction algorithm, identifying the one or more evaluation features having a predictive reliability metric exceeding a threshold.

17. The system of claim 16, wherein the analysis module is configured to:

computationally predict, based on the one or more evaluation features, thread lengths for one or more annotations within the testing set; and

adjust parameters of the model based on the predictions.

18. The system of claim 16, wherein the prediction algorithm is a classification tree.

19. The system of claim 18, wherein the prediction algorithm is a random forest comprising a plurality of regression trees.

20. The system of claim 15, wherein the analysis module is configured to produce the plurality of seed features by applying natural-language processing to annotations within the training set.

21. The system of claim 14, wherein the discussion server hosts a plurality of simultaneous discussions each visible only to a discussion group consisting of a subset of the students enrolled in the class.

22. The system of claim 14, wherein the first and second discussion group each comprise a different subset of students simultaneously enrolled in the class.

23. The system of claim 14, wherein the first discussion group corresponds to a first session of the class and the second discussion group comprises students enrolled in a second, subsequent session of the class.

24. The system of claim 14, wherein the analysis module is configured to:

computationally identify clusters of high-quality annotations relating to a same portion or related portions of the educational resource;

for each cluster, extract and summarize text from the annotations indicative of a topic to which the annotations relate; and

combine, in an electronically represented document, the extracted and summarized text and (i) at least some of the annotations and the identified portion or portions of the educational resource or (ii) clickable links thereto.