Instructional support platform for interactive learning environments

• US 10,438,498 B2
• Filed: 11/30/2016
• Issued: 10/08/2019
• Est. Priority Date: 12/01/2015
• Status: Active Grant

• Current Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Original Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Inventors: King, Gary, Mazur, Eric, Miller, Kelly, Lukoff, Brian
• Primary Examiner(s): Coburn, Corbett B
• Application Number:
  US15/365,019
• Publication Number:
  US 20170154543A1
• Time in Patent Office: 1,042 Days
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G09B 5/12 different stations being capable of presenting different information simultaneously G09B5/14 takes precedence
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

In various embodiments, subject matter for improving discussions in connection with an educational resource is identified and summarized by analyzing annotations made by students assigned to a discussion group to identify high-quality annotations likely to generate responses and stimulate discussion threads, identifying clusters of high-quality annotations relating to the same portion or related portions of the educational resource, extracting and summarizing text from the annotations, and combining, in an electronically represented document, the extracted and summarized text and (i) at least some of the annotations and the portion or portions of the educational resource or (ii) clickable links thereto.

Claims

1. A method of identifying and summarizing subject matter for improving discussions in connection with an educational resource provided to students over network-connected devices, the method comprising the steps of:

(a) distributing an interactive educational resource over a network to a plurality of student devices, the student devices being associated with students currently enrolled in a class utilizing the educational resource;

(b) hosting, at a discussion server, an online discussion for receiving and making visible, to student devices assigned to a discussion group, annotations concerning the educational resource received by the discussion server from the student devices assigned to the discussion group;

(c) computationally analyzing annotations to identify high-quality annotations likely to generate responses and stimulate discussion threads;

(d) computationally identifying clusters of high-quality annotations relating to the same portion or related portions of the educational resource;

(e) for each cluster, extracting and summarizing text from the annotations indicative of a topic to which the annotations relate; and

(f) combining, in an electronically represented document, the extracted and summarized text and (i) at least some of the annotations and the portion or portions of the educational resource or (ii) clickable links thereto.

2. The method of claim 1, further comprising, prior to step (c):

receiving an initial set of annotations at the discussion server, each of the initial set of annotations having a discussion thread associated therewith, wherein at least a portion of the initial set of annotations constitutes a training set;

extracting portions of annotations within the training set, thereby producing a plurality of seed features; and

computationally deriving, from the seed features, one or more evaluation features predictive of thread lengths of discussion threads associated with annotations in the training set.

3. The method of claim 2, wherein step (c) comprises using a machine-learning model to predict a thread length associated with each annotation based on the one or more evaluation features, the model being predictive in accordance with a prediction algorithm and generated by steps comprising:

dividing the initial set of annotations into the training set and a testing set, each of the training set and the testing set comprising a plurality of annotations and thread lengths associated therewith; and

identifying the one or more evaluation features based on predictive reliability in accordance with the prediction algorithm.

4. The method of claim 3, further comprising the steps of:

computationally predicting, based on the one or more evaluation features, thread lengths for one or more annotations within the testing set; and

adjusting parameters of the model based on the predictions prior to computationally analyzing annotations not within the testing set or training set to identify high-quality annotations.

5. The method of claim 3, wherein the prediction algorithm is a classification tree.

6. The method of claim 5, wherein the prediction algorithm is a random forest comprising a plurality of regression trees.

7. The method of claim 2, wherein producing the plurality of seed features comprises applying natural-language processing to annotations within the training set.

8. The method of claim 1, wherein the text from each of the clusters is represented in the document in the form of a panel.

9. The method of claim 1, further comprising, after step (c), making the identified annotations visible to student devices associated with students who are not assigned to the discussion group.

10. The method of claim 1, wherein the discussion server hosts a plurality of simultaneous discussions each visible only to a discussion group consisting of a subset of the students enrolled in the class.

11. The method of claim 10, wherein the annotations are analyzed within each discussion group and identified annotations within one discussion group are made visible to student devices associated with students who are (i) in one or more of the other discussion groups, and/or (ii) not assigned to the discussion group.

12. The method of claim 11, wherein the discussion group corresponds to a first session of the class and the students who are not assigned to the discussion group are enrolled in a second, subsequent session of the class.

13. An educational system comprising:

a plurality of student devices for executing an interactive educational resource received over a network, the student devices being configured to receive student annotations associated with the educational resource and transmit at least some of the annotations to a discussion server;

a student database;

a resource server in electronic communication with the student devices, the resource server comprising a communication module and being configured to make the resource available to student devices associated with students enrolled in a class;

a discussion server, in electronic communication with the student devices, for receiving and making visible, to student devices assigned to a discussion group in the student database, annotations concerning the educational resource received from the student devices assigned to the discussion group; and

an analysis module configured to (i) computationally analyze annotations to identify high-quality annotations likely to generate responses and stimulate discussion threads, (ii) computationally identify clusters of high-quality annotations relating to the same portion or related portions of the educational resource, (iii) for each cluster, extract and summarize text from the annotations indicative of a topic to which the annotations relate, and (iv) combine, in an electronically represented document, the extracted and summarized text and (a) at least some of the annotations and the portion or portions of the educational resource or (b) clickable links thereto.

14. The system of claim 13, wherein the analysis module is configured to:

extract portions of annotations within a training set of annotations, thereby producing a plurality of seed features; and

computationally derive, from the seed features, one or more evaluation features predictive of thread lengths of discussion threads associated with annotations in the training set.

15. The system of claim 14, wherein the analysis module uses a machine-learning model to predict a thread length associated with each annotation based on the one or more evaluation features, the model being predictive in accordance with a prediction algorithm and generated by steps comprising:

dividing an initial set of annotations into the training set and a testing set, each of the training set and the testing set comprising a plurality of annotations and thread lengths associated therewith; and

identifying the one or more evaluation features based on predictive reliability in accordance with the prediction algorithm.

16. The system of claim 15, wherein the analysis module is configured to:

computationally predict, based on the one or more evaluation features, thread lengths for one or more annotations within the testing set; and

adjust parameters of the model based on the predictions.

17. The system of claim 15, wherein the prediction algorithm is a classification tree.

18. The system of claim 17, wherein the prediction algorithm is a random forest comprising a plurality of regression trees.

19. The system of claim 14, wherein the analysis module is configured to produce the plurality of seed features by applying natural-language processing to annotations within the training set.

20. The system of claim 13, wherein the discussion server is configured to make the identified annotations visible to student devices associated with students who are not assigned to the discussion group.

21. The system of claim 13, wherein the discussion server hosts a plurality of simultaneous discussions each visible only to a discussion group consisting of a subset of the students enrolled in the class.

22. The system of claim 21, wherein the analysis module is configured to analyze annotations within each discussion group, the discussion server making identified annotations within one discussion group visible to student devices associated with students who are (i) in one or more of the other discussion groups, and/or (ii) not assigned to the discussion group.

23. The system of claim 22, wherein the discussion group corresponds to a first session of the class and the students who are not assigned to the discussion group are enrolled in a second, subsequent session of the class.