Method and system for designing adaptive, diagnostic assessments

• US 7,828,552 B2
• Filed: 02/22/2006
• Issued: 11/09/2010
• Est. Priority Date: 02/22/2005
• Status: Expired due to Fees

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Almond, Russell G., Shute, Valerie J., Hansen, Eric G.
• Primary Examiner(s): Thai; Xuan M
• Assistant Examiner(s): Hu; Kang
• Application Number:
  US11/359,232
• Publication Number:
  US 20060286533A1
• Time in Patent Office: 1,721 Days
• Field of Search: 434/118, 434/236, 434/322, 434/362, 434/353, 434/323, 434/327, 434/350, 434/354
• US Class Current: 434/118
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

A method and system for administering an assessment to a student are disclosed. The expected weight of evidence may be calculated for each of one or more tasks based on a student model pertaining to a student. A task may be selected based on the calculated expected weights of evidence. The selected task may be administered to the student, and evidence may be collected regarding the selected task. The student model pertaining to the student may be updated based on the evidence. A determination of whether additional information is required to assess the student may be made. If additional information is required to assess the student, the above steps may be repeated. Otherwise, a proficiency status may be assigned to the student based on the student model.

Claims

1. A computer-implemented method of administering an assessment to a student, the method comprising:

for one or more tasks, calculating an expected weight of evidence using a computer processor for the task based on a student model pertaining to a particular student, wherein the student model comprises one or more variables, wherein each of the one or more variables corresponds to a proficiency of the student and is based on student specific information collected prior to the assessment, wherein each of the one or more variables includes a probability of a plurality of probabilities, wherein each of the plurality of probabilities corresponds to a likelihood that the student has a particular proficiency level, wherein the expected weight of evidence is calculated based on the one or more variables corresponding to the proficiency for the particular student and the student specific information collected prior to the assessment;

selecting one of the one or more tasks based on the calculated expected weights of evidence using the computer processor;

administering the selected task to the student;

collecting evidence regarding the selected task;

updating the student model pertaining to the student based on the evidence using the computer processor;

determining whether additional information is required to assess the student using the computer processor;

if so, repeating the above steps; and

if not, assigning a proficiency status to the student based on the student model using the computer processor.

2. The method of claim 1 wherein the evidence comprises a scored response to the selected task.

3. The method of claim 1, further comprising:

scoring a response to the selected task.

4. The method of claim 1 wherein the student model comprises a Bayesian inference network.

5. The method of claim 1 wherein determining whether additional information is required to assess the student comprises determining whether a threshold has been passed.

6. The method of claim 1 wherein determining whether additional information is required to assess the student comprises determining whether a time limit has been exceeded.

7. The method of claim 1 wherein a status of the proficiency comprises one or more of the following:

a high level of proficiency;

a medium level of proficiency; and

a low level of proficiency.

15. The method of claim 1 wherein calculating the expected weight of evidence comprises calculating

17. The method of claim 1 comprising calculating expected weights of evidence for remaining tasks of the assessment based upon the updated student model, the student model being updated based upon evidence received in response to a question of the assessment.

18. The method of claim 17 comprising selecting a next task relating to a same proficiency as a prior task, next task having a different representation than the prior task, the student model being updated based upon evidence associated with the prior task.

8. A non-transitory computer readable storage medium containing one or more program instructions for administering an assessment to a student, the programming instructions, when executed, causing a computer processor to execute steps comprising:

for one or more tasks, calculating an expected weight of evidence for the task based on a student model pertaining to a particular student, wherein the student model comprises one or more variables, wherein each of the one ore more variables corresponds to a proficiency of the student and is based on student specific information collected prior to the assessment, wherein each of the one or more variables includes a probability of a plurality of probabilities, wherein each of the plurality of probabilities corresponds to a likelihood that the student has a particular proficiency level, wherein the expected weight of evidence is calculated based on the one or more variables corresponding to the proficiency for the particular student and the student specific information collected prior to the assessment;

selecting one of the one or more tasks based on the calculated expected weights of evidence;

administering the selected task to the student;

collecting evidence regarding the selected task;

updating the student model pertaining to the student based on the evidence;

determining whether additional information is required to assess the student;

if so, repeating the above steps; and

if not, assigning a proficiency status to the student based on the student model.

9. The computer readable storage medium of claim 8 wherein the evidence comprises a scored response to the selected task.

10. The computer readable storage medium of claim 8, further containing one or more programming instructions for scoring a response to the selected task.

11. The computer readable storage medium of claim 8 wherein the student model comprises a Bayesian inference network.

12. The computer readable storage medium of claim 8 wherein determining whether additional information is required to assess the student comprises one or more programming instructions for determining whether a threshold has been passed.

13. The computer readable storage medium of claim 8 wherein determining whether additional information is required to assess the student comprises one or more programming instructions for determining whether a time limit has been exceeded.

14. The computer readable storage medium of claim 8 wherein a status of the proficiency comprises one or more of the following:

a high level of proficiency;

a medium level of proficiency; and

a low level of proficiency.

16. The computer readable storage medium of claim 8 wherein calculating the expected weight of evidence comprises one or more programming instructions for calculating

19. The computer readable storage medium of claim 8 comprising calculating expected weights of evidence for remaining tasks of the assessment based upon the updated student model, the student model being updated based upon evidence received in response to a question of the assessment.

20. The computer readable storage medium of claim 19, the processing instructions, when executed, causing the processor to select a next task relating to a same proficiency as a prior task, next task having a different representation than the prior task, the student model being updated based upon evidence associated with the prior task.