System and method for improved cumulative assessment

• US 20070009871A1
• Filed: 05/26/2006
• Published: 01/11/2007
• Est. Priority Date: 05/28/2005
• Status: Abandoned Application

• Current Assignee: Ctb Mcgraw Hill
• Original Assignee: Ctb/Mcgraw-Hill
• Inventors: Lewis, Daniel, Tidwell-Scheuring, Sylvia
• Application Number:
  US11/441,449
• Publication Number:
  US 20070009871A1
• Time in Patent Office: Days
• US Class Current: 434/350
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

A system and method for improved cumulative assessment provide for automatically, e.g., programmatically, determining an evaluation of two or more assessments including two or more related items in a cumulative manner. In one embodiment, an initial assessment including initial assessment items is administered at time T1 and scored to produce an ability estimate. At least one successive assessment is also administered and scored to produce an ability estimate. Selected ones of the administered assessments or included items are determined (“included assessments”), and the included assessments are scored to produce a simultaneous maximum likelihood ability estimate for the included assessments, for example, in view of all of the included assessments.

Claims

1. A method for generating an ability estimate for an assessment subject comprising:

administering to the assessment subject a first assessment at a first time T1, the first assessment including one or more items;

scoring responses by the assessment subject to the items of the first assessment;

administering to the assessment subject one or more subsequent assessments at one or more subsequent times T₂- T_N, each subsequent assessment including one or more items;

scoring responses by the assessment subject to the items of each of the subsequent assessments;

selecting a group of included items comprising one or more items from said first assessment and one or more items from each of at least one of said subsequent assessments, wherein the included items are related to the ability being estimated; and

computing an ability estimate for the assessment subject at any time of the administered assessments T₁-T_N based on scores of the group of included items.

2. The method of claim 1, wherein the selecting step comprises applying predetermined selection criteria for selecting the included items.

3. The method of claim 1, wherein the included items are associated with learning targets of a learning map that share pre-cursor or post-cursor relationships with each other.

4. The method of claim 1, wherein the selecting step includes cluster analysis for identifying items forming a related group of items.

5. The method of claim 4, further comprising utilizing a learning map having learning targets with which the related group of items are associated to determine relationships between items within the related group of items.

6. The method of claim 1, wherein the selecting step is performed by reference to a scale on which the included items are represented.

7. The method of claim 1, wherein the ability estimate is computed using a probabilistic model that predicts an ability estimate based on item response results.

8. The method of claim 7, wherein the probabilistic model comprises a modeling function selected from the group comprising unidimensional item response theory models, multidimensional IRT models, Learning Map Analytics, and Bayesian Networks.

9. The method of claim 8, wherein the unidimensional item response theory models comprise a model selected from the group comprising:

3-parameter logistic model, 2-parameter logistic model, 1-parameter logistic model, and Rasch model.

10. The method of claim 1, wherein said first and subsequent assessments are administered as paper-based assessments on which students are instructed to provide hand-written responses to assessment items.

11. The method of claim 10, further comprising converting the hand-written responses into computer-readable data.

12. The method of claim 1, wherein said first and subsequent assessments are administered as computer-based assessments on which students are instructed to enter responses to assessment items on a computer input device.

13. A system for generating an ability estimate for an assessment subject comprising:

a test administration module adapted to administer to the assessment subject a first assessment at a first time T₁, the first assessment including one or more items, and to administer to the assessment subject one or more subsequent assessments at one or more subsequent times T₂-T_N, each additional assessment including one or more items;

a scoring module adapted to score responses by the assessment subject to the items of the first and subsequent assessments;

an item selection module adapted to select a group of included items comprising one or more items from said first assessment and one or more items from each of at least one of said additional assessments, wherein the included items are related to the ability being estimated; and

an ability estimate engine adapted to compute an ability estimate for the assessment subject at any time of the administered assessments T₁-T_N based on scores of the group of included items.

14. The system of claim 13, wherein said test administration module comprises an assessment presentation device and a user input device adapted to enable the assessment subject to input responses to items.

15. The system of claim 14, wherein said presentation device comprises one or more of a display monitor, speakers, and actuators, and said user input device comprises one or more of a mouse, keyboard, microphone, and pen.

16. A method for generating a cumulative ability estimate for an assessment subject comprising:

administering to the assessment subject an initial assessment at an initial time, the initial assessment including initial assessment items;

generating an initial ability estimate for the assessment subject for the initial time based on responses to the initial assessment items related to the ability being estimated;

administering to the assessment subject at least one successive assessment at a time different from the initial time, the successive assessment including successive assessment items including items having measurement goals that are related to measurement goals of the initial assessment items;

generating a successive ability estimate for the assessment subject for the different time based on responses to the successive assessment items related to the ability being estimated;

selecting two or more assessments of the initial and at least one successive assessment to be included in an improved likelihood ability estimate;

selecting assessment items from the two or more selected assessments to be included in the improved likelihood ability estimate and excluding non-selected items from the improved likelihood ability estimate; and

generating improved likelihood ability estimates for the assessment subject for the initial time and for the different time based on the responses to the selected assessment items.

17. The method of claim 16, wherein each of the items of the initial and successive assessments correspond with at least one learning target of a learning map and wherein items are selected to be included in the improved likelihood ability estimate according to precursor-postcursor relationships existing between learning targets to which the items correspond.

18. The method of claim 16, wherein the ability estimates are computed using a probabilistic model that predicts an ability estimate based on item response results.

19. The method of claim 16, wherein said initial and successive assessments are administered as paper-based assessments on which students are instructed to provide hand-written responses to assessment items.

20. The method of claim 19, further comprising converting the hand-written responses into computer-readable data.

21. The method of claim 16, wherein said initial and successive assessments are administered as computer-based assessments on which students are instructed to enter responses to assessment items on a computer input device.

22. The method of claim 18, wherein the probabilistic model comprises a modeling function selected from the group comprising unidimensional item response theory models, multidimensional IRT models, Learning Map Analytics, and Bayesian Networks.

23. The method of claim 22, wherein the unidimensional item response theory models comprise a model selected from the group comprising:

3-parameter logistic model, 2-parameter logistic model, 1-parameter logistic model, and Rasch model.