Computerized system and method for teaching and assessing the holistic scoring of open-ended questions

• US 6,970,677 B2
• Filed: 07/27/2004
• Issued: 11/29/2005
• Est. Priority Date: 12/05/1997
• Status: Expired due to Fees

• Current Assignee: Harcourt Assessment Incorporated (Pearson plc)
• Original Assignee: Harcourt Assessment Incorporated (Pearson plc)
• Inventors: Mellott, David K., Jongsma, Eugene A.
• Primary Examiner(s): Harris, Chanda L.
• Application Number:
  US10/900,004
• Publication Number:
  US 20050014123A1
• Time in Patent Office: 490 Days
• Field of Search: 434/353, 434/327, 434/323
• US Class Current: 434/353
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: Y10S 706/927 Education or instruction

Abstract

A tutorial method for teaching the scoring of open-ended questions holistically includes displaying a student response to a scorer and permitting the scorer to access a rubric containing the rules for scoring that response. The scorer can choose a display form from a handwritten form and a typed text form that retains and originally present errors. Following the scorer'"'"'s having entered a score, a model score is displayed so that a scoring efficacy may be determined. Annotations prepared by expert scorers may be accessed to enhance the learning process. In addition, a running correlation between the model and entered scores is calculated and displayed for the scorer over a tutorial session that includes attempts at scoring different responses. The system includes a processor, a workstation, and software for performing the above-described method.

Claims

1. A system for use by a scorer to learn to holistically score an answer to an open-ended question comprising:

a display device and a scorer input device;

a database housing a plurality of model responses to an open-ended question, a model response for each score level, the database further housing a scoring rubric for the question;

a processor in signal communication with the display device, the input device, and the database;

software resident on the processor having code segments adapted to;

display a scoring practice module to the scorer;

receive an election from the scorer to engage in the scoring practice module;

receive a choice from the scorer of a text presentation display mode selected from an original handwritten form and a typed text form retaining original errors;

display to the scorer a model student response to an open-ended question;

receive a choice from the scorer to access a scoring rubric for the question, and, if the choice is made, to display the scoring rubric to the scorer; and

receive from the scorer via the input device a score.

2. The system recited in claim 1, wherein the software further comprises a code segment adapted to display a model score to the scorer for permitting the scorer to assess a scoring efficacy based upon a comparison between the entered score and the model score.

3. The system recited in claim 2, wherein the software further comprises a code segment adapted to, following the receipt of a plurality of scores, display a summary table of the entered scores and the model scores.

4. The system recited in claim 3, wherein the software further comprises a code segment adapted to calculate and to include in the summary table a summary scoring efficacy comprising a percentage of agreement between the model scores and the respective entered scores.

5. The system recited in claim 1, wherein the database further houses a tutorial overview, and wherein the software further comprises code segments adapted to receive from the scorer a selection to view the tutorial overview prior to the display of the scoring practice module and to display the tutorial overview to the scorer.

6. The system recited in claim 1, wherein the software further comprises code segments adapted to, prior to receiving the score, receive from the scorer an election to view one of a plurality of model papers, each model paper associated with a score assigned thereto, and to display the model paper to the scorer.

7. The method recited in claim 6, wherein the software further comprises code segments adapted to receive from the scorer an election to view annotations made by an expert scorer on the viewed model paper.

8. The system recited in claim 1, wherein the scoring rubric comprises a plurality of scoring rubrics, one scoring rubric for each score level.

9. The system recited in claim 8, wherein each scoring rubric comprises a plurality of features preferably possessed by an answer at the associated score level.