Method of distribution of digitized materials and control of scoring for open-ended assessments

• US 6,256,399 B1
• Filed: 09/27/1999
• Issued: 07/03/2001
• Est. Priority Date: 07/08/1992
• Status: Expired due to Term

• Current Assignee: NCS Pearson Incorporated (Pearson plc)
• Original Assignee: NCS Pearson Incorporated (Pearson plc)
• Inventors: Poor, David D. S.
• Primary Examiner(s): PATEL, JAYANTI K
• Application Number:
  US09/405,953
• Time in Patent Office: 645 Days
• Field of Search: 382/100, 382/305, 434/322-327, 434/350-363
• US Class Current: 382/100
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 7/066 with answer indicating cards, blocks

Abstract

Methods are disclosed for distributing digitized electronic representations of representative responses to non-objective assessment tasks and established scores for the responses to readers in order to train the readers to determine scores for responses in accordance with a scoring criteria, to qualify the readers to determine scores for responses in accordance with the scoring criteria, and to monitor how well a reader is determining scores for such responses in accordance with the scoring criteria. In disclosed embodiments, the methods provide such distribution for multiple scorers on a network located at a single site, for multiple scorers on a network remote from a central site; and scoring by individual scorers remote from the central site. Distribution may be accomplished by transferring data from electronic files using diverse techniques such as magnetic storage discs, CD-ROM, cable or high-speed public communications network, in accordance with the needs of the particular job and setting.

Claims

1. A method of distributing digitized electronic representations of responses to non-objective assessment tasks to a reader to train the reader to determine scores for such responses in accordance with a scoring criteria, comprising the steps of:

providing a criteria for scoring responses to a non-objective assessment task;

providing a batch of representative responses to the non-objective assessment task;

providing an established score for each of the representative responses in accordance with the scoring criteria;

creating a digitized electronic representation of said responses;

storing the digitized representations of the responses in an electronic file;

storing the established scores for the responses in an electronic file;

providing a reader station having a display screen for displaying digitized information to a reader;

distributing the digitized representations of responses and the established scores to the reader station;

training a reader to score in accordance with the scoring criteria by the substeps of;

(a) displaying digitized representations of responses to the reader on the display screen;

(b) recording the score assigned by the reader to each response in the batch;

(c) providing to the reader the established scores for the responses for comparison to the reader'"'"'s scores;

(d) repeating training steps (a) through (c) with different digitized representations of responses until the reader has demonstrated an ability to assign scores corresponding to the established scores with an acceptable consistency.

2. A method as in claim 1, wherein the step of providing the established score to the reader is accomplished by displaying the established score on the display screen following the step of recording the score assigned by the reader.

3. A method as in claim 1, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a magnetic storage disc, delivering the disc to the scorer station, and reading digitized information from the disc to be displayed on the display screen.

4. A method as in claim 1, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a CD-ROM storage device, delivering the CD-ROM to the scorer station, and reading digitized information from the CD-ROM to be displayed on the display screen.

5. A method as in claim 1, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes connecting the scorer station as a node on a network, storing the digitized representations and scores on a server for the network server, and serving digitized information from the server to the scorer station to be displayed on the display screen.

6. A method as in claim 1, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via modem over a public telecommunications network to the scorer station.

7. A method as in claim 1, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via private high-speed data line to the scorer station.

8. A method of distributing digitized electronic representations of responses to non-objective assessment tasks to a reader to qualify the reader to determine scores for such responses in accordance with a scoring criteria, comprising the steps of:

providing a scoring criteria for scoring responses to a non-objective assessment task;

providing a set of representative responses to the non-objective assessment task;

providing an established score to each of the representative responses in accordance with the scoring criteria;

creating a digitized electronic representation of said responses;

storing the digitized representations of responses in an electronic file;

storing the established scores for the responses in an electronic file;

providing a reader station having a display screen for displaying digitized information to a reader;

distributing the digitized representations of responses and the established scores to the reader station;

qualifying a reader to score in accordance with the scoring criteria by the substeps of;

(a) displaying digitized representation of representative responses to the reader on the display screen;

(b) recording a score assigned by the reader to each response;

(c) comparing the scores assigned by the reader to the established scores for the responses.

9. A method as in claim 8, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a magnetic storage disc, delivering the disc to the scorer station, and reading digitized information from the disc to be displayed on the display screen.

10. A method as in claim 8, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a CD-ROM storage device, delivering the CD-ROM to the scorer station, and reading digitized information from the CD-ROM to be displayed on the display screen.

11. A method as in claim 8, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes connecting the scorer station as a node on a network, storing the digitized representations and scores on a server for the network server, and serving digitized information from the server to the scorer station to be displayed on the display screen.

12. A method as in claim 8, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via modem over a public telecommunications network to the scorer station.

13. A method as in claim 8, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via private high-speed data line to the scorer station.

14. A method of distributing digitized electronic representations of responses to non-objective assessment tasks to a reader to monitor how well the reader is determining scores for such responses in accordance with a scoring criteria, comprising the steps of:

providing a scoring criteria for scoring responses to a non-objective assessment task;

providing a set of representative responses to the non-objective assessment task;

providing an established score for each of the representative responses in accordance with the scoring criteria;

creating a digitized electronic representation of said responses;

storing the digitized representations of responses in an electronic file;

storing the established scores for the responses in an electronic file;

providing a reader station having a display screen for displaying digitized information to a reader;

distributing the digitized representations of the representative responses and the established scores to the reader station;

displaying representative responses interspersed into a sequence of responses on the display screen;

recording a score assigned by the reader to the responses;

monitoring how well the reader determines scores in accordance with the scoring criteria by comparing the score assigned by the reader to the established score for the representative responses.

15. A method as in claim 14, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a magnetic storage disc, delivering the disc to the scorer station, and reading digitized information from the disc to be displayed on the display screen.

16. A method as in claim 14, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes storing the digitized representations and scores on a CD-ROM storage device, delivering the CD-ROM to the scorer station, and reading digitized information from the CD-ROM to be displayed on the display screen.

17. A method as in claim 14, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes connecting the scorer station as a node on a network, storing the digitized representations and scores on a server for the network server, and serving digitized information from the server to the scorer station to be displayed on the display screen.

18. A method as in claim 14, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via modem over a public telecommunications network to the scorer station.

19. A method as in claim 14, wherein the step of distributing the digitized representations of responses and established scores to the scorer station includes transmitting the digitized representations and scores via private high-speed data line to the scorer station.