System and method for scoring test answer sheets having open-ended questions

• US 6,311,040 B1
• Filed: 07/31/1997
• Issued: 10/30/2001
• Est. Priority Date: 07/31/1997
• Status: Expired due to Term

• Current Assignee: Harcourt Assessment Incorporated (Pearson plc)
• Original Assignee: Psychological Corp.
• Inventors: Sonnenschein, Joan, Gonzalez, Jose, Kucinski, Bernard
• Primary Examiner(s): Martin-Wallace, Valencia
• Assistant Examiner(s): Harris, Chanda
• Application Number:
  US08/903,753
• Time in Patent Office: 1,552 Days
• Field of Search: 434/322, 434/350
• US Class Current: 434/350
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G09B 7/066 with answer indicating cards, blocks

Abstract

A scoring system and method include a distribution application for delivering an answer image to a qualified reader. The distributor prefetches a batch of answer images from long-term storage and caches them for retrieval and scoring. The scoring process includes displaying an answer based upon an answer image form along with a scoring protocol. Geometric tools are also available for assisting in scoring geometry tests. A routing system permits a reader to electronically pose a query to a supervisor and to continue working on another answer while the query is being addressed. A reader'"'"'s effectiveness and efficiency can be monitored by distributing a calibration answer to the reader in transparent fashion and comparing the resulting score with a target score.

Claims

1. A method for distributing an answer pages to a reader for scoring, the method comprising the steps of:

fetching a batch of visual images of answer pages containing answer images to a test question from a storage device, the answer image at least partially contained within a sector on the answer page;

electronically determining a qualification required of a scorer for the answer batch;

receiving an electronic request from a scorer workstation for an answer to be scored;

electronically determining if the scorer is qualified to score the fetched answer batch;

if the scorer is qualified, electronically delivering an answer image from the fetched answer batch to the scorer workstation for scoring; and

permitting the scorer to access another sector on the answer page not contained within the current sector of the answer page.

2. The method recited in claim 1, further comprising the step, following the fetching step, of storing the fetched answer images in a temporary storage device.

3. The method recited in claim 1, wherein the determining step comprises comparing a set of reader qualifications with a set of required qualifications for scoring the answer to the test question.

4. The method recited in claim 1, further comprising the steps, following the delivering step, of receiving a score from the reader on the answer and storing the score.

5. The method recited in claim 4, further comprising the steps, following the score storing step, of:

determining whether additional answer images remain in the batch for scoring; and

if additional answer images remain, electronically delivering a second answer from the fetched answer batch to the scorer workstation for scoring.

6. The method recited in claim 4, further comprising the step, following the score receiving step, of maintaining workflow statistics on the reader.

7. The method recited in claim 1, further comprising the step, preceding the fetching step, of selecting a batch of answer images based upon a set of predetermined criteria, and the fetching step comprises fetching the selected batch of answer images.

8. A system for providing a test answer image to a reader for scoring, the system comprising:

a server having stored therein;

a visual image of an answer page containing an answer to a test question at least partially within a first sector on the answer page; and

a qualification criterion for scoring the answer;

means for electronically transmitting a request for an answer image from a reader to the server;

means for determining whether the reader making the request meets the qualification criterion;

if the reader making the request meets the qualification criterion, means for electronically routing an answer image to the reader for scoring; and

means for permitting the reader to access another sector on the answer page in spaced relation from the first sector and not contained within the current sector of the answer page.

9. The system recited in claim 8, further comprising means for storing an answer image on a long-term basis and means for transferring the answer image from the long-term storage means to the server.

10. The system recited in claim 8, wherein the server further has stored therein an answer image form delineating a layout of the answer image and comprises means for formatting the answer image based upon the answer image form for creating a desired display format upon a reader display means.

11. The system recited in claim 10, wherein the server further has stored therein a scoring criterion for the answer and wherein the formatting means further comprises means for creating an interactive scoring element upon the reader display means simultaneously with the desired display format.

12. The system recited in claim 10, wherein the formatting means comprises means for displaying a selected area of the answer image.

13. A method for displaying a test answer page comprising at least one answer to an open-ended question for scoring, the method comprising the steps of:

providing a visual image of an answer page containing an S answer to an open-ended question;

providing a forms database containing a layout of the page, the layout including a location of a sector on the page containing a space for the answer to the open-ended question;

locating the answer sector on the page image by referencing the forms database;

formatting the page image to initially display the answer sector to a reader; and

permitting access to another sector on the page outside the answer sector.

14. The method recited in claim 13, further comprising the steps of:

providing a scoring protocol for the answer;

displaying score entry indicia to the reader corresponding to the scoring protocol.

15. The method recited in claim 13, wherein the image providing step comprises the steps of:

fetching the image from a long-term storage unit; and

temporarily storing the image in a server having faster response characteristics than the long-term storage unit.

16. The method recited in claim 13, wherein the answer comprises a geometrical image, and further comprising the steps of:

displaying a geometric measurement tool superimposed upon the visual image; and

manipulating the geometric measurement tool to score the answer.

17. A system for displaying a test answer page comprising at least one answer to an open-ended question for scoring, the system comprising:

a server having resident therein;

a visual image of an answer page containing an answer to an open-ended question; and

a forms database containing a layout of the page, the layout including a location of a sector on the page containing a space for the answer to the open-ended question;

software means for locating the answer sector on the page image by referencing the forms database;

software means for formatting the page image to initially display the answer sector to a reader; and

display means having means for permitting access to another sector on the page outside the answer sector.

18. The system recited in claim 17, wherein the server further has resident therein a scoring protocol for the answer, and further comprising means for displaying score entry indicia to the reader corresponding to the scoring protocol.

19. A system for displaying a test answer page comprising at least one answer to an open-ended question for scoring, the system comprising:

a server having resident therein;

a visual image of an answer page containing an answer to an open-ended question, comprising a geometrical image;

a forms database containing a layout of the page, the layout including a location of a sector on the page containing a space for the answer to the open-ended question; and

a scoring protocol for the answer;

software means for locating the answer sector on the page image by referencing the forms database;

software means for formatting the page image to initially display the answer sector to a reader;

means for displaying score entry indicia to the reader corresponding to the scoring protocol;

display means having means for permitting access to another sector on the page outside the answer sector;

software means for displaying a geometric measurement tool superimposed upon the visual image; and

means for manipulating the geometric measurement tool to score the answer.