Collaborative and quality control scoring system

• US 5,466,159 A
• Filed: 08/12/1994
• Issued: 11/14/1995
• Est. Priority Date: 02/05/1993
• Status: Expired due to Term

• Current Assignee: NCS Pearson Incorporated (Pearson plc)
• Original Assignee: National Computer Systems, Inc. (Pearson plc)
• Inventors: Clark, Jay V., Moyer, Judith, Bramlett, William L. Jr.
• Primary Examiner(s): Cheng, Joe H.
• Application Number:
  US08/290,019
• Time in Patent Office: 459 Days
• Field of Search: 434/156, 434/157, 434/169, 434/185, 434/307 R, 434/308, 434/322-365, 434/118, 364/419.1, 382/10, 382/48, 382/59, 358/444, 273/273, 273/430, 379/88-97
• US Class Current: 434/322
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 7/066 with answer indicating cards, blocks
• CPC Class Codes: Y10S 706/927 Education or instruction

Abstract

A system for managing resolvers of test answers to maintain consistent and efficient scoring of test answers. Collaborative scoring allows two test resolvers to score the same test answer and resolves potential discrepancies in the scores provided by the resolvers. Quality items monitor a test resolver'"'"'s scoring performance against a known standard. The system transmits a quality-control test item, having a known score, to a test resolver. After receiving a score from the resolver, the system compares the known score with the resolver'"'"'s score and can generate a quality profile for the resolver.

Claims

1. A system for verifying test answers between two or more test resolvers, the system comprising:

a) first receive means for electronically receiving an answer to a test question, the answer comprising an electronic representation of at least a portion of a test answer sheet;

b) transmit means for electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers;

c) second receive means for electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and

d) verification means for electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.

2. The system of claim 1 further comprising display means for electronically displaying the test question to the first and second test resolvers at the first and second test resolver terminals.

3. The system of claim 1 wherein the verification means further comprises collaboration means for providing an electronic communication link between the first and second test resolver terminals and, if the first and second test scores contain the discrepancy based on the predefined criteria, for signalling the first and second test resolvers to interactively determine and enter a joint score using the electronic communication link, the verification means further comprising recording means for recording the joint score.

4. The system of claim 3 wherein the collaboration means further comprises means for tracking average scores of the first and second test resolvers and for preventing the first and second test resolvers from scoring a same test item if the first and second test resolvers each have average scores within predefined scoring criteria.

5. The system of claim 3 wherein the verification means further comprises means for preventing the first and second test resolver terminals from receiving another answer to be scored until the first and second test resolvers have entered the joint score.

6. The system of claim 3 wherein the verification means further comprises means for transmitting the test questions to a third test resolver terminal to be scored by a third test resolver, if the first and second test resolvers do not enter the joint score within a predetermined time period.

7. The system of claim 1 wherein the verification means further comprises means for transmitting the answer to a third test resolver terminal to be scored by a third test resolver, if the first and second test scores contain the discrepancy based on the predefined criteria, and for recording a third test score, as determined by the third test resolver, received from the third test resolver terminal.

8. The system of claim 7 wherein the means for transmitting the answer to a third test resolver terminal further comprises means for providing the third test resolver terminal with the first and second test scores.

9. The system of claim 7 wherein the means for transmitting the answer to a third test resolver terminal further comprises means for preventing the third test resolver from viewing the first and second test scores on the third test resolver terminal before the third test resolver records the third test score.

10. The system of claim 1, wherein the verification means further comprises means for electronically linking the first and second test resolver terminals when the first and second test resolvers are located in different geographical locations.

11. The system of claim 1 wherein the verification means comprises means for selectively recording, as the value, a greater of the first and second test scores, a lower of the first and second test scores, and an average of the first and second test scores.

12. A system for evaluating the performance of test resolvers by routing quality-control questions for scoring to test resolvers, the quality-control questions comprising a test answer with a score known to quality-control personnel, the system comprising:

a) transmit means for electronically transmitting to a test resolver terminal a quality-control question to be scored by a test resolver, the quality-control question comprising a test answer with a score known to quality-control personnel;

b) receive means for electronically receiving from the test resolver terminal a resolver score, as determined by the test resolver, for the quality-control question; and

c) processing means for electronically comparing the score known to the quality-control personnel and the resolver score and for reporting a discrepancy based on predetermined criteria between the known score and the resolver score.