Test question response verification system
First Claim
1. In a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the method comprising:
- a. selecting from each electronic image of test question responses at least one image clip;
b. applying image recognition to generate data electronically from the at least one image clip;
c. determining whether verification is needed of the data derived from the at least one image clip;
d. matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
e. recompiling selected image clips into coherent image portions of the test question responses; and
f. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
whereby, through the transfer of the recompiled coherent image portions to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention relates to a system and method for verifying and editing optically imaged responses received from a response provider.
-
Citations
23 Claims
-
1. In a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the method comprising:
-
a. selecting from each electronic image of test question responses at least one image clip;
b. applying image recognition to generate data electronically from the at least one image clip;
c. determining whether verification is needed of the data derived from the at least one image clip;
d. matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
e. recompiling selected image clips into coherent image portions of the test question responses; and
f. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
whereby, through the transfer of the recompiled coherent image portions to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
a. electronically assigning an individual digital identifier to each of the image clips, wherein each individual digital identifier represents at least a portion of information related to the test taker or the test;
b. the step of matching image clip information to the particular skills of the human editors comprises processing the individual digital identifiers related to each coherent image portion against the particular skills of the human editors;
whereby, through the transfer of the recompiled image portions tagged with the digital codes to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses.
-
-
10. The method of claim 1, further comprising the following steps:
-
a. recompiling again selected image clips already viewed by appropriate human editors into coherent image portions of the test question response;
b. selectively presenting the coherent image portions to appropriate human editors located at the remote locations but not to the same human editors who previously viewed the coherent image portions; and
c. electronically recording at least one entry generated by the appropriate human editor for each presented coherent image portion.
-
-
11. In a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the method comprising:
-
a. selecting from each electronic image of test question responses at least one image clip;
b. applying image recognition to generate data electronically from the at least one image clip;
c. determining whether verification is needed of the data derived from the at least one image clip;
d. matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
e. recompiling selected image clips into coherent image portions of the test question responses;
f. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
g. analyzing through observation by an appropriate human editor the portion of the response disclosed by the coherent image portions;
h. electronically recording a decision as to the human editors analysis, the decision being generated by the selected human editor for each distributed coherent image portion;
whereby, through the transfer of the recompiled coherent image portions to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses.
-
-
12. In a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the method comprising:
-
a. selecting from each electronic image of test question responses at least one image clip;
b. electronically assigning an individual digital identifier to each of the image clips, wherein each individual digital identifier represents at least a portion of information related to the test taker or the test;
c. applying image recognition to generate data electronically from the at least one image clip;
d. determining whether verification is needed of the data derived from the at least one image clip;
e. processing the individual digital identifier related to each image clip against the particular skills of the human editors in order to select human editors having the particular appropriate skills;
f. recompiling selected image clips into coherent image portions of the test question response;
g. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
h. analyzing through observation by an appropriate human editor the portion of the response disclosed by the coherent image portions;
i. electronically recording a decision verifying the particular data, the decision being generated by the selected human editor for each distributed coherent image portion;
whereby, through the transfer of the recompiled coherent image portions tagged with the digital codes to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring for viewing relatively-large, unclipped electronic images of test question responses. - View Dependent Claims (13, 14)
-
-
15. A system for improving the productive use of geographically-distributed human editors having particular appropriate skills by enhancing the speed and accuracy of data within a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising
a. a server subsystem having at least one server computer; -
b. a client subsystem having at least one client computer used by human editors located at the client computers and coupled to the server subsystem;
c. the at least one client computer configured for selecting from each electronic image of test question responses at least one image clip;
d. the at least one client computer configured to apply image recognition to generate data electronically from the at least one image clip;
e. the at least one server computer configured to match image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
f. the at least one server computer configured to recompile selected image clips into coherent image portions of the test question responses; and
g. the at least one client computer configured to selectively present the coherent image portions to appropriate human editors located at the remote locations. - View Dependent Claims (16, 17, 18, 19, 20)
a. recompile again selected image clips already viewed by appropriate human editors into coherent image portions of the test question response;
b. selectively present the coherent image portions to appropriate human editors located at the remote locations but not to the same human editors who previously viewed the coherent image portions; and
c. electronically record at least one entry generated by the appropriate human editor for each presented coherent image portion.
-
-
21. A computer program product readable by a computing system for a computer process for a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the computer program product comprising encoded instructions either embedded on a computer-readable media or transmitted in a propagating wave, the encoded instructions being arranged and configured to be read by the computing system for causing the computing system to perform the steps of:
-
a. selecting from each electronic image of test question responses at least one image clip;
b. applying image recognition to generate data electronically from the at least one image clip;
c. determining whether verification is needed of the data derived from the at least one image clip;
d. matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
e. recompiling selected image clips into coherent image portions of the test question responses; and
f. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
whereby, through the transfer of the recompiled coherent image portions to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses.
-
-
22. A computer program product readable by a computing system for a computer process for a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising a server subsystem having at least one server computer and a client subsystem having one or more client computers used by human editors located at the client computers, each human editor having particular skills for examining particular test portion subject matter, a method for enhancing the speed and accuracy of data within the workflow system in order to effect productive use of geographically-distributed human editors having particular appropriate skills, the computer program product comprising encoded instructions either embedded on a computer-readable media or transmitted in a propagating wave, the encoded instructions being arranged and configured to be read by the computing system for causing the computing system to perform the steps of:
-
a. selecting from each electronic image of test question responses at least one image clip;
b. applying image recognition to generate data electronically from the at least one image clip;
c. determining whether verification is needed of the data derived from the at least one image clip;
d. matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
e. recompiling selected image clips into coherent image portions of the test question responses;
f. selectively presenting the coherent image portions to appropriate human editors located at the remote locations;
g. analyzing through observation by an appropriate human editor the portion of the response disclosed by the coherent image portions;
h. electronically recording a decision as to the human editors analysis, the decision being generated by the selected human editor for each distributed coherent image portion;
whereby, through the transfer of the recompiled coherent image portions to appropriate human editors, widely-distributed human editors of greatly varying skills can be effectively used at the remote locations without the inefficiency of transferring relatively-large, unclipped electronic images of test question responses.
-
-
23. A system for improving the productive use of geographically-distributed human editors having particular appropriate skills by enhancing the speed and accuracy of data within a computerized test processing workflow system configured for analyzing electronic images of test question responses provided by a test taker, the system comprising:
-
a. means for selecting from each electronic image of test question responses at least one image clip;
b. means for applying image recognition to generate data electronically from the at least one image clip;
c. means for matching image clip information to the particular skills of the human editors in order to select human editors having the particular appropriate skills;
d. means for recompiling selected image clips into coherent image portions of the test question responses; and
e. means for present the coherent image portions to appropriate human editors located at the remote locations.
-
Specification