Unified web-based system for the delivery, scoring, and reporting of on-line and paper-based assessments

• US 20060003306A1
• Filed: 07/02/2004
• Published: 01/05/2006
• Est. Priority Date: 07/02/2004
• Status: Abandoned Application

• Current Assignee: Vantage Technologies Knowledge Assessment LLC
• Original Assignee: Vantage Technologies Knowledge Assessment LLC
• Inventors: Elliott, Scott M., McGinley, Michael P., Maguire, Brian C., Patrylak, Robert, Allen, George T. III
• Application Number:
  US10/884,404
• Publication Number:
  US 20060003306A1
• Time in Patent Office: Days
• US Class Current: 434/350
• CPC Class Codes: G09B 3/00 Manually or mechanically operated teaching appliances working with questions and answers electrically-operated G09B7/00; e.g. G09B7/063 advertising or displaying in general G09F

Abstract

A hardware and software implemented, standardize testing, scoring and reporting system has a network interface, being either LAN or the Internet. A test is selected from a library, assigned to a group of examinees, and a record is made of the assignment. The test is available on-line, or downloaded, or both. Downloaded tests are saved as PDF files and printed into booklets. Examinees record answers on bubble sheets, which are scanned at the customer site or shipped to a processing plant. Individual scanned answer sheets provided data for individual scanned records. Scanning software performs an initial auto-correct and then performs an initial check for errors in scanned records. Tests are batched for later inspection. Scanned records with errors are classified as invalid and the test papers are separated. Prompts are generated for a human test paper inspection and scanned record editing. Accepted batches of scanned records are sent to a scanned records server where a second scanned records error check is performed. Prompts are sent to the appropriate scanning station for further human editing of invalid scanned records. Once all records are valid they are scored and the test results stored in a database, which is accessed by a reporting subsystem associated with an on-line server.

Claims

1. A unified on-line and paper, web-based testing and reporting system, comprising:

at least one on-line assessment station;

at least one on-line paper test downloading and scanned test record editing station;

at least one production line test record scanning and editing station; and

a server connected via the Internet to each said assessment station, each said scanned test record editing station, and via a LAN to each said production line scanning and editing station, for accepting scanned and edited test records, for scoring said accepted test records, and storing assessments of said accepted test records;

wherein said server makes reports of said stored assessments selectively available on the Internet.

2. A unified assessment testing and reporting system for distributed sources, comprising:

at least one test paper scanning workstation for providing digitized records of each test answer sheet scanned into a scanned test record, said scanning workstation having software for selectively editing each said scanned test record;

a server connected via a communication connection to each said scanning workstation, for accepting scanned and edited test records, for scoring said accepted test records, and storing said scored assessment test records;

wherein said server makes reports of said stored assessment test records selectively available via a communication connection.

3. A unified assessment testing and reporting system for distributed sources, comprising:

at least one test paper computerized scanning workstation for providing digitized records of each test answer sheet scanned into as a scanned test record, said scanning workstation being operated by an operator;

software resident in said computerized scanning workstation for error checking each said scanned test record for errors and for prompting said operator for editing each error containing scanned test record;

a server connected via a communication connection to each said computerized scanning workstation, for accepting scanned and edited test records, for scoring said accepted test records, and storing said scored assessment test records;

wherein said server makes reports of said stored assessment test records selectively available via a communication connection.

4. A unified test delivery, scoring and reporting system, comprising:

a server connected on-line to the Internet, said server accepting digitized test result records and analyzing each said test result record for editing said record as necessary;

at least one on-line testing station, for testing an on-line examinee, said on-line testing station providing test result records to said server via the Internet;

wherein said server analyzes each said on-line testing station test result record for conformity, and prompts the examinee when there is non-conformity, and wherein after said responses conform, said server scores said response and generates a database of score assessments;

a least one on-line editing station, being operable by an operator, said on-line editing station delivering digitized test result records to said server via the Internet;

wherein said server analyzes each delivered test result record from each said on-line editing station and prompts said operator for editing said test result records; and

wherein after a said test results is successfully edited said server scores said test results and generates a database of score assessments;

wherein said sever makes reports available on the Internet of score assessments in said database.

5. A unified system for reporting on-line and paper based assessments, comprising:

a server;

a plurality of dispersed editing workstations each having an operator, said workstations being connected to said server and each including means for digitizing assessment test records, and for displaying said digitized assessment test records to said operator;

software associated with at least some of said plurality of editing workstations for analyzing said digitized assessment test records, and for prompting said operator for editing said test records at said workstation; and

software associated with said server, for analyzing said digitized assessment test records on at least a portion of said plurality of said editing workstations, for prompting for human editing thereof at said workstation.

6. A system for digitizing paper based assessments, comprising:

an OMR scanner for scanning answer sheets and having at least two output trays and a gate for directing scanned answer sheet paper to either tray;

an OMR reading head on said scanner for scanning answer sheets for generating digitized output as a function of scanned marks on said scanned answer sheets, said digitized output defining a scanned test record for each scanned answer sheet;

a control circuit connected to said scanner for controlling the gating operation of said scanner;

a stored software template associated with said control circuit defining the format, layout and configuration of the scanned answer sheet;

stored software associated with said control circuit including stored conformity data for each said scanned record, said software reading each said scanned test record and comparing it to said software template and said conformity data and directing said scanner to gate each said scanned answer sheet to an appropriate accepted or rejected output tray.

7. A system for digitizing paper based assessments into records with reduced errors, comprising:

means for scanning said paper based assessment and creating a scanned test record from each said scanned assessment, each said scanned test record comprising as a plurality of data fields;

means connected to the output of said scanning for auto-correcting certain errors detected in said data fields of each scanned record;

means connected to the output of said scanning for analyzing said scanned test records for irregularities, and prompting a human editor for editing said irregularities in said scanned test records.

8. A unified system for processing digitized assessment records from a distributed network, comprising:

a server connected to the Internet;

relational database software resident within said server;

web page software resident within said server; and

error checking and prompting software resident within said server;

wherein said server receives digitized assessment records, determines if there is an apparent error in any record and sends an editing prompt to the sender of said records.

9. An assessment workstation, comprising:

a computer connected to the Internet or a LAN having an output therefrom and operator input thereto;

communication software on said computer;

a scanner connected to said computer;

a printer connected to said computer;

error checking software on said computer;

wherein said computer is capable of downloading an assessment test and printing same of an examinee to take;

wherein said scanner is capable of scanning an examinee assessment test answer sheet to provide a scanned test record to said computer; and

wherein said error checking software checks each scanned test record provided from said scanner and provides an editing prompt to said computer operator output.

10. An assessment workstation, comprising:

a computer connected to a communication network having an output therefrom and operator input thereto;

communication software resident in said computer;

a scanner connected to said computer;

error checking software resident in said computer;

wherein said scanner is capable of scanning an examinee assessment test answer sheet to provide a scanned test record to said computer; and

wherein said error checking software in said computer checks each scanned test record provided from said scanner and provides a status of the condition of said scanned test record.

11. A method of providing, over a communications network, assessment test results derived from scanned test records, comprising the steps of:

receiving scanned test records;

reading each said scanned test record and auto-correcting data each said record;

initial checking each scanned test record for errors in data entries thereof and providing a editing prompt for each detected error thereof;

editing of data entry errors prompted;

additional checking each scanned test record for additional errors in data entries and providing a prompt for additional editing thereof;

editing of additional errors prompted;

scoring each scanned test record free of errors; and

providing a report being an assessment of test results.

20. The method of claim 11, wherein said scanned test records are received over a communications network from a customer.

21. The method of claim 20, wherein said communications network is the Internet.

22. The method of claim 11, wherein said received scanned test records are a received from a scanner which has scanned bubble sheets, one at a time, whereof said scanned records are recorded in a database.

23. The method of claim 22, wherein said reading and auto-correcting step is conducted on each scanned test record as said bubble sheet passes through the scanner.

24. The method of claim 23, wherein said initial checking each scanned test record and providing an editing prompt is conducted as said bubble sheet passes through the scanner.

25. The method of claim 24, wherein providing an editing prompt includes providing a spreadsheet of scanned test records with detected errors visually indicated.

26. The method of claim 25, wherein said provided spread sheet has rows and columns, wherein a scanned test record appears in a row and a data field is defined by a column, and wherein said visually indicated error is a data cell which with a visual indicator.

12. A method of providing a unified web-based delivery, scoring and reporting of test assessments, comprising the steps of:

establishing a digital library of assessment tests;

establishing a database of customer information;

receiving an on-line customer request for a test;

constructing a test menu from log-on information, customer information and assessment test library information;

receiving a customer test selection and assignment information;

making the selected test available for on-line assessment to an assigned logged-on examinee;

performing data error checking for each examinee entry and prompting for necessary changes;

thereafter providing scoring and assessment data; and

reporting assessment data to each logged-on requester.

13. A method of providing a unified web-based delivery, scoring and reporting of paper based test assessments, comprising the steps of:

providing an on-line test a customer;

receiving paper bubble sheets of test answers for said selected test;

scanning said bubble sheets into memory as scanned test records;

detecting irregularities in the scanned test records;

prompting an operator to correct the detected irregularities;

scoring each test record which is free of errors; and

providing on-line reporting of each scored test record.

34. The method of claim 13, wherein the step of providing an on line-test to a customer includes the steps of:

establishing a digital library of assessment tests and/or test modules;

establishing a database of customer information;

receiving an on-line customer request for a test;

constructing a test menu from said log-on information, customer database information and digital library information;

receiving a customer test selection information; and

downloading a selected test to said customer.

35. The method of claim 13, wherein the steps of detecting irregularities in the scanned test records and prompting an operator to correct the detected irregularities, includes the steps of:

auto-correcting said irregularities detected on the scanned test record;

checking for initial errors in the data in each scanned test record;

prompting an operator for editing the detected initial errors;

checking for additional errors in the data in each scanned test record after said initial errors are corrected; and

prompting an operator for editing the detected additional errors.

36. The method of claim 13, wherein the step of scoring each test record free of errors also includes the steps thereafter of entering the error free test record into a database, and entering the test score into a database.

37. The method of claim 36, wherein the step of providing on-line reporting of each scored test record also includes:

assembling a series of on-line spread sheet reports at different population levels having results assembled from said test record and test score databases;

presenting said series of on-line spread sheet reports sequentially as a next spreadsheet is selected from a previous one.

38. The method of claim 37 wherein the different population levels includes a class of examinees.

39. The method of claim 38 wherein the different population levels includes an individual examinee.

14. A method of providing a unified, distributed network, web-based delivery, scoring and reporting of on-line and paper based assessments, comprising the steps of:

establishing a relational on-line database for test result assessments, customer information, examinee information, and tests;

distributing assessment tests on-line to requesting work stations;

receiving, editing, prompting and scoring on-line examinee assessments and storing test results in said relational database;

receiving paper based assessments and scanning same to produce digitized scanned assessment records;

checking each scanned assessment record for initial errors and providing editing prompts related to each;

additionally checking each initially edited scanned assessment record for additional errors and providing editing prompts related to each erroneous record;

scoring each record which is free of or edited free of detected errors;

storing assessments of each record scored in said relational database; and

providing on-line reports to customer requests.

40. The method of claim 14 wherein before the step of checking each scanned assessment record for initial errors and providing editing prompts related to each, there is included the steps of detecting irregularities in the scanned assessment records, and auto-correcting said detected irregularities in each scanned assessment record.

41. The method of claim 40, wherein the steps of detecting and auto-correcting irregularities in each scanned assessment record is conducted as said paper based assessment is scanned.

42. The method of claim 14, wherein the step of checking each scanned assessment record of initial errors and for providing editing prompts related to each includes the step of:

providing a spreadsheet of checked, additionally scanned assessment records with respective records containing additionally detected errors visually indicated.

43. The method of claim 42, wherein the step of additionally checking each initially edited scanned assessment record for additional errors and providing editing prompts related to each erroneous record includes the step of:

providing a spreadsheet of additionally checked, scanned assessment records where in each row is the scanned record is reported on individual examinee; and

visually indicating a row containing a detected additional error.

44. The method of claim 43, wherein the visual indication of detected errors includes the steps of providing the spreadsheet with each row being an individual canned assessment record and a column being a defined data field, whereof a cell reports respective data for a respective examinee, and providing highlighting on a cell containing detected erroneous data.

45. The method of claim 44, wherein the visual indication of detected errors also includes providing a status of each respective scanned record row.

46. The method of claim 45, wherein the status is a word indicator.

47. The method of claim 45, wherein the visual indication of additionally detected errors includes the steps of highlighting the row containing a respective additionally detected error, and providing a word prompt defining the additionally detected error.

15. A method of providing a unified, distributed network, web-based delivery, scoring and reporting of on-line and paper based assessments, comprising the steps of:

establishing a relational on-line database for test result assessments, customer information, examinee information, and tests;

distributing assessment tests on-line to requesting work stations;

receiving, editing, prompting and scoring on-line examinee assessments as and storing test results in said relational database;

receiving paper based assessments and scanning same to produce digitized scanned assessment records;

checking each scanned assessment record for initial errors and providing editing prompts related to each;

additionally checking each initially edited scanned assessment record for additional errors and providing additional editing prompts related to each erroneous record;

scoring each scanned assessment record which is free of or edited free of detected errors and storing test results in said relational database;

printing a distributed test at a customer site;

scanning paper based assessments at a customer site to provide digitized scanned assessment records;

error checking each scanned assessment record at said customer site for initial errors and providing editing prompts related to each;

receiving uploaded scanned assessment records and checking each for additional errors and providing additional editing prompts related to each erroneous record;

receiving scanned assessment records which are free of or edited free of errors and scoring each record which is free of errors or edited free of errors and storing test results in said relational database; and

reporting assessments, on-line, from said relational database in response to customer requests.

48. The method of claim 15, wherein before the step of checking each scanned record and providing editing prompts is included the step of auto-correcting each scanned record for detected irregularities, and wherein before the step of error checking each scanned assessment record and providing editing prompts is included the step of auto-correcting each scanned record for detected irregularities.

49. The method of claim 48, wherein the step of checking each scanned record and providing editing prompts and wherein the step of error checking each scanned assessment record and providing editing prompts each include the step of:

providing a spreadsheet of additionally checked, scanned assessment records where in each row is the scanned record is reported on individual examinee; and

visually indicating a row containing a detected additional error.

50. The method of claim 49, wherein the visual indication of detected errors includes the steps of providing the spreadsheet with each row being an individual canned assessment record and a column being a defined data field, whereof a cell reports respective data for a respective examinee, and providing highlighting on a cell containing detected erroneous data.

51. The method of claim 50, wherein the visual indication of detected errors also includes providing a status of each respective scanned record row.

52. The method of claim 51, wherein the status is a word indicator.

53. The method of claim 48, wherein the step of additionally checking each initially scanned record and providing editing prompts, and the step of receiving uploaded scanned assessment records and additionally checking each record and providing additional editing prompts, each include the steps of:

highlighting the row containing a respective additionally detected error; and

providing a word prompt defining the additionally detected error.

16. A method of scanning and editing assessment tests for providing digitized scanned records suitable for use by a delivery, scoring and reporting system, comprising the steps of:

obtaining a plurality of paper based test answer papers;

scanning each said test answer paper to generate a scanned record;

auto-correcting each scanned record for any easily resolved errors detected;

checking each auto-corrected scanned record for errors in field data;

separating each test answer paper for which field data errors are detected;

prompting an operator for editing field data errors from said separated test answer paper;

checking each scanned record for absence of field data errors and thereupon checking each scanned record for test assignment related errors;

prompting an operator for editing test assignment related errors; and

delivering each test record which is free of errors or edited free of error, in a digitized format, to a system capably of on-line delivery, scoring and reporting.

17. A web-based method of using a computer for editing errors from scanned test records, comprising the steps of:

presenting a report of the scanned test records;

visually indicating a individual scanned test record in which an error is present, including visually indicating each error present therein;

additionally presenting a report of additionally scanned test record; and

visually indicating an individual scanned test record in which an additional error is present.

54. The web-based method of claim 17, wherein after the step of visually indicating is included the additional steps of:

additionally presenting a report of additionally scanned test record; and

visually indicating an individual additionally scanned test record in which an additional error is present.

55. The web-based method of claim 54, wherein the step of visually presenting a report of the scanned test records includes the step of presenting a spreadsheet-like presentation of individual scanned test records with fields of data.

56. The web-base method of claim 55, wherein the step of visually indicating an individual scanned test record in which an error is present includes the step of visually indicating each respective error present in each record.

57. The web-based method of claim 56, wherein the step of visually indicating each individual additionally scanned test in which an additional error is present includes the step of providing a word prompt of each additional error.

58. The web-based method of claim 57, wherein the visual indication of each error appears on the provided spreadsheet-like presentation.

59. The web-based method of claim 58, wherein the step of providing a word prompt for each additional error includes the step of providing a word prompt on the provided spreadsheet-like presentation.

60. The web-based method of claim 59 also including the step of removing a visual indication and changing a word prompt when a respective error or additional error is edited out of a respective record.

18. A method using a computer for editing errors from scanned test records, comprising the steps of:

presenting a spreadsheet of the scanned test results of a group of interest with each row representing a subgroup;

visually indicating a row in which scanned test record errors appear;

selecting a visually indicated row to bring up a further spreadsheet of scanned test results of the subgroup of interest;

repeating the presenting, visually indicating and selecting steps until the spreadsheet is a group of individual examinees, with each row in that spreadsheet being the scanned test record of an examinee;

visually indicating a cell in the examinee spreadsheet which contains an error; and

editing the error thereby removing the visual indication.

19. The method of claim 18, also including repeating all the steps until all spreadsheet are clear of visual indications.

28. The method of claim 27, wherein said editing of data entry errors prompted includes the steps of:

inspecting the respective bubble sheet entries of the examinee for which visually indicated cell appears;

determining the correct data entry;

computer selecting the erroneous cell entry; and

replacing the selected erroneous entry with a correct entry.

29. The method of claim 28, wherein the additional checking and prompting for addition editing includes:

visually indicating the row representing the entire scanned record of an examinee in which an additional error is detected;

word prompting on the type of error;

determining the correct entry from the word prompt; and

replacing the incorrect entry with a correct entry.

30. The method of claim 29, wherein the visually indicating of a data cell is highlighting said data cell.

31. The method of claim 30, wherein the checking and prompting visual indication of detected errors also includes providing a status of each record.

32. The method of claim 31, wherein the additional checking and prompting visual indication of additional errors includes highlighting said erroneous record row.

33. The method of claim 31, wherein the additional checking and prompting visual indication of additional errors also includes providing a status of each record.