System and methods for computer based testing

• US 5,827,070 A
• Filed: 06/07/1995
• Issued: 10/27/1998
• Est. Priority Date: 10/09/1992
• Status: Expired due to Term

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Kershaw, Roger C., Ward, William C. Jr., Swanson, Leonard C., Romano, Frank J.
• Primary Examiner(s): CHENG, JOE H
• Application Number:
  US08/485,628
• Time in Patent Office: 1,238 Days
• Field of Search: 434/118, 434/307 R, 434/308, 434/322, 434/323, 434/327, 434/347, 434/350, 434/353, 434/362, 395/100, 395/118, 395/144, 395/152, 395/154, 395/327, 395/500, 395/502, 395/506, 395/561, 395/752-760, 395/927, 395/101, 395/173, 395/761, 395/806, 483/1, 483/23, 483/42, 381/51
• US Class Current: 434/322
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: Y10S 706/927 Education or instruction

Abstract

The present invention provides a system and method for computer based testing. The system of the present invention comprises a test development system for producing a computerized test, a test delivery system for delivering the computerized test to an examinee, and a workstation on which the computerized test is delivered to the examinee. The method of the present invention comprises producing a computerized test, delivering the computerized test to an examinee and recording examinee responses to questions presented to the examinee during the delivery of the computerized test. A method of producing a computerized test is also provided. This method comprises preparing a test document of items, computerizing each item, preparing scripts and other test components and packaging the items, scripts and other test components together to form the computerized test. A method of delivering a computerized test is also provided in which a standardized test is created, an electronic form of the test is then prepared, the items of the test are presented to an examinee on a workstations display and the examinee'"'"'s responses are accepted and recorded. A method of administering a computerized test is further provided in which a computerized test is installed on a workstation and then the delivery of the test to an examinee is initiated. A data distribution system is further provided. The system comprises an examinee performance database and a file processing component for receiving files from the computer based testing system and updating the examinee performance database with information from these files.

Claims

1. A data distribution system for use with a computer based testing (CBT) system, said CBT system delivering at least one computerized test to at least one examinee and said data distribution system processing information related to each said delivery, wherein the CBT system supports a plurality of program-specific tests, and said CBT system having a designated post processing system for scoring each examinee'"'"'s responses for each program-specific test, each said post processing system providing a definition file defining specific information said post processing system requires and a format in which said specific information is to be provided, the data distribution system comprising:

an examinee performance database having examinee performance records indicative of responses provided by each examinee during each said delivery;

a file processing component interfaced with said examinee performance database for receiving transmission files generated by said CBT system and having data indicative of said information, said file processing component generating from said data at least one said examinee performance record for at least some of said deliveries and updating said examinee performance database with each said examinee performance record so generated; and

a format post processing component interfaced with each post processing system and to said examinee performance database, said format post processing component retrieving at least some examinee performance records from said examinee performance file database and formatting said retrieved examinee performance records based on said definition file.

2. The system of claim 1, wherein said transmission files comprise examinee performance files having data indicative of events initiated by one said examinee during each said delivery, error log files having data indicative of CBT system errors, if any, occurring during any said delivery, and security log files having data indicative of security-related events occurring during any said delivery.

3. The system of claim 2, wherein said file processing component determines whether an accurate number of examinee performance files, error log files, and security log files have been received.

4. The system of claim 1, wherein said data distribution system is located at a central processing site and at least some of said computerized tests are delivered at at least one test center located remote from said central processing site.

5. The system of claim 4, wherein modem to modem communications is available between said central processing site and said at least one test center for transmitting said transmission files from said test centers to said central processing site.

6. The system of claim 4, wherein transmission files are stored on a magnetic media at said at least one test center and transported to said central processing site.

7. The system of claim 4, further comprising:

a computer based testing network (CBTN) database interfaced with said file processing component, said file processing component generating from said data contained in said transmission files at least one computer based testing record having data used for test center control and

wherein the format post processing component is interfaced with said CBTN database, said format post processing component retrieving at least some computer based testing records from said CBTN database and formatting said retrieved records based on said definition file.

8. The system of claim 7, wherein the transmission files contain compressed data.

9. The system of claim 7, wherein said file processing component checks the data of said transmission files so received for data errors.

10. The system of claim 1, wherein said information includes data indicative of security-related events, said security-related events being recorded in at least one security log file during the delivery of each said computerized test, further comprising:

a security log database having security log records indicative of security-related events; and

a security log processing component interfaced with said file processing component, said file processing component receiving an input of said security log files and transferring said security log files to said security log processing component, said security log processing component generating security log records from said security log files and updating said security log database with said security log records so generated.

11. The system of claim 10, further comprising:

a report processing component interfaced with the examinee performance database and the security log database for retrieving examinee performance records and security log records from said databases and generating at least one report from said retrieved records.

12. The system of claim 10, wherein said data distribution system is located at a central processing site and at least some of said computerized tests are delivered at at least one test center located remote from said central processing site.

13. The system of claim 12, further comprising:

a computer based testing network (CBTN) database interfaced with said file processing component, said file processing component generating from said data contained in said transmission files at least one computer based testing record having data used for test center control.

14. The system of claim 13, further comprising:

a report processing component interfaced with said examinee performance database, said security log database and said CBTN database for retrieving examinee performance records, security log records, and computer based testing records from said respective databases and generating at least one report from said retrieved records.

15. The system of claim 14, wherein said at least one report is generated for at least one said test center and provides information indicative of activities at each test center for which said reports are so generated.

16. The system of claim 14, wherein said at least one report is generated for at least one examinee and provides information permitting said examinee to track said examinee'"'"'s performance.

17. The system of claim 14, wherein said tests are delivered on workstations located at said test centers and at least one report is generated for at least one workstation and provides information indicative of said security-related events.

18. The system of claim 1, wherein said information includes data indicative of events initiated by each said examinee during each said delivery of one said computerized test, said events being recorded in an examinee performance file during the delivery of said one computerized test, further comprising:

an examinee performance file processing component interfaced with said file processing component, said file processing component receiving an input of said examinee performance files and transferring said examinee performance files to said examinee performance file processing component, said examinee performance file processing component analyzing said data contained therein for the existence of at least one predetermined condition.

19. The system of claim 18, wherein said at least one predetermined condition includes the existence of one of said examinee performance files so received having data duplicated by at least one examinee performance record existing in said examinee performance database.

20. The system of claim 18, wherein the existence of said at least one predetermined condition is based upon whether the data in the examinee performance files is valid.

21. The system of claim 18, wherein said information includes data indicative of security-related events, said security-related events being recorded in at least one security log file during the delivery of each said computerized test, further comprising:

a security log database having security log records indicative of security-related events; and

a security log processing component interfaced with said file processing component, said file processing component receiving an input of said security log files and transferring said security log files to said security log processing component, said security log processing component analyzing said security log files for at least said one predetermined condition and defining those files having at least one said predetermined condition as a reject record;

said security log processing component generating security log records from said security log files where no said predetermined conditions exist and updating said security log database with said security log records so generated.

22. The system of claim 21, further comprising:

a rejection and resolution processing component interfaced with said examinee performance file processing component and said security log processing component, said rejection and resolution processing component receiving an input of each said examinee performance and security log files for which at least one of said predetermined conditions exists, said reject and resolution processing component altering each of those files to eliminate any existing predetermined condition associated therewith.

23. The system of claim 1, further comprising:

a report processing component interfaced with the examinee performance database for retrieving information from said examinee performance database and generating at least one report from said retrieved information.

24. A method of delivering a computerized test for standardized testing using a computer based testing (CBT) system having at least one workstation at which test questions are presented to examinees and operable to accept responses from examinees, wherein the standardized test comprises a session script defining the sequence of tasks to be performed in delivering the computerized test to an examinee by a data distribution system of the CBT system, the method comprising the steps of:

reading, by the examinee'"'"'s workstation, the session script to identify a test script defining the rules for determining a sequence of the questions to be delivered to said examinee, wherein the rules are based on a measurement model for linear and non-linear tests;

determining, by the examinee'"'"'s workstation, the sequence of the questions to be delivered as defined by the rules of said test script; and

presenting, by the data distribution system, the questions to said examinee in the same sequence at the examinee'"'"'s workstation.

25. A method of delivering a computerized test for standardized testing using a computer based testing (CBT) system having at least one workstation at which test questions are presented to examinees and operable to accept responses from examinees, wherein the standardized test comprises a session script defining the sequence of tasks to be performed in delivering the computerized test to an examinee by a data distribution system of the CBT system, the method comprising the steps of:

reading, by the examinee'"'"'s workstation, the session script to identify the delivery units to be delivered and the order in which said delivery units are to be presented at the examinee'"'"'s workstation, wherein the delivery units includes at least one of general information screens, tutorial units, break units, data collection units, scoring and reporting units, and testing units; and

invoking, by the data distribution system, each delivery units in the order specified by said session script.

26. A method of delivering a computerized test as recited in claim 25 wherein said general information screen contains information regarding the actual text and graphics that will be presented to the examinee, the type of dismissal, whether the dismissal is automatic or manual, and the time after which the dismissal should occur.

27. A method of delivering a computerized test as recited in claim 25 wherein said tutorial units contain references to a tutorial which presents test familiarization materials to the examinee including at least one of operation of a mouse, how to scroll a screen, how to use testing tools, and how to answer.

28. A method of delivering a computerized test as recited in claim 25 wherein said break units implement a scheduled break within a session.

29. A method of delivering a computerized test as recited in claim 25 wherein said data collection units obtain additional information from the examinee including at least one of demographic and debriefing information.

30. A method of delivering a computerized test as recited in claim 25 wherein said scoring and reporting units score and report the results of at least one testing unit delivered in a session.

31. A method of delivering a computerized test as recited in claim 25 wherein said testing units present a test based on the contents of a test script that may have been selected at runtime.

32. A method of delivering a computerized test as recited in claim 25 wherein said testing units contain information regarding whether dynamic runtime selection is to be used to select a test script and a reference to a test script which controls the sequence of events and options used during testing unit.

33. A method of delivering a computerized test for standardized testing using a computer based testing (CBT) system having a data distribution system and at least one workstation at which test questions are presented to examinees and operable to accept responses from examinees, the method comprising the steps of:

electronically and dynamically selecting, using an examinee'"'"'s workstation, a test script accessible to the examinee'"'"'s workstation, wherein the test script includes at least one testing unit specifying a predetermined delivery mode for the computerized test; and

presenting, by the data distribution system, the test questions to the examinee at the examinee'"'"'s workstation in a sequence based on the predetermined delivery mode.