System and method for computer based test assembly

• US 6,000,945 A
• Filed: 02/09/1998
• Issued: 12/14/1999
• Est. Priority Date: 02/09/1998
• Status: Expired due to Term

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Keener, Barbara, Kuntz, David, Willian, Kenneth, Burns, Richard, Driscoll, Gary, Tyler, Linda, Sanchez-Lazer, Teresa
• Primary Examiner(s): Cheng, Joe H.
• Application Number:
  US09/020,755
• Time in Patent Office: 673 Days
• Field of Search: 434/118, 434/322-325, 434/337, 434/350, 434/353-362, 434/365, 706/45, 706/52, 706/915, 706/927, 704/2-10, 707/100-104, 381/51
• US Class Current: 434/322
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: Y10S 706/927 Education or instruction

Abstract

A computer-based test assembly system and method for generating an assembly of test items. The test assembly process includes creating tests in electronic form from already-existing test items and allowing access by a plurality of assembly personnel and reviewers via a network. Assemblers select test items on-line for inclusion in an assembly. Electronic selection is accomplished by means of an automated item selection algorithm or through a manual, but electronic series of queries. The test assembly process also includes an automated test layout feature which automatically assembles a test from the selected items. When a test is assembled, an assembler uses an automated documentation feature which generates a comprehensive documentation of the completed assembly.

Claims

1. A method of assembling a test on-line in electronic form using an automated test assembly system, comprising the steps of:

(a) developing at least one query in order to search an inventory of available test items for desired test items, said at least one query defining characteristics of desired test items that are needed for said test;

(b) inputting one query at a time into said test assembly system so as to yield a plurality of test items that satisfy each query;

(c) selecting a plurality of test items on-line from said plurality of test items that satisfy each query for storing in a worksheet; and

(d) producing a test from said worksheet of test items, said test being a layout of the assembly of test items as it would appear during an administration of said test to test takers.

2. A method of assembling a test on-line in electronic form using an automated test assembly system, comprising the steps of:

(a) determining whether an inventory of available test items can satisfy a test assembly request;

(b) when said inventory is deemed satisfactory, creating a worksheet on-line for one of (1) a test and (2) a test section in which desired test items are to be assembled;

(c) selecting a plurality of test items on-line from said inventory for storing in said worksheet;

(d) sequencing said plurality of test items on-line in said worksheet so as to produce an assembly of test items;

(e) reviewing said assembly of test items on-line for one of (1) necessary changes in the assembly and (2) for problems with particular test items;

(f) making any necessary changes in the assembly and replacing test items having problems with new test items as necessary;

(g) producing a test, said test being a layout of the assembly of test items as it would appear during an administration of said test to test takers; and

(h) storing on-line at least said test and data from said worksheet.

3. The method of claim 2, comprising the further step of requesting that additional items be stored in the inventory if it is determined at step (a) that the inventory is unsatisfactory.

4. The method of claim 2, wherein step (c) further comprises the steps of:

(c1) developing at least one constraint, said constraint defining at least one characteristic of test items that are needed for said test; and

(c2) inputting said at least one constraint into an automated item selection algorithm which simultaneously finds test items that meet said at least one constraint and places said test items in a temporary worksheet for user evaluation.

5. The method of claim 4, wherein said step (c1) further comprises the step of placing an upper bound and a lower bound on the number of said test items for said automated item selection algorithm to find.

6. A method for assembling a test on-line in electronic form using an automated test assembly system, comprising the steps of:

developing at least one query in order to search an inventory of available test items for desired test items, said at least one query defining characteristics of desired test items that are needed for said test;

inputting one query at a time into said test assembly system so as to yield a plurality of test items that satisfy each query;

selecting a plurality of test items on-line from the plurality of test items that satisfy each query; and

placing said selected test items in a worksheet for evaluation, said worksheet storing one of (1) a test and (2) a test section in which test items are to be assembled.

7. The method of claim 6, wherein said inputting step comprises the step of selectively refining said at least one query to further reduce the number of desired test items.

8. A method for automatically assembling a test on-line in electronic form using an automated test assembly system having a test item inventory, said test being a layout of the assembly of test items as said test items would appear during an administration of said test to test takers, comprising the steps of:

executing a plurality of layout macros in said test assembly system which direct the system to retrieve camera-ready versions of each test item which have been locked in said test item inventory and to place the locked, camera-ready versions in a predetermined sequence on an on-line test template; and

editing said test template to produce a suitable layout of the test pages.

9. A computer based test assembly system for assembling a test on-line in electronic form, comprising:

an item repository for storing available test item information;

a statistics database for storing statistical information about available test items;

item selection software which selects a plurality of test items on-line for storing in a worksheet based on said statistical information;

a test layout feature for producing a test, said test being a layout of the assembly of said plurality of selected test items as said layout would appear during an administration of said test to test takers.

10. The test assembly system of claim 9, wherein the item selection software includes an automated item selection algorithm that yields test items that simultaneously meet a plurality of user-defined constraints.

11. A method of assembling a computer adaptive test on-line in electronic form using an automated test assembly system, the test being an adaptive layout of the assembly of test items as said layout would appear during an administration of said test to a particular test taker, the appearance of each successive test item being dependent on the manner in which the previous test item was answered, the method comprising the steps of:

(a) selecting a draft pool of test items through the specification of at least one constraint, said constraint defining at least one characteristic of test items that are needed for the test;

(b) setting up a simulation by selecting files to use for the simulation, whereby data from said files is collected for processing;

(c) performing analysis iterations on the draft pool of test items to determine fitness for use for the test;

(d) performing evaluation of the analysis iterations to determine an optimal pool; and

(e) packaging and preparing files of data from said optimal pool for inclusion in a test module to be administered.

12. The method of claim 11, wherein step (a) further comprises the step of:

(a1) inputting said at least one constraint into an automated item selection algorithm which simultaneously finds said test items that meet said at least one constraint and places said test items in a temporary worksheet for user evaluation.

13. The method of claim 12, wherein said step (a1) further comprises the step of placing an upper bound and a lower bound on the number of said test items for said automated item selection algorithm to find.

14. A computer based test assembly system for assembling a test on-line in electronic form, comprising:

an item repository for storing available test item information;

a statistics database for storing statistical information about available test items;

item selection software which selects a plurality of test items on-line for storing in a worksheet based on said statistical information;

a test layout feature for producing a test, said test being an adaptive layout of the assembly of test items as said layout would appear during an administration of said test to a particular test taker, the appearance of each successive test item being dependent on the manner in which the previous test item was answered.

15. The test assembly system of claim 14, wherein the item selection software includes an automated item selection algorithm that yields test items that simultaneously meet a plurality of user-defined constraints.