Extensible exam language (XXL) protocol for computer based testing

• US 7,494,340 B2
• Filed: 11/13/2002
• Issued: 02/24/2009
• Est. Priority Date: 11/13/2001
• Status: Expired due to Term

• Current Assignee: Prometric, LLC
• Original Assignee: Prometric Inc. (Educational Testing Service)
• Inventors: Hartley, Tronster Maxwell, Kvech, Kyle Michael, Garrison, William Howard, Bowers, Clarke Daniel
• Primary Examiner(s): MOSSER, KATHLEEN MICHELE
• Application Number:
  US10/292,911
• Publication Number:
  US 20030129573A1
• Time in Patent Office: 2,295 Days
• Field of Search: 434/323, 434/350, 434/353, 434/365, 434/307.R, 434/118, 707100-102, 717/124
• US Class Current: 434/118
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

A memory stores a plurality of first data structures, which includes element specific data objects indicating a classification of at least one of the plurality of segments of the test definition language, and second data structures, which include attribute specific data objects indicating at least one attribute of the segments of the test definition language implemented by a computer. A method for computer-based testing includes authoring a test specification and content of the at least one test using a test definition language, compiling the test specification and content of the at least one test to create a compiled test specification and content, which includes validating the test specification and content, storing the compiled test specification and content to a resource file, and retrieving the compiled test specification and content from the resource file during delivery of the test.

Claims

1. A computer readable medium storing a test definition language in extensible markup language format that characterizes at least one computer-based test delivered to an examinee using a test driver and is implemented by at least one computer, comprising:

the test definition language having a hierarchy of data structures;

the test driver delivering the at least one computer-based test to an examinee using a display device, managing the at least one computer-based test, controlling progression of the at least one computer-based test, controlling scoring of the at least one computer-based test, controlling timing of the at least one computer-based test, and controlling results reporting of the at least one computer-based test based on the test definition language; and

at least one plugin accessed by the test driver through at least one interface

the at least one plugin customizing the behavior the test driver by allowing the test driver to access the at least one plugin for additional data that expands the functionality of the test driver;

the heirarchy of data structures including;

at least one of a plurality of first data structures, the plurality of first data structures including elements indicating a classification of at least one of a plurality of segments of the test definition language, wherein the plurality of segments defines information comprising data content, presentation format, progression, scoring, timing, and results reporting of the at least one computer-based test;

at least one of a plurality of second data structures at least one of depending from and subordinate to the at least one of the plurality of first data structures, the plurality of second data structures including attributes indicating at least one attribute of the at least one of the plurality of segments of the test definition language implemented by the at least one computer; and

an exam resource file including the at least one of a plurality of first data structures and the at least one of a plurality of second data structures.

2. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises one of message boxes, scripts, data, plug-ins, templates, categories, items, groups, sections, forms, form groups, and exams.

3. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises message boxes and the at least one attribute of the at least one of the plurality of segments comprises at least one of ok, cancel, abort, retry, ignore, yes, no, and title.

4. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises scripts and the at least one attribute of the at least one of the plurality of segments comprises type.

5. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises data and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, a uniform resource identifier, and keep external.

6. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises plug-ins and the at least one attribute of the at least one of the plurality of segments comprises at least one of name and program identification.

7. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises templates and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, split, and size.

8. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises categories and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, duplicate, complete, and contents.

9. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises items and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, title, template, area, weight, scored, and skip allowed.

10. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises groups and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, title, and template.

11. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises sections and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, title, and skip allowed.

12. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises forms and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, version, title, restartable, skip allowed, begin section, and end section.

13. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises form groups and the at least one attribute of the at least one of the plurality of segments comprises at least one of name and selection.

14. The computer readable medium of claim 1, wherein the classification of the at least one of the plurality of segments of the test definition language comprises exams and the at least one attribute of the at least one of the plurality of segments comprises at least one of name, version, and title.

15. The computer readable medium of claim 1, further stores at least one of a plurality of third data structures at least one of depending from and subordinate to the at least one of the plurality of first data structures, the plurality of third data structures including data indicating at least one sub-classification of the at least one of the plurality of segments of the test definition language implemented by the at least one computer.

16. The computer readable medium of claim 15, wherein the classification of the at least one of the plurality of segments of the test definition language comprises one of plug-ins and items.

17. The computer readable medium of claim 1, further stores at least one of a plurality of third data structures at least one of depending from and subordinate to the at least one of the plurality of first data structures, the plurality of third data structures including elements indicating a sub-classification of the at least one of the plurality of segments of the test definition language, wherein the sub-classification further indicates at least one property specific to the at least one of the plurality of segments of the test definition language implemented by the at least one computer.

18. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises one of message boxes, plug-ins, templates, categories, items, groups, sections, forms, form groups, and exams.

19. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises message boxes and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of box size and button size.

20. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises plug-ins and the at least one sub-classification of the at least one of the plurality of segments comprises data.

21. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises templates and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of area and template.

22. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises categories and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of category, category reference, description, and scoring.

23. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises items and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of data, category reference, staff, finish, condition, and attribute.

24. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises groups and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of selection, scoring, attribute, groups, group reference, presentation, section, and section reference.

25. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises sections and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of navigation, timer, start finish, condition, attribute, category reference, group, and group reference.

26. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises forms and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of scoring, timer, minimum resolution, report, results, attribute, section reference, and section.

27. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises form groups and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of selection, form reference, and form.

28. The computer readable medium of claim 17, wherein the classification of the at least one of the plurality of segments of the test definition language comprises exams and the at least one sub-classification of the at least one of the plurality of segments comprises at least one of minimum resolution, attribute, form, form reference, and form group.

29. The computer readable medium of claim 17, further stores at least one of a plurality of fourth data structures at least one of depending from and subordinate to the at least one of the plurality of first data structures, the plurality of fourth data structures including groups indicating an order of an appearance of the at least one of the plurality of third data structures, a minimum occurrence of the appearance of the at least one of the plurality of third data structures, and a maximum occurrence of the appearance of the at least one of the plurality of third data structures.

30. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises one of groups, sections, forms, form groups, and exams.

31. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises groups and the sub-classification of the at least one of the plurality of third data structures comprises at least one of group, group reference, presentation, section, and section reference.

32. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises sections and the sub-classification of the at least one of the plurality of third data structures comprises at least one of group and group reference.

33. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises forms and the at least one of the plurality of third data structures comprises at least one of section reference and section.

34. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises form groups and the at least one of the plurality of third data structures comprises at least one of form reference and form.

35. The computer readable medium of claim 29, wherein the classification of the at least one of the plurality of segments of the test definition language comprises exams and the at least one of the plurality of third data structures comprises at least one of form, form reference, and form group.

47. The computer readable medium of claim 1, wherein the at least one plugin is customizable without changing the test driver.

48. The computer readable medium of claim 1, wherein the at least one plugin enables the test driver to support at least one of new item types, navigation algorithms, information displays, scoring algorithms, timing algorithms, test unit selection algorithms, results persistence reporting, printed score reporting, and helm types without changing the test driver'"'"'s executable code.

49. The computer readable medium of claim 1, wherein the at least one plugin is independent from the test driver.

36. A computer readable medium storing a schema for a test definition language in extensible markup language format that that characterizes at least one computer-based test delivered to an examinee using a test driver and is implemented by at least one computer, comprising:

the test definition language having a plurality of segments;

the test driver delivering the at least one computer-based test to an examinee using a display device, wherein the schema defines a permissible grammar for the test definition language; and

at least one plugin customizing the behavior of the test driver by allowing the test driver to access the at least one plugin through at least one interface for additional data that changes the functionality of the test driver, the computer readable medium storing;

at least one of a plurality of first data structures, the plurality of first data structures including elements defining an element classification of at least one of the plurality of segments of the schema, wherein the plurality of segments defines classification identification information comprising data content, presentation format, progression, scoring, timing, and results reporting of the at least one computer-based test;

at least one of a plurality of second data structures, the plurality of second data structures including attributes defining at least one attribute classification of the least one of the plurality of segments of the schema;

at least one of a plurality of third data structures, the plurality of third data structures including elements indicating at least one element sub-classification of the at least one of the plurality of segments of the schema;

at least one of a plurality of fourth data structures, the plurality of fourth data structures including attributes indicating at least one attribute of the at least one of the plurality of segments of the test definition language implemented by the at least one computer; and

an exam resource file including the at least one of a plurality of first data structures, the at least one of a plurality of second data structures, the at least one of a plurality of third data structures, and the at least one of a plurality of fourth data structures.

37. The computer readable medium of claim 36, wherein the at least one of the plurality of second data structures at least one of depends from and is subordinate to the at least one of the plurality of first data structures.

38. The computer readable medium of claim 36, wherein the at least one of the plurality of second data structures is independent of the at least one of the plurality of first data structures.

39. The computer readable medium of claim 36, wherein the at least one of the plurality of third data structures at least one of depends from and is subordinate to the at least one of the plurality of first data structures.

40. The computer readable medium of claim 36, wherein the at least one of the plurality of fourth data structures at least one of depends from and is subordinate to the at least one of the plurality of first data structures.

41. The computer readable medium of claim 36, wherein the element classification is defined by at least one of a plurality of definition attributes, the definition attributes comprising at least one of name, content, order, and model.

42. The computer readable medium of claim 36, wherein the at least one attribute classification is defined by at least one of a plurality of definition attributes, the definition attributes comprising at least one of name, type, required, values, and default.

43. The computer readable medium of claim 36, wherein the at least one element sub-classification is defined by at least one of a plurality of definition attributes, the definition attributes comprising at least one of type, minimum occurrences, and maximum occurrences.

44. The computer readable medium of claim 36, wherein the at least one attribute is defined by a definition attributes, the definition attributes comprising type.

45. The computer readable medium of claim 36, further stores at least one of a plurality of fifth data structures, the plurality of fifth data structures including groups indicating an order of an appearance of the at least one of the plurality of third data structures, a minimum occurrence of the appearance of the at least one of the plurality of third data structures, and a maximum occurrence of the appearance of the at least one of the plurality of third data structures.

46. The computer readable medium of claim 45, wherein the at least one of the plurality of fifth data structures at least one of depends from and is subordinate to the at least one of the plurality of first data structures.

50. The computer readable medium of claim 36, wherein the at least one plugin is customizable without changing the test driver.

51. The computer readable medium of claim 36, wherein the at least one plugin enables the test driver to support at least one of new item types, navigation algorithms, information displays, scoring algorithms, timing algorithms, test unit selection algorithms, results persistence reporting, printed score reporting, and helm types without changing the test driver'"'"'s executable code.

52. The computer readable medium of claim 36, wherein the at least one plugin is independent from the test driver.