System and Method for Reliable, Adaptive, and Objective Skills Based Testing

• US 20140113259A1
• Filed: 10/22/2013
• Published: 04/24/2014
• Est. Priority Date: 10/23/2012
• Status: Abandoned Application

• Current Assignee: TrueAbility, Inc.
• Original Assignee: TrueAbility, Inc.
• Inventors: Jones, Dusty, Owen, Luke, Mendler, Frederick, Robertson, Marcus
• Application Number:
  US14/060,585
• Publication Number:
  US 20140113259A1
• Time in Patent Office: Days
• US Class Current: 434/219
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Described herein is method and system for electronic skill-based testing. A computing device, which may be a cloud server, provides a live environment to the test-taker with some issues that must be resolved. The attempts of the test-taker to resolve the issue are scored according to the efficiency and speed of the test-taker'"'"'s approach, success or failure of the test-taker to resolve the issue, and by reference to the difficulty level of the task and the relative degree of achievement of other test takers.

Claims

1. A method for electronic skill-based testing, the method comprising:

creating, by a computing device, a live computing environment having at least one issue;

receiving, by the computing device, from the test-taker, instructions attempting to resolve the at least one issue; and

calculating, by the computing device, a score evaluating the instructions.

2. A method according to claim 1, wherein creating the live computing environment further comprises:

providing a live computing environment that is functioning correctly; and

introducing at least one issue into the live computing environment.

3. A method according to claim 1, wherein creating the live computing environment further comprises:

maintaining, in memory accessible to the computing device, a set of examination scenarios, each scenario comprising instructions to create a live computing environment having at least one issue;

selecting a scenario from the set of examination scenarios; and

executing the instructions of which the scenario is comprised.

4. A method according to claim 3, wherein the scenarios are grouped by difficulty level, and wherein selecting comprises selecting a test-taker difficulty level and selecting a scenario from a group corresponding to that test-taker difficulty level.

5. A method according to claim 4, wherein selecting the test-taker difficulty level further comprises:

maintaining, in memory accessible to the computing device, data reflecting past performance of the test-taker; and

determining, using that data, a test-taker difficulty level for the test taker.

6. A method according to claim 3, wherein selecting further comprises randomly selecting a scenario from the set of scenarios.

7. A method according to claim 1, wherein creating further comprises creating the live environment immediately before providing access to the test-taker.

8. A method according to claim 1, wherein receiving further comprises recording the inputs of the test-taker.

9. A method according to claim 1, wherein receiving further comprises recording the steps performed by the test taker.

10. A method according to claim 1, wherein receiving further comprises recording the time spent by the test-taker.

11. A method according to claim 1, wherein calculating further comprises analyzing the live computing environment and determining that the test-taker resolved the at least one issue.

12. A method according to claim 1, wherein calculating further comprises assessing the robustness of a resolution the test-taker provided for the issue.

13. A method according to claim 1, wherein calculating further comprises determining how long the live computing environment, as resolved by the test-taker, takes to perform a task, as a function of a parameter of the task.

14. A method according to claim 1, wherein calculating further comprises ranking the score calculated for the test-taker relative to the score of at least one additional test-taker.

15. A method according to claim 1, further comprising:

maintaining, in memory accessible to the computing device, a set of model instructions that together resolve the issue;

comparing the instructions entered by the test-taker to the model instructions; and

increasing the score for each instruction entered by the test-taker that matches a model instruction.

16. A method according to claim 1, further comprising providing data concerning the score to a test-sponsor.

17. A method according to claim 16, further comprising accepting, by the computing device, modifications to the score from the test-sponsor.

18. A system for electronic skill-based testing, the system comprising:

a computing device;

a live environment creator, executing on the computing device and creating a live computing environment having at least one issue;

a user interface, executing on the computing device and receiving, from the test-taker, instructions attempting to resolve the issue; and

a score calculator, executing on the computing device and calculating a score evaluating the instructions.