System and method for simulating a computer environment and evaluating a user's performance within a simulation

• US 20030176993A1
• Filed: 12/26/2002
• Published: 09/18/2003
• Est. Priority Date: 12/28/2001
• Status: Active Grant

• Current Assignee: Testout Corporation (The Computing Technology Industry Association, Inc.)
• Original Assignee: Testout Corporation (The Computing Technology Industry Association, Inc.)
• Inventors: Chen-Wright, Kathy, Lines, Vardell
• Granted Patent: US 7,089,172 B2
• Time in Patent Office: Days
• US Class Current: 703/22
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation

Abstract

A system and method is provided for simulating a computer environment and evaluating a user'"'"'s performance within a simulation which uses the computer environment. The method reads base data which defines a simulated component within a simulated computer environment. Initial condition data may also be read to modify a simulated component. The simulated computer environment is represented by executing one or more simulators to simulate actual software modules. A simulation is presented to a user. The simulation includes a scenario and a set of tasks to be completed. As the user interacts with the simulators, tasks completed by the user are recorded. Finally, the set of completed tasks are evaluated to determine a user'"'"'s performance within the simulation. The system and method allows a simulation author to automatically define a set of possible tasks for a simulation by performing the desired tasks in the simulated computer environment. The system and method provides a robust evaluation process for simulations. Partial scores, weighted scores, partially weighted scores, and binary scores are provided, as well as rescoring of simulations and simulation-based exam question seeding.

Claims

1. A method for simulating a hypothetical computer environment using a simulated computer environment and evaluating user performance within the simulated computer environment, the method comprising:

reading base data which defines the simulated computer environment, the simulated computer environment comprising a simulated component that emulates an actual component and initial condition data which defines a modification of a characteristic of the simulated component;

representing the simulated computer environment by executing a simulator which simulates an actual software module of the hypothetical computer environment, the simulator managing the simulated component and modifying a characteristic of the simulated component with the modification defined in the initial condition data;

presenting a simulation to the user using the simulator, the simulation comprising a scenario having a set of required tasks for the user to complete in the simulated computer environment;

recording a set of tasks completed by the user interfacing with the simulator; and

evaluating the set of completed tasks to determine the user'"'"'s performance within the simulation.

2. The method of claim 1, wherein evaluating the set of completed tasks comprises:

reviewing the set of required tasks, the set of completed tasks, and evaluation criteria to determine a score for the user'"'"'s performance within the simulation; and

associating the score with the user and the simulation.

3. The method of claim 2, wherein the score is a full credit score which is determined when each of the required tasks corresponds to a completed task in the set of completed tasks.

4. The method of claim 2, wherein the evaluation criteria comprises a single point value assigned to each of the required tasks, and wherein when less than all of the required tasks have corresponding completed tasks, the score is for partial credit and comprises the sum of point values assigned to each of the required tasks which have corresponding completed tasks in the set of completed tasks.

5. The method of claim 2, wherein the evaluation criteria comprises a variety of point values with each required task being assigned one of the point values and wherein when less than all of the required tasks have corresponding completed tasks, the score is for partial credit and comprises the sum of point values assigned to each of the required tasks which have corresponding completed tasks in the set of completed tasks.

6. The method of claim 2, wherein based on the evaluation criteria, the score comprises a no-score assessment such that the score for the simulation has no affect on the user'"'"'s overall score.

7. The method of claim 2, further comprising:

modifying the set of required tasks;

reviewing the modified set of required tasks, the set of completed tasks, and evaluation criteria to determine a re-score for the user'"'"'s performance within the simulation; and

associating the re-score with the user and the simulation.

8. The method of claim 1, wherein the simulation is incorporated in a software package which is administered by an administration engine and wherein evaluating the set of completed tasks comprises:

receiving a score inquiry from the administration engine;

reviewing the set of required tasks, the set of completed tasks, and evaluation criteria to determine a score for the user'"'"'s performance within the simulation; and

reporting the score to the administration engine.

9. The method of claim 8, wherein the software package is a certification exam and the administration engine is a certification exam engine.

10. The method of claim 8, wherein the software package is a training module and the administration engine is a training engine.

11. The method of claim 1, wherein the simulator comprises a user interface comprising substantially identical user interface controls, windows, error messages, pop-up windows, and user interface navigation functions as the actual user interface for the actual software module which the simulator emulates.

12. The method of claim 1, wherein the simulator simulates an actual operating system and corresponding user interface for the actual operating system.

13. A method for drafting a simulation for a hypothetical computer environment using a simulated computer environment wherein the simulation is defined by an author interfacing with the simulated computer environment, the method comprising:

executing a simulation editor for drafting simulations for the simulated computer environment;

in response to a command from the author, reading base data which defines the simulated computer environment comprising a simulated component that emulates an actual component;

representing the simulated computer environment by executing a simulator which simulates an actual software module of the hypothetical computer environment;

recording tasks completed by the author within the simulated computer environment as a set of available tasks for a simulation currently being defined;

presenting the set of available tasks to the author in the simulation editor;

receiving a selection of at least one available task which is designated by the author and converting the at least one selected available task to a required task for the simulation.

14. The method of claim 13, wherein the simulation is defined in a simulation file, the method further comprising:

packaging the base data, initial condition data, simulation file, and simulator in a distribution file;

certifying the authenticity of the distribution file; and

storing the distribution file on computer-readable media for presentation of the simulation within a certification exam.

15. A system for simulating a hypothetical computer environment using a simulated computer environment and evaluating user performance within the simulated computer environment, the system comprising:

a data module for reading base data which defines the simulated computer environment, the simulated computer environment comprising a simulated component that emulates an actual component and initial condition data which defines a modification of a characteristic of the simulated component;

a simulation engine in communication with the data module, the simulation engine representing the simulated computer environment by executing a simulator which simulates an actual software module of the hypothetical computer environment, the simulator managing the simulated component and modifying a characteristic of the simulated component with the modification defined in the initial condition data;

a presentation module in communication with the simulation engine, the presentation module presenting a simulation to the user using the simulator, the simulation comprising a scenario having a set of required tasks for the user to complete in the simulated computer environment;

a recorder for recording a set of tasks completed by the user interfacing with the simulator; and

an evaluation engine for evaluating the set of completed tasks to determine the user'"'"'s performance within the simulation.

16. The system of claim 15, wherein the evaluation engine reviews the set of required tasks, the set of completed tasks, and evaluation criteria to determine a score for the simulation and associates the score with the user and the simulation.

17. The system of claim 16, wherein the score comprises a full credit score which is determined when each of the required tasks corresponds to a completed task in the set of completed tasks.

18. The system of claim 16, wherein the evaluation criteria comprises a single point value assigned to each of the required tasks, and wherein when less than all of the required tasks have corresponding completed tasks, the score is for partial credit and comprises the sum of point values assigned to each of the required tasks which have corresponding completed tasks in the set of completed tasks.

19. The system of claim 16, wherein the evaluation criteria comprises a variety of point values with each required task being assigned one of the point values, and wherein when less than all of the required tasks have corresponding completed tasks, the score is for partial credit and comprises the sum of point values assigned to each of the required tasks which have corresponding completed tasks in the set of completed tasks.

20. The system of claim 16, wherein based on the evaluation criteria, the score comprises a no-score assessment such that the score for the simulation has no affect on the user'"'"'s overall score.

21. The system of claim 15, wherein the simulation is incorporated in a software package which is administered by an administration engine and wherein the evaluation engine receives a score inquiry from the administration engine, reviews the set of required tasks, the set of completed tasks, and evaluation criteria to determine a score for the user'"'"'s performance within the simulation, and reports the determined score to the administration engine.

22. The system of claim 21, wherein the software package is a certification exam and the administration engine is a certification exam engine.

23. The system of claim 21, wherein the software package is a training module and the administration engine is a training engine.

24. The system of claim 15, wherein the simulator comprises a user interface comprising substantially identical user interface controls, windows, error messages, pop-up windows, and user interface navigation functions to the actual user interface for the actual software module which the simulator emulates.

25. The system of claim 15, wherein the simulator simulates an actual operating system and corresponding user interface for the actual operating system.

26. A system for drafting a simulation for a hypothetical computer environment using a simulated computer environment wherein the simulation is defined by an author interfacing with the simulated computer environment, the system comprising:

a simulation editor for drafting simulations for the simulated computer environment;

a data module for reading base data which defines the simulated computer environment comprising a simulated component corresponding to an actual component within the computer environment;

a simulation engine in communication with the data module, the simulation engine representing the simulated computer environment by executing a simulator which simulates an actual software module of the computer environment;

a recorder for recording tasks completed by the author within the simulated computer environment as a set of available tasks for a simulation currently being defined;

wherein the simulation editor presents the set of available tasks to the author, receives a selection of at least one available task from the author, and converts the selected available task to a required task for the simulation.

27. The system of claim 26, wherein the simulation is defined in a simulation file, the system further comprising:

a packager for packaging the base data, initial condition data, simulation file, and simulator in a distribution file;

a certification module for certifying the authenticity of the distribution file; and

a storage module for storing the distribution file on computer-readable media for presentation of the simulation within a certification exam.

28. A computer-readable medium containing instructions for simulating a hypothetical computer environment using a simulated computer environment and evaluating user performance within the simulated computer environment, wherein the instructions comprise executable instructions for implementing a method comprising:

reading base data which defines the simulated computer environment, the simulated computer environment comprising a simulated component that emulates an actual component and initial condition data which defines a modification of a characteristic of the simulated component;

representing the simulated computer environment by executing a simulator which simulates an actual software module of the hypothetical computer environment, the simulator managing the simulated component and modifying a characteristic of the simulated component with the modification defined in the initial condition data;

presenting a simulation to the user using the simulator, the simulation comprising a scenario having a set of required tasks for the user to complete in the simulated computer environment;

recording a set of tasks completed by the user interfacing with the simulator; and

evaluating the set of completed tasks to determine the user'"'"'s performance within the simulation.

29. A computer-readable medium containing instructions for drafting a simulation for a hypothetical computer environment using a simulated computer environment wherein the simulation is defined by an author interfacing with the simulated computer environment, wherein the instructions comprise executable instructions for implementing a method comprising:

executing a simulation editor for drafting simulations for the simulated computer environment;

in response to a command from the author, reading base data which defines the simulated computer environment comprising a simulated component that emulates an actual component;

representing the simulated computer environment by executing a simulator which simulates an actual software module of the hypothetical computer environment;

recording tasks completed by the author within the simulated computer environment as a set of available tasks for a simulation currently being defined;

presenting the set of available tasks to the author in the simulation editor; and

receiving a selection of at least one available task which is designated by the author and converting the at least one selected available task to a required task for the simulation.