Simulated training environments based upon fixated objects in specified regions

• US 9,652,032 B2
• Filed: 08/29/2006
• Issued: 05/16/2017
• Est. Priority Date: 03/02/2004
• Status: Active Grant

• Current Assignee: Brian T. Mitchell
• Original Assignee: Brian T. Mitchell
• Inventors: Mitchell, Brian T.
• Primary Examiner(s): Musselman, Timothy
• Assistant Examiner(s): CARLOS, ALVIN LEABRES
• Application Number:
  US11/511,707
• Publication Number:
  US 20060290663A1
• Time in Patent Office: 3,913 Days
• Field of Search: 434247, 345156-158
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Objects associated with fixation regions are used to support training tasks that are characterized by operation within a dynamic 3D environment, problem-solving that involves user multi-tasking, fast decisions, heavy workloads, potential for unexpected events, and the need for high-performance skills. The invention is ideally suited to measure the nuances associated with maintaining situation awareness in such tasks where dynamic 3D environments that support problem solving with high cognitive fidelity have already been developed. Potential training application areas include urban warfighting, night vision goggle use, aircraft piloting, helicopter operation, remote control of vehicles and robots, night driving, air traffic control, and quarterback decision-making.

Claims

1. A method of generating an eye event, comprising the steps of:

obtaining user fixation parameters;

selecting a fixation region on a display device;

iteratively drawing one or more objects on the display device;

analyzing an attribute of each drawn object to determine if the user is fixated on that object;

generating an eye event as a function of the fixated objects, if any;

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

2. The method of claim 1, wherein the goals are educational research goals.

3. The method of claim 1, wherein the computed measures that are derived using subject workload metrics.

4. The method of claim 1, wherein the simulated learning environment involves vehicle or aircraft control.

5. The method of claim 1, wherein the simulated learning environment is a computer application.

6. The method of claim 1, wherein the simulated learning environment is a computer game.

7. The method of claim 1, wherein the simulated learning environment is a cooperative problem solving environment.

8. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters within a rendered scene;

b) selecting a fixation region within the rendered scene;

c) selecting an object in the region, the object being rendered as a set of pixels;

d) redrawing the object into buffer;

e) comparing the pixels of the object before and after the redraw to determine if there were changes and, if so, marking the object as fixated;

f) iterating through steps c) through e) for each object in the region;

g) constructing an eye event based upon the fixated objects, if any.

9. The method of claim 8, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives;

and modifying the learning environment in accordance with the correlation.

10. The method of claim 9, wherein the goals are educational research goals.

11. The method of claim 9, wherein the computed measures that are derived using subject workload metrics.

12. The method of claim 9, wherein the simulated learning environment involves vehicle or aircraft control.

13. The method of claim 9, wherein the simulated learning environment is a computer application.

14. The method of claim 9, wherein the simulated learning environment is a computer game.

15. The method of claim 9, wherein the simulated learning environment is a cooperative problem solving environment.

16. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters within a rendered scene;

b) selecting a fixation region within the rendered scene;

c) constructing a data structure having one or more object pointers, each indexed to an object key;

d) as the object is drawn, writing the object key into a field in the non-display pixel bits for each pixel of the object that is drawn;

e) after the object is drawn, retrieving the object keys for the pixels in the fixation region and using the keys to lookup the object pointers in the data structure; and

f) generating an eye event based upon the object pointers.

17. The method of claim 16, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives;

and modifying the learning environment in accordance with the correlation.

18. The method of claim 17, wherein the goals are educational research goals.

19. The method of claim 17 wherein the computed measures that are derived using subject workload metrics.

20. The method of claim 17, wherein the simulated learning environment involves vehicle or aircraft control.

21. The method of claim 17, wherein the simulated learning environment is a computer application.

22. The method of claim 17, wherein the simulated learning environment is a computer game.

23. The method of claim 17, wherein the simulated learning environment is a cooperative problem solving environment.

24. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters;

b) selecting a fixation region;

c) constructing an imaginary eye ray through the fixation region;

d) intersecting the eye ray with any object associated with the fixation region;

e) ordering the intersecting object(s) along the eye ray;

f) identifying a fixation objects as the intersecting object closest to a viewer; and

g) generating an eye event based upon the fixated objects, if any.

25. The method of claim 24, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

26. The method of claim 25, wherein the goals are educational research goals.

27. The method of claim 25, wherein the computed measures that are derived using subject workload metrics.

28. The method of claim 25, wherein the simulated learning environment involves vehicle or aircraft control.

29. The method of claim 25, wherein the simulated learning environment is a computer application.

30. The method of claim 25, wherein the simulated learning environment is a computer game.

31. The method of claim 25, wherein the simulated learning environment is a cooperative problem solving environment.