Simulated training environments based upon fixated objects in specified regions

• US 20060290663A1
• Filed: 08/29/2006
• Published: 12/28/2006
• Est. Priority Date: 03/02/2004
• Status: Active Grant

• Current Assignee: Brian T. Mitchell
• Original Assignee: Brian T. Mitchell
• Inventors: Mitchell, Brian T.
• Granted Patent: US 9,652,032 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Objects associated with fixation regions are used to support training tasks that are characterized by operation within a dynamic 3D environment, problem-solving that involves user multi-tasking, fast decisions, heavy workloads, potential for unexpected events, and the need for high-performance skills. The invention is ideally suited to measure the nuances associated with maintaining situation awareness in such tasks where dynamic 3D environments that support problem solving with high cognitive fidelity have already been developed. Potential training application areas include urban warfighting, night vision goggle use, aircraft piloting, helicopter operation, remote control of vehicles and robots, night driving, air traffic control, and quarterback decision-making.

Claims

1. A method of generating an eye event, comprising the steps of:

a) obtaining user fixation parameters;

b) selecting a fixation region on a display device;

c) iteratively drawing one or more objects on the display device;

d) analyzing an attribute of each drawn object to determine if the user is fixated on that object; and

e) generating an eye event as a function of the fixated objects, if any.

2. The method of claim 1, including the step of:

drawing each objects into buffer; and

wherein the step of analyzing an attribute of each drawn object includes the step of determining if any pixels of the object were drawn outside the buffer.

3. The method of claim 1, wherein the step of analyzing an attribute of each drawn object includes the steps of:

placing a pointer to that object in an object table;

setting an object reference value to an object key that corresponds to the location of that object in the table;

as the object is drawn, coding the object reference value into a field in the non-display pixel bits for each pixel that is altered;

after all the object are drawn, retrieving the object keys for the pixels in the fixation region and using the keys to lookup the object pointers in the object table; and

organizing the object pointers into the fixated-objects list.

4. The method of claim 1, wherein the step of analyzing an attribute of each drawn object includes the steps of:

constructing an imaginary eye rays through the fixation region;

intersecting each eye ray with all the objects;

ordering the intersecting object(s) along the eye ray; and

identifying the fixation objects as the intersecting objects closest to a viewer on each ray.

5. The method of claim 1, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

6. The method of claim 5, wherein:

the goals are research goals; and

the simulation manager is an experiment manager.

7. The method of claim 5, wherein the computed measures that are derived using subject workload metrics.

8. The method of claim 5, wherein the simulated learning environment involves vehicle or aircraft control.

9. The method of claim 5, wherein the simulated learning environment is a computer application.

10. The method of claim 5, wherein the simulated learning environment is a computer game.

11. The method of claim 5, wherein the simulated learning environment is a cooperative problem solving environment.

12. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters;

b) selecting a fixation region;

c) choosing an object in the region, the object being rendered as a set of pixels;

d) drawing the object into buffer;

e) comparing the pixels of the object before and after the redraw to determine if there were changes and, if so, marking the object as fixated;

f) iterating through steps c) through e) for each object in the region;

g) constructing an eye event based upon the fixated objects, if any.

13. The method of claim 12, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

14. The method of claim 12, wherein:

the goals are research goals; and

the simulation manager is an experiment manager.

15. The method of claim 12, wherein the computed measures that are derived using subject workload metrics.

16. The method of claim 12, wherein the simulated learning environment involves vehicle or aircraft control.

17. The method of claim 12, wherein the simulated learning environment is a computer application.

18. The method of claim 12, wherein the simulated learning environment is a computer game.

19. The method of claim 12, wherein the simulated learning environment is a cooperative problem solving environment.

20. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters;

b) selecting a fixation region;

c) setting an object reference value to an object key that corresponds to the location of that object in the table;

d) as the object is drawn, coding the object reference value into a field in the non-display pixel bits for each pixel that is altered;

e) after the object are drawn, retrieving the object keys for the pixels in the fixation region and using the keys to lookup the object pointers in the object table; and

f) organizing the object pointers into the fixated-objects list; and

g) generating an eye event based upon the pointers.

21. The method of claim 20, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

22. The method of claim 20, wherein:

the goals are research goals; and

the simulation manager is an experiment manager.

23. The method of claim 20, wherein the computed measures that are derived using subject workload metrics.

24. The method of claim 20, wherein the simulated learning environment involves vehicle or aircraft control.

25. The method of claim 20, wherein the simulated learning environment is a computer application.

26. The method of claim 20, wherein the simulated learning environment is a computer game.

27. The method of claim 20, wherein the simulated learning environment is a cooperative problem solving environment.

28. A method of generating an eye event, comprising the steps of:

a) obtaining fixation parameters;

b) selecting a fixation region;

c) constructing an imaginary eye rays through the fixation region;

d) intersecting each eye ray with all the objects;

e) ordering the intersecting object(s) along the eye ray;

f) identifying the fixation object as the intersecting object closest to a viewer; and

g) generating an eye event based upon the fixated objects, if any.

29. The method of claim 28, further including the steps of:

defining a set of instructional goals including training objectives;

simulating a learning environment using a display with pixels and a user interface;

computing various measures based upon the trainee interactions, including the eye event and simulation state;

recording and time-stamping the computed measures;

recording current user skill level based upon the computed measures;

correlating current trainee skill level with the training objectives; and

modifying the learning environment in accordance with the correlation.

30. The method of claim 28, wherein:

the goals are research goals; and

the simulation manager is an experiment manager.

31. The method of claim 28, wherein the computed measures that are derived using subject workload metrics.

32. The method of claim 28, wherein the simulated learning environment involves vehicle or aircraft control.

33. The method of claim 28, wherein the simulated learning environment is a computer application.

34. The method of claim 28, wherein the simulated learning environment is a computer game.

35. The method of claim 28, wherein the simulated learning environment is a cooperative problem solving environment.