METHOD AND APPARATUS FOR FACILITATING ATTENTION TO A TASK

• US 20140178843A1
• Filed: 12/20/2012
• Published: 06/26/2014
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: The United States of America As Represented By The Secretary of Agriculture
• Original Assignee: US Army Research Laboratory (Government of the United States of America)
• Inventors: Smyth, Christopher C.
• Granted Patent: US 9,842,511 B2
• Time in Patent Office: Days
• US Class Current: 434/238
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

A method and apparatus for facilitating attention to a task are disclosed. The method may include: detecting, with a sensor, one or more movements, estimating the task attention state based on the one or more movements, determining the workload based on the estimated attention state, and determining based on the workload, the optimal format for the relay of operational information that best facilitates attention to the task for increased ease of task performance.

Claims

1. A method for facilitating attention of an operator to perform a task with the aid of a display device, the following steps of the method being executed by one or more computer processors and comprising:

a. determining the state of attention of the operator;

b. determining one or more tasks to be performed;

c. determining operational information and control activations for the one or more tasks to be performed;

d. determining one or more display formats for display on a display device of operational information and/or control activations related to the one or more tasks, wherein the one or more display formats are determined in a manner to improve visual task attention of the operator'"'"'s interaction with the display device when performing a task, based on the determined state of attention of the operator; and

e. determining a schedule for the performance of the one or more tasks, wherein;

the schedule specifies a sequence of displaying the one or more determined display formats, and of associated control activations for performance of the one or more tasks; and

the determining of the schedule based on the determined state of attention of the operator using a model of task attention.

2. The method of claim 1, wherein determining the state of attention of the operator, comprises:

a. determining eye-movements and estimating eye-gaze;

b. determining manual activities in performance of the tasks; and

c. determining the state of the task performance.

3. The method of claim 1, wherein determining the state of attention of the operator comprises detecting attributes, wherein the attributes comprise one or more attributes of eye-movements and fixations, physiological attributes of the electrocardiogram, and physiological attributes of electroencephalogram measurements of cortical brain activity.

4. The method of claim 3, wherein determining the state of attention of the operator further comprises detecting attributes for the state comprising one or more of confounded, task orienting, task recall, task focus, and task execution.

5. The method of claim 1, wherein using the model of task attention in the determination of the schedule comprises taking into account model subtask times and workloads as task cost elements so as to minimize strategy cost according to a control strategy.

6. The method of claim 5, wherein the model of task attention comprises task elements of orientation, task recall, task focus, and task execution, where the task elements constitute a micro-model of attention at the millisecond level.

7. The method of claim 6, wherein determining the schedule comprises scheduling a sequence of control activations enabled within a time period, where the control activations to be enabled are determined from the micro-model sub-task times and workload elements.

8. The method of claim 7, wherein determining the schedule comprises scheduling a sequence of display formats for display of operational information for control activations, where the sequence of formats is chosen to best facilitate task attention as determined from the micro-model sub-task times and workload elements.

9. The method of claim 8, wherein the scheduling comprises determining a sequence of concurrently occurring multiple tasks and involves switching between shared operations of one task and monitoring of others.

10. The method of claim 8, wherein the scheduling comprises an iterative process that with access to a model knowledge base of micro-subtask times and workloads, computes associated cost variables for task combinations of display formats of operational information for control activations;

and computes the corresponding cost functions and selects the minimal cost schedule to best facilitate task attention.

21. The method of claim 1, wherein determining the one or more display formats comprises:

determining the size, grouping, arrangement, sequence, selection choices, the number, or any combination thereof, of the operational information and/or control activations which are to be displayed on the display device.

22. The method of claim 1, wherein the operational information and/or control activations comprises:

one or more display windows and/or menus displayed on the display device.

23. The method of claim 1, further comprising:

determining the time of the display of the one or more display formats on the display device.

24. The method of claim 1, further comprising:

ranking the attention of the operator to tasks by the degree of cognitive involvement in consecutive control activation functions from task initiation to task completion.

11. A system for facilitating attention of an operator to perform a task with the aid of a display device, comprising:

one or more computer processors configured to have a plurality of electronic modules, which include;

a. an electronic module configured to determine the state of attention of the operator;

b. an electronic module configured to specify one or more tasks to be performed, and the associated operational information and control activations needed to perform the one or more tasks;

c. an electronic module configured to determine one or more display formats for display on a display device of the operational information and/or control activations related to the one or more tasks, wherein the one or more display formats are determined in a manner to improve visual task attention of the operator'"'"'s interaction with the display device when performing a task, based on the determined state of attention of the operator; and

d. an electronic scheduler configured to determine a sequence of displaying the one or more determined display formats, and of control activations for performance of the one or more tasks based on the determined state of attention of the operator using a model of task attention.

12. The system of claim 11, wherein the electronic module configured to determine the state of attention of the operator, comprises:

a. a device for tracking eye-movements and predicting eye-gaze;

b. a device for tracking manual activities appropriate for performance of the tasks;

or

c. a device for determining the state of the task performance.

13. The system of claim 11, wherein the electronic module configured to determine the state of attention of the operator is configured to detect attributes, wherein the attributes comprise one or more attributes of eye-movements and fixations, physiological attributes of the electrocardiogram, and physiological attributes of electroencephalogram measurements of cortical brain activity.

14. The system of claim 13, wherein the electronic module configured to determine the state of attention of the operator is configured to detect attributes for the state comprising one or more of confounded, task orienting, task recall, task focus, task execution.

15. The system of claim 11, wherein the model of task attention, comprises:

a. a table of subtask elements comprising one or more of task orientation, task recall, task focus, and task execution; and

b. a data base of corresponding subtask times and workload as cost variables, where the task elements constitute a micro-model of attention at the millisecond level.

16. The system of claim 11, wherein the electronic scheduler is configured to take into account subtask times and workloads as task cost elements so as to minimize strategy cost according to a control strategy in determining the sequence.

17. The system of claim 15 wherein the model of task attention comprises task elements of task orientation, task recall, task focus, and task execution, wherein the task elements constitute a micro-model of attention at the millisecond level.

18. The system of claim 11, the electronic scheduler schedules a sequence of control activations enabled within a time period, and a sequence of display formats for display of operational information for the control activations, where the sequence is chosen to best facilitate task attention as determined from the micro-model sub-task times and workload elements.

19. The system of claim 18, where the electronic scheduler comprises:

a. a sequencer that with access to a task knowledge base, executes a scheduling process for task combinations suitable for the strategy, and computes associated cost variables; and

b. an optimizer that computes the corresponding cost functions and selects the minimal cost schedule for output to a task display driver.

20. (canceled)

25. A method for facilitating attention of an operator to perform a task via the aid of a display device, the following steps of the method being executed by one or more computer processors and comprising:

a. determining the state of attention of the operator;

b. determining one or more tasks to be performed;

c. determining operational information and/or control activations for the one or more tasks to be performed; and

d. determining one or more display formats for display on a display device of operational information and/or control activations related to the one or more tasks, wherein the one or more display formats are determined in a manner to improve visual task attention of the operator'"'"'s interaction with the display device when performing a task, based on the determined state of attention of the operator.