Eyetracker control of heads-up displays

• US 5,689,619 A
• Filed: 08/09/1996
• Issued: 11/18/1997
• Est. Priority Date: 08/09/1996
• Status: Expired due to Fees

• Current Assignee: United States Of Americal As Represented By The Secretary Of The Army
• Original Assignee: The United States Of America As Represented By The Secretary Of The Army
• Inventors: Smyth, Christopher C.
• Primary Examiner(s): Downs, Robert W.
• Assistant Examiner(s): Smith, Jeffrey S.
• Application Number:
  US08/695,847
• Time in Patent Office: 466 Days
• Field of Search: 395/10, 351/210, 345/156
• US Class Current: 706/45
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

The invention consists of an eyetracker, a display driver, an adaptive display interface controller, and a digital computer. A user of the invention can operate a heads-up display (or a head mounted display) with an eyetracker, leaving his hands free to perform a manual control task. The user selects a display element from an array of choices on the heads-up display by gazing at an element while he activates a confirming mechanism. Both tasks share a common visual working area with overlaid visual images, and task interference is reduced since eye-movements and visual attention are dedicated to the same display surface.

The adaptive display interface automatically aids the user in display operations during high manual task loading. The adaptive interface is driven by an electronic Expert System on human visual processes programmed as a Fuzzy Logic controller. The controller generates display cueing aids, under computer control, in response to the erratic eye movement patterns that occur during cognitive conflict. The cueing aids help the user to operate the display with a sequence of discrete eye gazes and confirming actions which only momentarily interrupt the manual control task.

As cueing aids, the invention provides a timing logic for element enhancement, and display markers for task feedback. While the aids would be disruptive at low cognitive loading, they provide necessary timing and memory prompts at high loadings. The element enhancement timing logic allows the user to momentarily look at the display and then return his attention to the tracking task before executing confirmation. The display cues show the user where he is looking on the display relative to the element to be selected.

Claims

1. A method for automated adaptive display interface which automates the presentation of gaze aiding cues in response to periods of high cognitive loading of a person performing tasks, comprising the steps of:

maintenance of a record of said person'"'"'s short term eye movements by a computer to produce a short term eye movement history;

automatic analysis by said computer of said short term eye movement history for determining erratic eye movement patterns about a display element when said short term eye movements fall both into and beyond a predetermined space surrounding said display, element for a certain period of time;

recognition by said computer of said erratic eye movement patterns as representative of periods of high cognitive loading; and

instruction by said computer to a video display for the presentation of a gaze cue marker on said video display near said display element and display capture windows on said video display surrounding said display elements during said periods of high cognitive loading.

2. The method of claim 1 wherein said record of said person'"'"'s short term eye movements contains the locations, start times, and durations of the visual fixation points on said video display.

3. The method of claim 2 wherein said short term eye movement history is computed from said record of said person'"'"'s short term eye movements from the preceding 10 seconds.

4. The method of claim 3 wherein said gaze cue marker comprises cross-hairs.

5. The method of claim 3 wherein said gaze cue marker comprises a dot.

6. The method of claim 3 wherein each said capture window comprises a rectangle surrounding each said display element.