Eye tracking as a method to improve the user interface

• US 9,377,852 B1
• Filed: 08/29/2013
• Issued: 06/28/2016
• Est. Priority Date: 08/29/2013
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Original Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Inventors: Lorch, Nicholas M., Shapiro, Geoffrey A.
• Primary Examiner(s): Bolotin, Dmitriy
• Application Number:
  US14/013,883
• Time in Patent Office: 1,034 Days
• Field of Search: 345/7, 345/8, 345/156
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

The present disclosure is directed to a method for managing a user interface. The method may include the step of detecting a gaze of a user within a display. The method also includes the step of correlating the gaze of the user to an item displayed on the display. A further step of the method entails receiving an input from the user related to the item.

Claims

1. A method for managing a user interface, comprising:

detecting a gaze of a user within a display via an eye tracking sensor, the eye tracking sensor measuring eye movement or eye motion;

correlating the gaze of the user to a first location associated with an item displayed on the display;

receiving an input from the user related to adjusting the item relative to a second location or adjusting a function associated with the item relative to the second location; and

communicating the input to at least one of;

an on board aircraft communication system and an off board aircraft communication system, wherein the adjusting of the item or the adjusting of the function is only performed if the input and the gaze correspond to the second location associated with the item displayed on the display.

2. The method as claimed in claim 1, wherein receiving an input from the user related to adjusting the item relative to the second location or adjusting the function associated with the item relative to the second location includes receiving an input via at least one of:

a voice recognition system, a tactile feedback system, a haptic technology system, a touch screen, a multi-touch surface, a pressure-triggered screen with a stylus, a joystick, a keyboard, a cursor control panel, or a mouse.

3. The method as claimed in claim 1, wherein the display includes at least one of a Head Up display or a head mounted display.

4. The method as claimed in claim 1, wherein the method is implemented on a goal based Flight Management System.

5. The method as claimed in claim 1, further comprising:

highlighting the item on the display.

6. The method as claimed in claim 1, further comprising:

adjusting a position of the item on the display.

7. The method as claimed in claim 6, wherein the first location is a location on a map and the second location is a center location on the display, and adjusting the position of the item on the display includes centering the first location to the second location on the display, and wherein receiving the input from the user related to adjusting the item relative to the second location includes at least one of:

decrementing an area of the first location via a zoom in request or incrementing the area of the first location via a zoom out request.

8. The method as claimed in claim 6, wherein the input is a voice command, wherein the second location is a location corresponding to a desired speed and adjusting the position of the item on the display includes adjusting a current speed to the desired speed only after the voice command is confirmed, wherein the voice command is confirmed by determining the gaze of the user is within a confirmation area associated with the second location corresponding to the desired speed, and wherein the voice command includes an acceleration command.

9. The method as claimed in claim 6, wherein adjusting a position of the item on the display includes at least one of:

scrolling through a list or paging through a list.

10. The method as claimed in claim 1, wherein the receiving an input from the user related to adjusting the item relative to the second location or adjusting the function associated with the item relative to the second location comprises receiving an additional input related to the voice command from the user.

11. The method as claimed in claim 1, further comprising:

determining a meaning of a voice command based on at least;

the voice command and the item.

12. A user interface system, comprising:

an eye tracking sensor, the eye tracking sensor configured to detect a gaze of a user viewing a display and to measure eye movement or eye motion;

a processor in communication with the eye tracking sensor and in communication with at least one of an on board aircraft communication system and an off board aircraft communication system, the processor configured to receive the gaze of the user, the processor further configured to correlate the gaze of the user to a first area associated with an item displayed on the display; and

an input device, the input device in communication with the processor, the input device configured to receive an input from the user related to adjusting the item relative to a second area or adjusting a function associated with the item relative to the second area, wherein the adjusting the item or the adjusting the function associated with the item is only performed by the processor if the input and the gaze of the user correspond to the second area associated with the item.

13. The system as claimed in claim 12, wherein the input is a voice command, wherein the input device is further configured to receive a second input related to the voice command from the user, and wherein the input device includes a voice recognition system and at least one of:

a tactile feedback system, a haptic technology system, a touch screen, a multi-touch surface, a pressure-triggered screen with a stylus, a keyboard, or a mouse.

14. The system as claimed in claim 12, wherein the eye tracking sensor includes a non-contact eye tracking sensor for tracking the eye motion or a contact lens for tracking the eye movement.

15. The system as claimed in claim 12, wherein the display includes at least one of:

a head up display or a helmet mounted display.

16. The system as claimed in claim 12, wherein the second area includes a confirmation area, and wherein the processor is further configured to highlight the item or highlight the confirmation area.

17. The system as claimed in claim 12, wherein the processor is further configured to adjust a location of the item relative to the second area within the display.

18. An input processing method, comprising:

receiving a voice command from a user related to an item displayed on the display;

detecting a gaze of the user within the display;

measuring an eye movement;

correlating the gaze of the user to a first location associated with the item displayed on the display according to the eye movement; and

confirming the voice command and (i) adjusting the item relative to a second location or (ii) adjusting a function associated with the item relative to the second location when the second location associated with the item and the gaze correspond to the voice command.

19. The input processing method as claimed in claim 18, further comprising:

determining a meaning of the voice command based on at least;

the voice command and the gaze of the user within the display.

20. The input processing method as claimed in claim 18, wherein confirming the voice command and (i) adjusting the item relative to the second location, or (ii) adjusting the function associated with the item relative to the second location when the second location associated with the item and the gaze correspond to the voice command includes determining if the gaze of the user falls within a predetermined confirmation area.