Gaze driven interaction for a vehicle

• US 10,324,527 B2
• Filed: 11/10/2017
• Issued: 06/18/2019
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: George-Svahn, Erland, Lannsjo, Rebecka, Skogo, Marten
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US15/809,014
• Publication Number:
  US 20180224933A1
• Time in Patent Office: 585 Days
• Field of Search: 345 61- 69, 345102, 345156-158, 349 61- 69, 362 971- 974, 701 1- 344, 715810-845, 715863
• CPC Class Codes: B60K 2360/122 Instrument input devices with reconfigurable control functions, e.g. reconfigurable menus
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 26/02 of initiating means or elements
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/285 for improving awareness by directing driver's gaze direction or eye points
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

According to the invention, a method for changing information on a display in a vehicle based on a gaze direction of a driver is disclosed. The method may include displaying information on the display in the vehicle. The method may also include receiving gaze data indicative of the gaze direction of a user. The method may further include changing the display based at least in part on the gaze data.

Claims

1. A method for controlling a vehicle system based on a gaze direction of a user, the method comprising:

receiving a first activation signal from an input device activated by a first non-gaze interaction with a user;

in response to receiving the first activation signal, determining a gaze position of the user on a display of a vehicle;

determining a vehicle system to control based on the gaze position determined in response to receiving the first activation signal;

receiving a second activation signal from the input device activated by a second non-gaze interaction with the user; and

in response to receiving the second activation signal, modifying operation of the vehicle system.

2. The method for controlling the vehicle system based on the gaze direction of the user of claim 1, wherein determining the vehicle system to control comprises:

identifying an icon on the display of the vehicle that is located at the gaze position; and

determining the vehicle system associated with the icon.

3. The method for controlling the vehicle system based on the gaze direction of the user of claim 1, wherein determining the vehicle system to control comprises:

identifying an icon on the display of the vehicle that is located within a threshold distance of the gaze position; and

determining the vehicle system associated with the icon.

4. The method for controlling the vehicle system based on the gaze direction of the user of claim 1, wherein the vehicle system is selected from a group consisting of:

a climate control system;

a stereo system;

a telephone system;

a television system;

a vehicle control system;

an on board computer; and

a safety system.

5. The method for controlling the vehicle system based on the gaze direction of the user of claim 1, wherein determining the gaze position of the user comprises:

determining a series of gaze positions of the user over a period of time.

6. The method for controlling the vehicle system based on the gaze direction of the user of claim 5, wherein modifying operation of the vehicle system is further based on:

the series of gaze positions of the user over the period of time.

7. The method for controlling the vehicle system based on the gaze direction of the user of claim 1, wherein modifying operation of the vehicle system is further based on:

verbal or tactile commands from the user.

8. A non-transitory computer readable medium having instructions stored thereon executable by a computing device to cause the computing device to perform operations comprising:

receiving a first activation signal from an input device activated by a first non-gaze interaction with a user;

in response to receiving the first activation signal, determining a gaze position of the user on a display of a vehicle;

determining a vehicle system to control based on the gaze position determined in response to receiving the first activation signal;

receiving a second activation signal from the input device activated by the user a second time; and

in response to receiving the second activation signal, controlling operation of the vehicle system.

9. The non-transitory computer readable medium of claim 8, wherein the vehicle system is selected from a group consisting of:

a climate control system;

a stereo system;

a telephone system;

a television system;

a vehicle control system;

an on board computer; and

a safety system.

10. The non-transitory computer readable medium of claim 8, wherein determining the vehicle system to control comprises:

identifying an icon on the display of the vehicle that is located at the gaze position; and

identifying the vehicle system associated with the icon.

11. The non-transitory computer readable medium of claim 8, wherein determining the vehicle system to control comprises:

identifying an icon on the display of the vehicle that is located within a threshold distance of the gaze position; and

identifying the vehicle system associated with the icon.

12. The non-transitory computer readable medium of claim 8, further comprising:

providing audible or tactile feedback while modifying operation of the vehicle system.

13. The non-transitory computer readable medium of claim 8, wherein:

at least one of the first non-gaze interaction or the second non-gaze interaction are verbal expressions by the user.

14. The non-transitory computer readable medium of claim 8, wherein:

at least one of the first activation signal or the second activation signal is generated in response to receiving a physical input at the input device.

15. A system for controlling a vehicle system based on a gaze direction of a user comprising:

an eye tracking device;

one or more processors configured to;

receive a first activation signal from an input device activated by a first non-gaze interaction with a user;

in response to receiving the first activation signal, determine a gaze position of the user using the eye tracking device;

determine a vehicle system to control based on the gaze position determined in response to receiving the first activation signal;

receive a second activation signal from the input device activated by a second non-gaze interaction with the user; and

in response to receiving the second activation signal, modify operation of the vehicle system.

16. The system for controlling the vehicle system based on the gaze direction of the user of claim 15, wherein:

determining a gaze position of the user comprises determining a series of gaze positions of the user over a period of time; and

modifying operation of the vehicle system is further based on the series of gaze positions of the user over the period of time.

17. The system for controlling the vehicle system based on the gaze direction of the user of claim 15, wherein modifying operation of the vehicle system is further based on:

verbal or tactile commands from the user.

18. The system for controlling the vehicle system based on the gaze direction of the user of claim 15, wherein the vehicle system is selected from a group consisting of:

a climate control system;

a stereo system;

a telephone system;

a television system;

a vehicle control system;

an on board computer; and

a safety system.

19. The system for controlling the vehicle system based on the gaze direction of the user of claim 15, wherein the system further comprises:

the input device; and

wherein at least one of the first activation signal or the second activation signal is generated in response to receiving a physical input at the input device.

20. The system for controlling the vehicle system based on the gaze direction of the user of claim 15, wherein:

at least one of the first non-gaze interaction or the second non-gaze interaction are verbal expressions by the user.