Gaze driven interaction for a vehicle

• US 10,035,518 B2
• Filed: 02/23/2017
• Issued: 07/31/2018
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: George-Svahn, Erland, Lannsjo, Rebecka, Skogo, Marten
• Primary Examiner(s): Wong, Yuen H
• Application Number:
  US15/441,043
• Publication Number:
  US 20170291613A1
• Time in Patent Office: 523 Days
• CPC Class Codes: B60K 2360/122 Instrument input devices with reconfigurable control functions, e.g. reconfigurable menus
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/194 by directing line of sight
• CPC Class Codes: B60K 26/02 of initiating means or elements
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/285 for improving awareness by directing driver's gaze direction or eye points
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

According to the invention, a method for modifying operation of at least one system of a vehicle based at least in part on a gaze direction of a driver is disclosed. The method may include receiving gaze data indicative of the gaze direction of a driver of a vehicle. The method may also include modifying operation of at least one system of the vehicle based at least in part on the gaze data.

Claims

1. A system for controlling operation of at least one subsystem of a vehicle on a roadway based on gaze directions of a driver of the vehicle over a period of time, wherein the system comprises:

an eye tracking device for receiving gaze data indicative of a series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle; and

a processor configured to;

determine, based on the gaze data, that the driver is gazing at locations inside and outside the vehicle;

determine, based on the gaze data, at least one of;

a relative frequency at which the driver is gazing at locations inside relative to outside the vehicle;

or

a relative length of time during which the driver is gazing at locations inside relative to outside the vehicle; and

controlling operation of at least one subsystem of the vehicle based at least in part on an intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time, and the series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle.

2. The system of claim 1, wherein the relative frequency at which the driver is gazing at locations inside relative to outside the vehicle comprises:

a total amount of time during which the driver is gazing at locations inside the vehicle relative to a total amount of time during which the driver is gazing at locations outside the vehicle.

3. The system of claim 1, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative frequency exceeding a predetermined frequency.

4. The system of claim 1, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative length of time exceeding a predetermined length of time.

5. The system of claim 1, wherein the at least one subsystem of the vehicle is selected from a group consisting of:

a safety subsystem;

a braking subsystem;

a steering subsystem; and

an accelerator limiting system.

6. A method for controlling operation of at least one subsystem of a vehicle on a roadway based on gaze directions of a driver of the vehicle over a period of time, wherein the method comprises:

receiving gaze data indicative of a series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle;

determining, based on the gaze data, that the driver is gazing at locations inside and outside the vehicle;

determining, based on the gaze data, at least one of;

a relative frequency at which the driver is gazing at locations inside relative to outside the vehicle;

or

a relative length of time during which the driver is gazing at locations inside relative to outside the vehicle; and

controlling operation of at least one subsystem of the vehicle based at least in part on an intention of the drive regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time, and the series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle.

7. The method of claim 6, wherein the relative frequency at which the driver is gazing at locations inside relative to outside the vehicle comprises:

a total amount of time during which the driver is gazing at locations inside the vehicle relative to a total amount of time during which the driver is gazing at locations outside the vehicle.

8. The method of claim 6, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative frequency exceeding a predetermined frequency.

9. The method of claim 6, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative length of time exceeding a predetermined length of time.

10. The method of claim 6, wherein the at least one subsystem of the vehicle is selected from a group consisting of:

a safety subsystem;

a braking subsystem;

a steering subsystem; and

an accelerator limiting system.

11. A non-transitory machine readable medium having instructions stored thereon for controlling operation of at least one subsystem of a vehicle on a roadway based on gaze directions of a driver of the vehicle over a period of time, wherein the instructions are executable by one or more processors for at least:

receiving gaze data indicative of a series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle;

determining, based on the gaze data, that the driver is gazing at locations inside and outside the vehicle;

determining, based on the gaze data, at least one of;

a relative frequency at which the driver is gazing at locations inside relative to outside the vehicle;

or

a relative length of time during which the driver is gazing at locations inside relative to outside the vehicle; and

controlling operation of at least one subsystem of the vehicle based at least in part on an intention of the drive regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time, and the series or sequence of gaze directions of the driver of the vehicle towards locations inside and outside the vehicle.

12. The non-transitory machine readable medium of claim 11, wherein the relative frequency at which the driver is gazing at locations inside relative to outside the vehicle comprises:

a total amount of time during which the driver is gazing at locations inside the vehicle relative to a total amount of time during which the driver is gazing at locations outside the vehicle.

13. The non-transitory machine readable medium of claim 11, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative frequency exceeding a predetermined frequency.

14. The non-transitory machine readable medium of claim 11, wherein controlling operation of at least one subsystem of the vehicle based at least in part on the intention of the driver regarding travel of the vehicle on the roadway as determined from at least the relative frequency or the relative length of time comprises:

controlling operation of at least one subsystem of the vehicle based at least in part on the relative length of time exceeding a predetermined length of time.