SYSTEMS, METHODS, AND APPARATUS FOR CONTROLLING DEVICES BASED ON A DETECTED GAZE

• US 20140348389A1
• Filed: 12/29/2011
• Published: 11/27/2014
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Graumann, David L.
• Granted Patent: US 9,517,776 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 37/00 Dashboards as road-vehicle superstructure sub-units B62D25/14
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/00 Input parameters relating to occupants
• CPC Class Codes: B60W 2540/21 Voice
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 50/08 Interaction between the driver and the control system
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Certain embodiments of the invention may include systems, methods, and apparatus for controlling devices based on a detected gaze. According to an example embodiment of the invention, a method is provided for executing computer executable instructions by one or more processors for controlling one or more devices attached to a vehicle based at least in part on a detected direction of observation of an occupant of the vehicle, the method further includes receiving one or more images from at least one camera attached to the vehicle; locating, from the one or more images, one or more body features associated with the occupant; determining a direction of gaze from the one or more located body features; and controlling the one or more devices based at least in part on the direction of gaze.

Claims

1. A method comprising executing computer executable instructions by one or more processors for controlling one or more devices attached to a vehicle based at least in part on a detected direction of observation of an occupant of the vehicle, the method further comprising:

receiving one or more images from at least one camera attached to the vehicle;

locating, from the one or more images, one or more body features associated with the occupant;

determining a direction of gaze from the one or more located body features;

controlling the one or more devices based at least in part on the direction of gaze.

2. The method of claim 1, wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

3. The method of claim 1, wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

4. The method of claim 1, further comprising determining a gesture of the occupant from the one or more body features and controlling the one or more devices based at least in part on the gesture.

5. The method of claim 4, wherein the gesture comprises one or more of a hand movement or a voice command.

6. The method of claim 1, wherein controlling the one or more devices comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

7. The method of claim 1, wherein determining the direction of gaze comprises tracking the occupant'"'"'s head attitude duration and stability.

8. A vehicle comprising:

at least one camera attached to the vehicle;

one or more devices attached to the vehicle;

at least one memory for storing data and computer-executable instructions; and

one or more processors configured to access the at least one memory and further configured to execute computer executable instructions for;

receiving one or more images from the at least one camera;

locating, from the one or more images, one or more body features associated with an occupant of the vehicle;

determining a direction of gaze from the one or more located body features;

controlling the one or more devices based at least in part on the direction of gaze.

9. The vehicle of claim 8, wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

10. The vehicle of claim 8, wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

11. The vehicle of claim 8, wherein the one or more processors are further configured for determining a gesture of the occupant from the one or more body features and controlling the one or more devices based at least in part on the gesture.

12. The vehicle of claim 11, wherein the gesture comprises one or more of a hand movement or a voice command.

13. The vehicle of claim 8, wherein controlling the one or more devices comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

14. The vehicle of claim 8, wherein determining the direction of gaze comprises tracking the occupant'"'"'s head attitude duration and stability.

15. The vehicle of claim 8, wherein the one or more devices comprise one or more of a video screen, a mirror with an integrated display, a stereo system, a sun roof, a car window, a lock, a car seat, a climate control system, or a dashboard console.

16. An apparatus comprising:

at least one memory for storing data and computer-executable instructions; and

one or more processors configured to access the at least one memory and further configured to execute computer executable instructions for;

receiving one or more images from at least one camera attached to a vehicle;

locating, from the one or more images, one or more body features associated with an occupant of the vehicle;

determining a direction of gaze from the one or more located body features;

controlling the one or more devices based at least in part on the direction of Gaze.

17. The apparatus of claim 16, wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

18. The apparatus of claim 16, wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

19. The apparatus of claim 16, wherein the one or more processors are further configured for determining a gesture of the occupant from the one or more body features and controlling the one or more devices based at least in part on the gesture.

20. The apparatus of claim 19, wherein the gesture comprises one or more of a hand movement or a voice command.

21. The apparatus of claim 16, wherein controlling the one or more devices comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

22. The apparatus of claim 16, wherein determining the direction of gaze comprises tracking the occupant'"'"'s head attitude duration and stability.

23. The apparatus of claim 16, wherein the one or more devices comprise one or more of a video screen, a mirror with an integrated display, a stereo system, a sun roof, a car window, a lock, a car seat, a climate control system, or a dashboard console.

24. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for controlling one or more devices attached to a vehicle based at least in part on a detected direction of observation of an occupant of the vehicle, the method further comprising:

receiving one or more images from at least one camera attached to the vehicle;

locating, from the one or more images, one or more body features associated with the occupant;

determining a direction of gaze from the one or more located body features;

controlling the one or more devices based at least in part on the direction of gaze.

25. The computer program product of claim 24, wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

26. The computer program product of claim 24, wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

27. The computer program product of claim 24, further comprising determining a gesture of the occupant from the one or more body features and controlling the one or more devices based at least in part on the gesture.

28. The computer program product of claim 27, wherein the gesture comprises one or more of a hand movement or a voice command.

29. The computer program product of claim 24, wherein controlling the one or more devices comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

30. The computer program product of claim 24, wherein determining the direction of gaze comprises tracking the occupant'"'"'s head attitude duration and stability.