Eye tracking control of vehicle entertainment systems

• US 8,928,585 B2
• Filed: 09/06/2012
• Issued: 01/06/2015
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: Thales Avionics Incorporated (Thales SA)
• Original Assignee: Thales Avionics Incorporated (Thales SA)
• Inventors: Mondragon, Christopher K., Bleacher, Brett
• Primary Examiner(s): LEE, NICHOLAS J
• Application Number:
  US13/605,176
• Publication Number:
  US 20130063340A1
• Time in Patent Office: 852 Days
• Field of Search: None
• US Class Current: 345/156
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/0028 of a body part, e.g. of an arm or a leg
• CPC Class Codes: B60N 2210/24 Cameras
• CPC Class Codes: B64D 11/0015 Arrangements for entertainment or communications, e.g. radio, television
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

An in-flight entertainment system includes a video display unit facing a user seat. The video display unit includes a display surface that displays images to a user who is seated on the user seat. A light emitter illuminates eyes of the user. A camera outputs a video signal containing reflections from the illuminated eyes. A processor processes the video signal to determine a viewing location on the display surface at which the eyes are directed, and controls at least one function for how images are displayed on the display surface responsive to the determined viewing location.

Claims

1. A video display unit comprising:

a display surface configured to display images to a user;

a light emitter configured to illuminate eyes of the user while seated in front of the video display unit;

a camera configured to output a video signal containing reflections from the illuminated eyes;

a processor configured to process the video signal to determine a viewing location on the display surface at which the eyes are directed, and to control at least one function for how images are displayed on the display surface responsive to the determined viewing location; and

a camera angle actuator configured to rotate a viewing angle of the camera relative to the display surface,

wherein the processor is further configured to process the video signal to detect a location of the user'"'"'s face relative to the camera, and to control the camera angle actuator to rotate the camera toward the user'"'"'s face to receive reflections from the illuminated eyes.

2. The video display unit of claim 1, wherein the processor is further configured to:

display target indicia in at least two opposite corner areas of the display surface;

process the video signal to detect when the eyes are looking at a first one of the target indicia, and to responsively determine a first angle to the eyes;

process the video signal to detect when the eyes are looking at a second one of the target indicia that is located within an opposite corner area of the display surface relative to the first target indicia, and to responsively determine a second angle to the eyes;

determine a location of the eyes using the first and second angles; and

control the camera angle actuator to rotate the camera toward the location of the eyes.

15. The video display unit of claim 1, wherein:

the video display unit is part of an in-flight entertainment system and the video display unit is attached to a back surface of a user seat with the display surface to display images to a user who is seated on another user seat facing the user seat.

3. A video display unit comprising;

a display surface configured to display images to a user;

a light emitter configured to illuminate eyes of the user while seated in front of the video display unit;

a camera configured to output a video signal containing reflections from the illuminated eyes; and

a processor configured to;

process the video signal to determine a viewing location on the display surface at which the eyes are directed,

display a plurality of user selectable indicia on the display surface, wherein a user'"'"'s selection of different ones of the user selectable indicia causes the processor to perform a corresponding different function controlling how images are displayed on the display surface;

detect that the user has selected a first one of the user selectable indicia in response to determining that the viewing location corresponds to a location of the first user selectable indicia for at least a threshold selection dwell time

display the user selectable indicia with no less than a defined minimum dimensional size value on the display surface;

determine detection accuracy over a plurality of user'"'"'s attempted selections of at least one of the user selectable indicia; and

control the defined minimum dimensional size value of the user selectable indicia displayed on the display surface in response to the determined detection accuracy.

4. The video display unit of claim 3, wherein the processor is further configured to:

detect that the user has selected the first one of the user selectable indicia in response to further determining, from processing of the video signal, that the user'"'"'s eyes remained closed while blinking for at least a threshold blink time while the previously determined viewing location corresponds to the location of the first user selectable indicia.

5. The video display unit of claim 3, wherein the processor is further configured to:

receive information identifying an angle of recline of a seat back surface on which the user rests; and

control the defined minimum dimensional size value in response to the angle of recline of the seat back surface.

6. The video display unit of claim 3, wherein the processor is further configured to:

display the user selectable indicia with no less than a defined minimum dimensional size value on the display surface;

process the video signal to determine distance between the user'"'"'s eyes; and

control the defined minimum dimensional size value in response to the distance between the user'"'"'s eyes.

16. The video display unit of claim 3, wherein:

the video display unit is part of an in-flight entertainment system and the video display unit is attached to a back surface of a user seat with the display surface to display images to a user who is seated on another user seat facing the user seat.

7. A video display unit comprising:

a display surface configured to display images to a user;

a light emitter configured to illuminate eyes of the user while seated in front of the video display unit;

a camera configured to output a video signal containing reflections from the illuminated eyes;

an acceleration sensor that generates a vibration signal that is indicative of an amount of vibration experienced by the display surface, wherein the video display unit is part of an in-flight entertainment system and the display surface is attached to a back surface of a user seat to display images to a user who is seated on another user seat facing the user seat; and

a processor configured to;

process the video signal to determine a viewing location on the display surface at which the eyes are directed,

display a plurality of user selectable indicia on the display surface, wherein a user'"'"'s selection of different ones of the user selectable indicia causes the processor to perform a corresponding different function controlling how images are displayed on the display surface;

detect that the user has selected a first one of the user selectable indicia in response to determining that the viewing location corresponds to a location of the first user selectable indicia for at least a threshold selection dwell time;

display the plurality of user selectable indicia with no less than a defined minimum dimensional size value on the display surface; and

enlarge the defined minimum dimensional size of the plurality of user selectable indicia concurrently displayed on the display surface from a first value to a second value in response to detecting a threshold amount of vibration indicated by the vibration signal.

17. The video display unit of claim 7, wherein subsequent to enlarging the defined minimum dimensional size value of the plurality of user selectable indicia concurrently displayed on the display surface from a first value to a second value in response to detecting the threshold amount of vibration indicated by the vibration signal, the processor is further configured to:

return the defined minimum dimensional size value of the plurality of user selectable indicia concurrently displayed on the display surface back to the first value in response to detecting less than the threshold amount of vibration indicated by the vibration signal.

8. A video display unit comprising:

a display surface configured to display images to a user;

a light emitter configured to illuminate eyes of the user while seated in front of the video display unit;

a camera configured to output a video signal containing reflections from the illuminated eyes;

a processor configured to process the video signal to determine a viewing location on the display surface at which the eyes are directed, and to control at least one function for how images are displayed on the display surface responsive to the determined viewing location;

a plurality of the light emitters spaced apart and arranged horizontally across the video display unit to form a horizontal array; and

a plurality of the light emitters spaced apart and arranged vertically along the video display unit to form a vertical array,

wherein the processor is further configured to sequentially pulse different ones of the light emitters and to process the video signal to identify one of the light emitters in the horizontal array that creates a brightest eye reflection and to identify one of the light emitters in the vertical array that creates a brightest eye reflection, and to determine the viewing location on the display surface responsive to the identified ones of the light emitters in the horizontal and vertical arrays that create the brightest eye reflections.

9. The video display unit of claim 8, wherein:

the horizontal array of the light emitters is arranged adjacent to a bottom surface of the display area; and

the vertical array of the light emitters is arranged adjacent to a side surface of the display area.

10. The video display unit of claim 8, wherein the horizontal array comprises a first horizontal array, and the vertical array comprises a first vertical array, and further comprising:

a plurality of the light emitters spaced apart and arranged horizontally across the video display unit to form a second horizontal array that is spaced apart on an opposite side of the display surface from the first horizontal array; and

a plurality of the light emitters spaced apart and arranged vertically along the video display unit to form a second vertical array that is spaced apart on an opposite side of the display surface from the first vertical array,

wherein the processor is further configured to sequentially pulse different ones of the light emitters and process the video signal to identify one of the light emitters in the first horizontal array that creates a brightest eye reflection, to identify one of the light emitters in the second horizontal array that creates a brightest eye reflection, to identify one of the light emitters in the first vertical array that creates a brightest eye reflection, to identify one of the light emitters in the second vertical array that creates a brightest eye reflection, and to determine the viewing location on the display surface responsive to the identified ones of the light emitters in the first horizontal array, the second horizontal array, the first vertical array, and the second vertical array that create the brightest eye reflections.

11. The video display unit of claim 8, wherein:

the processor is further configured to process the video signal to estimate an offset distance of the eyes from the display surface, and to further determine the viewing location on the display surface responsive to the offset distance.

12. The video display unit of claim 8, wherein the processor is further configured to:

process the video signal to estimate an offset distance of the eyes from the display surface;

select a number of the light emitters within the horizontal array to pulse as a group at a same time responsive to the offset distance;

select a number of the light emitters within the vertical array to pulse as a group at a same time responsive to the offset distance; and

process the video signal to identify which group of the light emitters in the horizontal array creates a brightest eye reflection and to identify which group of the light emitters in the vertical array creates a brightest eye reflection, and to determine the viewing location on the display surface responsive to the identified groups of the light emitters in the horizontal and vertical arrays that create the brightest eye reflections.

13. The video display unit of claim 12, wherein:

the processor is further configured to select a greater number of the light emitters within the horizontal array to pulse as a group at a same time responsive to an increased offset distance, and to select a greater number of the light emitters in the vertical array to pulse as a group at a same time responsive to the increased offset distance.

14. The video display unit of claim 8, wherein:

the processor is further configured to display a prompt to the user seated on a user seat within an aircraft to turn-off overhead lighting and/or to close a side window blind responsive to the processor detecting more than a threshold level of lighting in the video signal which interferes with the determination of the viewing location on the display surface.

18. The video display unit of claim 8, wherein:

the video display unit is part of an in-flight entertainment system and the video display unit is attached to a back surface of a user seat with the display surface to display images to a user who is seated on another user seat facing the user seat.