Controlling vehicle entertainment systems responsive to sensed passenger gestures

• US 9,037,354 B2
• Filed: 09/06/2012
• Issued: 05/19/2015
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: Thales Avionics Incorporated (Thales SA)
• Original Assignee: Thales Avionics Incorporated (Thales SA)
• Inventors: Mondragon, Christopher K., Bleacher, Brett, Allen, William W., Reichert, David
• Primary Examiner(s): Edwards, Jerrah
• Application Number:
  US13/605,137
• Publication Number:
  US 20130066526A1
• Time in Patent Office: 985 Days
• Field of Search: 701/3, 701/48, 701/49
• US Class Current: 701/48
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B64D 11/0015 Arrangements for entertainment or communications, e.g. radio, television
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A vehicle entertainment system is controlled responsive to gestures that are formed by a passenger of the vehicle. The vehicle entertainment system includes a display device, at least one gesture control camera, and a processor. The gesture control camera generates a camera signal responsive to light reflected from at least one object within a field of view of the at least one gesture control camera. The processor analyzes the camera signal to identify a gesture made by a passenger moving the at least one object, and controls at least one operation of the vehicle entertainment system responsive to the identified gesture.

Claims

1. A vehicle entertainment system, comprising:

a display device;

at least one gesture control camera configured to generate a camera signal responsive to light reflected from at least one object within a field of view of the at least one gesture control camera; and

a processor configured to;

analyze the camera signal to identify a gesture made by a passenger moving the at least one object,

control at least one operation of the vehicle entertainment system responsive to the identified gesture,

respond to an indication that the passenger has becoming seated in a seat facing the display device by performing an initial calibration to identify a distance to the passenger'"'"'s body and identify locations of at least one object in the foreground between the passenger'"'"'s body and the at least one gesture control camera, and

determine when the passenger'"'"'s body has moved at least a threshold distance away from or toward the at least one gesture control camera while remaining seated, and respond to the determination by performing a calibration to identify a distance to the passenger'"'"'s body and identify locations of at least one object in the foreground between the passenger'"'"'s body and the at least one gesture control camera.

2. The vehicle entertainment system of claim 1,

further comprising a gesture library that identifies a plurality of different movement gestures and associated commands;

and

wherein the processor is configured to;

track movement of the at least one object;

match the tracked movement to one of the plurality of different movement gestures in the gesture library;

identify the command that is associated with the matched movement gesture; and

perform the associated command to control an operation of the vehicle entertainment system.

3. The vehicle entertainment system of claim 2, wherein the processor is configured to access the gesture library to identify and then perform one of the following commands to control the vehicle entertainment system:

select one of a plurality of menu item indicia that are displayed on the display device to cause indicia for sub-menu items to be displayed on the display device;

select one of a plurality of movie indicia that are displayed on the display device to initiate playing of an associated movie on the display device;

select one of a plurality of application indicia that are displayed on the display device to initiate execution of an associated application by the processor;

control audio volume through an audio interface of the vehicle entertainment system;

control playing, pausing, fast forwarding, and/or rewinding of a movie on the display device; and

/or

control operation of a game being executed by the processor.

4. The vehicle entertainment system of claim 2, wherein the processor is configured to access the gesture library to identify and then perform one of the following commands:

control a recline actuator to regulate a recline angle of a passenger seat facing the display device;

control a massage actuator within the passenger seat; and

/or

control a foot rest actuator to regulate a position of a foot rest of the passenger seat.

5. The vehicle entertainment system of claim 2, wherein the processor is configured to access the gesture library to identify and then perform one of the following commands:

control operation of a window shade actuator to open and close a window shade that is adjacent to a passenger seat facing the display device;

control overhead lighting for the passenger seat;

control a passenger service function within an aircraft vehicle; and

/or

control communication of requests and/or responses to a flight attendant communication panel and/or terminal of an aircraft vehicle.

6. The vehicle entertainment system of claim 2, wherein the processor is configured to access the gesture library to identify one of the plurality of different movement gestures and associated commands by:

determining a location on the display device where a passenger'"'"'s finger and/or hand is pointed;

distinguishing between swiping motions at different defined angles made by the passenger'"'"'s finger and/or hand relative to the display device;

identifying changing distances between two or more fingers or between a pair of hands that are held in front of the display device; and

/or

identifying rotation of two or more fingers or rotation of a hand in front of the display device.

7. The vehicle entertainment system of claim 1, wherein the processor is configured to:

scroll in a defined direction a list of menu items that is displayed on the display device in response to tracking movement as a hand or finger swipes in a corresponding direction relative to the display device; and

select one of the displayed menu items in response to detecting that a finger is pointed at a location on the display device where the menu item is displayed.

8. The vehicle entertainment system of claim 1, wherein the processor is configured to:

identify a level of alertness of the passenger by tracking movement of at least a portion of the passenger'"'"'s body;

respond to tracking less than a threshold level of movement by performing at least one of the following commands to control the vehicle entertainment system;

turn-off and/or dim brightness of overhead lighting for a passenger seat facing the display device;

notify a flight attendant that the passenger is sleeping; and

/or

control operation of a window shade actuator to close a window shade that is adjacent to the passenger seat; and

respond to tracking more than the threshold level of movement by performing at least one of the following commands to control the vehicle entertainment system;

increase brightness of the overhead lighting;

notify a flight attendant that the passenger is awake; and

/or

control operation of a window shade actuator to open the window shade.

9. The vehicle entertainment system of claim 1, wherein the processor is configured to:

identify a level of alertness of the passenger by tracking movement of at least a portion of the passenger'"'"'s body;

respond to tracking less than a threshold level of movement by performing at least one of the following commands to control the vehicle entertainment system;

turn-off and/or dim brightness of backlighting of the display device; and

pause playback of a video stream through the display device; and

respond to tracking more than the threshold level of movement by performing at least one of the following commands to control the vehicle entertainment system;

increase brightness of backlighting of the display device; and

resuming playback of the video stream through the display device from a previously paused location.

10. The vehicle entertainment system of claim 1, wherein:

the at least one gesture control camera comprises a pair of gesture control cameras that are spaced apart from each other and directed to provide an overlapping field of view in an area in front of the display device for sensing the at least one object moved by the passenger;

each of the gesture control cameras includes at least one lens that is configured to restrict the depth of focus to a defined distance from the gesture control camera to create a passenger gesture control space within which object movement is tracked by the processor and outside of which object movement is not tracked by the processor when identifying a gesture made by the passenger.

11. A vehicle entertainment system, comprising:

a display device;

at least one gesture control camera configured to generate a camera signal responsive to light reflected from at least one object within a field of view of the at least one gesture control camera;

an acceleration sensor that outputs an acceleration signal that indicates a level of acceleration of the vehicle entertainment system; and

a processor configured to;

analyze the camera signal to identify a gesture made by a passenger moving the at least one object,

control at least one operation of the vehicle entertainment system responsive to the identified gesture, and

use the acceleration signal to at least partially compensate for effect of acceleration on the at least one object moved by the passenger when identifying the gesture made by the passenger, by

identifying presence of turbulence in response to the acceleration signal indicating at least a threshold level of acceleration; and

increasing a distance that the passenger needs to move the at least one object to generate a gesture that is identified by the processor, responsive to identifying the presence of turbulence.

12. The vehicle entertainment system of claim 11, wherein the processor is further configured to:

generate a velocity compensation vector responsive to integration of the acceleration signal over a defined time period; and

subtract the velocity compensation vector from a contemporaneous motion of the at least one object identified in the camera signal.

13. A vehicle entertainment system, comprising:

a display device;

a pair of gesture control cameras each configured to generate a camera signal responsive to light reflected from at least one object within a field of view of the gesture control camera, the pair of gesture control cameras are spaced apart from each other and tilted toward each other to provide a narrowed overlapping field of view in front of the display device for sensing the at least one object moved by a passenger when identifying a gesture made by the passenger; and

a processor configured to;

analyze the camera signals from the pair of gesture control cameras to identify a gesture made by the passenger moving the at least one object,

control at least one operation of the vehicle entertainment system responsive to the identified gesture, and

ignore movement that occurs within the field of view of only one, but not both, of the pair of gesture control cameras when identifying the gesture made by the passenger.

14. The vehicle entertainment system of claim 13, wherein:

the pair of gesture control cameras are spaced apart horizontally across a face of a housing that surrounds side surfaces of the display device, and are located on opposite sides of and equally distant from a vertical centerline of the display device.

15. A vehicle entertainment system, comprising:

a display device;

a pair of gesture control cameras each configured to generate a camera signal responsive to light reflected from at least one object within a field of view of the gesture control camera, the pair of gesture control cameras are spaced apart from each other and tilted away from each other to provide a narrowed overlapping field of view in front of the display device for sensing the at least one object moved by a passenger when identifying a gesture made by the passenger; and

a processor configured to;

analyze the camera signals from the pair of gesture control cameras to identify a gesture made by the passenger moving the at least one object,

control at least one operation of the vehicle entertainment system responsive to the identified gesture, and

ignore movement that occurs within the field of view of only one, but not both, of the pair of gesture control cameras when identifying the gesture made by the passenger.

16. The vehicle entertainment system of claim 15, wherein:

the pair of gesture control cameras are spaced apart less than 4 inches from each other and are located on opposite sides of and equally distant from a vertical centerline of the display device.

17. A vehicle entertainment system, comprising:

a display device;

a pair of gesture control cameras each configured to generate a camera signal responsive to light reflected from at least one object within a field of view of the gesture control camera, the pair of gesture control cameras are spaced apart from each other; and

a processor configured to;

analyze the camera signals from the pair of gesture control cameras to identify a gesture made by a passenger moving the at least one object,

control at least one operation of the vehicle entertainment system responsive to the identified gesture, and

determine distance between the at least one object and the pair of gesture control camera, to ignore object movement that occurs within the field of view of only one but not both of the gesture control cameras when identifying a gesture made by the passenger, and to further ignore object movement that is beyond a threshold distance of the gesture control cameras although being within the field of view of both of the gesture control cameras when identifying a gesture made by the passenger.