GESTURE BASED IMAGE CAPTURING SYSTEM FOR VEHICLE

• US 20150199019A1
• Filed: 01/16/2014
• Published: 07/16/2015
• Est. Priority Date: 01/16/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation, Denso International America Incorporated (DENSO Corporation)
• Original Assignee: DENSO Corporation, Denso International America Incorporated (DENSO Corporation)
• Inventors: Steers, Michael
• Granted Patent: US 9,430,046 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/176 Camera images
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/146 Input by gesture
• CPC Class Codes: B60K 2370/1464 3D-gesture
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/1529 Head-up displays
• CPC Class Codes: B60K 2370/176 Camera images
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 37/06 of controls, e.g. controls knobs
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

An image capturing system for a vehicle captures an image of a target area based on a predefined gesture performed by an occupant of the vehicle. A gesture capture device determines a gesture position of a gesture, which is the predefined gesture, with respect to a reference point. An eye tracking device determines an eye position of the occupant based on a viewpoint of the occupant. A target area determination module determines a target vector extending from the eye position toward the gesture position and aligns an on-board camera with the target vector, such that the target vector passes through an image field-of-view of the on-board camera. The on-board camera captures an image of the target area which is within the image field-of-view, and the image is stored in a computer-readable medium.

Claims

1. An image capturing system for a vehicle for capturing an image of a target area based on a predefined gesture performed by an occupant of the vehicle, the image capturing system comprising:

an on-board camera disposed on a body of the vehicle and capturing an image of the target area, wherein the on-board camera has an image field-of-view within which the image is captured and the image field-of-view overlaps with the target area;

a gesture capture device determining a gesture position of a gesture performed by the occupant with respect to a reference point, wherein the gesture is the predefined gesture;

an eye tracking device determining an eye position of the occupant;

a target area determination module determining a target vector extending from the eye position toward the gesture position and aligning the on-board camera in accordance with the target vector, such that the target vector passes through the image field-of-view of the on-board camera; and

a memory unit storing the image captured by the on-board camera, wherein the memory unit is a computer-readable medium.

2. The image capturing system of claim 1 wherein the target area determination module transmits the image to a display module of the vehicle, wherein the display module displays the image.

3. The image capturing system of claim 1 wherein the target area determination module transmits the image to a portable device via a wireless communication module disposed within the vehicle.

4. The image capturing system of claim 1 wherein the target area determination module transmits the image to a data server via a wireless communication module disposed within the vehicle.

5. The image capturing system of claim 1 wherein the gesture capture device further comprises:

a gesture camera that captures movement within a space defined by a field of view of the gesture camera; and

a gesture determination module that determines whether the movement captured is of the predefined gesture.

6. The image capturing system of claim 1 wherein the eye tracking device determines a viewpoint of the occupant and determines the eye position based on the viewpoint.

7. The image capturing system of claim 6 wherein

the eye tracking device determines the eye position as a position of a right eye when the viewpoint is from the right eye, determines the eye position as a position of a left eye when the viewpoint is from the left eye, and determines the eye position as a position of a midpoint between the right eye and the left eye when the viewpoint includes both the right eye and the left eye.

8. The image capturing system of claim 1 wherein

the gesture capture device determines a size of the gesture, and

the target area determination module adjusts a zoom level and a size of the image field-of-view based on the size of the gesture, such that the target vector passes through the image field-of-view.

9. The image capturing system of claim 1 wherein

the on-board camera detects an object within the image field-of-view, and

the target area determination module adjusts a zoom level and a size of the image field-of-view based on the object detected by the on-board camera, such that the target vector passes through the image field-of-view after the zoom level and the size of the image field-of-view is adjusted.

10. An image capturing method for a vehicle for capturing an image of a target area with an on-board camera disposed on the vehicle, the method comprising:

determining whether a gesture performed by an occupant is a predefined gesture for capturing the image;

determining a position of the gesture when the gesture is the predefined gesture, wherein the position of the gesture is with respect to a reference point;

determining an eye position of the occupant with respect to the reference point;

determining a target vector based on the eye position and the position of the gesture, wherein the target vector extends from the eye position towards the position of the gesture;

aligning the on-board camera in accordance with the target vector, such that the target vector passes an image field-of-view of the on board camera;

capturing an image of the image field-of-view with the on-board camera; and

storing the image in a memory, wherein the memory is a computer readable medium.

11. The image capturing method of claim 10 further comprising:

using a display module disposed in the vehicle, displaying the image captured by the on-board camera.

12. The image capturing method of claim 10 further comprising:

using a wireless communication module disposed in the vehicle, transmitting the image to a portable device.

13. The image capturing method of claim 10 further comprising,

using a wireless communication module disposed in the vehicle, transmitting the image to a data server external of the vehicle.

14. The image capturing method of claim 10 wherein determining the eye position further comprises:

determining a viewpoint of the occupant as a right eye, a left eye, or both the right and the left eye; and

determining the eye position based on the viewpoint, wherein the eye position is a position of the right eye when the viewpoint is from the right eye, the eye position is a position of the left eye when the viewpoint is from the left eye, and the eye position is a position of a midpoint between the left eye and the right eye when the viewpoint is from both the right eye and the left eye.

15. An image selection system for a vehicle for selecting an image displayed along a plane by a display module based on a predefined gesture performed by an occupant of a vehicle, the system comprising:

a gesture capture device determining features of a gesture performed by an occupant, the features including a position of a center of the gesture with respect to a reference point, wherein the gesture is the predefined gesture;

an eye tracking device determining a viewpoint of the occupant and an eye position of the occupant based on the viewpoint, wherein the eye position is the position of the viewpoint with respect to the reference point; and

a target area determination module determining a target vector extending from the eye position toward the center of the gesture, and determining a position along the plane at which the target vector intersects the plane.

16. The image selection system of claim 15 wherein the target area determination module transmits the position to the display module which determines whether the position correlates with the image being displayed.

17. The image selection system of claim 15 wherein:

the eye tracking device determines the viewpoint based on whether the occupant is using the right eye, the left eye or both the right eye and the left eye, and

the eye tracking device determines the eye position as a position of a right eye when the viewpoint is from the right eye, determines the eye position as a position of a left eye when the viewpoint is from the left eye, and determines the eye position as a position of a midpoint between the right eye and the left eye when the viewpoint includes both the right eye and the left eye.