TOUCHLESS HUMAN MACHINE INTERFACE

• US 20150316998A1
• Filed: 07/15/2015
• Published: 11/05/2015
• Est. Priority Date: 02/10/2011
• Status: Abandoned Application

• Current Assignee: Continental Automotive Systems, Inc. (Continental AG)
• Original Assignee: Continental Automotive Systems, Inc. (Continental AG)
• Inventors: Tang, Leung Yu
• Application Number:
  US14/800,050
• Publication Number:
  US 20150316998A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/11 Instrument graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1868 according to driving situations
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/11 Graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2370/146 Input by gesture
• CPC Class Codes: B60K 2370/1868 according to driving situations
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/1365 Matching; Classification

Abstract

A system and method for receiving input from a user is provided. The system includes at least one camera configured to receive an image of a hand of the user and a controller configured to analyze the image and issue a command based on the analysis of the image.

Claims

1. In a non-transitory computer readable storage medium having stored therein instructions executable by a programmed processor for receiving input to a vehicle system from a user of the system, the storage medium comprising instructions for:

capturing a series of background images of the interior of a vehicle over a predetermined time to produce a rolling window of background images;

after the rolling window of background averages is produced, capturing a plurality of captured images of background and an image of a finger print of a user'"'"'s finger;

subtracting the rolling window of background images from the plurality of captured images of the background and the image of a finger print of the user'"'"'s finger to produce a plurality of images of the finger print of user'"'"'s finger;

comparing the plurality of images of the finger print of the user'"'"'s finger to a stored finger print pattern to identify the user'"'"'s finger and to determine a position, orientation, and three-dimensional path of the user'"'"'s finger;

analyzing the position, orientation, and three dimensional path of the user'"'"'s finger to determine a gesture;

issuing a command to the vehicle system responsive to the determined gesture.

2. The computer readable storage medium according to claim 1, wherein the image is one of a plurality of images captured by a plurality of cameras on a front console of a vehicle.

3. The computer readable storage medium according to claim 1, further comprising instructions for determining positions of at least two fingers relative to one another in each image of a series of images to determine a gesture based on each finger path of the at least two fingers in three dimensional space.

4. The computer readable storage medium according to claim 3, further comprising instructions for determining the positions of the at least two fingers relative to one another in each image based on triangulation and the position and orientation of each camera.

5. The computer readable storage medium according to claim 4, further comprising instructions for determining the positions of the at least two fingers by generating an average of a plurality of background images and calculating a difference between each image in the series of images and the average of the plurality of background images.

6. A system for receiving input from a user of a vehicle, the system comprising:

at least two cameras configured to selectively capture images, a first image being an image of a background, a second image being captured after the first image and including both background and an image of a finger print of a user'"'"'s finger;

and

a controller coupled to the cameras and configured to subtract the first image from the second image to produce a difference image comprising an image of the finger print of the user, the controller being additionally configured to compare the image of the finger print to a stored pattern and determine from said comparison the identity of a finger of a user, the controller being additionally configured to determine the location of the finger of the user within a field of view of at least one of the cameras, the controller being additionally configured to determine a three dimensional path of the finger based on a model of the finger print in three dimensional space without the finger contacting a surface.

7. The system according to claim 6, wherein the controller is configured to match captured images of finger prints obtained from at least two fingers, to stored images of finger prints and wherein the controller is configured to determine the positions of the at least two fingers relative to one another in a plurality of successive images based on triangulation.

8. The system according to claim 7, wherein the controller is configured to determine the positions of the at least two fingers on the hand by first generating an average of a plurality of background images that do not include the hand and thereafter calculating a difference between each image in the series of images that include the hand and captured images of the finger prints of the at least two fingers.

9. A system for receiving input command gestures from a user in a vehicle, the system comprising:

at least one camera located in the vehicle and configured to receive a first series of images of a background obtained from inside the vehicle and, after the at least one camera receives the first series of images of background, receive a second image of a finger print of a user'"'"'s finger captured in the first series of images; and

a controller configured to average the first series of images of the background captured prior to capturing the second image and to subtract the average of the first series of images of the background from the second image to produce an image of the finger print of a user'"'"'s finger, the controller additionally configured to determine the identity of the user'"'"'s finger by comparing the image of the finger print of the user'"'"'s finger to a stored pattern of a finger print and issue a command based on whether the image of the finger print of the user'"'"'s finger matches the stored pattern of a finger print, the controller being additionally configured to determine a three dimensional path of the finger based on a model of the finger print to determine a gesture of the finger in three dimensional space.

10. The system according to claim 9, wherein the controller is configured to determine the position of one of more fingers in the image.

11. The system according to claim 10, further comprising a display screen in the vehicle, and wherein the at least one camera is a plurality of cameras located in predetermined positions inside the vehicle, each camera being configured and located to capture a series of images of finger prints, the controller being configured to identify the positions of the one or more fingers responsive to a determination that images of the finger prints of one or more fingers match stored patterns of finger prints.

12. The system according to claim 11, wherein the controller is configured to analyze each series of images to identify positions of the one or more fingers in three dimensional space.

13. The system according to claim 11, wherein the controller is configured to determine the positions of at least two fingers relative to one another.

14. The system according to claim 9, wherein the controller is configured to determine the positions of at least one finger in each image of a series of images to determine a finger path.

15. The system according to claim 9, wherein the controller is configured to determine the positions of at least two fingers relative to one another in each image of a series of images to determine a gesture based on each finger path of the at least two fingers in three dimensional space.

16. The system according to claim 9, wherein the controller is configured to determine the positions of at least two fingers relative to one another in each image based on triangulation and a position and orientation of each camera.

17. The system according to claim 9, wherein the controller is configured to determine the positions of at least two fingers by generating an average of a plurality of background images and calculating a difference between each image of a series of images and the average of the plurality of background images.

18. The system according to claim 9, wherein the controller is configured to match a stored model of a finger print to a finger print on at least one of the fingers.