INTERACTING WITH VEHICLE CONTROLS THROUGH GESTURE RECOGNITION

• US 20130204457A1
• Filed: 02/06/2012
• Published: 08/08/2013
• Est. Priority Date: 02/06/2012
• Status: Abandoned Application

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: King, Anthony Gerald, Remillard, Jeffrey Thomas, Greenberg, Jeff Allen
• Application Number:
  US13/366,388
• Publication Number:
  US 20130204457A1
• Time in Patent Office: Days
• US Class Current: 701/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/148 Instrument input by voice
• CPC Class Codes: B60K 2360/595 Data transfer involving internal databases
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture-based recognition system obtains a vehicle occupant'"'"'s desired command inputs through recognition and interpretation of his gestures. An image of the vehicle'"'"'s interior section is captured and the occupant'"'"'s image is separated from the background, in the captured image. The separated image is analyzed and a gesture recognition processor interprets the occupant'"'"'s gesture from the image. A command actuator renders the interpreted desired command to the occupant along with a confirmation message, before actuating the command. When the occupant confirms, the command actuator actuates the interpreted command. Further, an inference engine processor assesses the occupant'"'"'s state of attentiveness and conveys signals to a drive assist system if the occupant in inattentive. The drive-assist system provides warning signals to the inattentive occupant if any potential threats are identified. Further, a driver recognition module readjusts a set of vehicle'"'"'s personalization functions to pre-stored settings, on recognizing the driver.

Claims

1. A gesture-based recognition system for interpreting a vehicle occupant'"'"'s gesture and obtaining the occupant'"'"'s desired command inputs through gesture recognition, the system comprising:

a means for capturing an image of the vehicle'"'"'s interior section;

a gesture recognition processor adapted to separate the occupant'"'"'s image from the captured image, and further adapted to interpret occupant'"'"'s gestures from the image and generate an output; and

a command actuator coupled to the gesture recognition processor and adapted to receive the output therefrom, interpret a desired command, and actuate the command based on a confirmation received from the occupant.

2. A system of claim 1, wherein the means includes a camera configured to obtain a two dimensional image or a three dimensional depth-map of the vehicle'"'"'s interior section.

3. A system of claim 1, wherein the command actuator includes a user interface configured to display the desired command and a corresponding confirmation message, prompting the occupant to provide the confirmation.

4. A system of claim 1, wherein the command actuator includes a communication module configured to verbally communicate the interpreted occupant'"'"'s gesture to the occupant, and a voice-recognition module configured to recognize a corresponding verbal confirmation from the occupant.

5. A system of claim 1, wherein the gesture recognition processor includes a database storing a set of pre-determined gesture images corresponding to different gesture-based commands.

6. A system of claim 5, wherein the pre-determined images include at least the images corresponding to knob-adjustment, zoom-in and zoom-out controls, click to select, scroll-through, flip-through, and click to drag.

7. A system of claim 1, wherein the gesture-recognition processor further comprises an inference engine processor configured to assess the occupant'"'"'s attentiveness;

the system further comprising a drive-assist system coupled to the inference engine processor to receive inputs therefrom, if the occupant is inattentive.

8. A system of claim 6, further comprising a collision detection system coupled to the drive-assist system and the inference engine processor, the collision detection system being adapted to assess any potential threats and provide corresponding threat signals to the drive assist system.

9. A system of claim 1, wherein the gesture recognition processor includes a driver recognition module configured to recognize the driver'"'"'s image and re-adjust a set of personalization functions to a set of pre-stored settings corresponding to the driver, based on the recognition.

10. A system of claim 9, wherein the driver recognition module includes a facial database containing a set of pre-stored images, and is configured to compare features from the captured image with the images in the facial database.

11. A method of interpreting a vehicle occupant'"'"'s gesture and obtaining occupant'"'"'s desired command inputs through gesture-recognition, the method comprising:

capturing an image of the vehicle'"'"'s interior section;

separating the occupant'"'"'s image from the captured image, analyzing the separated image, and interpreting the occupant'"'"'s gesture from the separated image;

interpreting the occupant'"'"'s desired command, generating a corresponding confirmation message and delivering the message to the occupant; and

obtaining the confirmation from the occupant and actuating the command.

12. A method of claim 11, wherein capturing the image includes obtaining a two-dimensional image or a three-dimensional depth map of the vehicle'"'"'s interior.

13. A method of claim 11, further comprising rendering the interpreted desired command along with a corresponding confirmation message through a user interface.

14. A method of claim 11, further comprising verbally communicating the interpreted desired command and receiving a verbal confirmation from the occupant through voice-based recognition.

15. A method of claim 11, further comprising obtaining the confirmation from the occupant through gesture recognition.

16. A method of claim 11, further comprising comparing the captured image or the separated image with a set of pre-stored images corresponding to a set of pre-defined gestures, to interpret the occupant'"'"'s gesture.

17. A method of claim 11, further comprising assessing the occupant'"'"'s state of attentiveness and any potential threats, and providing warning signals to the occupant based on occupant'"'"'s state of attentiveness.

18. A method of claim 11, further comprising detecting a potential collision threat and providing warning signals to the occupant based on the detection.

19. A method of claim 11, further comprising recognizing the driver'"'"'s image in the separated image, and re-adjusting a set of personalization functions to a set of pre-stored settings.

20. A method of claim 19, wherein recognizing the driver'"'"'s image comprises comparing features of the captured image with the features of a set of pre-stored images in a facial database.