Gesture based input system in a vehicle with haptic feedback

• US 9,248,840 B2
• Filed: 12/20/2013
• Issued: 02/02/2016
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Truong, Dat
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US14/137,540
• Publication Number:
  US 20150175172A1
• Time in Patent Office: 774 Days
• Field of Search: 701/36
• US Class Current: 1/1
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A system for controlling vehicle systems includes a haptic actuator, a gesture detection device, a controller, and a plurality of vehicle systems. The gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger. The controller is coupled to the gesture detection device and receives a signal from the gesture detection device indicative of the gesture. The controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture and sends a signal to the haptic actuator to provide the selected haptic effect. The controller is also coupled to the vehicle systems, and based on the detected gesture, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.

Claims

1. A system for controlling one or more of a plurality of vehicle components comprising:

a haptic actuator configured to provide haptic feedback to a vehicle driver or passenger;

a gesture detection device disposed in an interior of the vehicle, wherein the gesture detection device is configured to capture movements of the vehicle driver or passenger and output data representing a gesture made by the vehicle driver or passenger; and

a controller coupled to the gesture detection device and configured to receive a signal from the gesture detection device indicative of the gesture, wherein the controller is coupled to a haptic effect feedback system and the plurality of vehicle components,

wherein based on the signal indicative of the gesture, the controller is configured to send a signal to select a haptic effect from an effect table and configured to send a signal to the haptic actuator to provide the selected haptic effect, and

wherein based on the signal indicative of the gesture, the controller is further configured to send a signal to at least one of the plurality of vehicle components to perform a particular action, wherein the particular action of the at least one of the plurality of vehicle components is matched with the selected haptic effect.

2. The system of claim 1, wherein the at least one of the plurality of vehicle components is a display and the signal from the controller to the display highlights an icon on the display.

3. The system of claim 2, wherein the controller is further configured to send a signal to a second one of the plurality of vehicle components to perform a second particular action, wherein the second particular action of the second one of the plurality of vehicle components is matched with the particular action of the display and the haptic effect.

4. The system of claim 1, wherein the plurality of vehicle components are selected from the group consisting of a display, radio, a CD player, a climate control system, a navigation system, a Bluetooth®

wireless communication system, and an auxiliary device system.

5. The system of claim 1, wherein the haptic actuator is selected from group consisting of electromagnetic motor actuators, eccentric rotating mass, linear resonant actuators, shape memory alloy actuators, electro-active polymer actuators, vibrotactile actuators, inertial actuators, piezoelectric actuators, and moving magnet actuators.

6. The system of claim 1, wherein the haptic actuator is coupled to a portion of an interior of the vehicle.

7. The system of claim 6, wherein the haptic actuator is coupled to at least one of a seat, a steering wheel, an armrest, a door rest, or a seat belt.

8. The system of claim 1, wherein the haptic actuator is coupled to an object that is wearable by the vehicle driver or passenger.

9. The system of claim 8, wherein the object that is wearable by the vehicle driver or passenger is at least one of a glove, a hat, a wrist band, or a wristlet.

10. The system of claim 1, wherein the gesture detection device is at least one camera.

11. The system of claim 1, further comprising:

an audio detection device disposed in an interior of the vehicle, wherein the audio detection device is configured to receive voice commands of the vehicle driver or passenger and to output data representing the voice command;

wherein the controller is coupled to the audio detection device and is configured to receive a signal from the audio detection device indicative of the voice command,

wherein based on the signal indicative of the voice command, the controller is configured to send a signal to select a haptic effect from the effect table and configured to send a signal to the haptic actuator to provide the selected haptic effect, and

wherein based on the signal indicative of the voice command, the controller is further configured to send a signal to the at least one of the plurality of vehicle components to perform a particular action, wherein the particular action of the at least one of the vehicle components is matched with the selected haptic effect.

12. The system of claim 11, wherein the audio detection device comprises a microphone.

13. A method of controlling one or more of a plurality of vehicle components comprising the steps of:

tracking movements of a driver or passenger of the vehicle to detect a gesture by the driver or passenger from a table of gestures;

selecting a haptic effect from a table of effects based on the detected gesture;

providing the haptic effect to the driver or passenger through a haptic actuator;

controlling at least one of the plurality of vehicle components to perform a particular action based on the detected gesture, wherein the particular action of the at least one of a plurality of vehicle components is matched with the selected haptic effect.

14. The method of claim 13, wherein the at least one of the vehicle components is a display and the particular action is highlighting an icon on the display based on the detected gesture.

15. The method of claim 14, further comprising the step of controlling at least a second one of a plurality of vehicle components to perform a second particular action, wherein the second particular action of the second one of the plurality of vehicle components is matched with the particular action of the display and the haptic effect.

16. The method of claim 13, wherein the plurality of vehicle systems includes a plurality of vehicle components selected from the group consisting of a display, radio, a CD player, a climate control system, a navigation system, a Bluetooth®

wireless communication system, and an auxiliary device system.

17. The method of claim 13, wherein the haptic actuator is selected from group consisting of electromagnetic motor actuators, eccentric rotating mass, linear resonant actuators, shape memory alloy actuators, electro-active polymer actuators, vibrotactile actuators, inertial actuators, piezoelectric actuators, and moving magnet actuators.

18. The method of claim 13, wherein the haptic actuator is coupled to an interior of the vehicle.

19. The method of claim 18, wherein the haptic actuator is coupled to at least one of a seat, a steering wheel, an armrest, a door rest, or a seat belt.

20. The method of claim 13, wherein the haptic actuator is coupled to an object that is wearable by the driver or passenger.

21. The method of claim 13, wherein the step of tracking movements of the driver or passenger is performed via a camera.

22. The method of claim 13, further comprising:

detecting sound in the vehicle to detect a voice command from the vehicle driver or passenger;

selecting a haptic effect from a table of effects based on the detected voice command;

providing the haptic effect through the haptic actuator;

controlling at least one of the plurality of vehicle components to perform a particular action based on the detected voice command, wherein the particular action of the at least one of a plurality of vehicle components is matched with the selected haptic effect.

23. The method of claim 22, wherein the step of detecting sound in the vehicle is performed via a microphone.