Gesture based input system in a vehicle with haptic feedback

• US 10,449,972 B2
• Filed: 07/31/2017
• Issued: 10/22/2019
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Truong, Dat
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US15/664,192
• Publication Number:
  US 20180015933A1
• Time in Patent Office: 813 Days
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A vehicle haptic feedback system includes a haptic actuator, a detection device, and a controller. The haptic actuator is configured to provide haptic feedback to a vehicle driver. The detection device is disposed in an interior of the vehicle and is configured to capture movements of the vehicle driver and output data representing the movements. The controller is coupled to the detection device and is configured to receive a signal from the detection device indicative of the movements. The controller is also coupled to a haptic effect feedback system. Based on the signal, the controller is configured to send a signal to select a haptic effect from an effect table and is configured to send a signal to the haptic actuator to provide the selected haptic effect. The haptic actuator is coupled to a driver'"'"'s seat, a steering wheel, a driver'"'"'s seat belt, and/or an object wearable by the driver.

Claims

1. A vehicle haptic feedback system comprising:

a haptic actuator configured to provide haptic feedback to a haptic feedback recipient, the haptic feedback recipient being one of a first or a second vehicle occupant;

a detection device disposed in an interior of the vehicle, wherein the detection device is configured to capture movements of a gesture provider and output a gesture signal representing the movements made by the gesture provider, the gesture provider being one of the first or the second vehicle occupant; and

a controller configured to

receive the gesture signal from the detection device indicative of the movements, wherein the controller is further configured to provide a haptic signal to the haptic actuator,

select a haptic effect based on the gesture signal indicative of the movements, and send the haptic signal to the haptic actuator to provide the selected haptic effect,

wherein the haptic actuator is coupled to at least one of the haptic feedback recipient'"'"'s seat, a steering wheel, the haptic feedback recipient'"'"'s seat belt, and/or an object that is wearable by the haptic feedback recipient.

2. The system of claim 1, wherein the movements of the gesture provider are hand and/or finger movements.

3. The system of claim 2, wherein the hand and/or finger movements are non-contact with respect to vehicle input devices.

4. The system of claim 1, wherein the haptic actuator is selected from the group consisting of electromagnetic motor actuators, eccentric rotating mass, linear resonant actuators, shape memory alloy actuators, electro-active polymer actuators, vibrotactile actuators, inertial actuators, piezoelectric actuators, and moving magnet actuators.

5. The system of claim 1, wherein the detection device is at least one camera.

6. The system of claim 1, wherein the controller is further configured to cause a vehicle component to perform at least one action in response to the gesture signal from the detection device.

7. The system of claim 6, further comprising at least one microphone configured to receive an audio command from the gesture provider, wherein the controller is further configured to receive an audio signal corresponding to the audio command and cause the vehicle component to perform the at least one action in response to the gesture signal from the detection device and the audio signal.

8. The system of claim 6, wherein the controller is configured to select the haptic effect according to the at least one action caused to be performed by the vehicle component.

9. The system of claim 1, wherein the controller is configured to select the haptic effect as confirmation that the movements of the gesture provider were captured.

10. The system of claim 1, further comprising at least one microphone configured to receive an audio command from the gesture provider, wherein the controller is further configured to receive an audio signal corresponding to the audio command and select the haptic effect according to the gesture signal and the audio signal.

11. A method of providing haptic feedback to a haptic feedback recipient including one of a first or a second vehicle occupant, the method comprising:

capturing movements of a gesture provider via a detection device disposed in an interior of the vehicle, wherein the gesture provider is one of the first or the second vehicle occupant;

outputting, by the detection device, a gesture signal representing the movements made by the gesture provider;

receiving, via a controller, the gesture signal from the detection device indicative of the movements;

selecting, via the controller, a haptic effect based on the gesture signal indicative of the movements;

sending, via the controller, a haptic signal to a haptic actuator to provide the selected haptic effect, and

providing the selected haptic effect to the haptic feedback recipient via the haptic actuator, the haptic actuator being coupled to at least one of the haptic feedback recipient'"'"'s seat, a steering wheel, the haptic feedback recipient'"'"'s seat belt, and/or an object that is wearable by the haptic feedback recipient.

12. The method of claim 11, wherein the movements of the gesture provider are hand and/or finger movements.

13. The method of claim 12, wherein the hand and/or finger movements are non-contact with respect to vehicle input devices.

14. The method of claim 11, wherein the haptic actuator is selected from the group consisting of electromagnetic motor actuators, eccentric rotating mass, linear resonant actuators, shape memory alloy actuators, electro-active polymer actuators, vibrotactile actuators, inertial actuators, piezoelectric actuators, and moving magnet actuators.

15. The method of claim 11, wherein the detection device is at least one camera.

16. The method of claim 11, further comprising causing, by the controller, a vehicle component to perform at least one action in response to the gesture signal from the detection device.

17. The method of claim 16, further comprising:

receiving, via at least one microphone, an audio command from the gesture provider;

receiving, via the controller, an audio signal corresponding to the audio command; and

causing the vehicle component to perform the at least one action in response to the gesture signal from the detection device and the audio signal.

18. The method of claim 16, further comprising selecting the haptic effect according to the at least one action caused to be performed by the vehicle component.

19. The method of claim 11, further comprising selecting the haptic effect as confirmation that the movements of the gesture provider were captured.

20. The method of claim 11, further comprising:

receiving, via at least one microphone, an audio command from the gesture provider;

receiving, via the controller, an audio signal corresponding to the audio command; and

selecting, by the controller, the haptic effect according to the gesture signal and the audio signal.