Method, system, and device for converting audio signal to one or more haptic effects

• US 10,431,057 B2
• Filed: 08/13/2018
• Issued: 10/01/2019
• Est. Priority Date: 02/11/2011
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Bhatia, Satvir Singh, Gandhi, Kanav, Ullrich, Christopher J., Cruz-Hernandez, Juan Manuel, Timone, Herve Thu, Lau, Jason Hoi Fun
• Primary Examiner(s): Tweel, Jr., John A
• Application Number:
  US16/102,464
• Publication Number:
  US 20190005782A1
• Time in Patent Office: 414 Days
• Field of Search: 3404071
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04

Abstract

A haptic conversion system is provided that intercepts frames of audio data, such as a digital audio signal, converts the frames into a haptic signal, and plays the created haptic signal through an actuator to produce haptic effects. The haptic signal is based on a maximum value of each audio data frame, which defines a magnitude of the haptic signal. The haptic signal is applied to the actuator configured to receive the haptic signal, where the actuator utilizes the haptic signal to generate the one or more haptic effects.

Claims

1. A method for converting an audio signal into one or more haptic effects, the method being performed by a processor and comprising:

receiving a set of samples of the audio signal;

performing frequency filtering on the set of samples to generate a filtered set of samples;

generating a haptic signal by mixing the filtered set of samples with a periodic waveform; and

sending the haptic signal to an actuator to generate the one or more haptic effects.

2. The method of claim 1, wherein the periodic waveform has a frequency that is a resonant frequency of the actuator.

3. The method of claim 1, wherein the filtered set of samples forms a filtered audio data frame, and wherein the periodic waveform has a frequency that is equal to a frequency of the filtered audio data frame.

4. The method of claim 1, wherein the periodic waveform has an amplitude that is based on a maximum value of the filtered set of samples.

5. The method of claim 1, wherein mixing the filtered set of samples with the periodic waveform comprises generating a weighted sum of the filtered set of samples and the periodic waveform.

6. The method of claim 5, wherein the weighted sum is equal to a first waveform added to a second waveform, wherein the first waveform is equal to a first number multiplied by the filtered set of samples, and the second waveform is equal to a second number multiplied by the periodic waveform, wherein the second number is equal to 1 minus the first number, the first number being a decimal value greater than 0 and less than 1.

7. The method of claim 1, wherein the frequency filtering is performed with a filter that is configured to extract, from the set of samples of the audio signal, frequency content that is lower than 500 Hz.

8. The method of claim 1, wherein the frequency filtering is performed with a filter that is configured to extract, from the set of samples of the audio signal, frequency content that is lower than 200 Hz.

9. The method of claim 1, wherein the frequency filtering is performed with a frequency filter selected by a user input.

10. The method of claim 1, wherein the haptic signal is sent to the actuator only when a maximum value of the set of samples is greater than a specified threshold.

11. The method of claim 1, wherein the set of samples are resampled before the frequency filtering is performed.

12. The method of claim 1, wherein the periodic waveform is one of a plurality of periodic waveforms that have different respective frequencies, and wherein the haptic signal is generated by mixing the filtered set of samples with the plurality of periodic waveforms.

13. The method of claim 1, wherein the set of samples are received from an audio buffer.

14. The method of claim 1, wherein the periodic waveform is a sine carrier waveform.

15. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, causes the processor

to receive a set of samples of the audio signal;

to perform frequency filtering on the set of samples to generate a filtered set of samples;

to generate a haptic signal by mixing the filtered set of samples with a periodic waveform; and

to send the haptic signal to an actuator to generate the one or more haptic effects.

16. A method for converting an audio signal into one or more haptic effects, the method comprising:

receiving a set of samples of the audio signal;

determining a maximum value of the set of samples;

performing frequency filtering on the set of samples to generate a filtered set of samples;

determining a first waveform that is based on the filtered set of samples;

determining a second waveform that is based on a periodic waveform, wherein the periodic waveform has an amplitude that is based on the maximum value of the set of samples;

generating a haptic signal that is a sum of the first waveform and the second waveform, and

sending the haptic signal to an actuator to generate the one or more haptic effects.

17. The method of claim 16, wherein the first waveform is equal to a first number multiplied by the filtered set of samples, and wherein the second waveform is equal to a second number multiplied by the periodic waveform, wherein the first number is a decimal value that is greater than 0 and less than 1, and wherein the second number is equal to 1 minus the first number.

18. The method of claim 16, wherein the periodic waveform has a frequency that is a resonant frequency of the actuator.

19. The method of claim 18, wherein the frequency filtering is performed with a filter that is configured to extract, from the set of samples of the audio signal, frequency content that is lower than 500 Hz.