Sound to haptic effect conversion system using mapping

• US 9,818,271 B2
• Filed: 05/12/2016
• Issued: 11/14/2017
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Cruz-Hernandez, Juan Manuel, Ullrich, Christopher J.
• Primary Examiner(s): Monikang, George
• Application Number:
  US15/152,699
• Publication Number:
  US 20160260298A1
• Time in Patent Office: 551 Days
• Field of Search: 381 58- 59, 381 61- 62, 381118
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

A system generates a haptic signal. The system receives an audio signal, pre-processes the audio signal by modifying the audio signal to create a modified audio signal, and maps the modified audio signal to a haptic signal. The system then sends the haptic signal to an actuator to generate one or more haptic effects.

Claims

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to generate a haptic signal, the processor:

receiving an audio signal;

pre-processing the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;

mapping the modified audio signal to a haptic signal based on the one or more parameters; and

sending the haptic signal to an actuator to generate one or more haptic effects.

2. The non-transitory computer-readable medium of claim 1, wherein the modifying of the audio signal comprises under-sampling the audio signal.

3. The non-transitory computer-readable medium of claim 1, wherein the modifying of the audio signal comprises fusing an average of two or more channels of the audio signal.

4. The non-transitory computer-readable medium of claim 1, wherein the modifying of the audio signal comprises filtering the audio signal.

5. The non-transitory computer-readable medium of claim 1, wherein one or more frequencies in the haptic signal are selected based on one or more characteristics of the actuator.

6. The non-transitory computer-readable medium of claim 5, wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a low range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

7. The non-transitory computer-readable medium of claim 5, wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a middle range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

8. The non-transitory computer-readable medium of claim 5, wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a high range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

9. The non-transitory computer-readable medium of claim 1, wherein the mapping of the modified audio signal to the haptic signal comprises:

mapping the modified audio signal to one or more haptic signals; and

combining the one or more haptic signals into the haptic signal.

10. The non-transitory computer-readable medium of claim 1, wherein the mapping of the modified audio signal to the haptic signal comprises modifying the haptic signal based on one or more characteristics of the actuator.

11. The non-transitory computer-readable medium of claim 10, wherein the modifying of the haptic signal comprises filtering the haptic signal.

12. The non-transitory computer-readable medium of claim 10, wherein the mapping of the haptic signal is based on a dynamic range, a frequency response, an amplitude, or a physical location of the actuator.

13. The non-transitory computer-readable medium of claim 1, wherein the processor further maps the haptic signal to the actuator based on one or more characteristics of the actuator.

14. The non-transitory computer-readable medium of claim 13, wherein the one or more characteristics of the actuator include a dynamic range, a frequency response, an amplitude, or a physical location of the actuator.

15. The non-transitory computer-readable medium of claim 1, wherein the sending of the haptic signal to the actuator comprises sending the haptic signal to a haptic effect player, wherein the haptic effect player sends the haptic signal to the actuator to generate the one or more haptic effects.

16. A computer-implemented method for generating a haptic signal, comprising:

receiving an audio signal;

pre-processing the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;

mapping the modified audio signal to a haptic signal based on the one or more parameters; and

sending the haptic signal to an actuator to generate one or more haptic effects.

17. The method of claim 16, wherein the modifying of the audio signal comprises under-sampling the audio signal.

18. The method of claim 16, wherein the modifying of the audio signal comprises fusing an average of two or more channels of the audio signal.

19. The method of claim 16, wherein the modifying of the audio signal comprises filtering the audio signal.

20. A haptic signal generation system comprising:

a memory configured to store a haptic module;

a processor configured to execute the haptic module stored on the memory; and

an actuator configured to output one or more haptic effects;

wherein the haptic module is configured to;

receive an audio signal;

pre-process the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;

map the modified audio signal to a haptic signal based on the one or more parameters; and

send the haptic signal to the actuator to generate the one or more haptic effects.