Systems and Methods For Generating Haptic Effects Associated WIth An Envelope In Audio Signals

• US 20170162009A1
• Filed: 01/12/2017
• Published: 06/08/2017
• Est. Priority Date: 09/06/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Saboune, Jamal, Cruz-Hernandez, Juan Manuel
• Granted Patent: US 9,934,660 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G10L 21/16 Transforming into a non-visible representation devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04
• CPC Class Codes: H04L 27/265 Fourier transform demodulators, e.g. fast Fourier transform [FFT] or discrete Fourier transform [DFT] demodulators H04L27/26524 takes precedence

Abstract

Systems and methods for generating haptic effects associated with envelopes in audio signals are disclosed. One disclosed system for outputting haptic effects includes a processor configured to: receive an audio signal; determine an envelope associated with the audio signal; determine a haptic effect based in part on the envelope; and output a haptic signal associated with the haptic effect.

Claims

1-20. -20. (canceled)

21. A system for outputting haptic effects comprising:

a processor configured to;

receive an audio signal;

receive data associated with a haptic track;

synchronize the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determine a haptic effect based in part on the audio signal and the haptic track;

output a haptic signal associated with the haptic effect; and

output the audio signal to an audio output device configured to output an audible effect.

22. The system of claim 21, wherein the haptic effect comprises one or more of:

a variation in coefficient of friction, a simulated texture, or a vibration.

23. The system of claim 21, wherein the processor is configured to determine the envelope signal by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

26. The system of claim 21, wherein the processor is configured to determine the envelope signal by:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

24. The system of claim 24, wherein the time windows are overlapping.

25. The system of claim 24, wherein the envelope signal is the positive values of the normalized signal.

27. A method for outputting haptic effects comprising:

receiving an audio signal;

receiving data associated with a haptic track;

synchronizing the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determining a haptic effect based in part on the audio signal and the haptic track;

outputting a haptic signal associated with the haptic effect; and

outputting the audio signal to an audio output device configured to output an audible effect.

28. The method of claim 27, wherein the haptic effect comprises one or more of:

a variation in coefficient of friction, a simulated texture, or a vibration.

29. The method of claim 27, wherein the envelope signal associated with the audio signal comprises a signal determined by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

30. The method of claim 29, wherein the time windows are overlapping.

31. The method of claim 29, wherein the envelope signal is the positive values of the normalized signal.

32. The method of claim 27, wherein the envelope signal associated with the audio signal comprises a signal determined by:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

33. The method of claim 32, wherein the envelope signal is the interpolated and normalized sample signal.

34. The method of claim 27, wherein the processor is further configured to filter the envelope signal.

35. The method of claim 34, wherein filtering the envelope signal comprises one or more of:

filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

36. A non-transitory computer readable medium comprising program code, which when executed by a processor, is configured to cause the processor to:

receive an audio signal;

receive data associated with a haptic track;

synchronize the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determine a haptic effect based in part on the audio signal and the haptic track;

output a haptic signal associated with the haptic effect; and

output the audio signal to an audio output device configured to output an audible effect.

37. The non-transitory computer readable medium of claim 36, wherein the haptic effect comprises one or more of:

a variation in coefficient of friction, a simulated texture, or a vibration.

38. The non-transitory computer readable medium of claim 36, wherein the envelope signal associated with the audio signal comprises a signal determined by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

39. The non-transitory computer readable medium of claim 38, wherein the time windows are overlapping.

40. The non-transitory computer readable medium of claim 38, wherein the envelope signal is the positive values of the normalized signal.