Haptic effect conversion system using granular synthesis

• US 9,733,710 B2
• Filed: 01/14/2016
• Issued: 08/15/2017
• Est. Priority Date: 06/14/2012
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Wu, Liwen, Cruz-Hernandez, Juan Manuel, Modarres, Ali, Birnbaum, David
• Primary Examiner(s): TRIEU, VAN THANH
• Application Number:
  US14/995,858
• Publication Number:
  US 20160162027A1
• Time in Patent Office: 579 Days
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

A system is provided that converts an input, such as audio data, into one or more haptic effects. The system applies a granular synthesis algorithm to the input in order to generate a haptic signal. The system subsequently outputs the one or more haptic effects based on the generated haptic signal. The system can also shift a frequency of the input, and also filter the input, before the system applies the granular synthesis algorithm to the input.

Claims

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an input signal into a haptic effect, the converting comprising:

receiving the input signal;

segmenting the input into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;

combining each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on a touchscreen;

receiving a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal;

combining the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and

generating the haptic effect based on the cloud.

2. The non-transitory computer-readable medium of claim 1, wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

3. The non-transitory computer-readable medium of claim 1, wherein the second grain is mapped to the position on the touchscreen.

4. The non-transitory computer-readable medium of claim 1, wherein the position signal is generated by a sensor configured to detect a form of energy.

5. The non-transitory computer-readable medium of claim 4, wherein the form of energy comprises a force, pressure or strain.

6. The non-transitory computer-readable medium of claim 1, further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.

7. The non-transitory computer-readable medium of claim 1, wherein the touchscreen comprises a plurality of pixels, wherein each pixel represents a position on the touchscreen.

8. A system for converting an input signal into a haptic effect, the system comprising:

a memory configured to store a haptic conversion module;

a processor configured to execute the haptic conversion module stored on the memory; and

a touchscreen coupled to the processor;

wherein the haptic conversion module is configured to receive the input signal;

wherein the haptic conversion module is further configured to segment the input signal into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;

wherein the haptic conversion module is further configured to combine each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on the touchscreen;

wherein the haptic conversion module is further configured to receive a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal

wherein the haptic conversion module is further configured to combine the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and

wherein the haptic conversion module is further configured to generate the haptic effect based on the cloud.

9. The system of claim 8, wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

10. The system of claim 8, wherein the second grain is mapped to the position on the touchscreen.

11. The system of claim 8, wherein the position signal is generated by a sensor configured to detect a form of energy.

12. The system of claim 11, wherein the form of energy comprises a force, pressure or strain.

13. The system of claim 8, further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.

14. The system of claim 8, wherein the touchscreen is comprises of a plurality of pixels, wherein each pixel represents a position on the touchscreen.

15. A method to convert an input signal into a haptic effect, the converting comprising:

receiving the input signal;

segmenting the input into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;

combining each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on a touchscreen;

receiving a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal;

combining the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and

generating the haptic effect based on the cloud.

16. The method of claim 15, wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

17. The method of claim 15, wherein the second grain is mapped to the position on the touchscreen.

18. The method of claim 15, wherein the position signal is generated by a sensor configured to detect a form of energy.

19. The method of claim 18 wherein the form of energy comprises a force, pressure or strain.

20. The method of claim 15, further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.