SOUND TO HAPTIC EFFECT CONVERSION SYSTEM USING MULTIPLE ACTUATORS

• US 20130265286A1
• Filed: 04/04/2012
• Published: 10/10/2013
• Est. Priority Date: 04/04/2012
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: HEUBEL, Robert, CRUZ-HERNANDEZ, Juan Manuel, GRANT, Danny, DA COSTA, Henry, TIMONE, Herve, RANK, Steve
• Granted Patent: US 9,715,276 B2
• Time in Patent Office: Days
• US Class Current: 345/177
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

A haptic conversion system is provided that analyzes an audio signal, generates a plurality of haptic signals based on the analysis of the audio signal, and plays the generated plurality of haptic signals through a plurality of actuators to produce one or more haptic effects. The haptic conversion system maps the generated plurality of haptic signals to the plurality of actuators based on one or more audio characteristics of the audio signal. Each generated haptic signal includes one or more haptic parameters, and is played at its mapped actuator to generate the one or more haptic effects.

Claims

1. A computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an audio signal into one or more haptic effects, the instructions comprising:

analyzing the audio signal;

generating a plurality of haptic signals based on one or more audio characteristics of the audio signal, wherein each haptic signal comprises one or more haptic parameters;

mapping the plurality of haptic signals to a plurality of actuators, wherein each haptic signal is mapped to a corresponding actuator;

sending each haptic signal to its mapped actuator; and

playing each haptic signal at its mapped actuator to generate a haptic effect of the one or more haptic effects.

2. The computer-readable medium of claim 1, the instructions further comprising:

selecting one or more pre-defined haptic signals from a universal haptic layer library based on the one or more audio characteristics of audio signal;

mapping the one or more pre-defined haptic signals to the plurality of actuators, wherein each pre-defined haptic signal is mapped to a corresponding actuator; and

sending each pre-defined haptic signal to its mapped actuator.

3. The computer-readable medium of claim 1, the instructions further comprising:

generating one or more secondary haptic signals based on a stored definition, wherein the definition comprises a conditional statement that comprises a condition and a statement, wherein the condition is the generation of one or more primary haptic signals that are sent to a primary actuator, and the statement is an instruction to generate the one or more secondary haptic signals that are sent to a secondary actuator;

mapping the one or more secondary haptic signals to the secondary actuator; and

sending each secondary haptic signal to the secondary actuator based on the stored definition.

4. The computer-readable medium of claim 1, the instructions further comprising:

dividing at least one haptic signal into a plurality of haptic signal components;

mapping the plurality of haptic signal components to the plurality of actuators, wherein each haptic signal component is mapped to the corresponding actuator; and

sending the plurality of haptic signal components to its mapped actuator.

5. The computer-readable medium of claim 1, the instructions further comprising:

combining at least two signals of the one or more haptic signals into a combined haptic signal;

mapping the combined haptic signal to an actuator; and

sending the combined haptic signal to the actuator.

6. The computer-readable medium of claim 1, wherein the one or more haptic parameters comprise at least one of:

an amplitude parameter, a duration parameter, or a frequency parameter.

7. The computer-readable medium of claim 1, wherein the one or more haptic parameters are determined based on the one or more audio characteristics of the audio signal.

8. The computer-readable medium of claim 7, wherein the one or more audio characteristics comprise at least one of:

an amplitude of an audio data frame, a frequency of an audio data frame, or a duration of an audio data frame.

9. The computer-readable medium of claim 1,

wherein the mapping is system-defined, and

wherein the mapping is based on either the one or more audio characteristics of the analyzed audio signal, or a combination of the one or more audio characteristics and one or more characteristics of each actuator of the plurality of actuators.

10. The computer-readable medium of claim 1, wherein the mapping is user-defined.

11. A computer-implemented method for converting an audio signal into one or more haptic effects, the computer-implemented method comprising:

analyzing the audio signal;

generating a plurality of haptic signals based on one or more audio characteristics of the audio signal, wherein each haptic signal comprises one or more haptic parameters;

mapping the plurality of haptic signals to a plurality of actuators, wherein each haptic signal is mapped to a corresponding actuator;

sending each haptic signal to its mapped actuator; and

playing each haptic signal at its mapped actuator to generate a haptic effect of the one or more haptic effects.

12. The computer-implemented method of claim 11, further comprising:

selecting one or more pre-defined haptic signals from a universal haptic layer library based on the one or more audio characteristics of audio signal;

mapping the one or more pre-defined haptic signals to the plurality of actuators, wherein each pre-defined haptic signal is mapped to a corresponding actuator; and

sending each pre-defined haptic signal to its mapped actuator.

13. The computer-implemented method of claim 11, further comprising:

generating one or more secondary haptic signals based on a stored definition, wherein the definition comprises a conditional statement that comprises a condition and a statement, wherein the condition is the generation of one or more primary haptic signals that are sent to a primary actuator, and the statement is an instruction to generate the one or more secondary haptic signals that are sent to a secondary actuator;

mapping the one or more secondary haptic signals to the secondary actuator; and

sending each secondary haptic signal to the secondary actuator based on the stored definition.

14. The computer-implemented method of claim 11, further comprising:

dividing at least one haptic signal into a plurality of haptic signal components;

mapping the plurality of haptic signal components to the plurality of actuators, wherein each haptic signal component is mapped to the corresponding actuator; and

sending the plurality of haptic signal components to its mapped actuator.

15. The computer-implemented method of claim 11, further comprising:

combining at least two signals of the one or more haptic signals into a combined haptic signal;

mapping the combined haptic signal to an actuator; and

sending the combined haptic signal to the actuator.

16. A haptic conversion system comprising:

a memory configured to store a haptic conversion module;

a processor configured to execute the haptic conversion module stored on the memory; and

a plurality of actuators, each actuator of the plurality of actuators configured to output one or more haptic effects;

wherein the haptic conversion module is configured to;

analyze the audio signal;

generate one or more haptic signals based on one or more audio characteristics of the audio signal, wherein each haptic signal of the one or more haptic signals comprises one or more haptic parameters;

map the plurality of haptic signals to a plurality of actuators, wherein each haptic signal is mapped to a corresponding actuator;

send each haptic signal to its mapped actuator; and

play each haptic signal at its mapped actuator to generate a haptic effect of the one or more haptic effects.

17. The haptic conversion system of claim 16, wherein the haptic conversion module is further configured to:

select one or more pre-defined haptic signals from a universal haptic layer library based on the one or more audio characteristics of audio signal;

map the one or more pre-defined haptic signals to the plurality of actuators, wherein each pre-defined haptic signal is mapped to a corresponding actuator; and

send each pre-defined haptic signal to its mapped actuator.

18. The haptic conversion system of claim 16, wherein the haptic conversion module is further configured to:

generate one or more secondary haptic signals based on a stored definition, wherein the definition comprises a conditional statement that comprises a condition and a statement, wherein the condition is the generation of one or more primary haptic signals that are sent to a primary actuator, and the statement is an instruction to generate the one or more secondary haptic signals that are sent to a secondary actuator;

map the one or more secondary haptic signals to the secondary actuator; and

send each secondary haptic signal to the secondary actuator based on the stored definition.

19. The haptic conversion system of claim 16, wherein the haptic conversion module is further configured to:

divide at least one haptic signal into a plurality of haptic signal components;

map the plurality of haptic signal components to the plurality of actuators, wherein each haptic signal component is mapped to the corresponding actuator; and

send the plurality of haptic signal components to its mapped actuator.

20. The haptic conversion system of claim 16, wherein the haptic conversion module is further configured to:

combine at least two signals of the one or more haptic signals into a combined haptic signal;

map the combined haptic signal to an actuator; and

send the combined haptic signal to the actuator.

21. A device, comprising:

a surface;

a plurality of actuators located within the surface, wherein each actuator is located in a different region of the surface, and wherein each actuator is configured to output one or more audio effects and one or more haptic effects;

wherein the device is configured to analyze an audio signal used to cause an actuator to output an audio effect, generate a haptic signal used to cause the actuator to output a haptic effect, and play the haptic signal at the actuator to generate the haptic effect.

22. The device of claim 21, wherein the surface comprises a touchscreen.