Stream-independent sound to haptic effect conversion system

• US 9,092,059 B2
• Filed: 10/26/2012
• Issued: 07/28/2015
• Est. Priority Date: 10/26/2012
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Bhatia, Satvir Singh
• Primary Examiner(s): Swarthout, Brent
• Application Number:
  US13/661,140
• Publication Number:
  US 20140118125A1
• Time in Patent Office: 1,005 Days
• Field of Search: 340/407.1, 704/235, 345/156, 345/173, 700/94, 710/6, 710/58
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

A haptic conversion system is provided that receives a plurality of audio streams of an audio signal. The haptic conversion system further determines, for each stream of the plurality of streams, whether at least one parameter of the one or more parameters indicates that the corresponding stream is to be converted into a haptic effect, and thus, identifies one or more streams that include at least one parameter that indicates the corresponding stream is to be converted into the haptic effect. The haptic conversion system further generates, for the identified streams, a haptic signal based on each corresponding stream. The haptic conversion system further sends, for the identified streams, each haptic signal to an actuator to generate a haptic effect.

Claims

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert one or more streams of an audio signal into one or more haptic effects, the converting comprising:

receiving a plurality of streams of the audio signal, wherein each stream comprises a plurality of parameters, and wherein the plurality of parameters comprise a haptic conversion indication parameter that indicates whether the corresponding stream is to be converted into a haptic effect and one or more haptic parameters that configure a haptic signal that is generated based on the stream;

determining, for each stream, whether the haptic conversion indication parameter indicates that the corresponding stream is to be converted into a haptic effect;

identifying one or more streams that include the haptic conversion indication parameter that indicates the corresponding stream is to be converted into the haptic effect, wherein a number of the one or more identified streams is less than a number of the plurality of streams;

generating one or more haptic signals based on the one or more identified streams; and

sending the one or more haptic signals to an actuator to generate one or more haptic effects.

2. The non-transitory computer-readable medium of claim 1, the converting further comprising configuring at least one haptic signal based on the one or more haptic parameters.

3. The non-transitory computer-readable medium of claim 2, wherein the one or more haptic parameters comprise at least one of a strength parameter, a density parameter, a magnitude parameter, a frequency parameter, an actuator parameter, or a fail-safe parameter;

and

the configuring further comprising at least one of;

configuring a strength of the at least one haptic signal, configuring a density of the at least one haptic signal, configuring a magnitude of the at least one haptic signal, configuring a frequency of the at least one haptic signal, configuring one or more specific actuators that the at least one haptic signal is sent to, or executing a fail-safe operation.

4. The non-transitory computer-readable medium of claim 2, wherein the configuring the at least one haptic signal is performed before the generating of the at least one haptic signal.

5. The non-transitory computer-readable medium of claim 2, wherein the configuring the at least one haptic signal is performed during the generating of the one or more haptic signals.

6. The non-transitory computer-readable medium of claim 2, wherein the configuring the at least one haptic signal is performed after the generating of the one or more haptic signals.

7. The non-transitory computer-readable medium of claim 1, wherein the audio signal is a pulse-code modulation audio buffer, and the stream is a pulse-code modulation stream.

8. The non-transitory computer-readable medium of claim 1, the converting further comprising sending a stream of the audio signal to a speaker to generate an audio effect regardless of whether the haptic conversion indicator parameter indicates that the stream is to be converted into the haptic effect.

9. A computer-implemented method for converting a plurality of streams of an audio signal into one or more haptic effects, the computer-implemented method comprising:

receiving a plurality of streams of the audio signal, wherein each stream comprises a plurality of parameters, and wherein the plurality of parameters comprise a haptic conversion indication parameter that indicates whether the corresponding stream is to be converted into a haptic effect and one or more haptic parameters that configure a haptic signal that is generated based on the stream;

determining, by a processor and for each stream, whether the haptic conversion indication parameter indicates that the corresponding stream is to be converted into a haptic effect;

identifying, by the processor, one or more streams that include the haptic conversion indication parameter that indicates the corresponding stream is to be converted into the haptic effect, wherein a number of the one or more identified streams is less than a number of the plurality of streams;

generating, by the processor, one or more haptic signals based on the one or more identified streams; and

sending the one or more haptic signals to an actuator to generate one or more haptic effects.

10. The computer-implemented method of claim 9, further comprising configuring at least one haptic signal based on the one or more haptic parameters.

11. The computer-implemented method of claim 10, wherein the one or more haptic parameters comprise at least one of a strength parameter, a density parameter, a magnitude parameter, a frequency parameter, an actuator parameter, or a fail-safe parameter;

and

the configuring further comprising at least one of;

configuring a strength of the at least one haptic signal, configuring a density of the at least one haptic signal, configuring a magnitude of the at least one haptic signal, configuring a frequency of the at least one haptic signal, configuring one or more specific actuators that the at least one haptic signal is sent to, or executing a fail-safe operation.

12. The computer-implemented method of claim 10, wherein the configuring the at least one haptic signal is performed before the generating of the one or more haptic signals.

13. The computer-implemented method of claim 10, wherein the configuring the at least one haptic signal is performed during the generating of the one or more haptic signals.

14. The computer-implemented method of claim 10, wherein the configuring the at least one haptic signal is performed after the generating of the one or more haptic signals.

15. The computer-implemented method of claim 9, wherein the audio signal is a pulse-code modulation audio buffer, and the stream is a pulse-code modulation stream.

16. The computer-implemented method of claim 9, further comprising sending a stream of the audio signal to a speaker to generate an audio effect regardless of whether the haptic conversion indicator parameter indicates that the stream is to be converted into the haptic effect.

17. A haptic conversion system comprising:

a memory configured to store a stream-independent haptic conversion module;

a processor configured to execute the stream-independent haptic conversion module stored on the memory; and

an actuator configured to output one or more haptic effects;

wherein the stream-independent haptic conversion module is configured to;

receive a plurality of streams of an audio signal, wherein each stream comprises a plurality of parameters, and wherein the plurality of parameters comprise a haptic conversion indication parameter that indicates whether the corresponding stream is to be converted into a haptic effect and one or more haptic parameters that configure a haptic signal that is generated based on the stream;

determine, for each stream, whether the haptic conversion indication parameter indicates that the corresponding stream is to be converted into a haptic effect;

identify one or more streams that include the haptic conversion indication parameter that indicates the corresponding stream is to be converted into the haptic effect, wherein a number of the one or more identified streams is less than a number of the plurality of streams;

generate one or more haptic signals based on the one or more identified streams; and

send the one or more haptic signals to the actuator to generate one or more haptic effects.

18. The haptic conversion system of claim 17, wherein the stream-independent haptic conversion module is further configured to configure at least one haptic signal based on the one or more haptic parameters.

19. The haptic conversion system of claim 18, wherein the one or more haptic parameters comprise at least one of a strength parameter, a density parameter, a magnitude parameter, a frequency parameter, an actuator parameter, or a fail-safe parameter;

and

wherein the stream-independent haptic conversion module is further configured to perform at least one of;

configuring a strength of the at least one haptic signal, configuring a density of the at least one haptic signal, configuring a magnitude of the at least one haptic signal, configuring a frequency of the at least one haptic signal, configuring one or more specific actuators that the at least one haptic signal is sent to, or executing a fail-safe operation.

20. The haptic conversion system of claim 18, wherein the stream-independent haptic conversion module is further configured to configure the at least one haptic signal before it generates the one or more haptic signals.

21. The haptic conversion system of claim 18, wherein the stream-independent haptic conversion module is further configured to configure the at least one haptic signal while it generates the one or more haptic signals.

22. The haptic conversion system of claim 18, wherein the stream-independent haptic conversion module is further configured to configure the at least one haptic signal after it generates the one or more haptic signals.

23. The haptic conversion system of claim 17, wherein the audio signal is a pulse-code modulation audio buffer, and the stream is a pulse-code modulation stream.