Sound to haptic effect conversion system using waveform

• US 9,606,627 B2
• Filed: 05/12/2015
• Issued: 03/28/2017
• Est. Priority Date: 02/11/2011
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Bhatia, Satvir Singh, Gandhi, Kanav, Ullrich, Christopher J., Cruz-Hernandez, Juan Manuel, Timone, Herve Thu, Lau, Jason Hoi Fun
• Primary Examiner(s): Tweel, Jr., John A
• Application Number:
  US14/709,672
• Publication Number:
  US 20150241975A1
• Time in Patent Office: 686 Days
• Field of Search: 345/156, 340/407.1, 463/30, 715/702
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04

Abstract

A haptic conversion system is provided that intercepts frames of audio data, such as a digital audio signal, converts the frames into a haptic signal, and plays the created haptic signal through an actuator to produce haptic effects. The haptic signal is based on a maximum value of each audio data frame, which defines a magnitude of the haptic signal. The haptic signal is applied to the actuator configured to receive the haptic signal, where the actuator utilizes the haptic signal to generate the one or more haptic effects.

Claims

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an audio signal into one or more haptic effects, the converting comprising:

receiving a plurality of data frames of the audio signal;

determining a maximum amplitude value of each of the data frames;

generating a haptic signal based on the maximum amplitude value of each of the data frames; and

sending the haptic signal to an actuator to generate the one or more haptic effects.

2. The non-transitory computer-readable medium of claim 1, the converting further comprising:

transforming the data frame.

3. The non-transitory computer-readable medium of claim 2, the converting further comprising modulating the transformed data frame.

4. The non-transitory computer-readable medium of claim 3, the converting further comprising normalizing the modulated data frame.

5. The non-transitory computer-readable medium of claim 3, wherein the modulated data frame defines, at least in part, the haptic signal.

6. The non-transitory computer-readable medium of claim 2, wherein transforming the data frame comprises band-pass filtering the data frame.

7. The non-transitory computer-readable medium of claim 2, wherein transforming the data frame comprises low-pass filtering the data frame.

8. The non-transitory computer-readable medium of claim 2, the converting further comprising resampling the data frame before the data frame is transformed.

9. The non-transitory computer-readable medium of claim 2, the converting further comprising boosting the audio signal contained within the data frame after the data frame is transformed.

10. The non-transitory computer-readable medium of claim 1, wherein the haptic signal is generated only when the maximum amplitude value of the data frame is greater than a specified threshold.

11. A computer-implemented method for converting an audio signal into one or more haptic effects, the computer-implemented method comprising:

receiving a plurality of data frames of the audio signal;

determining a maximum amplitude value of each of the data frames;

generating a haptic signal based on the maximum amplitude value of each of the data frames; and

sending the haptic signal to an actuator to generate the one or more haptic effects.

12. The computer-implemented method of claim 11, further comprising:

transforming the data frame.

13. The computer-implemented method of claim 12, further comprising modulating the transformed data frame.

14. The computer-implemented method of claim 13, further comprising normalizing the modulated data frame.

15. The computer-implemented method of claim 11, wherein the haptic signal is generated only when the maximum amplitude value of the data frame is greater than a specified threshold.

16. A haptic conversion system comprising:

a memory configured to store a haptic conversion module;

a processor configured to execute the haptic conversion module stored on the memory; and

an actuator configured to output one or more haptic effects;

wherein the haptic conversion module is configured to receive a plurality of data frames of the audio signal;

wherein the haptic conversion module is further configured to determine a maximum amplitude value of each of the data frames;

wherein the haptic conversion module is further configured to generate a haptic signal based on the maximum amplitude value of each of the data frames; and

wherein the haptic conversion module is further configured to send the haptic signal to an actuator to generate the one or more haptic effects.

17. The haptic conversion system of claim 16,

wherein the haptic conversion module is further configured to transform the data frame.

18. The haptic conversion system of claim 17,

wherein the haptic conversion module is further configured to modulate the transformed data frame.

19. The haptic conversion system of claim 18, wherein the haptic conversion module is further configured to normalize the modulated data frame.

20. The haptic conversion system of claim 16, wherein the haptic signal is generated only when the maximum amplitude value of the data frame is greater than a specified threshold.