Sound to haptic effect conversion system using amplitude value

• US 9,448,626 B2
• Filed: 02/03/2012
• Issued: 09/20/2016
• Est. Priority Date: 02/11/2011
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Cruz-Hernandez, Juan Manuel, Timone, Herve Thu, Lau, Jason Hoi Fun
• Primary Examiner(s): Lim, Steven
• Assistant Examiner(s): LITTLEJOHN JR, MANCIL H
• Application Number:
  US13/365,984
• Publication Number:
  US 20120206246A1
• Time in Patent Office: 1,691 Days
• Field of Search: 340/407.1, 340/407.2, 704/224, 704/225, 704/E19.012, 704/200, 345/156, 345/161, 463/36, 463/30, 463/42, 369/2, 369/4, 369/5, 369/47.2, 369/275.3, 381/63, 381/119, 381/102, 381104-107, 381/56
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

A haptic conversion system is provided that intercepts frames of audio data, such as a digital audio signal, converts the frames into a haptic signal, and plays the created haptic signal through an actuator to produce haptic effects. The haptic signal is based on a maximum value of each audio data frame, which defines a magnitude of the haptic signal. The haptic signal is applied to the actuator configured to receive the haptic signal, where the actuator utilizes the haptic signal to generate the one or more haptic effects.

Claims

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an audio signal into one or more haptic effects, the converting comprising:

receiving a plurality of data frames of the audio signal;

sampling each data frame to generate one or more sample values of each of the plurality of data frames;

determining a maximum value of each of the data frames based on the one or more sample values of each of the data frames;

generating a haptic signal based on the maximum value of each of the data frames;

filtering the magnitude of the haptic signal using a parameter to define an amount of decay of the haptic signal, and

sending the haptic signal to an actuator to generate the one or more haptic effects.

2. The non-transitory computer-readable medium of claim 1, wherein the haptic signal comprises a periodic effect.

3. The non-transitory computer-readable medium of claim 1, wherein the one or more sample values of the data frame are absolute values.

4. The non-transitory computer-readable medium of claim 1, wherein the one or more sample values of the data frame are square values.

5. The non-transitory computer-readable medium of claim 1, wherein the maximum value is converted from an original maximum value using a linear conversion function.

6. The non-transitory computer-readable medium of claim 1, wherein a sharpness parameter is used to define a rate of decay to be applied to the magnitude of the haptic signal.

7. The non-transitory computer-readable medium of claim 1, wherein the audio signal is a pulse-code modulation audio buffer.

8. The non-transitory computer-readable medium of claim 1, wherein the generating and the sending the haptic signal further comprises passing the maximum value as a second parameter to an application programming interface of a haptic effect player module.

9. The non-transitory computer-readable medium of claim 1, the converting further comprising extracting a frequency content from the data frame using a fast fourier transformation of the data frame.

10. The non-transitory computer-readable medium of claim 1, the converting further comprising extracting a frequency content from the data frame using a band-pass filter.

11. The non-transitory computer-readable medium of claim 1, the converting further comprising:

dividing the data frame into low frequency content, middle frequency content, and high frequency content; and

extracting the middle frequency content and high frequency content using content of the audio signal.

12. A computer-implemented method for converting an audio signal into one or more haptic effects, the computer-implemented method comprising:

receiving a plurality of data frames of the audio signal;

sampling each data frame to generate one or more sample values of each of the plurality of data frames;

determining a maximum value of each of the data frames based on the one or more sample values of each of the data frames;

generating a haptic signal based on the maximum value of each of the data frames;

filtering the magnitude of the haptic signal using a parameter to define an amount of decay of the haptic signal and

sending the haptic signal to an actuator to generate the one or more haptic effects.

13. The computer-implemented method of claim 12, wherein the haptic signal comprises a periodic effect.

14. The computer-implemented method of claim 12, wherein the one or more sample values of the data frame are absolute values.

15. The computer-implemented method of claim 12, wherein the maximum value is converted from an original maximum value using a linear conversion function.

16. The computer-implemented method of claim 12, the computer-implemented method further comprising extracting a frequency content from the data frame using a fast fourier transformation of the data frame.

17. The computer-implemented method of claim 12, the computer-implemented method further comprising extracting a frequency content from the data frame using a band-pass filter.

18. The computer-implemented method of claim 12, the computer-implemented method further comprising:

dividing the data frame into low frequency content, middle frequency content, and high frequency content; and

extracting the middle frequency content and high frequency content using content of the audio signal.

19. A haptic conversion system comprising:

a haptic conversion module, wherein the haptic conversion module comprises a first sub-module and a second sub-module; and

an actuator configured to generate one or more haptic effects;

wherein the first sub-module is configured to receive a plurality of data frames of an audio signal;

wherein the second sub-module is configured to sample each data frame to generate one or more sample values of each of the data frames;

wherein the second sub-module is further configured to determine a maximum value of each of the data frames based on the one or more sample values;

wherein the first sub-module is further configured to generate a haptic signal based on the maximum value of each of the data frames;

wherein the first sub-module is further configured to filter the magnitude of the haptic signal using a parameter to define an amount of decay of the haptic signal and

wherein the first sub-module is further configured to send the haptic signal to the actuator.

20. The haptic conversion system of claim 19, wherein the haptic signal comprises a periodic effect.

21. The haptic conversion system of claim 19, wherein the one or more sample values of the data frame are absolute values.

22. The haptic conversion system of claim 19, wherein the second sub-module is further configured to convert the maximum value from an original maximum value using a linear conversion function.

23. The haptic conversion system of claim 19, wherein the second sub-module is further configured to extract a frequency content from the data frame using a fast fourier transformation of the data frame.

24. The haptic conversion system of claim 19, wherein the second sub-module is further configured to extract a frequency content from the data frame using a band-pass filter.

25. The haptic conversion system of claim 19, wherein the second sub-module is further configured to:

divide the data frame into low frequency content, middle frequency content, and high frequency content; and

extract the middle frequency content and high frequency content using content of the audio signal.

26. A haptic conversion system comprising:

a haptic conversion module, and

an actuator configured to output one or more haptic effects;

wherein the haptic conversion module is configured to;

receive a plurality of data frames of an audio signal;

sample each data frame to generate one or more sample values of each of the plurality of data frames;

determine a maximum value of each of the data frames based on the one or more sample values of each of the data frames;

generate a haptic signal based on the maximum value of each of the data frames;

filter the magnitude of the haptic signal using a parameter to define an amount of decay of the haptic signal; and

send the haptic signal to the actuator.