Haptic feedback sensations based on audio output from computer devices

• US 7,623,114 B2
• Filed: 10/09/2001
• Issued: 11/24/2009
• Est. Priority Date: 10/09/2001
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rank, Stephen D.
• Primary Examiner(s): Kumar; Srilakshmi K
• Application Number:
  US09/974,759
• Publication Number:
  US 20030067440A1
• Time in Patent Office: 2,968 Days
• Field of Search: 345156-181, 345/701, 345/702, 345/978, 273/148.B, 340/407.1, 340/407.2, 381/56, 381/58, 381/151, 381/301, 381/320, 434/112, 434/114, 434/307.R, 463/30, 463/35, 704/271, 704/276
• US Class Current: 345/156
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/6063 for sound processing
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

Triggering haptic sensations based on sound output from a computer device. A portion of sound data is stored that is output to a user as audio from an application program running on a computer. The portion of sound data is analyzed using intelligent heuristics to extract at least one sound feature from the sound data. The execution of at least one haptic effect is triggered based on the sound feature, where the haptic effect is commanded to the haptic feedback device approximately correlated to the output of the portion of sound to the user as audio. The haptic effect causes a haptic sensation to be output to the user. Different haptic effects can be associated with different sound features, frequency ranges, amplitudes, etc.

Claims

1. A method, comprising:

storing at least a portion of sound data in a memory buffer of a computer, wherein the sound data is to be analyzed by a processor to output a haptic effect from the analyzed sound data;

dividing at least a portion of the sound data into a plurality of frequency ranges, at least one of the frequency ranges associated with a periodic haptic effect;

analyzing each frequency range by the processor to determine one or more sound features corresponding to at least one of the frequency ranges; and

triggering execution of at least one haptic effect based on the determined one or more sound features.

2. The method of claim 1 wherein the portion of sound data is divided into a plurality of frequency ranges by applying a plurality of filters to the portion of sound data.

3. The method of claim 2, the plurality of filters having at least:

a low-pass filter; and

a high-pass filter.

4. The method of claim 2, the analyzing including separating the portion of sound data into a plurality of frequency components associated with a plurality of frequency ranges using a fast Fourier transform (FFT).

5. The method of claim 4, wherein a number of outputs from the fast Fourier transform are grouped to provide sound features associated with each frequency range from the plurality of frequency ranges.

6. The method of claim 1, wherein the at least one haptic effect was previously mapped to the at least one sound feature.

7. The method of claim 2, further comprising:

computing an average of sound magnitudes in each frequency range, wherein the determined one or more sound features have a sound magnitude above a threshold value from the average.

8. A computer readable medium encoded with a computer program having code capable of being read by a processor, the code comprising:

code to store at least a portion of sound data in a memory buffer of a computer wherein the sound data is to be analyzed by a processor to output a haptic from the analyzed sound data;

code to divide at least a portion of sound data into a plurality of frequency ranges, at least one of the frequency ranges associated with a periodic haptic effect;

code to cause the processor to analyze each frequency range to determine one or more sound features corresponding to at least one of the frequency ranges; and

code to trigger execution of at least one haptic effect based on the determined one or more sound features.

9. The computer readable medium of claim 8, wherein at least one haptic effect is associated with the at least one frequency component.

10. The computer readable medium of claim 8 wherein the portion of sound data is divided into a plurality of frequency ranges by applying a plurality of filters to the portion of sound data.

11. The computer medium readable of claim 10, the code to analyze including code to separate the portion of sound data into a plurality of frequency components associated with a plurality of frequency ranges using a fast Fourier transform (FFT).

12. The computer readable medium of claim 8, wherein the at least one haptic effect was previously mapped to the at least one sound feature.

13. The computer readable medium of claim 8, the code further comprising:

code to compute an average of sound magnitudes in each frequency range, wherein the determined one or more sound features have a sound magnitude above a threshold value from the average.

14. An apparatus, comprising:

means for storing at least a portion sound data in a memory buffer of a computer, wherein the sound data is to be analyzed by a processor to output a haptic effect from the analyzed sound;

means for dividing at least a portion of sound data into a plurality of frequency ranges, at least one of the frequency ranges associated with a periodic haptic effect;

means for analyzing each frequency range to determine one or more sound features corresponding to at least one of the frequency ranges, wherein said means for analyzing is performed by the processor; and

means for triggering execution of at least one haptic effect based on the determined one or more sound features.

15. The apparatus of claim 14, further comprising:

means for computing an average of sound magnitudes in each frequency range, wherein the determined one or more sound features have a sound magnitude above a threshold value from the average.