Haptic feedback sensations based on audio output from computer devices

• US 8,441,437 B2
• Filed: 11/23/2009
• Issued: 05/14/2013
• Est. Priority Date: 10/09/2001
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rank, Stephen D.
• Primary Examiner(s): Kumar, Srilakshmi K
• Application Number:
  US12/623,685
• Publication Number:
  US 20100066512A1
• Time in Patent Office: 1,268 Days
• Field of Search: 345156-181, 345/701, 345/702, 463/30, 340/407.1
• US Class Current: 345/156
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/6063 for sound processing
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

Triggering haptic sensations based on sound output from a computer device. A portion of sound data is stored that is output to a user as audio from an application program running on a computer. The portion of sound data is analyzed using intelligent heuristics to extract at least one sound feature from the sound data. The execution of at least one haptic effect is triggered based on the sound feature, where the haptic effect is commanded to the haptic feedback device approximately correlated to the output of the portion of sound to the user as audio. The haptic effect causes a haptic sensation to be output to the user. Different haptic effects can be associated with different sound features, frequency ranges, amplitudes, etc.

Claims

1. A method for triggering a haptic effect, comprising:

receiving, by at least one processor, sound data;

applying, by the processor, a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into a plurality of frequency ranges, and wherein the plurality of filters includes at least one bandpass filter;

analyzing, by the processor, at least two frequency ranges;

determining, by the processor, one or more sound features for each of the analyzed at least two frequency ranges; and

triggering, by the processor, at least a first haptic effect based on the determined one or more sound features for a first one of the at least two frequency ranges and at least a second haptic effect based on the determined one or more sound features for a second one of the at least two frequency ranges, wherein the first haptic effect is different than the second haptic effect.

2. A method for triggering a haptic effect, comprising:

receiving, by at least one processor, sound data;

analyzing, by the processor, a plurality of frequency ranges of the sound data, each frequency range of the sound data having a sound magnitude;

generating, by the processor, an average of sound magnitudes for each frequency range of the plurality of frequency ranges of the sound data;

determining, by the processor, one or more sound features from at least one frequency range of sound data; and

triggering, by the processor, at least one haptic effect when the one or more sound features from the at least one frequency range of sound data have a sound magnitude above a threshold value from the average for the corresponding frequency range.

3. The method of claim 2, wherein each average is a running average for each frequency range of the plurality of frequency ranges of the sound data.

4. The method of claim 2, wherein said determining includes applying a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into the plurality of frequency ranges, and wherein the one or more sound features are determined from at least one of the plurality of frequency ranges.

5. A method for triggering a haptic effect, comprising:

receiving, by at least one processor, sound data;

analyzing, by the processor, a first frequency range of the sound data and a second frequency range of the sound data;

determining, by the processor, a first sound feature from the analyzed first frequency range of the sound data and a second sound feature from the analyzed second frequency range of the sound data;

determining, by the processor, a difference in sound magnitude between the first sound feature and the second sound feature; and

triggering, by the processor, at least one haptic effect when the difference in sound magnitude exceeds a threshold value, wherein the threshold value comprises an average of sound magnitudes for at least one frequency range of the first frequency range and the second frequency range of the sound data.

6. The method of claim 5, wherein said determining the first sound feature and the second sound feature includes applying a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into the plurality of frequency ranges, and wherein the first sound feature and the second sound feature are determined from at least one of the plurality of frequency ranges.

7. A system for triggering a haptic effect, comprising:

one or more processors configured to;

receive sound data;

apply a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into a plurality of frequency ranges, and wherein the plurality of filters includes at least one bandpass filter;

analyze at least two frequency ranges;

determine one or more sound features based for each of the analyzed at least two frequency ranges; and

trigger at least a first haptic effect based on the determined one or more sound features for a first one of the at least two frequency ranges and at least a second haptic effect based on the determined one or more sound features for a second one of the at least two frequency ranges, wherein the first haptic effect is different than the second haptic effect.

8. A system for triggering a haptic effect, comprising:

one or more processors configured to;

receive sound data;

analyze a plurality of frequency ranges of the sound data, each frequency range of the sound data having a sound magnitude;

generate an average of sound magnitudes for each frequency range of the plurality of frequency ranges of the sound data;

determine one or more sound features from at least one frequency range of sound data; and

trigger at least one haptic effect when the one or more sound features from the at least one frequency range of sound data have a sound magnitude above a threshold value from the average for the corresponding frequency range.

9. The system of claim 8, wherein each average is a running average for each frequency range of the plurality of frequency ranges of the sound data.

10. The system of claim 8, wherein said determine one or more sound features includes an application of a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into the plurality of frequency ranges, and wherein the one or more sound features are determined from at least one of the plurality of frequency ranges.

11. A system for triggering a haptic effect, comprising:

one or more processors configured to;

receive sound data;

analyze a first frequency range of the sound data and a second frequency range of the sound data;

determine a first sound feature from the analyzed first frequency range of the sound data and a second sound feature from the analyzed second frequency range of the sound data;

determine a difference in sound magnitude between the first sound feature and the second sound feature; and

trigger at least one haptic effect when the difference in sound magnitude exceeds a threshold value, wherein the threshold value comprises an average of sound magnitudes for at least one frequency range of the first frequency range and the second frequency range of the sound data.

12. The system of claim 11, wherein said determine the first sound feature and the second sound feature includes an application of a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into the plurality of frequency ranges, and wherein the first sound feature and the second sound feature are determined from at least one of the plurality of frequency ranges.

13. A non-transitory computer readable medium encoded with a computer program comprising instructions that cause a processor to trigger a haptic effect, the triggering comprising:

receiving sound data;

applying a plurality of filters to the sound data, wherein the plurality of filters divide the sound data into a plurality of frequency ranges, and wherein the plurality of filters includes at least one bandpass filter;

analyzing at least two frequency ranges;

determining one or more sound features for each of the analyzed at least two frequency ranges; and

triggering at least a first haptic effect based on the determined one or more sound features for a first one of the at least two frequency ranges and at least a second haptic effect based on the determined one or more sound features for a second one of the at least two frequency ranges, wherein the first haptic effect is different than the second haptic effect.

14. A non-transitory computer readable medium encoded with a computer program comprising instructions that cause a processor to trigger a haptic effect, the triggering comprising:

receiving sound data;

analyzing a plurality of frequency ranges of the sound data, each frequency range of the sound data having a sound magnitude;

generating an average of sound magnitudes for each frequency range of the plurality of frequency ranges of the sound data;

determining one or more sound features from at least one frequency range of sound data; and

triggering at least one haptic effect when the one or more sound features from the at least one frequency range of sound data have a sound magnitude above a threshold value from the average for the corresponding frequency range.

15. A non-transitory computer readable medium encoded with a computer program comprising instructions that cause a processor to trigger a haptic effect, the triggering comprising:

receiving sound data;

analyzing a first frequency range of the sound data and a second frequency range of the sound data;

determining a first sound feature from the analyzed first frequency range of the sound data and a second sound feature from the analyzed second frequency range of the sound data;

determining a difference in sound magnitude between the first sound feature and the second sound feature; and

triggering at least one haptic effect when the difference in sound magnitude exceeds a threshold value, wherein the threshold value comprises an average of sound magnitudes for at least one frequency range of the first frequency range and the second frequency range of the sound data.