Dynamically generating haptic effects from audio data

• US 9,128,523 B2
• Filed: 12/20/2012
• Issued: 09/08/2015
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Buuck, David Charles, Zarraga, John Aaron, Yun, Tiffany Ann
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): FAHNERT, FRIEDRICH
• Application Number:
  US13/722,435
• Publication Number:
  US 20140176415A1
• Time in Patent Office: 992 Days
• Field of Search: 345/156, 345/173, 340/407.1, 463/30, 381/56, 381/61, 381/98, 381/111, 381/161, 381/162, 381/301, 381/333, 381/388, 700/94
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

Haptic effects are dynamically generated for content presentation on a device through analysis of the content. During content playback, audio data for the content may be analyzed to determine low frequency audio data. The low frequency audio data is mapped from a low frequency range to a haptic control frequency range of one or more haptic actuators included in the device. This mapping may be used to generate a control signal to drive the one or more haptic actuators. The haptic effects and the content may be synchronized to one another during the presentation of the content on the device. The haptic actuator control signal may be amplified proportionally to the amplitude of the low frequency audio data.

Claims

1. A system comprising:

at least one non-transitory computer-readable memory device storing computer-executable instructions; and

at least one hardware processor configured to access the at least one memory and execute the computer-executable instructions to;

receive content including at least one audio channel;

determine low frequency effects (LFE) audio data from the at least one audio channel, wherein the LFE audio data includes a first frequency range;

based on the LFE audio data, generate a haptic control signal by mapping the first frequency range to a second frequency range corresponding to at least one haptic actuator; and

send the haptic control signal to the at least one haptic actuator, enabling the at least one haptic actuator to create at least one haptic effect based on the second frequency range and corresponding to a presentation of the content.

2. The system of claim 1, the instructions further configured to adjust an amplitude of the haptic control signal prior to sending the haptic control signal to the at least one haptic actuator, such that the amplitude of the haptic control signal is proportional to an incoming amplitude of the LFE audio data.

3. The system of claim 1, the instructions further configured to adjust one or more of a timing of the haptic control signal or the presentation of the content, to substantially synchronize the at least one haptic effect with the presentation of the content.

4. The system of claim 1, wherein the content includes digital content data, and the instructions are further configured to perform a digital to analog conversion of the digital content data.

5. A computer-implemented method, comprising:

determining low frequency audio data based on content data, the low frequency audio data corresponding to sound in a first frequency range;

generating haptic control data in a second frequency range to control at least one haptic actuator in a device, wherein the second frequency range corresponds to the first frequency range of the low frequency audio data; and

sending the haptic control data to the at least one haptic actuator to create at least one haptic effect based on the second frequency range and corresponding with a presentation of the content data on the device.

6. The method of claim 5, wherein the content data includes one or more of audio content data, video content data, or game content data.

7. The method of claim 5, wherein the determining the low frequency audio data based on the content data includes:

identifying a subwoofer channel of the content data as including the low frequency audio data.

8. The method of claim 5, wherein the determining the low frequency audio data based on the content data includes applying a low pass filter to an audio portion of the content data.

9. The method of claim 5, wherein the generating the haptic control data includes mapping the low frequency audio data from the first frequency range to the second frequency range.

10. The method of claim 5, wherein the generating the haptic control data includes:

mapping the low frequency audio data from each of a first set of M frequency ranges to at least one corresponding range in a second set of N frequency ranges, wherein the first set of M frequency ranges at least partly spans the first frequency range, and wherein the second set of N frequency ranges at least partly spans the second frequency range.

11. The method of claim 10, wherein M=N, and wherein the mapping is a one-to-one mapping from each of the first set of M frequency ranges to one corresponding range in the second set of N frequency ranges.

12. The method of claim 5, further comprising adjusting an amplitude of the haptic control data prior to sending the haptic control data to the at least one haptic actuator, such that the amplitude of the haptic control data is proportional to an incoming amplitude of the low frequency audio data.

13. The method of claim 5, further comprising adjusting one or more of a timing of the haptic control data or the presentation of the content data, such that the sending of the haptic control data corresponds to the presentation of the content data.

14. The method of claim 5, wherein the generating the haptic control data further includes performing an active, full wave rectification on the low frequency audio data, and wherein the active, full wave rectification modifies a frequency of the low frequency audio data from the first frequency range to the second frequency range to form the haptic control data.

19. The method of claim 10, wherein sending the haptic control data to the at least one haptic actuator includes:

generating a first haptic control signal based on a first portion of the second set of N frequency ranges;

sending the first haptic control signal to a first haptic actuator;

generating a second haptic control signal based on a second portion of the second set of N frequency ranges; and

sending the second haptic control signal to a second haptic actuator.

15. A system comprising:

at least one non-transitory computer-readable memory device storing computer-executable instructions; and

at least one hardware processor configured to access the at least one memory and execute the computer-executable instructions to;

determine low frequency audio data based on content data, wherein the low frequency audio data corresponds to a first frequency range;

based at least partly on the first frequency range of low frequency audio data, generate haptic control data corresponding to a second frequency range associated with at least one haptic controller; and

send the haptic control data to the at least one haptic controller to drive the at least one haptic actuator during a presentation of the content data.

16. The system of claim 15, wherein the first frequency range is lower than the second frequency range.

17. The system of claim 15, wherein the instructions are further configured to generate the haptic control data by mapping the low frequency audio data from the first frequency range to the second frequency range.

18. The system of claim 15, wherein the instructions are further configured to:

generate the haptic control data by mapping the low frequency audio data from at least one of a first set of M frequency ranges to at least one corresponding range in a second set of N frequency ranges, wherein the first set of M frequency ranges at least partly spans the first frequency range, and wherein the second set of N frequency ranges at least partly spans the second frequency range; and

send the haptic control data by;

generating a first haptic control signal based on a first portion of the second set of N frequency ranges;

sending the first haptic control signal to a first haptic actuator;

generating a second haptic control signal based on a second portion of the second set of N frequency ranges; and

sending the second haptic control signal to a second haptic actuator.

20. The system of claim 15, wherein the instructions are further configured to generate the haptic control data by performing a rectification operation on the low frequency audio data to modify a frequency of the low frequency audio data from the first frequency range to the second frequency range.