Method and system for providing haptic effects based on information complementary to multimedia content

• US 9,928,701 B2
• Filed: 04/05/2017
• Issued: 03/27/2018
• Est. Priority Date: 09/06/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Modarres, Ali, Cruz-Hernandez, Juan Manuel, Saboune, Jamal
• Primary Examiner(s): Lau, Hoi
• Application Number:
  US15/479,398
• Publication Number:
  US 20170206755A1
• Time in Patent Office: 356 Days
• Field of Search: 340 412, 3404071, 3404072, 340 76, 345156, 345173, 707769, 715702, 386239, 386E09011
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

The present disclosure is generally directed to systems and methods for providing haptic effects based on information complementary to multimedia content. For example, one disclosed method includes the steps of receiving multimedia data comprising multimedia content and complementary data, wherein the complementary data describes the multimedia content, determining a haptic effect based at least in part on the complementary data, and outputting the haptic effect while playing the multimedia content.

Claims

1. A method comprising:

receiving, by a processor, a data stream comprising an audio track and a metadata container comprising haptic-effect data, the haptic-effect data being information usable to generate or modify one or more haptic effects;

determining, by the processor, a haptic effect based on the audio track;

determining, by the processor, a modified version of the haptic effect based on the haptic-effect data included in the metadata container; and

transmitting, by the processor, a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the audio track.

2. The method of claim 1, wherein the metadata container comprises a field and the haptic-effect data is stored in the field.

3. The method of claim 1, wherein the metadata container comprises a haptic-effect code corresponding to one or more haptic effects in a database.

4. The method of claim 1, further comprising:

synchronizing output of the haptic effect with an event in the audio track; and

outputting the haptic effect concurrently with the event in the audio track.

5. The method of claim 1, wherein the haptic-effect data comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

6. The method of claim 1, further comprising:

determining a haptic theme using the haptic-effect data; and

generating the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.

7. The method of claim 1, wherein the data stream further comprises a video track corresponding to the audio track and the metadata container.

8. A system comprising:

a processing device; and

a memory device on which instructions executable by the processing device are stored to cause the processing device to;

receive a data stream comprising an audio track and a metadata container comprising haptic-effect data, the haptic-effect data being information usable to generate or modify one or more haptic effects;

determine a haptic effect based on the audio track;

determine a modified version of the haptic effect based on the haptic-effect data included in the metadata container; and

transmit a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the audio track.

9. The system of claim 8, wherein the metadata container comprises a field and the haptic-effect data is stored in the field.

10. The system of claim 8, wherein the metadata container comprises a haptic-effect code corresponding to one or more haptic effects in a database.

11. The system of claim 8, wherein the memory device further includes instructions executable by the processing device to cause the processing device to:

synchronize output of the haptic effect with an event in the audio track; and

cause the haptic effect to be output concurrently with the event in the audio track.

12. The system of claim 8, wherein the haptic-effect data comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

13. The system of claim 8, wherein the memory device further includes instructions executable by the processing device to cause the processing device to:

determine a haptic theme using the haptic-effect data; and

generate the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.

14. The system of claim 8, wherein the data stream further comprises a video track corresponding to the audio track and the metadata container.

15. A non-transitory computer readable medium comprising processor-executable program code, which when executed by a processor is configured to cause the processor to:

receive a data stream comprising an audio track and a metadata container comprising haptic-effect data, the haptic-effect data being information usable to generate or modify one or more haptic effects;

determine a haptic effect based on the audio track;

determine a modified version of the haptic effect based on the haptic-effect data included in the metadata container; and

transmit a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the audio track.

16. The non-transitory computer readable medium of claim 15, wherein the metadata container comprises a field and the haptic-effect data is stored in the field.

17. The non-transitory computer readable medium of claim 15, wherein the metadata container comprises a haptic-effect code corresponding to one or more haptic effects in a database.

18. The non-transitory computer readable medium of claim 15, further comprising processor-executable program code executable by the processor to cause the processor to:

synchronize output of the haptic effect with an event in the audio track; and

cause the haptic effect to be output concurrently with the event in the audio track.

19. The non-transitory computer readable medium of claim 15, wherein the haptic-effect data comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

20. The non-transitory computer readable medium of claim 15, further comprising processor-executable program code executable by the processor to cause the processor to:

determine a haptic theme using the haptic-effect data; and

generate the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.