Method and system for providing haptic effects based on information complementary to multimedia content

• US 10,140,823 B2
• Filed: 02/06/2018
• Issued: 11/27/2018
• Est. Priority Date: 09/06/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Modarres, Ali, Cruz-Hernandez, Juan Manuel, Saboune, Jamal
• Primary Examiner(s): Lau, Hoi C
• Application Number:
  US15/889,301
• Publication Number:
  US 20180158291A1
• Time in Patent Office: 294 Days
• Field of Search: 340 412, 3404071, 3404072, 340 76, 345156, 345173, 707769, 715702, 386239, 386E09011
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

The present disclosure is generally directed to systems and methods for providing haptic effects based on information complementary to multimedia content. For example, one disclosed method includes the steps of receiving multimedia data comprising multimedia content and complementary data, wherein the complementary data describes the multimedia content, determining a haptic effect based at least in part on the complementary data, and outputting the haptic effect while playing the multimedia content.

Claims

1. A method comprising:

receiving, by a processor, a data stream comprising a video track and complementary information;

determining, by the processor, a haptic effect based on the video track;

determining, by the processor, a modified version of the haptic effect based on the complementary information; and

transmitting, by the processor, a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the video track.

2. The method of claim 1, further comprising using the complementary information to provide context to an automated haptification algorithm.

3. The method of claim 1, wherein the complementary information comprises a haptic effect code corresponding to one or more haptic effects in a database.

4. The method of claim 1, further comprising:

synchronizing output of the modified version of the haptic effect with an event in the video track; and

outputting the modified version of the haptic effect concurrently with the event in the video track.

5. The method of claim 1, wherein the complementary information comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

6. The method of claim 1, further comprising:

determining a haptic theme using the complementary information; and

generating the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.

7. The method of claim 1, wherein the data stream further comprises an audio track corresponding to the video track and the complementary information.

8. A system comprising:

a processing device; and

a memory device on which instructions executable by the processing device are stored to cause the processing device to;

receive a data stream comprising a video track and complementary information;

determine a haptic effect based on the video track;

determine a modified version of the haptic effect based on the complementary information; and

transmit a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the video track.

9. The system of claim 8, wherein the memory device further includes instructions executable by the processing device to cause the processing device to:

use the complementary information to provide context to an automated haptification algorithm.

10. The system of claim 8, wherein the complementary information comprises a haptic effect code corresponding to one or more haptic effects in a database.

11. The system of claim 8, wherein the memory device further includes instructions executable by the processing device to cause the processing device to:

synchronize output of the modified version of the haptic effect with an event in the video track; and

cause the modified version of the haptic effect to be output concurrently with the event in the video track.

12. The system of claim 8, wherein the complementary information comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

13. The system of claim 8, wherein the memory device further includes instructions executable by the processing device to cause the processing device to:

determine a haptic theme using the complementary information; and

generate the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.

14. The system of claim 8, wherein the data stream further comprises an audio track corresponding to the video track and the complementary information.

15. A non-transitory computer readable medium comprising processor-executable program code, which when executed by a processor is configured to cause the processor to:

receive a data stream comprising a video track and complementary information;

determine a haptic effect based on the video track;

determine a modified version of the haptic effect based on the complementary information; and

transmit a signal configured to cause a haptic output device to output the modified version of the haptic effect while playing the video track.

16. The non-transitory computer readable medium of claim 15, further comprising processor-executable program code that is executable by the processor to cause the processor to:

use the complementary information to provide context to an automated haptification algorithm.

17. The non-transitory computer readable medium of claim 15, wherein the complementary information comprises a haptic effect code corresponding to one or more haptic effects in a database.

18. The non-transitory computer readable medium of claim 15, further comprising processor-executable program code that is executable by the processor to cause the processor to:

synchronize output of the modified version of the haptic effect with an event in the video track; and

cause the modified version of the haptic effect to be output concurrently with the event in the video track.

19. The non-transitory computer readable medium of claim 15, wherein the complementary information comprises at least one of a title, artist, author, genre, rating, review, list of songs, comment, chapter data, or song type.

20. The non-transitory computer readable medium of claim 15, further comprising processor-executable program code that is executable by the processor to cause the processor to:

determine a haptic theme using the complementary information; and

generate the modified version of the haptic effect by modifying a characteristic of the haptic effect based on the haptic theme.