Method and system for providing haptic effects based on information complementary to multimedia content

• US 10,395,490 B2
• Filed: 10/22/2018
• Issued: 08/27/2019
• Est. Priority Date: 09/06/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Modarres, Ali, Cruz-Hernandez, Juan Manuel, Saboune, Jamal
• Primary Examiner(s): Lau, Hoi C
• Application Number:
  US16/166,434
• Publication Number:
  US 20190057583A1
• Time in Patent Office: 309 Days
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

The present disclosure is generally directed to systems and methods for providing haptic effects based on information complementary to multimedia content. For example, one disclosed method includes the steps of receiving multimedia data comprising multimedia content and complementary data, wherein the complementary data describes the multimedia content, determining a haptic effect based at least in part on the complementary data, and outputting the haptic effect while playing the multimedia content.

Claims

1. A server comprising:

a processor; and

a memory comprising program code that is executable by the processor to cause the processor to;

receive multimedia content from a first remote device;

determine a plurality of haptic effects corresponding to events occurring in the multimedia content by analyzing the multimedia content;

generate a haptic track based on the plurality of haptic effects; and

transmit the multimedia content with the haptic track to a second remote device, the second remote device being configured to receive the haptic track and responsively output the plurality of haptic effects specified in the haptic track while playing back the multimedia content.

2. The server of claim 1, wherein the multimedia content includes video content and audio content.

3. The server of claim 2, wherein the video content depicts video game activity.

4. The server of claim 1, further comprising embedding the haptic track into the multimedia content prior to transmitting the multimedia content to the second remote device.

5. The server of claim 1, wherein transmitting the multimedia content comprises streaming the multimedia content with the haptic track to the second remote device over the Internet.

6. The server of claim 1, wherein analyzing the multimedia content comprises analyzing video content or audio content within the multimedia content for the events corresponding to the plurality of haptic effects.

7. The server of claim 1, wherein the memory further comprises program code that is executable by the processor to cause the processor to determine the plurality of haptic effects at least in part by analyzing complementary data describing the multimedia content.

8. A method comprising:

receiving, by a processor, multimedia content from a first remote device;

determining, by the processor, a plurality of haptic effects corresponding to events occurring in the multimedia content by analyzing the multimedia content;

generating, by the processor, a haptic track based on the plurality of haptic effects; and

transmitting, by the processor, the multimedia content with the haptic track to a second remote device, the second remote device being configured to receive the haptic track and responsively output the plurality of haptic effects specified in the haptic track while playing back the multimedia content.

9. The method of claim 8, wherein the multimedia content includes video content and audio content.

10. The method of claim 9, wherein the video content depicts video game activity.

11. The method of claim 8, further comprising embedding the haptic track into the multimedia content prior to transmitting the multimedia content to the second remote device.

12. The method of claim 8, wherein transmitting the multimedia content comprises streaming the multimedia content with the haptic track to the second remote device over the Internet.

13. The method of claim 8, wherein analyzing the multimedia content comprises analyzing video content or audio content within the multimedia content for the events corresponding to the plurality of haptic effects.

14. The method of claim 8, further comprising determining the plurality of haptic effects at least in part by analyzing complementary data describing the multimedia content.

15. A non-transitory computer-readable medium comprising program code that is executable by a processor to cause the processor to:

receive multimedia content from a first remote device;

determine a plurality of haptic effects corresponding to events occurring in the multimedia content by analyzing the multimedia content;

generate a haptic track based on the plurality of haptic effects; and

transmit the multimedia content with the haptic track to a second remote device, the second remote device being configured to receive the haptic track and responsively output the plurality of haptic effects specified in the haptic track while playing back the multimedia content.

16. The non-transitory computer-readable medium of claim 15, wherein the multimedia content depicts video game activity.

17. The non-transitory computer-readable medium of claim 15, further comprising program code that is executable by the processor to cause the processor to embed the haptic track into the multimedia content prior to transmitting the multimedia content to the second remote device.

18. The non-transitory computer-readable medium of claim 15, wherein transmitting the multimedia content comprises streaming the multimedia content with the haptic track to the second remote device over the Internet.

19. The non-transitory computer-readable medium of claim 15, wherein analyzing the multimedia content comprises analyzing video content or audio content within the multimedia content for the events corresponding to the plurality of haptic effects.

20. The non-transitory computer-readable medium of claim 15, further comprising program code that is executable by the processor to cause the processor to determine the plurality of haptic effects at least in part by analyzing complementary data describing the multimedia content.