Synchronization of haptic effect data in a media transport stream

• US 9,912,842 B2
• Filed: 02/03/2017
• Issued: 03/06/2018
• Est. Priority Date: 10/19/2005
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Lacroix, Robert A., Rabemiarisoa, Andrianaivo, Da Costa, Henrique D., Timone, Herve Thu, Rank, Stephen D., Ullrich, Christopher J.
• Primary Examiner(s): TRAN, NAM T
• Application Number:
  US15/423,960
• Publication Number:
  US 20170150013A1
• Time in Patent Office: 396 Days
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56

Abstract

A method for synchronizing haptic effects with at least one media component in a media transport stream includes identifying a series of video frames containing imaging information and/or a series of audio frames containing sound information in the media transport stream; identifying a series of haptic frames containing force feedback information in the media transport stream; and synchronizing the force feedback information in response to the imaging information and/or sound information.

Claims

1. A method for synchronizing haptic effects, the method comprising:

receiving a media transport stream, wherein the media transport stream comprises haptic frames and media frames;

parsing the media transport stream into the haptic frames and the media frames;

synchronizing the haptic frames with the media frames, wherein the synchronizing is performed by a media synchronization layer and based on predetermined time stamps specified within a master timing table in the media transport stream;

assigning a particular time stamp from the predetermined time stamps to a articular haptic frame; and

executing the particular haptic frame at a time indicated by the particular time stamp to cause activation of an actuator to generate a haptic effect.

2. The method according to claim 1, wherein the media frames comprise at least one of a video frame and an audio frame.

3. The method according to claim 1, wherein each haptic frame of the haptic frames in the media transport stream is sized with substantially a same length of time.

4. The method according to claim 1, wherein at least one haptic frame of the haptic frames in the media transport stream comprises multiple haptic effects.

5. The method according to claim 4, wherein the multiple haptic effects are offset in time from one another relative to a start of the at least one haptic frame.

6. The method according to claim 1, wherein the haptic effect comprises at least one of intensity, magnitude, duration, periodicity, start-delay, attack intensity, attack timing, fade intensity, or fade timing.

7. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform synchronization of haptic effects, the performing comprising:

receiving a media transport stream, wherein the media transport stream comprises haptic frames and media frames;

parsing the media transport stream into the haptic frames and the media frames;

synchronizing the haptic frames with the media frames, wherein the synchronizing is performed by a media synchronization layer and based on predetermined time stamps specified within a master timing table in the media transport stream;

assigning a particular time stamp from the predetermined time stamps to a particular haptic frame; and

executing the particular haptic frame at a time indicated by the particular time stamp to cause activation of an actuator to generate a haptic effect.

8. The non-transitory computer readable medium according to claim 7, wherein the media frames comprise at least one of a video frame and an audio frame.

9. The non-transitory computer readable medium according to claim 7, wherein each haptic frame of the haptic frames in the media transport stream is sized with substantially a same length of time.

10. The non-transitory computer readable medium according to claim 7, wherein at least one haptic frame of the haptic frames in the media transport stream contains multiple haptic effects.

11. The non-transitory computer readable medium according to claim 10, wherein the multiple haptic effects are offset in time from one another relative to a start of the at least one haptic frame.

12. The non-transitory computer readable medium according to claim 7, wherein the haptic effect comprises at least one of intensity, magnitude, duration, periodicity, start-delay, attack intensity, attack timing, fade intensity, or fade timing.

13. A system for synchronizing haptic effects, the system comprising:

a media player configured to receive a media transport stream, wherein the media transport stream comprises haptic frames and media frames;

the media player further configured to;

parse the media transport stream into the haptic frames and the media frames and synchronize the haptic frames with the media frames, wherein the synchronizing is performed by a media synchronization layer and based on predetermined time stamps specified within a master timing table in the media transport stream,

assign a particular time stamp from the predetermined time stamps to a particular haptic frame, and

execute the particular haptic frame at a time indicated by the particular time stamp; and

an actuator configured to generate a haptic effect in response to execution of the particular haptic frame.

14. The system according to claim 13, wherein the media frames comprise at least one of a video frame and an audio frame.

15. The system according to claim 13, wherein each haptic frame of the haptic frames in the media transport stream is sized with substantially a same length of time.

16. The system according to claim 13, wherein at least one haptic frame of the haptic frames in the media transport stream comprises multiple haptic effects.

17. The system according to claim 16, wherein the multiple haptic effects are offset in time from one another relative to a start of the at least one haptic frame.