Synchronization of haptic effect data in a media transport stream

• US 9,615,002 B2
• Filed: 04/14/2014
• Issued: 04/04/2017
• Est. Priority Date: 10/19/2005
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Lacroix, Robert A., Rabemiarisoa, Andrianaivo, Da Costa, Henrique D., Timone, Herve Thu, Rank, Stephen D., Ullrich, Christopher J.
• Primary Examiner(s): TRAN, NAM T
• Application Number:
  US14/252,391
• Publication Number:
  US 20140226068A1
• Time in Patent Office: 1,086 Days
• Field of Search: 709231, 348460, 455 306, 463 30
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56

Abstract

A method for synchronizing haptic effects with at least one media component in a media transport stream includes identifying a series of video frames containing imaging information and/or a series of audio frames containing sound information in the media transport stream; identifying a series of haptic frames containing force feedback information in the media transport stream; and synchronizing the force feedback information in response to the imaging information and/or sound information.

Claims

1. A method for synchronizing haptic effects, the method comprising:

identifying by a media player, a series of video frames containing imaging information or a series of audio frames containing sound information, in a media transport stream;

identifying by the media player, a series of haptic frames containing force feedback information, in the media transport stream; and

synchronizing the force feedback information with the imaging information or the sound information, wherein the synchronizing is performed by a media synchronization layer and based on a plurality of predetermined time stamps specified within a master timing table in the media transport stream, wherein, when a time stamp is assigned to a haptic frame, the haptic frame is executed at a time indicated by the time stamp.

2. The method according to claim 1, wherein the synchronizing of the force feedback information comprises utilizing time stamps within the series of video frames or the series of audio frames to determine which haptic frame of the series of haptic frames to play back with a selected video frame or a selected audio frame.

3. The method according to claim 2, further comprising assigning the time stamps to the series of haptic frames by the media synchronization layer.

4. The method according to claim 1, wherein each haptic frame in the series of haptic frames is sized with substantially the same length of time.

5. The method according to claim 1, wherein each haptic frame in the series of haptic frames contains multiple haptic effects.

6. The method according to claim 5, wherein the multiple haptic effects are offset in time from one another relative to the start of a corresponding haptic frame.

7. The method according to claim 1, wherein the force feedback information comprises at least one of intensity, magnitude, duration, periodicity, start-delay, attack intensity, attack timing, fade intensity, or fade timing of a haptic effect.

8. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform synchronization of haptic effects, the performing comprising:

identifying by a media player, a series of video frames containing imaging information or a series of audio frames containing sound information, in a media transport stream;

identifying by the media player, a series of haptic frames containing force feedback information, in the media transport stream; and

synchronizing the force feedback information with the imaging information or the sound information, wherein the synchronizing is performed by a media synchronization layer and based on a plurality of predetermined time stamps specified within a master timing table in the media transport stream, wherein, when a time stamp is assigned to a haptic frame, the haptic frame is executed at a time indicated by the time stamp.

9. The non-transitory computer readable medium according to claim 8, wherein the synchronizing of the force feedback information comprises utilizing time stamps within the series of video frames or the series of audio frames to determine which haptic frame of the series of haptic frames to play back with a selected video frame or a selected audio frame.

10. The non-transitory computer readable medium according to claim 9, wherein the synchronizing of the force feedback information further comprises assigning the time stamps to the series of haptic frames by the media synchronization layer.

11. The non-transitory computer readable medium according to claim 8, wherein each haptic frame in the series of haptic frames is sized with substantially the same length of time.

12. The non-transitory computer readable medium according to claim 8, wherein each haptic frame in the series of haptic frames contains multiple haptic effects.

13. The non-transitory computer readable medium according to claim 12, wherein the multiple haptic effects are offset in time from one another relative to the start of a corresponding haptic frame.

14. The non-transitory computer readable medium according to claim 8, wherein the force feedback information comprises at least one of intensity, magnitude, duration, periodicity, start-delay, attack intensity, attack timing, fade intensity, or fade timing of a haptic effect.

15. A system for synchronizing haptic effects, the system comprising:

a processor configured to execute a collection of software modules, wherein the processor is further configured, when executing the collection of software modules, to;

identify by a media player, a series of video frames containing imaging information or a series of audio frames containing sound information, in a media transport stream;

identify by the media player, a series of haptic frames containing force feedback information, in the media transport stream; and

synchronize the force feedback information with the imaging information or the sound information, wherein the synchronizing is performed by a media synchronization layer and based on a plurality of predetermined time stamps specified within a master timing table in the media transport stream, wherein, when a time stamp is assigned to a haptic frame, the haptic frame is executed at a time indicated by the time stamp.

16. The system according to claim 15, wherein the synchronizing of the force feedback information comprises utilizing time stamps within the series of video frames or the series of audio frames to determine which haptic frame of the series of haptic frames to play back with a selected video frame or a selected audio frame.

17. The system according to claim 16, wherein the time stamps are assigned to the series of haptic frames by the media synchronization layer.

18. The system according to claim 15, wherein each haptic frame in the series of haptic frames is sized with substantially the same length of time.

19. The system according to claim 15, wherein each haptic frame in the series of haptic frames contains multiple haptic effects.

20. The system according to claim 19, wherein the multiple haptic effects are offset in time from one another relative to the start of a corresponding haptic frame.