Synchronization of haptic effect data in a media stream

• US 9,019,087 B2
• Filed: 10/16/2007
• Issued: 04/28/2015
• Est. Priority Date: 10/16/2007
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Bakircioglu, Munibe M., Rank, Stephen D., Ullrich, Christopher J.
• Primary Examiner(s): Bugg, George
• Assistant Examiner(s): SMALL, NAOMI J
• Application Number:
  US11/975,120
• Publication Number:
  US 20090096632A1
• Time in Patent Office: 2,751 Days
• Field of Search: 340/407.1, 340/691.1, 463/30, 348/473
• US Class Current: 340/407.1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 5/18 Timing circuits for raster scan displays specially adapted for television H04N ; synchronisation between the display unit and other display units, videodisc player G09G5/12
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 1/72442 for playing music files
• CPC Class Codes: H04M 19/047 Vibrating means for incoming calls
• CPC Class Codes: H04M 19/048 Arrangements providing optical indication of the incoming call, e.g. flasher circuits
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/81 Monomedia components thereof
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Haptic information in a series of frames of a media file is identified and time stamps corresponding thereto are determined in accordance with information embedded in a tag either preceding or appending the media file. The haptic effect information is automatically synchronized to the media file based on the information in one or more frames of the tag, whereby an actuator outputs the defined haptic effects in the tag in a manner synchronized with corresponding audio and/or video content of the media file.

Claims

1. A method comprising:

receiving a media file having embedded haptic event data in a tag preceding content data in the media file, the haptic event data having time stamp information embedded in one or more frames of the haptic event data;

synchronizing the embedded haptic event data with the content data in the media file by assigning one or more time stamps to the one or more frames of the haptic event data according to a master time code embedded in the media file, wherein each time stamp is synchronized with a corresponding frame of the content data, wherein each time stamp is embedded within the tag preceding the content data in the media file, wherein at least one time stamp of the one or more time stamps is a sub-time stamp that is an offset of another time stamp within a single frame of the haptic event data; and

outputting one or more haptic effects via an actuator in a playback device, wherein each outputted haptic effect is synchronized with a corresponding media event of the content data, each media event and corresponding haptic event occurring at a substantially same time stamp, and wherein at least one haptic effect is output at the offset based on an offset parameter indicated within the haptic event data.

2. The method of claim 1, wherein the frame of the embedded haptic event data is in a private frame of the tag.

3. The method of claim 1, wherein the frame of the embedded haptic event data is in a lyrics frame of the tag.

4. The method of claim 1, wherein the content data includes audio data.

5. The method of claim 1, wherein the content data includes video data.

6. The method of claim 1, wherein the embedded haptic event data further comprises a start-time, a magnitude, a frequency, and a duration.

7. The method of claim 1, wherein the media file is stored prior to being received at the playback device.

8. The method of claim 1, wherein the media file is streamed from a source and received by the playback device.

9. The method of claim 1, wherein the master time code comprises one or more time stamps independent of the content data.

10. An apparatus comprising:

means for receiving a media file having embedded haptic event data in a tag preceding content data in the media file, the haptic event data having time stamp information embedded in one or more frames of the haptic event data;

means for synchronizing the embedded haptic event data with the content data in the media file by assigning one or more time stamps to the one or more frames of the haptic event data according to a master time code embedded in the media file, wherein each time stamp is synchronized with the master time code, wherein each time stamp is embedded within the tag preceding the content data in the media file, and wherein at least one time stamp of the one or more time stamps is a sub-time stamp that is an offset of another time stamp within a single frame of the haptic event data;

means for outputting one or more haptic effects via an actuator, wherein each haptic effect is associated with each time stamp of the haptic event data, and wherein at least one haptic effect is output at the offset based on an offset parameter indicated within the haptic event data.

11. The apparatus of claim 10, wherein the frame of the embedded haptic event data is in a private frame of the tag.

12. The apparatus of claim 10, wherein the frame of the embedded haptic event data is in a lyrics frame of the tag.

13. The apparatus of claim 10, wherein the content data includes audio data.

14. The apparatus of claim 10, wherein the content data includes video data.

15. The apparatus of claim 10, wherein the media file is in a MP3 format.

16. The apparatus of claim 10, wherein the embedded haptic event data further comprises a start-time, a magnitude, a frequency, and a duration.

17. The apparatus of claim 10, wherein the media file is stored prior to being received at the playback device.

18. The apparatus of claim 10, wherein the media file is streamed from a source and received by the playback device.

19. The apparatus of claim 10, wherein the master time code comprises one or more time stamps independent of the content data.

20. Logic encoded in one or more tangible media for execution by a machine and when executed operable to perform a method, the method comprising:

receiving a media file having embedded haptic event data in a tag preceding content data in the media file, the haptic event data having time stamp information embedded in one or more frames of the haptic event data;

synchronizing the embedded haptic event data with the content data in the media file by assigning one or more time stamps to the one or more frames of the haptic event data according to a master time code embedded in the media file, wherein each time stamp is synchronized with a corresponding frame of the content data, wherein each time stamp is embedded within the tag preceding the content data in the media file, and wherein at least one time stamp of the one or more time stamps is a sub-time stamp that is an offset of another time stamp within a single frame of the haptic event data; and

outputting one or more haptic effects via an actuator in a playback device, wherein each outputted haptic effect is synchronized with a corresponding media event of the content data, each media event and the corresponding haptic event occurring at a substantially same time stamp, and wherein at least one haptic effect is output at the offset based on an offset parameter indicated within the haptic event data.