Transport hint table for synchronizing delivery time between multimedia content and multimedia content descriptions

• US 20020198905A1
• Filed: 03/28/2002
• Published: 12/26/2002
• Est. Priority Date: 05/29/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Tabatabai, Ali, Rising, Hawley K. III, Visharam, Mohammed Z., Walker, Toby
• Granted Patent: US 7,734,997 B2
• Time in Patent Office: Days
• US Class Current: 715/201
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

The content description of multimedia content contains a plurality of descriptor values that pertain to specific events within the media. A Transport Hint Table (THT) provides a set of synchronization events that indicate the relation between timed events in the media and their representation in the content'"'"'s description data. The THT allows for synchronized delivery of presentation of content description data with the associated events in the media. The THT provides a set of sync events based on the existence of “time” related events obtained from the content description data, which are relevant to particular events in the media. The THT allows the transport layer encoder to use the time-related events to synchronize the scheduled delivery times of media content segments and their associated descriptions. The THT also provides information for fragmenting a description for delivery and for managing resources at the decoder.

Claims

1. A method comprising:

accessing the content description of a media content, the content description containing a plurality of descriptor values, each descriptor value pertaining to a specific event in the media content; and

generating a set of sync events, each sync event indicating an association between an event in a content description and a temporal interval of the sync event.

2. The method of claim 1, wherein each sync event is an association between a reference element and a location element, the reference element specifying a path to a particular event in the content description, the location element specifying the temporal interval of the sync event.

3. The method of claim 2, wherein each sync event is ordered based on their time of occurrence in the content description.

4. The method of claim 2, wherein the content description is an MPEG-7 content description.

5. The method of claim 4, wherein generating a set of sync events comprises:

parsing the content description to extract MPEG-7 XML tags that provide information related to a temporal order of the events.

6. The method of claim 5, wherein the MPEG-7 XML tags are selected from the group consisting of MediaTime, MediaLocator, TemporalSegmentLocator, and ImageLocator.

7. A method comprising:

transmitting a set of sync events, each sync event an association between a reference element and a location element, the reference element specifying the path to a particular event in the content description, the location element specifying the temporal interval in the sync event, along with the corresponding media content segment; and

updating the location element of each sync event to reflect a presentation time stamp and the duration of a corresponding media segment.

8. The method of claim 7, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery transport mechanism selected from the group consisting of an MPEG-2 data stream, an MPEG-4 ES for a media object, and MPEG-7 decoder configuration information.

9. A method comprising:

receiving a set of sync events, each sync event indicating an association between a reference element and a location element, the reference element specifying the path to a to a particular event in the content description, the location element specifying temporal interval of the sync event of the sync event;

receiving the media content; and

mapping a presentation time stamp of a segment of the media content to a location element of a corresponding sync event.

10. The method of claim 9, wherein mapping the presentation time stamp of the segment of media content to the location element of a corresponding sync event is effected through use of an update command,

11. The method of claim 10, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

13. The machine-readable medium of claim 12, wherein each sync event is an association between a reference element and a location element, the reference element specifying a path to a particular event in the content description, the location element specifying the temporal interval of the sync event.

14. The machine-readable medium of claim 13, wherein each sync event is ordered based on their time of occurrence in the content description.

15. The machine-readable medium of claim 13, wherein the content is a MPEG-7 content description.

16. The machine-readable medium of claim 15, wherein generating a set of sync events comprises:

parsing the content description to extract the MPEG-7 XML tags that provide information related to a temporal order of the events.

17. The machine-readable medium of claim 16, wherein the MPEG-7 XML tags are selected from the group consisting of MediaTime, MediaLocator, TemporalSegmentLocator, and ImageLocator.

18. The machine-readable medium of claim 13, wherein the method further comprises:

transmitting a set of sync events, each sync event an association between a reference element and a location element, the reference element specifying the path to a particular event in the content description, the location element specifying the temporal interval in the sync event, along with the corresponding media content segment; and

updating the location element of each sync event to reflect a presentation time stamp and the duration of a corresponding media segment.

19. The machine-readable medium of claim 18, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery transport mechanism selected from the group consisting of an MPEG-2 data stream, MPEG-4 ES for a media object, and MPEG-7 decoder configuration information.

12. A machine-readable medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

accessing the content description of a media content, the content description containing a plurality of descriptor values, each descriptor value pertaining to a specific event in the media content; and

generating a set of sync events, each sync event indicating an association between an event in a content description and a temporal interval of the sync event.

20. A machine-readable medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

receiving a set of sync events, each sync event indicating an association between a reference element and a location element, the reference element specifying the path to a to a particular event in the content description, the location element specifying temporal interval of the sync event of the sync event;

receiving the media content; and

mapping a presentation time stamp of a segment of the media content to a location element of a corresponding sync event.

21. The machine-readable medium of claim 20, wherein mapping the presentation time stamp of the segment of media content to the location element of a corresponding sync event is effected through use of an update command.

22. The machine-readable medium of claim 21, wherein the update command distinguishes between updates to the set of sync events and updates to the content description

24. The apparatus of claim 23, wherein each sync event is an association between a reference element and a location element, the reference element specifying a path to a particular event in the content description, the location element specifying the temporal interval of the sync event.

25. The apparatus of claim 24, wherein each sync event is ordered based on their time of occurrence in the content description.

26. The apparatus of claim 24, wherein the content description is an MPEG-7 content description.

27. The apparatus of claim 26, wherein to generate a set of sync events further comprises:

parsing the content description to extract MPEG-7 XML tags that provide information related to a temporal order of the events.

28. The apparatus of claim 27, wherein the MPEG-7 XML tags are selected from the group consisting of MediaTime, MediaLocator, TemporalSegmenLocator, and ImageLocator.

29. The apparatus of claim 24 further comprising:

a transmitter, coupled to the processor, to transmit the set of sync events along with the corresponding media content segment; and

an encoder to update the location element of each sync event to reflect a presentation time stamp and the duration of a corresponding media segment.

30. The apparatus of claim 29, wherein the encoder is configured to map a presentation time stamp of a media segment to a location element of a corresponding sync event.

31. The apparatus of claim 30, wherein mapping the presentation time stamp of a media segment to the location element of a corresponding sync event is effected through use of an update command.

32. The apparatus of claim 31, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

33. The apparatus of claim 27, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery mechanism selected from the group consisting of an MPEG-2 data stream, an MPEG-4ES for a media object, and MPEG-7 decoder configuration information.

35. The apparatus of claim 34, wherein mapping the presentation time stamp of a segment of media content to the location element of a corresponding sync event is effected through use of an update command.

36. The apparatus of claim 35, wherein the update command distinguishing between updates to the set of sync events and updates to the content description.

38. The method of claim 37 further comprising transmitting the media content segment and corresponding sync event.

39. The method of claim 37, wherein the set of sync events is transmitted prior to updating.

23. An apparatus comprising:

a processor, and a memory coupled thereto, the memory having stored thereon executable instructions, which, when executed by the processor, cause the processor to access the content description of a media content, the content description containing a plurality of descriptor values, each descriptor value pertaining to a specific event in the media content, and generate a set of sync events, each sync event indicating an association between an event in a content description and a temporal interval of the sync event.

34. An apparatus comprising:

a processor, and a memory coupled thereto, the memory having stored thereon executable instructions, which, when executed by the processor, cause the processor to receive a set of sync events, each sync event indicating an association between a reference element and a location element, the reference element specifying the path to a to a particular event in the content description, the location element specifying temporal interval of the sync event of the sync event, receive the media content, and map a presentation time stamp of a segment of the media content to a location element of a corresponding sync event.

37. A method comprising:

extracting a set of sync events from a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value pertaining to specific media content, each sync event an association between a reference element and a location element, the reference element specifying the path to a particular event in the content description, the location element specifying the temporal interval in the sync event, along with the corresponding media content segment;

encoding media content for transmission, a presentation time stamp added to each of a plurality of media content segments; and

updating the location element of a sync event corresponding to a media content segment to reflect the presentation time stamp of the media content segment.

40. A machine-readable medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

extracting a set of sync events from a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value pertaining to specific media content, each sync event an association between a reference element and a location element, the reference element specifying the path to a particular event in the content description, the location element specifying the temporal interval in the sync event, along with the corresponding media content segment;

encoding media content for transmission, a presentation time stamp added to each of a plurality of media content segments; and

updating the location element of a sync event corresponding to a media content segment to reflect the presentation time stamp of the media content segment.

41. The machine-readable medium of claim 40 further comprising transmitting the media content segment and corresponding sync event.

42. The machine-readable medium of claim 40, wherein the set of sync events is transmitted prior to updating.

43. A method comprising:

receiving media content at a media presentation device, the media content having a content description containing a plurality of descriptor values, each descriptor value pertaining to specific event in the media content;

buffering a portion of the media content for subsequent presentation;

accessing a set of sync events, each sync event indicating a temporally ordered segment of the content description; and

discarding buffered portions of the media content based upon information contained in the set of sync events.

44. The method of claim 43, wherein each sync event is an association between a reference element and a location element, the reference element specifying the path to a particular event in the content description, the location element specifying the temporal interval in the sync event, along with the corresponding media content segment.

45. The method of claim 43, wherein portions of the media content are selected for display on the media presentation device based upon information contained in the set of sync events.

47. The machine-readable medium of claim 46, wherein each sync event is an association between a reference element and a location element, the reference element specifying the temporal interval in the sync event, along with the corresponding media content segment.

48. The machine-readable medium of claim 46, wherein portions of the media content are selected for display on the media presentation device based upon information contained in the set of sync events.

46. A machine-readable medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

receiving media content at a media presentation device, the media content having a content description containing a plurality of descriptor values, each descriptor value pertaining to specific media content;

buffering a portion of the media content for subsequent presentation;

accessing a set of sync events, each sync event indicating a temporally ordered segment of the content description; and

discarding or retaining buffered portions of the media content based upon information contained in the set of sync events.