Transport hint table for synchronizing delivery time between multimedia content and multimedia content descriptions

• US 7,734,997 B2
• Filed: 03/28/2002
• Issued: 06/08/2010
• Est. Priority Date: 05/29/2001
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Tabatabai, Ali, Rising, Hawley K. III, Visharam, Mohammed Z., Walker, Toby
• Primary Examiner(s): Stevens; Robert
• Application Number:
  US10/113,001
• Publication Number:
  US 20020198905A1
• Time in Patent Office: 2,994 Days
• Field of Search: 715/501.1, 715/500, 715/201, 715/202, 715/203, 715/204, 707/102, 707/104.1, 707/793, 707/803
• US Class Current: 715/203
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

The content description of multimedia content contains a plurality of descriptor values that pertain to specific events within the media. A Transport Hint Table (THT) provides a set of synchronization events that indicate the relation between timed events in the media and their representation in the content'"'"'s description data. The THT allows for synchronized delivery of presentation of content description data with the associated events in the media. The THT provides a set of sync events based on the existence of “time” related events obtained from the content description data, which are relevant to particular events in the media. The THT allows the transport layer encoder to use the time-related events to synchronize the scheduled delivery times of media content segments and their associated descriptions. The THT also provides information for fragmenting a description for delivery and for managing resources at the decoder.

Claims

1. A computerized method comprising:

accessing a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content; and

generating, by a computer, for transmission to a media presentation device, a set of sync events from the content description, wherein the set of sync events is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the timed events by the media presentation device, each sync event logically relating a timed event and a temporal interval that defines an occurrence of the timed event in the media content, wherein the set of sync events is subsequently transmitted as a table to the media presentation device for access by a user to control the presentation of the timed events and the transmission of the descriptor values, the table ordered according to fields in the sync events.

2. The method of claim 1, wherein each sync event comprises an association between a reference element and a location element, the reference element specifying a path in the content description to a descriptor value for a timed event, the location element specifying the temporal interval for the timed event.

3. The method of claim 2, wherein each sync event is ordered by its temporal interval.

4. The method of claim 2, wherein the content description is an MPEG-7 content description.

5. The method of claim 4, wherein generating a set of sync events comprises:

parsing the content description to extract MPEG-7 XML tags that provide information related to a temporal order of the timed events.

6. The method of claim 5, wherein the MPEG-7 XML tags are selected from the group consisting of Media Time, MediaLocator, TemporalSegmentLocator, and ImageLocator.

7. A computerized method comprising:

transmitting, to a media presentation device, a set of sync events as a table ordered according to fields in the sync events, each sync event comprising a reference element and a location element, the reference element specifying a path within a content description to a descriptor value that describes a timed event in a media content, the location element specifying a temporal interval that defines an occurrence of the timed event in the media content, the sync event generated from a content description containing a plurality of descriptor values, and wherein the sync event is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the time events by the media presentation device; and

synchronizing, by a computer, transmission of the descriptor values with presentation of the timed events by updating the location element of each sync event to be relative to a presentation time stamp for the media content, the presentation time stamp set during encoding of the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values.

8. The method of claim 7, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery transport mechanism selected from the group consisting of an MPEG-2 data stream, an MPEG-4 ES for a media object, and MPEG-7 decoder configuration information.

9. A computerized method comprising:

receiving a set of sync events as a table ordered according to fields in the sync events, each sync event comprising a reference element and a location element, the reference element specifying a path within a content description to a descriptor value that describes a timed event in a media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, the sync event generated from a content description containing a plurality of descriptor values;

receiving the media content; and

updating, by a computer, the location element of a sync event to be relative to a presentation time stamp for the media content, wherein the sync event is used by a multimedia presentation device to synchronize transmission of the descriptor values with presentation of the time events by the media presentation device, the presentation time stamp being set during encoding of the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values.

10. The method of claim 9, wherein the updating is caused by an update command.

11. The method of claim 10, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

12. A machine-readable storage medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

accessing the content description of a media content, the content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content; and

generating, for transmission to a media presentation device, a set of sync events from the content description, wherein the set of sync events is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the timed events by the media presentation device, each sync event logically relating a timed event and a temporal interval that defines an occurrence of the timed event in the media content, wherein the set of sync events is subsequently transmitted as a table to the media presentation device for access by a user to control the presentation of the timed events and the transmission of the descriptor values, the table ordered according to fields in the sync events.

13. The machine-readable storage medium of claim 12, wherein each sync event comprises an association between a reference element and a location element, the reference element specifying a path in the content description to a descriptor value for a timed event, the location element specifying the temporal interval for the timed event.

14. The machine-readable storage medium of claim 13, wherein each sync event is ordered based on its temporal interval.

15. The machine-readable storage medium of claim 13, wherein the content is a MPEG-7 content description.

16. The machine-readable storage medium of claim 15, wherein generating a set of sync events comprises:

parsing the content description to extract MPEG-7 XML tags that provide information related to a temporal order of the timed events.

17. The machine-readable storage medium of claim 16, wherein the MPEG-7 XML tags are selected from the group consisting of MediaTime, MediaLocator, TemporalSegmentLocator, and ImageLocator.

18. The machine-readable storage medium of claim 13, wherein the method further comprises:

transmitting, to a multimedia presentation device, a set of sync events as a table ordered according to fields in the sync events, each sync event comprising a reference element and a location element, the reference element specifying a path within the content description to a descriptor value that describes a timed event in a media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, the sync event generated from a content description containing a plurality of descriptor values, and wherein the sync event is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the time events by the media presentation device; and

updating the location element of each sync event to be relative to a presentation time stamp for the media content, wherein the sync event is used by a multimedia presentation device to synchronize transmission of the descriptor values with presentation of the timed events at the media presentation device, the presentation time stamp being set during encoding of the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values.

19. The machine-readable storage medium of claim 18, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery transport mechanism selected from the group consisting of an MPEG-2 data stream, MPEG-4 ES for a media object, and MPEG-7 decoder configuration information.

20. A machine-readable storage medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

receiving a set of sync events as a table ordered according to fields in the sync events, each sync event comprising a reference element and a location element, the reference element specifying a path within a content description to a descriptor value that describes a timed event in a media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, the sync event generated from a content description containing a plurality of descriptor values;

receiving the media content; and

updating the location element of a sync event to be relative to a presentation stamp for the media content, wherein the sync event is used by a multimedia presentation device to synchronize transmission of the descriptor values with presentation of the timed events by the media presentation device, the presentation time stamp being set during encoding of the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values.

21. The machine-readable storage medium of claim 20, wherein the updating is caused by an update command.

22. The machine-readable storage medium of claim 21, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

23. An apparatus comprising:

a processor, and a memory coupled thereto, the memory having stored thereon executable instructions, which, when executed by the processor, cause the processor to;

access a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content, and

generate for transmission to a media presentation device, a set of sync events from the content description, wherein the set of sync events is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the timed events by the media presentation device, each sync event logically relating a timed event and a temporal interval that defines an occurrence of the timed event in the media content, wherein the set of sync events is subsequently transmitted as a table to the media presentation device for access by a user to control the presentation of the timed events and the transmission of the descriptor values, the table ordered according to fields in the sync events.

24. The apparatus of claim 23, wherein each sync event comprises a reference element and a location element, the reference element specifying a path within the content description to a descriptor value for a timed event, the location element specifying the temporal interval for the timed event.

25. The apparatus of claim 24, wherein each sync event is ordered by its temporal interval.

26. The apparatus of claim 24, wherein the content description is an MPEG-7 content description.

27. The apparatus of claim 26, wherein to generate a set of sync events further comprises:

parsing the content description to extract MPEG-7 XML tags that provide information related to a temporal order of the timed events.

28. The apparatus of claim 27, wherein the MPEG-7 XML tags are selected from the group consisting of MediaTime, MediaLocator, TemporalSegmentLocator, and ImageLocator.

29. The apparatus of claim 24 further comprising:

a transmitter, coupled to the processor, to transmit the set of sync events as a table along with the media content, wherein each sync event comprises a location element that specifies the temporal interval and the table is ordered according to fields in the sync events; and

an encoder to update the location element of each sync event to be relative to a presentation time stamp for the media content set that is set by the encoder.

30. The apparatus of claim 29, wherein the encoder is configured to determine the presentation time stamp.

31. The apparatus of claim 30, wherein the updating is caused by an update command.

32. The apparatus of claim 31, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

33. The apparatus of claim 29, wherein the set of sync events is transmitted by including the set of sync events as an element of a delivery mechanism selected from the group consisting of an MPEG-2 data stream, an MPEG-4ES for a media object, and MPEG-7 decoder configuration information.

34. An apparatus comprising:

a processor, and a memory coupled thereto, the memory having stored thereon executable instructions, which, when executed by the processor, cause the processor to receive a set of sync events as a table ordered according to fields in the sync events, each sync event comprising a reference element and a location element, the reference element specifying a path within a content description to a descriptor value that describes a timed event in a media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, to receive the media content, the sync event generated from a content description containing a plurality of descriptor values, and to update the location element of a sync event to be relative to a presentation time stamp for the media content, wherein the sync event is used by a multimedia presentation device to synchronize transmission of the descriptor values with presentation of the timed events by the media presentation device, the presentation time stamp being set during encoding of the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values.

35. The apparatus of claim 34, wherein the updating is caused by an update command.

36. The apparatus of claim 35, wherein the update command distinguishes between updates to the set of sync events and updates to the content description.

37. A computerized method comprising:

generating, by a computer, for transmission to a media presentation device, a set of sync events from a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content, each sync event comprising a reference element and a location element, the reference element specifying a path within the content description to a descriptor value for a timed event in the media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, and wherein the set of sync events is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the time events by the media presentation device;

encoding the media content for transmission, a presentation time stamp added to the media content; and

synchronizing transmission of the descriptor values with presentation of the timed events by a media presentation device by updating the location element of a sync event to be relative to the presentation time stamp, wherein the set of sync events is subsequently transmitted as a table to the media presentation device for access by a user to control the presentation of the timed events and the transmission of the descriptor values, the table ordered according to fields in the sync events.

38. The method of claim 37 further comprising transmitting the media content and the set of sync events.

39. The method of claim 37, wherein the set of sync events is transmitted prior to updating.

40. A machine-readable storage medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

generating, for transmission to a media presentation device, a set of sync events from a content description of a media content, the content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content, each sync event comprising a reference element and a location element, the reference element specifying a path within the content description to a descriptor value for a timed event in the media content, the location element specifying a temporal interval for an occurrence of the timed event in the media content, and wherein the set of sync events is used by the media presentation device to synchronize transmission of the descriptor values with presentation of the time events by the media presentation device;

encoding media content for transmission, a presentation time stamp added to the media content; and

synchronizing transmission of the descriptor values with presentation of the timed events by updating the location element of a sync event to be relative to the presentation time stamp, wherein the set of sync events is subsequently transmitted as a table to the media presentation device for access by a user to control the presentation of the timed events and the transmission of the descriptor values, the table ordered according to fields in the sync events.

41. The machine-readable storage medium of claim 40 further comprising transmitting the media content and the set of sync events.

42. The machine-readable storage medium of claim 40, wherein the set of sync events is transmitted prior to updating.

43. A computerized method comprising:

receiving media content at a media presentation device, the media content having a content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content;

buffering a portion of the media content for subsequent presentation;

accessing, by a computer, a table of sync events, wherein the sync events are used by the media presentation device to synchronize transmission of the descriptor values with presentation of the timed events by a media presentation device, the table ordered according to fields in the sync events, each sync event logically relating a timed event and a temporal interval that defines an occurrence of the timed event in the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values; and

discarding buffered portions of the media content in accordance with duration values specified for the timed events in the temporal intervals of the set of sync events.

44. The method of claim 43, wherein each sync event comprises a reference element and a location element, the reference element specifying a path within the content description to a descriptor value for a timed event, the location element specifying the temporal interval for the timed event.

45. The method of claim 43, wherein timed events in the media content are selected for display on the media presentation device as a result of accessing the set of sync events by the user.

46. A machine-readable storage medium having executable instructions which, when executed by a processor cause the processor to perform a method, the method comprising:

receiving media content at a media presentation device, the media content having a content description containing a plurality of descriptor values, each descriptor value describing a timed event in the media content;

buffering a portion of the media content for subsequent presentation;

accessing a table of sync events, wherein the sync events are used by the media presentation device to synchronize transmission of the descriptor values with presentation of the timed events by a presentation device, the table ordered according to fields in the sync events, each sync event logically relating a timed event and a temporal interval that defines an occurrence of the timed event in the media content, wherein the set of sync events is accessible at the media presentation device by a user to control the presentation of the timed events and the transmission of the descriptor values; and

discarding buffered portions of the media content in accordance with duration values specified for the timed events in the temporal intervals of the set of sync events.

47. The machine-readable storage medium of claim 46, wherein each sync event comprises a reference element and a location element, the reference element specifying the path within the content description to a descriptor value for a timed event, the location element specifying the temporal interval for the timed event.

48. The machine-readable storage medium of claim 46, wherein timed events in the media content are selected for display on the media presentation device as a result of accessing the set of sync events by the user.