Marker-Based Inter-Destination Media Synchronization

• US 20150052571A1
• Filed: 03/29/2013
• Published: 02/19/2015
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Koninklijke KPN NV, Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Original Assignee: Koninklijke KPN NV, Nederlandese Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Inventors: Stokking, Hans Maarten, Van Brandenburg, Ray, Van Deventer, Mattijs Oskar
• Granted Patent: US 9,832,497 B2
• Time in Patent Office: Days
• US Class Current: 725/116
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/637 Control signals issued by the client directed to the server or network components
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A method and a system enabling inter-destination synchronization of processing of content of at least a first and second media stream, wherein said first and second media stream are associated with one or more markers for marking one or more content parts of said first and second media streams respectively. The method comprises the steps of: detecting a first marked content part in said first media stream and a second marked content part in said second media stream, said first and second market content part being associated with a first and second marker respectively; associating first timing information, preferably comprising a first clock time, with said detected first marked content part and associating second timing information, preferably comprising a second clock time, with said detected second marked content part; determining a timing difference on the basis of said first and second timing information; and, generating on the basis of said timing difference synchronization information for enabling a first and/or second synchronization client to adjust the processing of said first and/or second media stream respectively.

Claims

1. A method for enabling inter-destination synchronization of processing of content of at least a first media stream and a second media stream, said first media stream and said second media stream being associated with one or more markers for marking one or more content parts of said first media stream and said second media stream respectively, said method comprising:

detecting a first marked content part of said first media stream and a second marked content part of said second media stream, said first and second marked content part being associated with a first and second marker respectively;

associating first timing information, comprising a first clock time, with said detected first marked content part and second timing information, comprising a second clock time, with said detected second marked content part;

determining a timing difference on the basis of said first and second timing information; and

generating, on the basis of said timing difference, synchronization information, said synchronization information enabling at least one of a first device or a second device, comprising a first synchronization client or a second synchronization client respectively, to adjust the processing of content of at least one of said first media stream or said second media stream respectively.

2. The method according to claim 1 wherein said one or more markers comprise a marker identifier for identifying markers associated with marked content parts of said first and second media stream, said method further comprising:

comparing a first marker identifier associated with said first marked content part with a second marker identifier associated with said second marked content part.

3. The method according to claim 2, further comprising:

if said first marker identifier is identical to said second marker identifier, determining said timing difference on the basis of said first and second timing information; and

if said first marker identifier is not identical to said second marker identifier, determining said timing difference on the basis of said first and second timing information and on the basis of a temporal relation between said first marker identified by said first marker identifier and said second marker identified by said second marker identifier.

4. The method according to claim 1 further comprising:

providing said first timing information from a first receiver to a synchronization function, said first receiver being configured for receiving the first media stream and including a first synchronization client; and

providing said second timing information from a second receiver to said synchronization function, said second receiver being configured for receiving said second media stream and including a second synchronization client, said synchronization function being implemented in one of;

a synchronization server, a content source adapted for transmitting said first and said second media stream to said first and second synchronization client, or a receiver associated with at least one of said first synchronization client or said second synchronization client.

5. The method according to claim 1, wherein said one or more markers comprise a marker identifier for identifying markers associated with marked content parts of said first and second media stream, and a first marker identifier is associated with said first marker content part, the method further comprising:

said first synchronization client transmitting said first timing information to a synchronization function;

transmitting a request for said second timing information to said second synchronization client, said request comprising said first marker identifier; and

in response to said request, said second synchronization client transmitting said second timing information to said synchronization function.

6. The method according to claim 1 comprising:

receiving said first media stream at a first receiver comprising a first synchronization client;

transmitting a first marker notification associated with said first marked content part to a device comprising a synchronization function, said first marker notification comprising a first marker identifier; and

generating, via said synchronization function, said first timing information upon receipt of said first marker notification.

7. The method according to claim 1, wherein said one or more markers are transmitted in one or more signaling messages associated with at least one of said first media stream or said second media stream.

8. The method according to claim 7, wherein said one or more signaling messages comprises at least one of a RTP timestamp, a NTP timestamp, or a PTS type timestamp.

9. The method according to claim 1, wherein said first marked content part and said and/or second marked content part are a first segment and a second segment of a first segmented media stream and a second segmented media stream respectively, each of said first and second segments being identified in a manifest file data structure by a segment identifier.

10. The method according to claim 9 wherein said manifest file data structure further comprises location information, associated with said segment identifiers for locating one or more media servers configured to transmit segments identified by said segment identifiers, and, marker information comprising at least one marker, said at least one marker being associated with said segment identifier comprised in said manifest file data structure.

15. A computer program product, implemented on computer-readable non-transitory storage medium, the computer program product configured for, when run on a computer, executing the method steps according to claim 1.

16. The method according to claim 1 wherein said first and second marker are the same or relate to the same position on a content time-line common to the first media stream and the second media stream.

11. A receiver, comprising a synchronization client configured to receive at least a media stream associated with at least one marker for marking a content part of said media stream, said synchronization client being configured for:

detecting at least one marked content part of said media stream;

providing a synchronization function with timing information associated with said detected first marker; and

receiving synchronization information from said synchronization function, said synchronization information configured for instructing said synchronization client to adjust the processing of said media stream,

wherein said synchronization function is configured to;

receive or provide first timing information associated with a first receiver, said first receiver comprising a first synchronization client, said first timing information being associated with a first marked content part of a first media stream being received by said first receiver;

receive or provide second timing information associated with a second receiver, said second receiver comprising a second synchronization client, said second timing information being associated with a second marked content part of a second media stream being received by said second receiver;

determine a timing difference based on said first and second timing information; and

generate, on the basis of said timing difference, said synchronization information for instructing at least one of said first synchronization client or said second synchronization client to adjust the processing of said first media stream or said second media stream respectively.

17. The receiver according to claim 11, wherein said timing information comprises at least one of a first clock time or a marker identifier.

18. The receiver according to claim 11, further comprising a synchronization function configured to:

receive or provide first timing information associated with a first receiver, said first receiver comprising a first synchronization client, said first timing information being associated with a first marked content part of a first media stream being received by said first receiver;

receive or provide second timing information associated with a second receiver, said second receiver comprising a second synchronization client, said second timing information being associated with a second marked content part of a second media stream being received by said second receiver;

determine a timing difference based on said first and second timing information; and

generate, on the basis of said timing difference, said synchronization information for instructing at least one of said first synchronization client or said second synchronization client to adjust the processing of said first media stream or said second media stream respectively.

12. A device comprising a synchronization function, said synchronization function being configured for:

receiving or providing first timing information associated with a first receiver, said first receiver comprising a first synchronization client, said first timing information being associated with a first marked content part of a first media stream received by said first receiver;

receiving or providing second timing information associated with a second receiver, said second receiver comprising a second synchronization client, said second timing information being associated with a second marked content part of a second media stream received by said second receiver;

determining a timing difference based on said first and second timing information; and

,

generating on the basis of said timing difference synchronization information for instructing at least one of said first synchronization client or said second synchronization client to adjust the processing of said first media stream or said second media stream respectively.

13. A system for enabling inter-destination synchronization of processing of content of at least a first media stream and a second media stream, said system comprising:

a first receiver, comprising a first synchronization client configured for;

detecting a first marked content part of a first media stream,

associating said first marked content part with first timing information, and

transmitting at least one of said first timing information to a device comprising a synchronization function or transmitting a first marker notification to said synchronization function;

a second receiver, comprising a second synchronization client configured for;

detecting a second marked content part of a second media stream,

associating said second marked content with second timing information, and

at least one of transmitting said second timing information to said device comprising said synchronization function or transmitting a second marker notification to said synchronization function; and

said device comprising a synchronization function configured for;

receiving said first and/or second timing information and/or deriving said first and/or second timing information from said received first and/or second marker notification;

determining a timing difference based on said first timing information and second timing information; and

generating on the basis of said timing difference synchronization information for said first and/or second synchronization client to adjust the processing of said first and/or second media stream respectively.

14. A computer readable medium encoded with a manifest file data structure said data structure comprising:

one or more segment identifiers and location information associated with said one or more segment identifiers for locating one or more media servers configured to deliver one or more segments associated with said one or more segment identifiers; and

,

marker information, comprising at least one or more markers for marking said one or more segment identifiers in said data structure.

19. The computer readable medium according to claim 14, wherein said manifest file data structure further comprises a timing parameter indicating a time at which timing information should be transmitted to a device comprising a synchronization function, the synchronization function being configured to:

receive or provide first timing information associated with a first receiver, said first receiver comprising a first synchronization client, said first timing information being associated with a first marked content part of a first media stream being received by said first receiver;

receive or provide second timing information associated with a second receiver, said second receiver comprising a second synchronization client, said second timing information being associated with a second marked content part of a second media stream being received by said second receiver;

determine a timing difference based on said first and second timing information; and

generate, on the basis of said timing difference, said synchronization information for instructing at least one of said first synchronization client or said second synchronization client to adjust the processing of said first media stream or said second media stream respectively.

20. The computer readable medium according to claim 19, wherein said timing information comprises one or more marker identifiers.