Coordination of video stream timing in cloud-based video streaming system

• US 9,686,576 B2
• Filed: 05/08/2015
• Issued: 06/20/2017
• Est. Priority Date: 05/08/2015
• Status: Active Grant

• Current Assignee: iStreamPlanet Co. LLC (Warner Bros. Discovery, Inc.)
• Original Assignee: iStreamPlanet
• Inventors: VanAntwerp, Mark Daniel
• Primary Examiner(s): Alata, Yassin
• Application Number:
  US14/708,139
• Publication Number:
  US 20160330497A1
• Time in Patent Office: 774 Days
• Field of Search: 725109
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/26208 the scheduling operation being performed under constraints
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite

Abstract

A method to coordinate video stream timing is disclosed. In one embodiment, a request is received at a streaming platform coordinator (SPC) to create a plurality of synchronized media workflows for video sources. Using the plurality of video sources as input, the SPC transmits a request from the SPC to at least one video streaming platform to create the plurality of media workflows, which are assigned to a plurality of workers. The SPC causes the plurality of videos to be routed to the plurality of workers upon receiving the request to route them. The SPC then determines a set of timing offsets to synchronize the plurality of media workflows, and makes the set of timing offsets available to the plurality of workers so that the workers publish synchronized media outputs. A system to perform the method and a storage medium storing instructions to perform the method are disclosed too.

Claims

1. A method executed by an electronic device implementing a streaming platform coordinator (SPC) to coordinate processing of a plurality of video sources, originated from a plurality of video output devices to broadcast an event concurrently, the method comprising:

receiving a request to create a plurality of media workflows, where each of the plurality of video sources correspond to a separate one of the plurality of media workflows, wherein the plurality of media workflows are to be synchronized;

responsive to receiving the request at the SPC to create the plurality of media workflows, transmitting a request from the SPC to at least one video streaming platform to create the plurality of media workflows using the plurality of video sources as input, wherein the plurality of media workflows are assigned to a plurality of workers;

receiving requests from the at least one video streaming platform to route the plurality of video sources to the plurality of workers;

causing the plurality of video sources to be routed to the plurality of workers;

determining a set of timing offsets to synchronize the plurality of media workflows, where each one of the plurality of media workflows has a separate timing offset in the set of timing offsets; and

making the set of timing offsets available to the plurality of workers, wherein the plurality of workers apply the set of timing offsets so that output of the plurality of media workflows are synchronized, where the synchronized output has a matched timing correlated to a time of recording by each of the plurality of video sources.

2. The method of claim 1, wherein the request to create the plurality of media workflows using the plurality of video sources as input is to transmitted to at least two video streaming platforms that are located separately.

3. The method of claim 1, wherein determining the set of timing offsets to synchronize the plurality of media workflows comprising:

receiving packets of the plurality of video sources;

extracting explicit digital timing information embedded in the packets; and

calculating a timing offset for each media workflow based on the explicit digital timing information.

4. The method of claim 1, wherein determining the set of timing offsets to

synchronize the plurality of media workflows comprises:

receiving packets of the plurality of video sources;

identifying a sequence of audiovisual test patterns in each of the plurality of video sources, wherein the sequence of test patterns is sent simultaneously in all of the plurality of video sources; and

calculating a timing offset for each media workflow based on when the sequence of test patterns is found in each of the plurality of video sources.

5. The method of claim 1, wherein determining the set of timing offsets to synchronize the plurality of media workflows comprises:

making current timing offset information available to an external override system;

receiving modified timing offset information for one or more media workflows back from the external override system; and

retaining the modified timing offset information for any impacted media workflow.

6. The method of claim 1, wherein making the set of timing offsets available to the plurality of workers comprises:

receiving a timing offset request from each of the plurality of workers; and

transmitting a response containing one timing offset for one requesting worker for each timing offset request.

7. The method of claim 1, wherein each of the plurality of video output devices is a video camera.

8. The method of claim 7, wherein each of the plurality of video sources is one view point of the event.

9. An electronic device to serve as a streaming platform coordinator (SPC) to coordinate processing of a plurality of video sources, originated from a plurality of video output devices to broadcast an event concurrently, the electronic device comprising:

a processor and a non-transitory machine-readable storage medium coupled to the processor, the non-transitory machine-readable storage medium containing operations executable by the processor, wherein the electronic device is operative to;

receive a request to create a plurality of media workflows, where each of for the plurality of video sources correspond to a separate one of the plurality of media workflows, wherein the plurality of media workflows are to be synchronized;

responsive to receiving the request at the SPC to create the plurality of media workflows, transmit a request from the SPC to at least one video streaming platform to create the plurality of media workflows using the plurality of video sources as input, wherein the plurality of media workflows are assigned to a plurality of workers;

receive requests from the at least one video streaming platform to route the plurality of video sources to the plurality of workers;

cause the plurality of video sources to be routed to the plurality of workers;

determine a set of timing offsets to synchronize the plurality of media workflows, where each one of the plurality of media workflows has a separate timing offset in the set of timing offsets; and

make the set of timing offsets available to the plurality of workers, wherein the plurality of workers apply the set of timing offsets so that output of the plurality of media workflows are synchronized, where the synchronized output has a matched timing correlated to a time of recording by each of the plurality of video sources.

10. The electronic device of claim 9, wherein the determination of the set of timing offsets to synchronize the plurality of media workflows is for the electronic device to further operative to:

receive packets of the plurality of video sources;

extract explicit digital timing information embedded in the packets; and

calculate a timing offset for each media workflow based on the explicit digital timing information.

11. The electronic device of claim 9, wherein the determination of the set of timing offsets to synchronize the plurality of media workflows is for the electronic device to further operative to:

receive packets of the plurality of video sources;

identify a sequence of audiovisual test patterns in each of the plurality of video sources, wherein the sequence of test patterns is sent simultaneously in all of the plurality of video sources; and

calculate a timing offset for each media workflow based on when the sequence of test patterns is found in each of the plurality of video sources.

12. The electronic device of claim 9, wherein the determination of the set of timing offsets to synchronize the plurality of media workflows is for the electronic device to further operative to:

make current timing offset information available to an external override system;

receive modified timing offset information for one or more media workflows back from the external override system; and

retain the modified timing offset information for any impacted media workflow.

13. The electronic device of claim 9, wherein to make the set of timing offsets available to the plurality of workers include the electronic device to further operative to:

receive a timing offset request from each of the plurality of workers; and

transmit a response containing one timing offset for one requesting worker for each timing offset request.

14. The electronic device of claim 9, wherein each of the plurality of video output devices is a video camera.

15. A non-transitory machine-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations in an electronic device implementing a streaming platform coordinator (SPC) to coordinate processing of a plurality of video sources, originated from a plurality of video output devices to broadcast an event concurrently, the operations comprising:

receiving a request to create a plurality of media workflows, where each of for the plurality of video sources correspond to a separate one of the plurality of media workflows, wherein the plurality of media workflows are to be synchronized;

responsive to receiving the request at the SPC to create the plurality of media workflows, transmitting a request from the SPC to at least one video streaming platform to create the plurality of media workflows using the plurality of video sources as input, wherein the plurality of media workflows are assigned to a plurality of workers;

receiving requests from the at least one video streaming platform to route the plurality of video sources to the plurality of workers;

causing the plurality of video sources to be routed to the plurality of workers;

determining a set of timing offsets to synchronize the plurality of media workflows, where each one of the plurality of media workflows has a separate timing offset in the set of timing offsets; and

making the set of timing offsets available to the plurality of workers, wherein the plurality of workers apply the set of timing offsets so that output of the plurality of media workflows are synchronized, where the synchronized output has a matched timing correlated to a time of recording by each of the plurality of video sources.

16. The storage medium of claim 15, wherein the request to create the plurality of media workflows using the plurality of video sources as input is to transmitted to at least two video streaming platforms that are located separately.

17. The storage medium of claim 15, wherein determining the set of timing offsets to synchronize the plurality of media workflows comprising:

receiving packets of the plurality of video sources;

extracting explicit digital timing information embedded in the packets; and

calculating a timing offset for each media workflow based on the explicit digital timing information.

18. The storage medium of claim 15, wherein determining the set of timing offsets to synchronize the plurality of media workflows comprises:

receiving packets of the plurality of video sources;

identifying a sequence of audiovisual test patterns in each of the plurality of video sources, wherein the sequence of test patterns is sent simultaneously in all of the plurality of video sources; and

calculating a timing offset for each media workflow based on when the sequence of test patterns is found in each of the plurality of video sources.

19. The storage medium of claim 15, wherein determining the set of timing offsets to synchronize the plurality of media workflows comprises:

making current timing offset information available to an external override system;

receiving modified timing offset information for one or more media workflows back from the external override system; and

retaining the modified timing offset information for any impacted media workflow.

20. The storage medium of claim 15, wherein making the set of timing offsets available to the plurality of workers comprises:

receiving a timing offset request from each of the plurality of workers; and

transmitting a response containing one timing offset for one requesting worker for each timing offset request.