Webcasting method and system for time-based synchronization of multiple, independent media streams

• US 7,346,698 B2
• Filed: 12/20/2000
• Issued: 03/18/2008
• Est. Priority Date: 12/20/2000
• Status: Expired due to Fees

• Current Assignee: G. W. Hannaway & Associates
• Original Assignee: G. W. Hannaway & Associates
• Inventors: Hannaway, G. Wyndham
• Primary Examiner(s): Jaroenchonwanit; Bunjob
• Assistant Examiner(s): DOAN, DUYEN MY
• Application Number:
  US09/742,165
• Publication Number:
  US 20020103919A1
• Time in Patent Office: 2,645 Days
• Field of Search: 709/231, 348/385, 348/14.12, 348/14.07, 370/465, 370230-235, 725 87- 96
• US Class Current: 709/231
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2389 Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4385 Multiplex stream processing, e.g. multiplex stream decrypting
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/64761 directed to the server
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10

Abstract

A method for time-based synchronization of multiple media streams transmitted over a communications network, such as the Internet, by multiple, independent streaming media sources. First and second media streams of data packets are received from first and second media sources. Timing data is parsed from the two media streams, and first and second transmission delay values are determined by comparing the timing data with a reference time. A synchronized media stream is created by combining the first and second media streams into a time-synchronized media stream with adjustments to correct for calculated transmission delay values. Feedback signals are sent to the media sources to control transmission variables such as stream length, transmission rate, and transmittal time to manage the variable delay at the media source. The first and second media streams are decoded into intermediate media streams compatibly formatted to allow mixing of the streams and data packets.

Claims

1. A synchronization system for time-based synchronization of streaming media transmitted over a communications network, comprising:

an input interface adapted for linking to the communications network to receive a first and a second media stream, wherein the first and second media streams comprise a plurality of digital data packets being transmitted over the communications network from a first and a second media source, respectively, and wherein the first and the second media streams each include a streaming video portion;

a first data buffer for storing the data packets of the first media stream;

a second data buffer for storing the data packets of the second media stream; and

a controller communicatively linked to the first and the second data buffers for selectively retrieving the data packets of the first and second media streams to form a first and a second time-adjusted stream, wherein the controller determines a variable transmission delay for the first and the second media streams from the first and second media sources to the input interface and performs the selective retrieving based on the determined variable transmission delays;

wherein the controller is further configured for mixing the first and second time-adjusted streams into a composite media stream wherein the first and second time-adjusted streams are synchronized based on time.

2. The system of claim 1, wherein the streaming video portion of the first media stream is compressed based on a first compression format and the second media stream is compressed based on a second compression format, the second compression format differing from the first compression format.

3. The system of claim 2, further including a decoding device between the input interface and the first and second data buffers for processing compressed first and second media streams into a first decoded stream and a second decoded stream, respectively, for storage in the first data buffer and the second data buffer, wherein the first decoded stream and the second decoded stream have compatible formatting.

4. The system of claim 1, wherein the controller forms the composite media stream by combining the first and the second time-adjusted streams such that the second time-adjusted stream has a first data packet positioned at a start time adjacent a last data packet of the first time-adjusted stream positioned at an end time.

5. The system of claim 4, wherein the controller is communicatively-linked to an external timing reference for receiving a reference time value, and wherein the controller is adapted for using the reference time value to determine the start time and the end time.

6. The system of claim 4, wherein the controller determines a length of the first media stream, compares the length with the end time and the variable network delay, computes an edit length for the first media stream, and compresses or lengthens the first media stream to form the first time-adjusted stream, whereby the last data packet coincides with the end time.

7. The system of claim 1, further including a data parsing device in communication with the input interface configured for retrieving time data from the first and the second media streams and for transmitting the time data to the controller, wherein the controller uses the time data to determine variable transmission delays.

8. The system of claim 6, wherein the controller is adapted to create media server control signals based on the determined variable transmission delays and to transmit the signals over the communications network to the first and the second media sources to control transmission variables of the first and second media streams.

9. The system of claim 8, wherein the transmission variables are selected from a group consisting of transmission timing, transmission rate, and transmission length.

10. The system of claim 1, wherein the composite media stream comprises a streaming video portion having picture-in-picture or side by side portions formed with the data packets of the first and the second time-adjusted streams.

11. The system of claim 1, wherein the controller combines the first media stream and second media stream in the composite media stream such that a data packet transmitted in the first media stream from the first media source at a transmission time is matched with a corresponding data packet in the second media stream transmitted from the second media source at the transmission time.

12. The system of claim 11, wherein the combining is performed by the controller by selecting a transmission rate for the first and the second media streams to correct for the determined variable transmission delays.

13. The system of claim 1, further including an output interface for transmitting the composite media stream from the controller over the communications network and including an end-user node linked to the communications network for receiving the composite media stream wherein the end-user node comprises a synchronizer for determining a variable transmission delay between the controller and the end-user node and for performing time-based correction of the composite media stream to adjust for the variable transmission delay.

14. An apparatus for synchronizing media streams transmitted over a communication network, comprising:

an input interface linked to the communications network and configured for receiving a first and a second media stream transmitted by a first and a second media source, respectively, wherein the first media stream comprises a plurality of data packets of a video stream encoded to a first compression standard and the second media stream comprises a plurality of data packets of a video stream encoded to a second compression standard differing from the first compression standard;

a decoder for decoding the first and the second media streams into a first and a second intermediate media stream, respectively, wherein the first and second intermediate streams are compatibly formatted;

a streaming media processor for mixing the first and the second intermediate-format media streams into a composite media stream encoded according to an output compression standard; and

a controller in communication with the input interface and the streaming media processor adapted for determining a variable transmission delay for the first and the second media streams based on a transmission time for a data packet of the first media stream and a time of receipt at the input interface of the data packet and on a transmission time for a data packet of the second media stream and a time of receipt at the input interface of the data packet;

wherein the controller is further configuring for selectively retrieving the first intermediate-format media stream and the second intermediate-format media stream based on the variable transmission delay of the first and the second media stream to create a first and a second time-adjusted stream;

wherein the processor combines the first and the second time-adjusted stream to form the composite media stream with the first media stream data packets and the second stream data packets being positioned for concurrent delivery.

15. The apparatus of claim 14, wherein the time of receipt is determined based on a time reference signal received from an external timing reference.

16. A method for time-based synchronization of two or more media streams transmitted over a data communications network, comprising:

receiving a first media stream wherein the first media stream comprising a plurality of data packets from one or more video files transmitted over the communications network by a first media source;

receiving a second media stream wherein the second media stream comprising a plurality of data packets from one or more video files transmitted over the communications network by a second media source;

retrieving timing data from the first and second media stream;

comparing the timing data with a reference time to determine a first and a second transmission delay value;

adjusting the first and the second media streams to correct for the first and the second transmission delay values, wherein the adjusting includes matching the data packets of the first and the second media streams based on transmittal times from the first and the second media sources;

storing the data packets of the first media stream in a first data buffer and the data packets of the second media stream in a second data buffer and wherein the adjusting includes selectively retrieving the data packets of the first media stream from the first data buffer to correct for the first transmission delay value and selectively retrieving the data packets of the second media stream from the second butter to correct for the second transmission delay value

creating a synchronized media stream by mixing the first and the second media streams, wherein the first and the second media streams are presented in the synchronized media stream concurrently.

17. The method of claim 16, wherein the adjusting includes creating a first and a second control signal in response to the first and the second transmission delay values, respectively, and includes transmitting the first and the second control signals to the first and the second media source to control transmittal of the first and the second media streams.

18. The method of claim 16, wherein the first media stream is encoded to a first compression standard and the second media stream is encoded to a second compression standard, and further including forming a first intermediate data stream by decoding the first media stream and forming a second intermediate data stream by decoding the second media stream, wherein the first and second intermediate data streams are compatibly formatted.

19. The method of claim 18, wherein the first compression standard differs from the second compression standard such that the first and second media stream are incompatible prior to the decoding to be combined into a single media stream.

20. The method of claim 16, wherein the synchronized media stream concurrently provides two screens corresponding to each of the first and second media streams.

21. The method of claim 20, wherein the two screens are arranged as split screens.

22. The method of claim 20, wherein the two screens are arranged as picture-in-picture with a first of the two screens provided within a second of the two screens.

23. The method of claim 16, wherein the first media stream comprises a first live webcast and the second media stream comprises a second live webcast.