Media stream rate synchronization

• US 10,462,202 B2
• Filed: 03/30/2016
• Issued: 10/29/2019
• Est. Priority Date: 03/30/2016
• Status: Active Grant

• Current Assignee: JBF Interlude 2009 Ltd.
• Original Assignee: JBF Interlude 2009 Ltd.
• Inventors: Bloch, Jonathan, Feldman, Barak, Zubalsky, Tal, Hofshy, Yuval, Lahav, Tomer
• Primary Examiner(s): Sall, El Hadji M
• Application Number:
  US15/085,209
• Publication Number:
  US 20170289220A1
• Time in Patent Office: 1,308 Days
• Field of Search: 709219, 709231, 709248, 709200
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Media streams or files, such as audio and video streams, within a media presentation are synchronized by dynamically adjusting the playback rate of one or more of the streams relative to a reference (clock) stream. In one implementation, a video presentation including a plurality of media streams is received, with each media stream having a respective playback rate. One of the media streams is selected as a clock stream. The media streams are simultaneously played according to their respective playback rates, and a second media stream is synchronized with the clock stream.

Claims

1. A computer-implemented method comprising:

receiving a video presentation comprising a plurality of media streams, each media stream having a respective playback rate;

selecting a first one of the media streams as a clock stream;

playing simultaneously the media streams according to their respective playback rates; and

synchronizing at least a second one of the media streams with the clock stream, the synchronizing comprising;

determining that an end of a first time interval has been reached;

determining, at the end of the first time interval, a current playback time of the clock stream and a current playback time of the second media stream; and

modifying the playback rate of the second media stream based on the current playback time of the clock stream and the current playback time of the second media stream, the modifying comprising calculating a new playback rate for the second media stream as a function of (1) a difference in current playback times between the clock stream and the second media stream and (2) a length of time of a next interval.

2. The method of claim 1, wherein the synchronizing is performed for each of a plurality of time intervals.

3. The method of claim 2, wherein the synchronizing comprises modifying the playback rate of the second media stream at an end of a first time interval to cause a playback time of the second media stream to be substantially the same as a playback time of the clock stream at an end of a second, later time interval.

4. The method of claim 2, wherein a length of each of the time intervals is fixed.

5. The method of claim 1, wherein the modifying comprises increasing the playback rate of the second media stream if the current playback time of the second media stream is less than the current playback time of the clock stream.

6. The method of claim 1, wherein the modifying comprises decreasing the playback rate of the second media stream if the current playback time of the second media stream is greater than the current playback time of the clock stream.

7. The method of claim 1, wherein calculating the new playback rate for the second media stream is by:

8. The method of claim 1, wherein the first media stream and the second media stream are selected from the group consisting of a video stream and an audio stream.

9. The method of claim 1, wherein the first media stream and the second media stream are video streams that play in parallel and can be seamlessly switched between.

10. A system comprising:

at least one memory storing computer-executable instructions; and

at least one processor for executing the instructions stored on the memory, wherein execution of the instructions programs the at least one processor to perform operations comprising;

receiving a video presentation comprising a plurality of media streams, each media stream having a respective playback rate;

selecting a first one of the media streams as a clock stream;

playing simultaneously the media streams according to their respective playback rates; and

synchronizing at least a second one of the media streams with the clock stream, the synchronizing comprising;

determining that an end of a first time interval has been reached;

determining, at the end of the first time interval, a current playback time of the clock stream and a current playback time of the second media stream; and

modifying the playback rate of the second media stream based on the current playback time of the clock stream and the current playback time of the second media stream, the modifying comprising calculating a new playback rate for the second media stream as a function of (1) a difference in current playback times between the clock stream and the second media stream and (2) a length of time of a next interval.

11. The system of claim 10, wherein the synchronizing is performed for each of a plurality of time intervals.

12. The system of claim 11, wherein the synchronizing comprises modifying the playback rate of the second media stream at an end of a first time interval to cause a playback time of the second media stream to be substantially the same as a playback time of the clock stream at an end of a second, later time interval.

13. The system of claim 11, wherein a length of each of the time intervals is fixed.

14. The system of claim 10, wherein the modifying comprises increasing the playback rate of the second media stream if the current playback time of the second media stream is less than the current playback time of the clock stream.

15. The system of claim 10, wherein the modifying comprises decreasing the playback rate of the second media stream if the current playback time of the second media stream is greater than the current playback time of the clock stream.

16. The system of claim 10, wherein calculating the new playback rate for the second media stream is by:

17. The system of claim 10, wherein the first media stream and the second media stream are selected from the group consisting of a video stream and an audio stream.

18. The system of claim 10, wherein the first media stream and the second media stream are video streams that play in parallel and can be seamlessly switched between.