Method and device for stream synchronization of real-time multimedia transport over packet network

• US 20070081562A1
• Filed: 10/11/2006
• Published: 04/12/2007
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Ma, Hui
• Granted Patent: US 7,680,153 B2
• Time in Patent Office: Days
• US Class Current: 370/516
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6332 directed to client
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/64753 directed to the client
• CPC Class Codes: H04N 21/654 Transmission by server directed to the client

Abstract

A method for stream synchronization of real-time multimedia transport over a packet network, in which the multimedia includes a first stream and a second stream which are buffered and played through buffers. The method includes: measuring average transport delays of the first and second streams; measuring transport delay jitters of the first and second streams; and calculating a delay difference between the first and second streams which corresponds to the average transport delays and the transport delay jitters of the first and second streams, and setting buffer durations. A device for stream synchronization of real-time multimedia transport over a packet network is also provided.

Claims

1. A method for stream synchronization of real-time multimedia transport over a packet network, the multimedia comprising a first stream and a second stream which are buffered and played through respective buffers at receivers, comprising:

measuring average transport delays of the first and second streams respectively;

measuring transport delay jitters of the first and second streams respectively; and

calculating a delay difference between the first and second streams which corresponds to the average transport delays and the transport delay jitters of the first and second streams, and setting buffer durations in the first and second stream buffers so as to be adapted to the delay difference.

2. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 1, further comprising the steps of:

one of the first and second stream buffers periodically providing its play instant as a synchronization reference to the other buffer; and

the other buffer adjusting its play in accordance with the synchronization reference point.

3. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 2, wherein the average transport delays are calculated from transmitting and receiving instants of data packets of the first and second streams.

4. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 3, wherein the calculation of the average transport delays comprises:

calculating transport delay of a current multimedia data packet; and

modifying iteratively and enabling the average transport delay to approach the transport delay of the current multimedia data packet.

5. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 2, wherein the step of setting buffer durations in the first and second stream buffers so as to be adapted to the delay difference comprises:

if the average transport delays of the first and second streams have a difference within a tolerable synchronization offset range and are both smaller than maximum buffer durations in the corresponding buffers, setting the buffer durations in the buffers to the transport delays;

if the difference between the average transport delays of the first and second streams goes beyond the tolerable synchronization offset range, defining either of the average transport delays to enable the delay difference be within the synchronization offset range; and

if the average transport delays of the first and second streams go beyond the maximum buffer durations, defining the buffer durations in the buffers as the maximum buffer duration.

6. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 2, wherein the first stream is an audio stream, and the second stream is a video stream.

11. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 1, wherein the average transport delays are calculated from transmitting and receiving instants of data packets of the first and second streams.

12. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 11, wherein the calculation of the average transport delays comprises:

calculating transport delay of a current multimedia data packet; and

modifying iteratively and enabling the average transport delay to approach the transport delay of the current multimedia data packet.

13. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 1, wherein the step of setting buffer durations in the first and second stream buffers so as to be adapted to the delay difference comprises:

if the average transport delays of the first and second streams have a difference within a tolerable synchronization offset range and are both smaller than maximum buffer durations in the corresponding buffers, setting the buffer durations in the buffers to the transport delays;

if the difference between the average transport delays of the first and second streams goes beyond the tolerable synchronization offset range, defining either of the average transport delays to enable the delay difference be within the synchronization offset range; and

if the average transport delays of the first and second streams go beyond the maximum buffer durations, defining the buffer durations in the buffers as the maximum buffer duration.

14. The method for stream synchronization of real-time multimedia transport over a packet network according to claim 1, wherein the first stream is an audio stream, and the second stream is a video stream.

7. A device for stream synchronization of real-time multimedia transport over a packet network, the multimedia comprising a first stream and a second stream, and the device comprising a first stream buffer, a second stream buffer, a delay detection module and a synchronization module, wherein:

the first and second stream buffers are adapted to calculate transport delay jitters of the first and second streams respectively;

the delay detection module is adapted to calculate average transport delays of the first and second streams; and

the synchronization module is adapted to calculate buffer durations in the first and second stream buffers in accordance with the average transport delays and the transport delay jitters of the first and second streams.

8. The device for stream synchronization of real-time multimedia transport over a packet network according to claim 7, wherein one of the first and second stream buffers periodically provides its play instant as a synchronization reference point to the other buffer, and the other buffer adjusts its play in accordance with the synchronization reference point.

9. The device for stream synchronization of real-time multimedia transport over a packet network according to claim 8, wherein the delay detection module calculates the average transport delays in accordance with transmitting and receiving instants of data packets of the first and second streams.

10. The device for stream synchronization of real-time multimedia transport over a packet network according to claim 9, wherein the delay detection module calculates the average transport delays with an iterative algorithm.

15. The device for stream synchronization of real-time multimedia transport over a packet network according to claim 7, wherein the delay detection module calculates the average transport delays in accordance with transmitting and receiving instants of data packets of the first and second streams.

16. The device for stream synchronization of real-time multimedia transport over a packet network according to claim 15, wherein the delay detection module calculates the average transport delays with an iterative algorithm.