Scheme for supporting real-time packetization and retransmission in rate-based streaming applications

• US 7,164,680 B2
• Filed: 06/04/2001
• Issued: 01/16/2007
• Est. Priority Date: 06/04/2001
• Status: Expired due to Fees

• Current Assignee: Pendragon Wireless LLC (Pendrell Corporation)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Loguinov, Dmitri
• Primary Examiner(s): Vu, Huy D.
• Assistant Examiner(s): NG, CHRISTINE Y
• Application Number:
  US09/873,566
• Publication Number:
  US 20020181506A1
• Time in Patent Office: 2,052 Days
• Field of Search: 370/389, 370/392, 370/394, 370/473, 370/474
• US Class Current: 370/394
• CPC Class Codes: H04L 1/1671 the supervisory signal being transmitted together with control information
• CPC Class Codes: H04L 1/1809 Selective-repeat protocols
• CPC Class Codes: H04L 1/1877 for semi-reliable protocols, e.g. for less sensitive applications like streaming video buffer level management for video bitstream control arrangements H04N21/44004
• CPC Class Codes: H04L 47/43 Assembling or disassembling of packets, e.g. segmentation and reassembly [SAR]
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Disclosed is a system and method for supporting real-time packetization of multimedia information and retransmission of lost multimedia packets over a packet network, and the method includes the steps of: packetizing multimedia bitstrems into a plurality of transmission packets according to a predetermined scheme; storing copies of the transmission packets for a predetermined time period; transmitting the transmission packets in sequence to the client; detecting lost packets; sending retransmission requests to the server upon detection of a lost packet; and, retransmitting copies of lost packets, which are stored in the server memory within a predetermined time period, to the client system.

Claims

1. A method for transmitting a multimedia bitstream in real-time between a server and a client over a packet network, the method comprising the steps of:

(a) packetizing said multimedia bitstreams into a plurality of packets comprised of layers according to a predetermined scheme;

(b) storing copies of the plurality of packets for a predetermined time period, which is updated by the client;

(c) transmitting said stored packets in sequence to said client; and

, (d) sending retransmission requests to said server, wherein said retransmission requests are sent upon the detection of a lost packet, wherein said predetermined time period is based on an equation of the form;

MH=[∀

i;

max{GOP_i+MD_i+ADB_i}]+RTT_last

where GOP_i is Group of Pictures (GOP) length of layer i, MD_i is the maximum decoding delay of layer i, ADB_i is an actual delay introduced by said client system for layer i, and, RTT_last is the latest estimate of the round-trip delay between said client system and said server system.

2. The method of claim 1, further comprising the step of retransmitting copies of lost packets to said client system.

3. The method of claim 1, further comprising the step of removing copies of the plurality of said packets if the lifetime of one of said packets is greater than said predetermined time period.

4. The method of claim 1, wherein said ADB_i is computed based on the following relationship:

ADB_i=TE−

TF_i−

IS_i

where TE is the time when all layers signal readiness, TF_i is the first packet of layer i received in said client system, and IS_i is the ideal startup delay of layer i.

5. A method for supporting the real-time transmission and retransmission of a multimedia bitstream between a server and a client, the method comprising the steps of:

(a) receiving said multimedia bitstream;

(b) transforming said multimedia bitstream into a plurality of packets having prefixes;

(c) adding said packet prefixes in sequence to a list controlled by said server for a predetermined time period, which is updated by the client;

(d) forwarding the plurality of said packets from said server to said client;

(e) searching said list for the prefix corresponding to a lost packet upon receiving a message to retransmit said lost packet from said client; and

, (f) transmitting said searched packet to said client, wherein said predetermined time period is based on an equation of the form;

MH=[∀

i;

max{GOP_i+MD_i+ADB_i}]+RTT_last

where GOP_i is Group of Pictures (GOP) length of layer i, MD_i is the maximum decoding delay of layer i, ADB_i is an actual delay introduced by said client system for layer i, and, RTT_last is the latest estimate of the round-trip delay between said client system and said server system.

6. The method of claim 5, further comprising the step of assembling the plurality of said packets into a continuous bitstream.

7. The method of claim 5, further comprising the step of determining whether one of said packets is lost based on said packet prefixes received by said client.

8. The method of claim 5, wherein said ADB_i is computed based on the following relationship:

ADB_i=TE−

TF_i−

IS_i

where TE is the time when all layers signal readiness, TF_i is the first packet of layer i received in said client system, and IS_i is the ideal startup delay of layer i.

9. The method of claim 5, wherein said searched packet is transmitted to said client by way of a Real-Time Transport Protocol.

10. A client system for receiving a multimedia file in real-time from a remote buffer from a server system, said client system and said server system being connected to a packet network, said client system comprising:

a packet buffer operably coupled to store incoming packets comprised of layers sent by said server system;

a depacketizer for assembling said incoming packets into a continuous bitstream;

a packet processor operably coupled to said depacketizer for detecting lost packets;

a retransmission manager operably coupled to said packet processor for sending retransmission requests to said server system upon detection of said lost packets, and means for computing a presentation time for copies of said incoming packets in said remote buffer and providing the presentation time to the server system, wherein said presentation time is based on the following relationship;

MH=[∀

i;

max{GOP_i+MD_i+ADB_i}]+RTT_last

where GOP_i is Group of Pictures (GOP) length of layer i, MD_i is the maximum decoding delay of layer i, ADB_i is actual delay introduced by said client system for layer i, and, RTT_last is the latest estimate of the round-trip delay between said client system and said server system.

11. The client system of claim 10 wherein said ADB_i is computed based on the following relationship:

ADB_i=TE−

TF_i−

IS_i

where TE is the time when all layers signal readiness, TF_i is the first packet of layer i received in said client system, and IS_i is the ideal startup delay of layer i.

12. A server system for transmitting a multimedia file stored in said server to a client system, said client system and said server system being connected to a packet network, said server system comprising:

a packetizer for packetizing incoming multimedia bitstrems into a plurality of packets comprised of layers according to a predetermined scheme;

a packet buffer operably coupled to said packetizer for storing copies of the plurality of packets for a predetermined time period, which is determined by said client system;

a packet transmitter operably coupled to the packet buffer for transmitting said stored packets in sequence to said client system; and

, a retransmission processor operably coupled to said packet buffer and said packet transmitter for retransmitting copies of lost packets to said client system, wherein said predetermined time period is based on the following relationship;

MH=[∀

i;

max{GOP_i+MD_i+ADB_i}]+RTT_last

where GOP_i is Group of Pictures (GOP) length of layer i, MD_i is the maximum decoding delay of layer i, ADB_i is actual delay introduced by said client system for layer i, and, RTT_last is the latest estimate of the round-trip delay between said client system and said server system.

13. The server system of claim 12, wherein said retransmission processor removes copies of the plurality of said packets if the lifetime of one of said packets is greater than said predetermined time period.

14. The server system of claim 12, wherein said ADB_i is computed based on the following relationship:

ADB_i=TE−

TF_i−

IS_i

where TE is the time when all layers signal readiness, TF_i is the first packet of layer i received in said client system, and IS_i is the ideal startup delay of layer i.

15. A system for streaming real-time multimedia information from a server to a client over a packet network, consisting of:

(a) said server for providing a multimedia bitstream by streaming the real-time multimedia information and performing retransmission, said server comprising;

a packetizer for converting said multimedia bitstream into a plurality of transmission packets comprised of layers;

a packet buffer for storing said transmission packets for a predetermined time period, which is determined by the client;

a packet transmitter for transmitting said transmission packets;

a retransmission processor for initiating retransmission of copies of said packets upon receipt of a retransmission request from said client;

(b) said client comprising;

a packet buffer for storing said packets of said multimedia stream received from said server;

a depacketizer for assembling said transmission packets into a continuous bitstream;

a packet processor operably coupled to said depacketizer for detecting lost packets; and

, a retransmission manager operably coupled to said packet processor for sending said retransmission request to said server system upon detection of said lost packets, wherein said predetermined time period is based on the following relationship;

MH=[∀

i;

max{GOP_i+MD_i+ADB_i}]+RTT_last

where GOP_i is Group of Pictures (GOP) length of layer i, MD_i is the maximum decoding delay of layer i, ADB_i is actual delay introduced by said client system for layer i, and, RTT_last is the latest estimate of the round-trip delay between said client system and said server system.

16. The system of claim 15, wherein said retransmission processor removes copies of the plurality of said packets if the lifetime of one of said transmission packets is greater than said predetermined time period.

17. The system of claim 15, wherein said ADB_i is computed based on the following relationship:

ADB_i=TE−

TF_i−

IS_i

where TE is the time when all layers signal readiness, TF_i is the first packet of layer i received in said client system, and IS_i is the ideal startup delay of layer i.