SYSTEM AND METHOD FOR LOW-DELAY, INTERACTIVE COMMUNICATION USING MULTIPLE TCP CONNECTIONS AND SCALABLE CODING

• US 20080130658A1
• Filed: 12/10/2007
• Published: 06/05/2008
• Est. Priority Date: 07/20/2005
• Status: Active Grant

• Current Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Original Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Inventors: Eleftheriadis, Alexandros, Sasson, Roi, Shapiro, Ofer, Chakareski, Jacob
• Granted Patent: US 7,933,294 B2
• Time in Patent Office: Days
• US Class Current: 370/395.42
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 47/43 Assembling or disassembling of packets, e.g. segmentation and reassembly [SAR]
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Systems and methods for communication of scaleable-coded audiovisual signals over multiple TCP/IP connections are provided. The sender schedules and prioritizes transmission of individual scalable-coded data packets over the plurality of TCP connections according to their relative importance in the scalable coding structure for signal reconstruction quality and according to receiver feedback. Low-latency packet delivery over the multiple TCP/IP connections is maintained by avoiding transmission or retransmission of packets that are less important for reconstructed media quality.

Claims

1. An audiovisual communication system including a communication network that uses the IP protocol and has a plurality of TCP connections to carry audiovisual signal data packets between a sender and a receiver, wherein audiovisual signals are scalable-coded by the sender into a set of scalable layers according to a scalable coding structure for transmission over the plurality of TCP connections, and wherein portions of the scalable layers data are designated as key frames, the system comprising:

a Scheduling Inverse Multiplexer (S-IMUX) coupled to the plurality of TCP connections at the sender, wherein the S-IMUX is configured to schedule transmission of individual scalable-coded data packets over the plurality of TCP connections;

a Feedback Multiplexer (F-MUX) coupled to the plurality of TCP connections at the receiver, wherein the F-MUX is configured to reassemble individual scalable-coded data packets received over different TCP connections in a single packet stream for decoding.

2. The audiovisual communication system of claim 1, wherein the S-IMUX is configured to prioritize transmission of an individual scalable-coded data packet according to its relative importance in the scalable coding structure for reconstruction of the audiovisual signal at the receiver.

3. The audiovisual communication system of claim 1, wherein the receiver is configured to provide feedback or acknowledgement of the receipt of the scalable-coded data packets on individual TCP connections, and wherein the S-IMUX is further configured to prioritize transmission of an individual scalable-coded data packet according to its relative importance in the scalable coding structure for reconstruction of the audiovisual signal at the receiver and according to the receiver feedback.

4. The audiovisual communication system of claim 1, wherein the S-IMUX is configured to transmit a non-key frame only once over a TCP connection even if receipt of the non-key frame by the receiver is unacknowledged within a pre-determined time period ‘

T’

, and to discard all non-key frames following the unacknowledged non-key frame until the next available key frame in the audiovisual signal.

5. The audiovisual communication system of claim 1, wherein the S-IMUX is configured to perform the steps of:

(a) transmitting a key frame over a particular open TCP connection; and

(b) if receipt of the key frame over the particular open TCP connection is not acknowledged within a pre-determined time period ‘

T’

, transmitting the unacknowledged key frame over another open TCP connection,

wherein an open TCP connection is a TCP connection that is currently not waiting for an acknowledgement of the receipt of a previous transmission on the TCP connection.

6. The audiovisual communication system of claim 5, wherein the S-IMUX is configured to repeat step (b) until either receipt of the key frame is acknowledged on a TCP connection or a pre-determined time period ‘

T2’

expires.

7. The audiovisual communication system of claim 6, wherein the S-IMUX is configured to discard, after receipt of a repeat transmission of the key frame is acknowledged on any of the TCP connections used, all following non-key frames until the next available key frame in the audiovisual signal, and to transmit the next available key frame on an open TCP connection.

8. The audiovisual communication system of claim 1, wherein the S-IMUX is configured to discard all non-key frames until the first available key frame in the audiovisual signal, and when receipt of a transmitted non-key frame is not acknowledged by the receiver to discard all following non-key frames until the next available key frame in the audiovisual signal.

9. The audiovisual communication system of claim 1, wherein the F-MUX is configured to reassemble individual scalable-coded data packets received over different TCP connections in a single packet stream in proper order according to the scaleable coding structure for decoding and for removing duplicate received packets.

10. A method for audiovisual communication over a communication network that uses the IP protocol and has a plurality of TCP connections to carry audiovisual signal data packets between a sender and a receiver, wherein audiovisual signals are scalable-coded by the sender into a set of scalable layers according to a scalable coding structure for transmission over the plurality of TCP connections, and wherein portions of the scalable layers data are designated as key frames, the method comprising:

at the sender, scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections;

at the receiver, reassembling individual scalable-coded data packets received over different TCP connections in a single packet stream for decoding.

11. The method of claim 10, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises the prioritizing transmission of an individual scalable-coded data packet according to its relative importance in the scalable coding structure for reconstruction of the audiovisual signal at the receiver.

12. The method of claim 10, wherein the receiver is configured to provide feedback or acknowledgement of the receipt of the scalable-coded data packets on individual TCP connections, and wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises prioritizing transmission of an individual scalable-coded data packet according to its relative importance in the scalable coding structure for reconstruction of the audiovisual signal at the receiver and according to the receiver feedback.

13. The method of claim 10, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises:

transmitting a non-key frame only once over a TCP connection even if receipt of the non-key frame by the receiver is unacknowledged within a pre-determined time period ‘

T’

; and

,

discarding all non-key frames following the unacknowledged non-key frame until the next available key frame in the audiovisual signal.

14. The method of claim 10, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises:

(a) transmitting a key frame over a particular open TCP connection; and

(b) if receipt of the key frame over the particular open TCP connection is not acknowledged within a pre-determined time period ‘

T’

, transmitting the unacknowledged key frame over another open TCP connection,

wherein an open TCP connection that is a TCP connection that is currently not waiting for an acknowledgement of the receipt of a previous transmission on the TCP connection.

15. The method of claim 14, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises repeating step (b) until either receipt of the key frame is acknowledged on a TCP connection or a pre-determined time period ‘

T2’

expires.

16. The method of claim 15, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises:

after receipt of a repeat transmission of the key frame is acknowledged on any of the TCP connections used, discarding all following non-key frames until the next available key frame in the audiovisual signal; and

transmitting the next available key frame on an open TCP connection.

17. The method of claim 10, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises:

discarding all non-key frames until the first available key frame in the audiovisual signal; and

,

when receipt of a transmitted non-key frame is not acknowledged by the receiver discarding all following non-key frames until the next available key frame in the audiovisual signal.

18. The method of claim 10, wherein reassembling individual scalable-coded data packets received over different TCP connections in a single packet stream for decoding comprises:

ordering the received packets for decoding according to the scaleable coding structure; and

removing duplicate received packets.

19. Computer readable media comprising a set of instructions to perform the steps recited in at least one of the method claims 10-18.