System and method for low-delay, interactive communication using multiple TCP connections and scalable coding

• US 7,933,294 B2
• Filed: 12/10/2007
• Issued: 04/26/2011
• Est. Priority Date: 07/20/2005
• Status: Active Grant

• Current Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Original Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Inventors: Eleftheriadis, Alexandros, Sasson, Roi, Shapiro, Ofer, Chakareski, Jacob
• Primary Examiner(s): Rao; Seema S
• Assistant Examiner(s): HUYNH, KHOA B
• Application Number:
  US11/953,398
• Publication Number:
  US 20080130658A1
• Time in Patent Office: 1,233 Days
• Field of Search: None
• US Class Current: 370/474
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 47/43 Assembling or disassembling of packets, e.g. segmentation and reassembly [SAR]
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Systems and methods for communication of scaleable-coded audiovisual signals over multiple TCP/IP connections are provided. The sender schedules and prioritizes transmission of individual scalable-coded data packets over the plurality of TCP connections according to their relative importance in the scalable coding structure for signal reconstruction quality and according to receiver feedback. Low-latency packet delivery over the multiple TCP/IP connections is maintained by avoiding transmission or retransmission of packets that are less important for reconstructed media quality.

Claims

1. An audiovisual communication system including a communication network that uses the IP protocol and has a plurality of TCP connections to carry audiovisual signal data packets between a sender and a receiver, wherein audiovisual signals are scalable-coded by the sender into a set of scalable layers according to a scalable coding structure for transmission over the plurality of TCP connections, and wherein portions of the scalable layers data are designated as key frames, the system comprising:

a Scheduling Inverse Multiplexer (S-IMUX) coupled to the plurality of TCP connections at the sender, wherein the S-IMUX is configured to schedule transmission of individual scalable-coded data packets over the plurality of TCP connections;

a Feedback Multiplexer (F-MUX) coupled to the plurality of TCP connections at the receiver, wherein the F-MUX is configured to reassemble individual scalable-coded data packets received over different TCP connections in a single packet stream for decoding;

wherein the S-IMUX is configured to perform the steps of;

(a) transmitting a key frame over a particular open TCP connection, wherein the key frame is a picture or audio frame for which delivery is necessary in order to ensure an uninterrupted visual experience of media presentation at the receiver; and

(b) if receipt of the key frame over the particular open TCP connection is not acknowledged within a pre-determined time period ‘

T’

, transmitting the unacknowledged key frame over another open TCP connection,

wherein an open TCP connection is a TCP connection that is currently not waiting for an acknowledgement of the receipt of a previous transmission on the TCP connection.

2. The audiovisual communication system of claim 1, wherein the S-IMUX is configured to repeat step (b) until either receipt of the key frame is acknowledged on a TCP connection or a pre-determined time period ‘

T2’

expires.

3. The audiovisual communication system of claim 2, wherein the S-IMUX is configured to discard, after receipt of a repeat transmission of the key frame is acknowledged on any of the TCP connections used, all following non-key frames until the next available key frame in the audiovisual signal, and to transmit the next available key frame on an open TCP connection.

4. A method for audiovisual communication over a communication network that uses the IP protocol and has a plurality of TCP connections to carry audiovisual signal data packets between a sender and a receiver, wherein audiovisual signals are scalable-coded by the sender into a set of scalable layers according to a scalable coding structure for transmission over the plurality of TCP connections, and wherein portions of the scalable layers data are designated as key frames, the method comprising:

at the sender, scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections;

at the receiver, reassembling individual scalable-coded data packets received over different TCP connections in a single packet stream for decoding;

wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises;

(a) transmitting a key frame over a particular open TCP connection, wherein the key frame is a picture or audio frame for which delivery is necessary in order to ensure an uninterrupted visual experience of media presentation at the receiver; and

(b) if receipt of the key frame over the particular open TCP connection is not acknowledged within a pre-determined time period ‘

T’

, transmitting the unacknowledged key frame over another open TCP connection,

wherein an open TCP connection that is a TCP connection that is currently not waiting for an acknowledgement of the receipt of a previous transmission on the TCP connection.

5. The method of claim 4, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises repeating step (b) until either receipt of the key frame is acknowledged on a TCP connection or a pre-determined time period ‘

T2’

expires.

6. The method of claim 5, wherein scheduling transmission of individual scalable-coded data packets of the audiovisual signal over the plurality of TCP connections comprises:

after receipt of a repeat transmission of the key frame is acknowledged on any of the TCP connections used, discarding all following non-key frames until the next available key frame in the audiovisual signal; and

transmitting the next available key frame on an open TCP connection.