Network communications bandwidth control

• US 7,701,884 B2
• Filed: 11/19/2004
• Issued: 04/20/2010
• Est. Priority Date: 04/19/2004
• Status: Active Grant

• Current Assignee: Iocom UK Limited
• Original Assignee: inSORS Integrated Communications, Inc.
• Inventors: Chapweske, Adam, Swanson, Jon N.
• Primary Examiner(s): PHAM, BRENDA H
• Application Number:
  US10/993,268
• Publication Number:
  US 20050232151A1
• Time in Patent Office: 1,978 Days
• Field of Search: 370/229, 370/231, 370/232, 370/235, 370/235.1, 370/236, 370/253, 370/260, 370/389, 370/412, 370/401, 370/402, 370/428, 370/230, 370/261, 370/262, 370/263, 370/358, 370/265, 370/270, 370/252
• US Class Current: 370/260
• CPC Class Codes: H04L 1/0002 by adapting the transmission rate
• CPC Class Codes: H04L 1/0014 by adapting the source coding
• CPC Class Codes: H04L 1/0018 based on latency requirement
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

A method for controlling real time data communications over a packet based network comprises steps of dynamically measuring data packet loss between a receiver and a sender on the network, dynamically measuring data latency between the sender and receiver, and comparing the dynamically measured data packet loss to a loss limit and comparing the dynamically measured data latency to a latency limit. If one or both of the loss limit or the latency limit is exceeded, then the method may include reducing the rate of data transmission from the sender.

Claims

1. A method for audio and video data control for communicating streaming video data over a data network including a plurality of computers operated by users and linked to one another by at least one conference bridge, a plurality of cameras for generating said video data and a plurality of microphones for generating said audio data, the network operating under a protocol that provides reports, the method comprising steps of:

receiving video data from each of said plurality of cameras and each of said plurality of microphones, communicating said video and audio data to others of said users and displaying said video data using displays, and playing said audio data using speakers;

dynamically measuring data packet loss between said bridge and each of said users for one or more of said audio or video data, said bridge recording information in a memory including a sequence number and a reception time for each data packet it receives from each of said plurality of users, said bridge determining a number of lost packets from each of said plurality of users over a substantially identical time period;

dynamically measuring data latency between said users for one or more of said audio or video data;

comparing said dynamically measured data packet loss to a loss limit and comparing said dynamically measured data latency to a latency limit;

determining a rate of data reception of said receiver if either of said loss limit or said latency limit is exceeded, said rate of data reception determined by;

RRU=[(EHSR_i−

EHSR_[i-1])−

(LP_i−

LP_[i-1])]/(TS_[i]−

TS_[i-1])

where;

RRU=actual data reception rate (e.g., the number of packets per unit time received by said receiver)

EHSR_i=The most recent packet sequence number received from said report received from said receiver

EHSR_[i-1]=The previous packet sequence number received from said report received from said receiver

LP_i=The most recent cumulative number of packets lost from said report received from the receiver

LP_[i-1]=The previous cumulative number of packets lost from said report received from the receiver

TS_i=The timestamp of the report containing EHSR_i and LP_i.

TSi_i-1=The timestamp of the report containing EHSR_[i-1] and LP_[i-1],

and if one or both of said loss limit or said latency limit is exceeded, reducing the rate of data transmission from the sender for one or more of said audio or video data to not exceed said rate of data reception.

2. The audio and video data control system of claim 1 wherein said loss limit is 0.

3. The audio and video data control system of claim 1 wherein said latency limit is no more than about 0.1 seconds.

4. The audio and video data control system of claim 1 wherein the said packet based data is communicated using at least a data channel and a control channel, wherein time data is maintained by said sender that includes the time of transmission of each data packet, wherein reports are communicated from said receiver to said sender that include the highest sequence number of data packet received from said sender, and wherein the step of dynamically measuring said data latency time DL is determined according to the relationship:

DL=(CT−

TT_EHSR)−

(RTCP RTT)/2

where

CT=current system time;

TT=time of transmission of each data packet

EHSR=highest sequence number of data packet received from said sender

TT_EHSR=transmission time TT of the highest sequence number packet reported received in said report;

RTCP RTT=control channel round trip time between sender and receiver.

12. The audio and video data control system of claim 1 wherein the step of dynamically measuring data latency comprises measuring data loss at intervals of no more than about every 0.5 sec.

13. The audio and video data control system of claim 1 wherein said data communication includes real time audio and video data being streamed between a plurality of users on the network linked to one another by a repeater, and wherein the steps of the method are performed at said repeater and at each of said users whereby the step of reducing the rate of data transmission is carried out at said repeater to reduce the rate of transmission to any of said users and may be carried out at any of said users to reduce the rate of transmission to said repeater.

14. The audio and video data control system of claim 1 and further including the step of increasing said data communication rate after determining one or both of said data latency and data loss are at acceptable levels.

15. The audio and video data control system of claim 14 wherein the step of increasing said data communication rate includes increasing by an amount INCR₁ after a time interval INTRVL₁ wherein INCR₁ is some defined portion of said data communications rate.

16. The audio and video data control system of claim 15 and further including the step determining that one or both of said data loss and said data latency are acceptable after increasing said data communication rate by INCR₁, and one or both of setting INCR₂ larger than INCR₁ and setting INTRVL₂ smaller than INTRVL₁.

17. The audio and video data control system of claim 15 and further including the step determining that one or both of said data loss and said data latency are unacceptable after increasing said data communication rate by INCR₁, and one or both of setting INCR₂ smaller than INCR₁ and setting INTRVL₂ larger than INTRVL₁.

5. The audio and video data control system of 1 wherein the step of reducing said rate of data transmission includes intentionally dropping a portion data packets being transmitted from said sender.

6. The audio and video data control system of claim 5, wherein said data packets comprise real time streaming audio and real time streaming video, and wherein said portion of said data packets comprises only video data packets.

7. The audio and video data control system of claim 6, wherein said data packets comprise real time streaming audio and real time streaming video, and wherein said portion of said data packets comprises only audio data packets.

8. The audio and video data control system of claim 5, wherein said data packets comprise a plurality of real time streaming data streams, and further including the step of ranking at least a portion of said streams by priority and wherein the step of intentionally dropping a portion of said data packets comprises dropping packets only from the lowest priority of said data streams.

9. The audio and video data control system of claim 5, and wherein said data communications is made using a protocol that reports lost packets, and wherein the method further includes the step of accounting for said intentionally dropped packets.

10. The audio and video data control system of claim 5 wherein said data being communicated includes real time audio and video data being streamed between a plurality of users linked by a bridge, and wherein the step of intentionally dropping packets is performed at said bridge.

11. The audio and video data control system of claim 10 wherein each of said users includes a plurality of individual sources of streaming data, and wherein said bridge sums all of said sources at each of said users to a single user stream.

18. A videoconference computer program product comprising computer readable program instructions that are stored in a memory, the program product for use with a videoconference system, the program instructions when executed by one or more computers cause the one or more computers to perform a method for controlling real time audio and video data communications over the packet based network during a videoconference, wherein packet based audio and video data is communicated using at least a data channel and a control channel, wherein time data is maintained by a sender that includes the time of transmission of each data packet, wherein reports are communicated from a receiver to the sender that include the highest sequence number of data packet received from the sender, said method comprising the steps of:

dynamically measuring data packet loss between the receiver and the sender on the network;

dynamically measuring data latency time DL between said sender and receiver according to the relationship;

DL=(CT−

TT_EHSR)−

(RTCP RTT)/2

where

CT=current system time;

TT=time of transmission of each data packet

EHSR=highest sequence number of data packet received from said sender

TT_EHSR=transmission time TT of the highest sequence number packet reported received in said report;

RTCP RTT=control channel round trip time between sender and receiver;

comparing said dynamically measured data packet loss to a loss limit and comparing said dynamically measured data latency to a latency limit; and

,

if one or both of said loss limit or said latency limit is exceeded, reducing the rate of data transmission from the sender.

19. The videoconference computer program product of claim 18 wherein said loss limit is no more than about 0.5% of data packets.

20. The videoconference computer program product of claim 18 wherein each of said sender and said receiver have a system clock, and wherein the step of dynamically determining data latency includes the step of synchronizing said sender system clock with said receiver system clock.

21. The videoconference computer program product of claim 20 wherein said data communication comprises real time audio and video data being streamed from a plurality of users over the network participating in a videoconference, and wherein the step of synchronizing said system clocks occurs substantially at the start of the videoconference.

22. The videoconference computer program product of claim 20 wherein the step of synchronizing said system clocks is repeated whenever the network has conditions favorable to performing a synchronization.

23. The videoconference computer program product of claim 18 wherein the step of dynamically measuring data latency comprises measuring data loss at intervals of no more than about every 0.1 sec.

24. The videoconference computer program product of claim 18 wherein said data communication includes real time audio and video data being streamed between a plurality of users on the network linked to one another by a bridge, wherein the step of dynamically measuring data loss is performed between said bridge and each of said users and further includes the step of said bridge storing information including a sequence number and a reception time for each data packet it receives from each of said plurality of users.

25. The videoconference computer program product of claim 24 and further including the step of said bridge storing information including the total number of packets received from each of said plurality of users.

26. The videoconference computer program product of claim 24 and further including the step of said bridge determining a number of lost packets from each of said plurality of users over a substantially identical time period.

27. The videoconference computer program product of claim 24 wherein said information is stored for a series of data packets received over an interval of between about 5 and about 10 sec.

28. The videoconference computer program product of claim 18 wherein said data is being communicated between a plurality of users linked to one another by a repeater R, and wherein the step of dynamically measuring data loss includes determining:

L_RI=S_RI−

Σ

(E_JI−

L_JI)

Where;

L_RI=loss from the output of the repeater to user I taken over all users J such that J≠

I,

S_RI=The number of packets repeater R sent to user I,

E_JI=The number of packets user J expected from user I,

L_JI=loss from user J to user I (as reported by user I), and

Σ

(E_JI−

L_JI) is taken over all J and I.

29. A videoconference computer program product, comprising computer readable program instructions that are stored in a memory, the program product for use with a videoconference system, the program instructions when executed by one or more computers cause the one or more computers to perform a method for controlling real time data communications over a packet based network being communicated between a plurality of users linked to one another by a repeater R during a videoconference, said method comprising the steps of:

dynamically measuring data packet loss between a receiver and a sender on the network by determining;

L_RI=S_RI−

Σ

(E_JI−

L_JI)

where;

L_RI=loss from the output of the repeater to user I taken over all users J such that J≠

I,

S_RI=The number of packets repeater R sent to user I,

E_JI=The number of packets user J expected from user I,

L_JI=loss from user J to user I (as reported by user I), and

Σ

(E_JI−

L_JI) is taken over all J and I;

dynamically measuring data latency between said sender and receiver;

comparing said dynamically measured data packet loss to a loss limit and comparing said dynamically measured data latency to a latency limit; and

,

if one or both of said loss limit or said latency limit is exceeded, reducing the rate of data transmission from the sender.

30. The videoconference computer program product of claim 29 wherein said latency limit is no more than about 0.5 seconds.

31. The videoconference computer program product of claim 29, wherein said data packets comprise real time streaming audio and real time streaming video data packets being communicated between each of a plurality of participants in a real time videoconference being conducted over said network, wherein said sender is a bridge linking said plurality of users to one another, and wherein said receiver is any of said participants.

32. The videoconference computer program product of claim 29 wherein the said packet based data is communicated using at least a data channel and a control channel, and wherein the step of dynamically determining data latency between said sender and said receiver is done over said data channel.

33. The videoconference computer program product of claim 29 wherein the step of reducing said data transmission rate is performed at the application level.

34. A method for communicating streaming video data over a data network comprising the steps of:

linking a plurality of user computers to one another over a packet based network by at least one conference bridge; and

linking a plurality of cameras to each of the user computers for generating video data, the plurality of user computers communicating real time streaming audio and video data to one another over said bridge;

dynamically measuring data packet loss between a receiver and a sender on the network, said data being communicated between a plurality of users linked to one another by a repeater R, where the dynamic measurement of data loss comprises determining;

L_RI=S_RI−

Σ

(E_JI−

L_JI)

where;

L_RI=loss from the output of the repeater to user I taken over all users J such that J≠

I,

S_RI=the number of packets repeater R sent to user I,

E_JI=the number of packets user J expected from user I,

L_JI=loss from user J to user I (as reported by user I),

and

Σ

(E_JI−

L_JI) is taken over all J and I;

dynamically measuring data latency between said sender and receiver;

comparing said dynamically measured data packet loss to a loss limit and comparing said dynamically measured data latency to a latency limit;

if one or both of said loss limit or said latency limit is exceeded, reducing the rate of data transmission from the sender and increasing said data communication rate after determining one or both of said data latency and data loss are at acceptable levels, and wherein said data communication rate is increased by an amount INCR₁ after a time interval wherein INCR₁ is some defined portion of said data communications rate.