Intelligent buffering process for network conference video

• US 7,133,362 B2
• Filed: 11/14/2001
• Issued: 11/07/2006
• Est. Priority Date: 11/14/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ben-Shachar, Ido, Chu, Lon-Chan, Leichtling, Ivan J., Collins, Leonard Alan, Xu, Yikang, Giloi, Claus T.
• Primary Examiner(s): Ngo, Ricky Q.
• Assistant Examiner(s): Wong, Warner
• Application Number:
  US09/992,731
• Publication Number:
  US 20030091000A1
• Time in Patent Office: 1,819 Days
• Field of Search: 709/204, 709/207, 709/235, 348/419, 348/14.11, 348 1408- 141, 348/14.09, 370/230, 370/235, 370260-263, 370412-418, 37524028- 9
• US Class Current: 370/230
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

In order to minimize the delay of the video images viewed by a network conference attendee, an intelligent buffering process (IB process) selectively discards video frames from at least one point, and in an embodiment, many network points, such as at “in” and/or “out” buffers of clients, servers, routers, etc. Packets of video frame data arrive at a buffer, which can fill to a predetermined limit if the packets cannot be forwarded due to delays or slow connections. To forward the most current video images, old frames in the buffer are discarded rather than forwarded. In a particular embodiment, when the buffer is full, the next arriving delta frame packet is discarded. To avoid distortion, each subsequent delta frame is discarded until a new key frame eventually arrives. If the buffer is still full when the key frame arrives, the buffer is purged and the new key frame is added.

Claims

1. An intelligent buffering process to reduce lag in a conference video stream after a data forwarding delay, the stream including a plurality of packets defining either a key frame or a delta frame, each of the key frames representing an entire screen image, each of the delta frames representing image changes relative to a previous frame, the process comprising the steps of:

receiving a newly arrived packet;

determining whether the newly arrived packet is a key frame; and

if the packet is a key frame;

detecting whether a buffer has reached a predetermined limit and deleting packets in the buffer if the predetermined limit has been reached; and

adding the newly arrived packet to the buffer;

discarding the newly arrived packet if the packet is a delta frame and if the buffer contents have reached the predetermined limit; and

determining whether the previous packet was discarded, and if so, discarding each newly arrived packet which contains delta frame if until another packet containing key frame arrives.

2. The intelligent buffering process of claim 1, further comprising the step of adding the newly arrived packet to the buffer if the packet is a delta frame, if the buffer has not reached the predetermined limit and if the previous frame was not discarded.

3. The intelligent buffering process of claim 1, further comprising the step of forwarding each packet from the buffer in the order received.

4. The intelligent buffering process of claim 3, wherein the forwarding step occurs as soon as downstream data traffic permits.

5. The intelligent buffering process of claim 3, wherein both the receiving step and forwarding step are executed above a network protocol level.

6. The intelligent buffering process of claim 3, wherein after the forwarding step the process includes transmitting the packets over a network.

7. The intelligent buffering process of claim 1, wherein the receiving step includes receiving packets sent from a conference attendee computer over the network.

8. An intelligent buffering process to reduce lag in a conference video stream after a data forwarding delay, the stream including a plurality of packets defining either a key frame or a delta frame, each of the key frames representing an entire screen image, each of the delta frames representing image changes relative to a previous frame, the process comprising the steps of:

receiving a newly arrived packet;

determining whether the newly arrived packet is a delta frame, and if so;

detecting whether the previous packet was discarded,

discarding the newly arrived delta frame if the previous packet was discarded;

detecting whether a buffer have reached a predetermined limit; and

discarding the newly arrived packet if the buffer has reached the predetermined limit.

9. The intelligent buffering process of claim 8, further comprising the step of adding the newly arrived delta frame packet to the buffer if the previous frame was not discarded and if the buffer has not reached the predetermined limit.

10. The intelligent buffering process of claim 8, further comprising:

deleting packets from the buffer if the newly arrived packet is a key frame, and if the buffer has reached the predetermined limit; and

adding the key frame to the buffer.

11. A process for managing a video conference among a plurality of attendee computers, the process comprising:

providing a plurality of buffers at a conference server, each of the buffers corresponding to a respective attendee destination;

receiving a newly arrived packet of a video stream, the packets defining respective key frames and a delta frames, each of the key frames representing an entire screen image, each of the delta frames representing image changes relative to a previous frame; and

for each of the buffers;

determining whether the newly arrived packet is a key frame;

if the packet is a key frame;

deleting the packets in the buffer if the buffer contents have reached a predetermined limit;

adding the key frame to the buffer;

forwarding non-deleted packets from the buffer to the corresponding attendee destination;

discarding the newly arrived frame if the frame is a delta frame and if the buffer contents have reached the predetermined limit; and

determining whether the previous packet was discarded, and if so, discarding each newly arrived delta frame packet if until a new key frame packet arrives.

12. The process of claim 11, independently for each of the buffers the process further comprising the step adding a newly arrived packet to the buffer if the packet is a delta frame, if the buffer has not reached the predetermined limit and if the previous packet was not discarded.

13. The process of claim 11, wherein after the forwarding step the process includes transmitting the packets over a network to an attendee destination according to a verified-delivery protocol.

14. The process of claim 13, wherein the protocol is TCP/IP.

15. The process of claim 11, wherein the process further comprises providing at least one buffer at at least one of the attendee computers, said receiving, determining, deleting, adding steps being performed by the attendee computer on the packets forwarded from the server.

16. A process for managing a video conference among a plurality of attendee computers, the process comprising:

providing a plurality of buffers at a conference server, each of the buffers corresponding to a respective attendee destination;

receiving a newly arrived packet of a video stream, the packets defining respective key frames and a delta frames, each of the key frames representing an entire screen image, each of the delta frames representing image changes relative to a previous frame; and

for each of the buffers;

determining whether the newly arrived packet is a key frame;

if the packet is a key frame;

deleting the packets in the buffer if the buffer contents have reached a predetermined limit;

adding the key frame to the buffer;

forwarding non-deleted packets from the buffer to the corresponding attendee destination; and

providing at least one buffer at at least one of the attendee computers, the attendee computer performing said receiving, determining, deleting, adding and forwarding steps on the packets prior to the receiving of packets by the server.

17. A computer-readable medium having computer-executable instructions for performing a process for managing a buffer, the process comprising:

receiving packets of a video stream data, the packets defining a plurality of key frames and delta frames, each of the key frames representing an entire screen, each of the delta frames representing image changes relative to a previous frame;

determining whether a newly arrived packet contains a key frame; and

if the packet is a key frame;

detecting whether the buffer contents have reached a predetermined limit and deleting packets in the buffer if the predetermined limit has been reached; and

adding the key frame to the buffer;

discarding each packet not containing a key frame which arrives after the buffer contents have reached said predetermined limit;

determining whether the previous packet was discarded; and

discarding each packet not containing a key frame which arrives subsequent to a discarded packet until the arrival of a new key frame.