REDUCING EFFECTS OF PACKET LOSS IN VIDEO TRANSMISSIONS

• US 20080225735A1
• Filed: 03/14/2007
• Published: 09/18/2008
• Est. Priority Date: 03/14/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Moore, Timothy M., Qiu, Jingyu, Jayasundar, Arvind, Crinon, Regis, Yuan, Zong Zong, Shieh, Guo-Wei
• Granted Patent: US 7,957,307 B2
• Time in Patent Office: Days
• US Class Current: 370/252
• CPC Class Codes: H04L 1/0009 by adapting the channel coding H04L1/1812 takes precedence
• CPC Class Codes: H04L 1/007 Unequal error protection for format H04L1/0078; for codes per se H03M13/35
• CPC Class Codes: H04N 19/65 using error resilience
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

An error correction system determines a level of error correction protection to apply to a frame of video data to be transmitted by a sending endpoint to a receiving endpoint based on the predicted impact of packet loss as well as the importance of the frame based on inter-frame dependencies, frame size, packet loss probability, historical packet loss pattern, central processing unit (CPU) load, and available network bandwidth. At the receiving endpoint, when packet loss is detected for a particular frame, the receiving endpoint will attempt to recover the frame using protection packets received along with the video data.

Claims

1. A method in a computer system for reducing the effects of packet loss when sending data between a sending endpoint and a receiving endpoint, the method comprising:

receiving a data size and an indication of a rate of packet loss during a previous period;

determining a level of protection to apply based on the received data size and the indication of the rate of packet loss;

sending to the receiving endpoint one or more data packets containing the data; and

sending to the receiving endpoint one or more protection packets based on the determined level of protection, wherein the one or more protection packets comprise information with which the receiving endpoint can recover a lost data packet, such that when a data packet is lost the receiving endpoint can obtain all of the data using the protection packets without additional communication with the sending endpoint.

2. The method of claim 1 wherein at least one level of protection applies an exclusive OR operation to the data.

3. The method of claim 1 wherein at least one level of protection applies the Reed-Solomon algorithm to the data.

4. The method of claim 1 including creating a table that maps levels of protection to frame size and rate of packet loss, and wherein determining a level of protection to apply comprises looking up the level of protection in the table.

5. The method of claim 1 wherein the protection packets include staged levels of protection.

6. The method of claim 1 wherein receiving a rate of packet loss comprises receiving a packet loss report from the receiving endpoint.

7. The method of claim 1 wherein determining a level of protection to apply comprises determining a type of a video frame, and including applying a higher level of protection to a frame type on which other frames depend.

8. The method of claim 1 wherein determining a level of protection to apply comprises not applying protection to data having a size below a predetermined threshold or containing video data below a predetermined bitrate.

9. The method of claim 1 wherein the sending endpoint is a video conferencing server serving multiple receiving endpoints.

10. A computer system for reducing the effects of packet loss on videoconferencing data transmitted from a sending endpoint to a receiving endpoint through a videoconferencing server over a network, comprising:

a video conference data receiving component configured to receive video conference data from the sending endpoint;

an error correction component configured to produce error correction data based on the video conference data, wherein the error correction data provides information for recovering a lost portion of the video conference data and is divided into packets based on one or more levels of protection;

a video conference data sending component configured to forward the video conference data received from the sending endpoint to multiple receiving endpoints, wherein the video conference data sending component determines which of the packets of the error correction data to send to each receiving endpoint based on characteristics of the video conference data and the connection with the receiving endpoint, such that the error correction data is not separately produced for each receiving endpoint.

11. The system of claim 10 wherein the error correction component receives the error correction data from the sending endpoint.

12. The system of claim 10 wherein the error correction component creates the error correction data at the videoconferencing server.

13. The system of claim 10 wherein the sending endpoint creates a portion of the error correction data and the videoconferencing server creates another portion of the error correction data.

14. The system of claim 10 wherein the error correction component sends an indication of a desired level of protection to the sending endpoint and receives error correction data from the sending endpoint based on the desired level of protection.

15. The system of claim 10 wherein the video conference data sending component periodically re-determines which of the packets of the error correction data to send to each receiving endpoint.

16. The system of claim 10 wherein a packet sent to the receiving endpoint comprises a format adaptation header that contains information about the presence and location of error correction data in a digital video bitstream.

17. The system of claim 10 wherein the error correction component determines whether error correction data should be produced for a particular video frame based on a type of the video frame.

18. The system of claim 10 wherein the error correction component produces error correction data based on statistics about a historical condition of the network.

19. The system of claim 10 wherein at least one receiving endpoint determines whether to use error correction data to recover one or more invalid packets based on information received from the videoconferencing server.

20. The system of claim 10 wherein the video conference data includes version information, such that a legacy client can interoperate with a newer videoconferencing server.

21. The system of claim 10 including a media transport packetizer that forms packets that include a header and a payload, wherein the header indicates to a receiving endpoint any dependency of the current frame on other frames in a video bitstream.

22. A computer-readable medium containing instructions for controlling a computer system to recover lost packets at a receiving endpoint, by a method comprising:

receiving video data from a sending endpoint, wherein the video data is divided into one or more data packets;

receiving error correction data from the sending endpoint, wherein the error correction data is divided into one or more protection packets;

determining that at least one data packet sent by the sending endpoint was not received by the receiving endpoint;

applying the error correction data from the one or more protection packets to recover the at least one data packet that was not received.

23. The computer-readable medium of claim 22 including adding padding to a data packet based on information in a header of the data packet.

24. The computer-readable medium of claim 22 including periodically sending an indication of packet loss to the sending endpoint.

25. The computer-readable medium of claim 22 wherein determining that at least one data packet was not received comprises detecting a sequence number for each received data packet.

26. The computer-readable medium of claim 22 wherein each protection packet includes an indication of the level of protection provided by the error correction data in the protection packet.