Methods and systems for forward error correction based loss recovery for interactive video transmission
First Claim
1. A method for transmitting video data over a lossy packet-based network, the method comprising:
- at a transmitter,(a) encoding a first video frame into k data packets, k being an integer;
(b) determining n minus k (n−
k) forward error correction (FEC) packets for the first video frame, n being an integer;
(c) transmitting the k data packets over a lossy packet-based network to a receiver during a first frame interval;
(d) transmitting the n−
k FEC packets to the receiver during frame intervals after the first frame interval;
at the receiver,(e) receiving some of the k data packets, decoding the received data packets to reproduce the first video frame, and displaying the first video frame with errors;
(f) receiving the n−
k FEC packets and using the n−
k FEC packets to restore the first video frame; and
(g) using the first video frame as a reference frame for decoding at least one subsequent video frame.
1 Assignment
0 Petitions
Accused Products
Abstract
Real-time interactive video transmission in the current Internet has mediocre quality because of high packet loss rates. Loss of packets belonging to a video frame is evident not only in the reduced quality of that frame but also in the propagation of that distortion to successive frames. This error propagation problem is inherent in any motion-based video codec because of the interdependence of encoded video frames. Since packet losses in the best-effort Internet environment cannot be prevented, minimizing the impact of these packet losses to the final video quality is important. A new forward error correction (FEC) technique effectively alleviates error propagation in the transmission of interactive video. The technique is based on a recently developed error recovery scheme called Recovery from Error Spread using Continuous Updates (RESCU). RESCU allows transport level recovery techniques previously known to be infeasible for interactive video transmission applications to be successfully used in such applications. The FEC technique can be very useful when the feedback channel from the receiver is highly limited, or transmission delay is high. Both simulation and Internet experiments indicate that the FEC technique effectively alleviates the error spread problem and is able to sustain much better video quality than H.261 or other conventional FEC schemes under various packet loss rates.
-
Citations
36 Claims
-
1. A method for transmitting video data over a lossy packet-based network, the method comprising:
-
at a transmitter, (a) encoding a first video frame into k data packets, k being an integer;
(b) determining n minus k (n−
k) forward error correction (FEC) packets for the first video frame, n being an integer;
(c) transmitting the k data packets over a lossy packet-based network to a receiver during a first frame interval;
(d) transmitting the n−
k FEC packets to the receiver during frame intervals after the first frame interval;
at the receiver, (e) receiving some of the k data packets, decoding the received data packets to reproduce the first video frame, and displaying the first video frame with errors;
(f) receiving the n−
k FEC packets and using the n−
k FEC packets to restore the first video frame; and
(g) using the first video frame as a reference frame for decoding at least one subsequent video frame. - View Dependent Claims (2, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
3. The method of 1 comprising determining a time interval between the n−
- k FEC packets required to protect the n−
k FEC packets from burst losses and wherein transmitting the n−
k FEC packets to the receiver includes spacing transmission of each of the n−
k FEC packets by the time interval. - View Dependent Claims (4, 5, 6)
- k FEC packets required to protect the n−
-
15. A method for optimized cascaded error recovery when transmitting video over a lossy packet-based network, the method comprising:
-
(a) receiving, over a lossy packet-based network, a plurality of data packets associated with a first video frame Fi, i being an integer;
(b) decoding the data packets to reproduce the first video frame Fi;
(c) displaying the first video frame Fi with errors;
(d) after display of the first video frame Fi, receiving error correction packets associated with the first video frame Fi;
(e) repairing the first video frame Fi using the error correction packets;
(f) receiving data packets associated with periodic frames Fi+n, n being an integer, transmitted after the first video frame Fi;
(g) using motion estimation to determine the effect of errors in the first video frame Fi on the subsequent periodic frames Fi+n and (h) restoring the subsequent periodic frames Fi+n using the restored first video frame Fi and the motion estimation. - View Dependent Claims (16, 17, 18)
-
-
19. A computer program product comprising computer-executable instructions embodied in a computer readable medium for performing steps comprising:
-
at a transmitter, (a) encoding a first video frame into k data packets, k being an integer;
(b) determining n minus k (n−
k) forward error correction (FEC) packets for the first video frame, n being an integer;
(c) transmitting the k data packets over a lossy packet-based network to a receiver during a first frame interval;
(d) transmitting the n−
k FEC packets to the receiver during frame intervals after the first frame interval;
at the receiver, (e) receiving some of the k data packets, decoding the received data packets to reproduce the first video frame and displaying the first video frame with errors;
(f) receiving the n−
k FEC packets and using the n−
k FEC packets to restore the first video frame; and
(g) using the first video frame as a reference frame for decoding at least one subsequent video frame. - View Dependent Claims (20, 25, 26, 27, 28, 29, 30, 31, 32)
-
-
21. The computer program product of 19 comprising determining a time interval between FEC packets required to protect the FEC packets from burst losses and wherein transmitting the n−
- k FEC packets to the receiver includes spacing transmission of each of the n−
k FEC packets by the time interval. - View Dependent Claims (22, 23, 24)
- k FEC packets to the receiver includes spacing transmission of each of the n−
-
33. A computer program product comprising computer-executable instructions embodied in a computer readable medium for performing steps comprising:
-
(a) receiving, over a lossy packet-based network, a plurality of data packets associated with a first video frame Fi, i being an integer;
(b) decoding the data packets to reproduce the first video frame Fi;
(c) displaying the first video frame Fi with errors;
(d) after display of the first video frame Fi, receiving error correction packets associated with the first video frame Fi;
(e) repairing the first video frame Fi using the error correction packets;
(f) receiving data packets associated with periodic frames Fi+n, n being an integer, transmitted after the first video frame Fi;
(g) using motion estimation to determine the effect of errors in the first video frame Fi on the subsequent periodic frames Fi+n and (h) restoring the subsequent periodic frames Fi+n using the restored first video frame Fi and the motion estimation. - View Dependent Claims (34, 35, 36)
-
Specification