Simultaneous processing of media and redundancy streams for mitigating impairments

• US 8,958,486 B2
• Filed: 07/31/2007
• Issued: 02/17/2015
• Est. Priority Date: 07/31/2007
• Status: Expired due to Fees

• Current Assignee: Interdigital Vc Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Rodriguez, Arturo A., VerSteeg, William C., Kernen, Thomas
• Primary Examiner(s): Kim, Jung
• Assistant Examiner(s): Ho, Thomas
• Application Number:
  US11/831,912
• Publication Number:
  US 20090034633A1
• Time in Patent Office: 2,758 Days
• Field of Search: 375/240.27, 375/240.25, 375/240.16, 375/240.13
• US Class Current: 375/240.27
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04L 1/0045 Arrangements at the receiver end
• CPC Class Codes: H04L 1/0047 Decoding adapted to other signal detection operation in conjunction with sequence estimation or equalization H04L25/03286
• CPC Class Codes: H04L 1/08 by repeating transmission, e.g. Verdan system H04L1/1858 and H04L1/189 take precedence
• CPC Class Codes: H04L 1/22 using redundant apparatus to increase reliability
• CPC Class Codes: H04N 19/166 concerning the amount of transmission errors, e.g. bit error rate [BER]
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/4425 Monitoring of client processing errors or hardware failure

Abstract

In one embodiment, a method comprises receiving a primary stream of encoded frames and a separate stream of redundant frames. The method further comprises decoding and reconstructing in parallel the frames in the primary stream and the separate stream of redundant frames, on a real-time basis, in accordance with a specified common clock reference. The method further comprises, upon determining that a frame in the primary stream exhibits an error or impairment, determining a decoded redundant frame in the separate stream that corresponds to the impaired frame, and substituting at least a portion of the information in the decoded redundant frame for a corresponding decoded version of the impaired frame.

Claims

1. A method comprising:

receiving a primary stream of encoded frames and a separate stream of redundant frames, wherein the separate stream of redundant frames comprises only reference frames redundant of frames in the primary stream, wherein the reference frames are selected based on a combination of;

     1) a minimum number of other frames depending on the reference frame in relation to other reference frames,

     2) a level of information propagation via indirect dependence, and

     3) a longevity of the information propagation;

decoding and reconstructing in parallel the frames in the primary stream and the separate stream of redundant frames, on a real-time basis, in accordance with a specified common clock reference; and

upon determining that a frame in the primary stream exhibits an error or impairment, determining a decoded redundant frame in the separate stream that corresponds to the impaired frame, and substituting at least a portion of the information in the decoded redundant frame for a corresponding decoded version of the impaired frame.

2. The method of claim 1, further comprising:

determining the corresponding frame in the separate stream of redundant frames based on a picture or slice type.

3. The method of claim 1, further comprising:

determining the corresponding frame in the separate stream of redundant frames based on a frame number that is relative to a sequence parameter set.

4. The method of claim 1, further comprising:

determining the corresponding frame in the separate stream of redundant frames based on matching a presentation time stamp in the frame in the primary stream with a presentation time stamp in the frame in the redundant stream.

5. The method of claim 1, further comprising:

if the frame in the primary stream exceeds an error criteria, determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received; and

if the FEC frame has been received, using the FEC frame to correct the frame in the primary stream.

6. The method of claim 1, further comprising:

if the frame in the primary stream exceeds an error criteria, determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received; and

if the FEC frame has not been received, requesting retransmission of the frame in the primary stream.

7. The method of claim 1, further comprising:

if the frame in the primary stream exceeds an error criteria, determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received;

if the FEC frame has not been received, evaluating whether decoding with the frame in the primary stream will produce a picture of acceptable quality; and

if the evaluation determines that the decoding will produce unacceptable quality, requesting retransmission of the frame in the primary stream.

8. The method of claim 1, further comprising:

if the frame in the primary stream exceeds an error criteria, determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received;

if the FEC frame has not been received, evaluating whether decoding with the frame in the primary stream will produce a picture of acceptable quality; and

if the evaluation determines decoding will produce acceptable quality, supplying the frame in the primary stream for decoding.

9. A system comprising:

means for receiving a primary stream of encoded frames and a redundant stream of encoded frames, wherein each encoded frame in the redundant stream has added information comprising a number that is relative to a particular predetermined point in the primary stream, wherein the redundant frames are selected based on a combination of;

     1) a minimum number of other frames depending on the redundant frame in relation to other reference frames,

     2) a level of information propagation via indirect dependence, and

     3) a longevity of the information propagation;

means for determining if the frame in the primary stream exceeds an error criteria;

means for determining a decoded corresponding frame in the redundant stream that corresponds to the frame in the primary stream and supplying the corresponding frame, responsive to the primary stream exceeding the error criteria; and

means for if the frame does not exceed the error criteria, supplying the frame in the primary stream for decoding.

10. The system of claim 9, further comprising:

means for determining the corresponding frame in the separate stream of redundant frames based on a picture or slice type.

11. The system of claim 9, further comprising:

means for determining the corresponding frame in the redundant stream based on a frame number that is relative to a sequence parameter set.

12. The system of claim 9, further comprising:

means for determining the corresponding frame in the redundant stream based on matching a presentation time stamp in the frame in the primary stream with a presentation time stamp in the frame in the redundant stream.

13. The system of claim 9, further comprising:

means for determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received, responsive to the frame in the primary stream exceeding the error criteria; and

means for using the FEC frame to correct the frame in the primary stream, responsive to determining the FEC frame has been received.

14. The system of claim 9, further comprising:

means for determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received; and

means for requesting retransmission of the frame in the primary stream, responsive to determining the FEC frame has not been received.

15. The system of claim 9, further comprising:

means for determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received, responsive to the frame in the primary stream exceeding the error criteria;

means for evaluating whether decoding with the frame in the primary stream will produce a picture of acceptable quality, responsive to determining that the FEC frame has not been received;

means for requesting retransmission of the frame in the primary stream, responsive to the evaluation that decoding will produce unacceptable quality.

16. The system of claim 9, further comprising:

determining whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received, responsive to the frame in the primary stream exceeding the error criteria;

evaluating whether decoding with the frame in the primary stream will produce a picture of acceptable quality, responsive to determining that the FEC frame has not been received; and

supplying the frame in the primary stream for decoding, responsive to the evaluation that decoding will produce acceptable quality.

17. An apparatus comprising:

a network interface configured to receive a combined stream including a primary stream of encoded frames and a redundant stream of encoded frames, wherein the redundant stream of encoded frames includes only redundant frame identified as frames of interest, wherein the redundant frames are selected based on a combination of;

     1) a minimum number of other frames depending on the redundant frame in relation to other reference frames,

     2) a level of information propagation via indirect dependence, and

     3) a longevity of the information propagation;

a demultiplexer configured to separate the primary stream and the redundant stream;

a decoder; and

a redundant stream processor configured to determine whether a frame in the primary stream exceeds an error criteria and to determine a corresponding decoded frame in the redundant stream for the frame in the primary stream and to supply the corresponding frame if the frame does not exceed the error criteria.

18. The apparatus of claim 17, wherein the redundant stream processor is further configured to determine the corresponding frame in the redundant stream based on matching a presentation time stamp in the frame in the primary stream with a presentation time stamp in the frame in the redundant stream.

19. The apparatus of claim 17, wherein the redundant stream processor is further configured to, when the frame in the primary stream exceeds the error criteria, determine whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received and if the FEC frame has been received, to correct the frame in the primary stream using the FEC frame.

20. The apparatus of claim 17, wherein the redundant stream processor is further configured to, if the frame in the primary stream exceeds the error criteria, determine whether a forward error correction (FEC) frame associated with the frame in the primary stream has been received and if the FEC frame has not been received, to request retransmission of the frame in the primary stream.