Error concealment guided robustness

• US 8,819,525 B1
• Filed: 06/14/2012
• Issued: 08/26/2014
• Est. Priority Date: 06/14/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Holmer, Stefan
• Primary Examiner(s): MCMAHON, DANIEL F
• Application Number:
  US13/523,072
• Time in Patent Office: 803 Days
• Field of Search: None
• US Class Current: 714/775
• CPC Class Codes: H03M 13/33 Synchronisation based on error coding or decoding
• CPC Class Codes: H03M 13/35 Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
• CPC Class Codes: H03M 13/45 Soft decoding, i.e. using symbol reliability information H03M13/41 takes precedence
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04L 1/0045 Arrangements at the receiver end
• CPC Class Codes: H04L 1/007 Unequal error protection for format H04L1/0078; for codes per se H03M13/35
• CPC Class Codes: H04N 19/67 involving unequal error protection [UEP], i.e. providing protection according to the importance of the data
• CPC Class Codes: H04N 19/895 in combination with error concealment
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002

Abstract

Error concealment guided robustness may include identifying a current portion of a current video stream. Identifying the current portion may include identifying a feature, or a vector of features, for the current portion. An estimated vulnerability metric may be identified based on the feature and an associated learned feature weight. An error correction code for the current portion may be generated based on the estimated vulnerability metric. Error concealment guided robustness may include generating learned feature weights based on one or more training videos by generating vulnerability metrics for the training videos and identifying relationships between features of the training videos and the vulnerability metrics generated for the training videos.

Claims

1. A method of generating error correction for a video stream, the method comprising:

identifying a current portion of a current video stream, the current portion having a feature;

identifying an estimated vulnerability metric based on a learned feature weight associated with the feature, the estimated vulnerability metric being a value based on comparing;

a first decoded portion of an encoded training portion of at least one training video stream, the encoded portion including the feature; and

a second decoded portion of the encoded training portion of the at least one training video stream, the second decoded portion based on the encoded training portion with a simulated transmission error; and

generating an error correction code for the current portion based on the estimated vulnerability metric.

2. The method of claim 1, further comprising:

receiving a plurality of learned feature weights including the learned feature weight.

3. The method of claim 1, wherein the generating the error correction code includes using the estimated vulnerability metric to determine an amount of error correction data to include in the error correction code.

4. The method of claim 1, wherein the generating the error correction code includes:

generating a plurality of packets by packetizing at least the current portion; and

generating an error correction code for each packet in the plurality of packets.

5. The method of claim 1, wherein the feature includes a vector of features, wherein each feature in the vector of features indicates a value.

6. The method of claim 1, the learned feature weight is generated by:

identifying the training portion of a training video stream;

generating the encoded training portion;

generating the first decoded portion by decoding the encoded training portion;

generating the second decoded portion by;

generating a transmission error portion by applying the simulated transmission error to the encoded training portion, and

decoding the transmission error portion using at least one error concealment technique;

determining a vulnerability metric based on the first decoded portion and the second decoded portion; and

determining the learned feature weight based on the feature and the vulnerability metric.

7. The method of claim 6, wherein the generating the learned feature weight includes generating a plurality of learned feature weights based on a plurality of training video streams including the training video stream.

8. The method of claim 6, wherein the generating the learned feature weight includes using machine learning.

9. The method of claim 6, wherein the determining the learned feature weight includes identifying the learned feature weight such that the learned feature weight multiplied by the feature is approximately equal to the vulnerability metric.

22. The method of claim 1 wherein the encoded training portion with the simulated transmission error comprises at least one of:

the encoded training portion with some data deleted;

or

the encoded training portion with some data altered.

10. An apparatus for generating error correction for a video stream, the apparatus comprising:

a memory; and

a processor configured to execute instructions stored in the memory to;

identify a current portion of a current video stream, the current portion having a feature;

identify an estimated vulnerability metric based on a learned feature weight associated with the feature, the estimated vulnerability metric being a value based on comparing;

a first decoded portion of an encoded training portion of at least one training video stream, the encoded portion including the feature; and

a second decoded portion of the encoded training portion of the at least one training video stream, the second decoded portion based on the encoded portion with a simulated transmission error; and

generate an error correction code for the current portion based on the estimated vulnerability metric.

11. The apparatus of claim 10, wherein the processor is configured to execute instructions stored in the memory to receive a plurality of learned feature weights including the learned feature weight.

12. The apparatus of claim 10, wherein the processor is configured to execute instructions stored in the memory to generate the error correction code by using the estimated vulnerability metric to determine an amount of error correction data to include in the error correction code.

13. The apparatus of claim 10, wherein the processor is configured to execute instructions stored in the memory to generate the error correction code by:

generating a plurality of packets by packetizing the current portion; and

generating an error correction code for each packet in the plurality of packets.

14. The apparatus of claim 10, wherein the feature includes a vector of features, wherein each feature in the vector of features indicates a value.

15. The apparatus of claim 10, wherein learned feature weight is generated by:

identifying the training portion of a training video stream;

generating the encoded training portion;

generating the first decoded portion by decoding the encoded training portion;

generating the second decoded portion by;

generating a transmission error portion by applying the simulated transmission error to the encoded training portion, and

decoding the transmission error portion using at least one error concealment technique;

determining a vulnerability metric based on the first decoded portion and the second decoded portion; and

determining the learned feature weight based on the feature and the vulnerability metric.

16. The apparatus of claim 15, wherein the processor is configured to execute instructions stored in the memory to generate the learned feature weight by generating a plurality of learned feature weights based on a plurality of training video streams including the training video stream.

17. The apparatus of claim 15, wherein the determining the learned feature weight includes identifying the learned feature weight such that the learned feature weight multiplied by the feature is approximately equal to the vulnerability metric.

23. The apparatus of claim 10 wherein the encoded training portion with the simulated transmission error comprises at least one of:

the encoded training portion with some data deleted;

or

the encoded training portion with some data altered.

18. A method for use in error correction, the method comprising:

generating a learned feature weight by;

identifying a training portion of a training video stream, the training portion having a feature;

generating an encoded training portion;

generating a decoded control portion by decoding the encoded training portion;

generating a decoded corrupted portion by;

generating a transmission error portion by applying a simulated transmission error to the encoded training portion, and

decoding the transmission error portion using at least one error concealment technique;

determining a vulnerability metric based on the decoded control portion and the decoded corrupted portion; and

determining the learned feature weight based on the feature and the vulnerability metric.

19. The method of claim 18, wherein the determining the learned feature weight includes identifying the learned feature weight such that the learned feature weight multiplied by the feature is approximately equal to the vulnerability metric.

20. The method of claim 19, wherein the generating the learned feature weight includes generating a plurality of learned feature weights based on a plurality of training video streams including the training video stream.

21. The method of claim 19, wherein the learned feature weight is based on a plurality of training video streams including the training video stream.