Video encoding and decoding using un-equal error protection

• US 9,185,429 B1
• Filed: 04/30/2012
• Issued: 11/10/2015
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gu, Qunshan, Wu, Ronghua
• Primary Examiner(s): Diep, Nhon
• Application Number:
  US13/460,047
• Time in Patent Office: 1,289 Days
• US Class Current: 1/1
• CPC Class Codes: H03M 13/35 Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/67 involving unequal error protection [UEP], i.e. providing protection according to the importance of the data

Abstract

Systems, apparatuses and methods for encoding and decoding a video stream having a plurality of frames are disclosed. When encoding, for example, blocks are selected in spiral fashion until contiguous blocks of at least a central section of the frame are selected. The selected blocks are encoded in an order in which they were selected. A first level of error protection is applied to encoded blocks within the central section. A second, different level of error protection is applied to encoded blocks outside the central section. By using a spiral scan and selective error protection, blocks most likely to have information of interest to a viewer are identified and can be encoded and error-protected to provide better image quality in the area of interest when decoded.

Claims

1. A method for encoding a video stream including a plurality of video frames, each frame comprising a plurality of blocks, the method comprising:

selecting a first block substantially central to a frame and then selecting subsequent blocks in a substantially spiral manner such that each subsequent block adjoins a previously selected block;

continuing the selecting of subsequent blocks until the selected blocks form a contiguous group of blocks occupying at least a central section of the frame;

encoding the selected blocks in an order in which the selected blocks were selected using a processor;

applying a first level of error protection to encoded blocks within the central section; and

if any encoded blocks are located outside the central section, applying a second level of error protection, different than the first level of error protection, to the encoded blocks outside the central section.

2. The method of claim 1, where the first level of error protection comprises forward error correction and the second level of error protection is no error protection.

3. The method of claim 1, where the first level of error protection comprises forward error correction applied at a first ratio to the encoded blocks within the central section, and the second level of error protection comprises forward error correction applied at a second ratio to the encoded blocks outside the central section, where the first ratio is greater than the second ratio.

4. The method of claim 1, wherein encoding the selected blocks further comprises:

for encoded blocks within the central section, performing at least one of motion search and motion compensation based on data from blocks only within the central section; and

for blocks outside the central section, performing at least one of motion search and motion compensation based on data from both blocks within the central section and blocks outside the central section.

5. The method of claim 1 wherein:

the first level of error protection comprises retransmitting data corresponding to blocks within the central section responsive to the data being lost in transmission to a decoder; and

the second level of error protection comprises not retransmitting data corresponding to blocks outside the central section responsive to the data being lost in transmission to the decoder.

6. The method of claim 1, further comprising:

receiving a signal indicating loss of a data packet; and

wherein;

applying the first level of error protection comprises retransmitting the data packet when the data packet includes data related to the encoded blocks within the central section; and

applying the second level of error protection comprises ignoring the signal when the data packet includes data related to the encoded blocks outside the central section.

7. An apparatus for encoding a video stream having a plurality of video frames, each frame having a plurality of blocks, the apparatus comprising:

memory; and

a processor programmed to execute instructions contained in the memory to;

select a first block substantially central to a frame and then select subsequent blocks in a substantially spiral manner such that each subsequent block adjoins a previously selected block;

continue the selecting of subsequent blocks until the selected blocks form a contiguous group of blocks occupying at least a central section of the frame;

encode the selected blocks in an order in which the selected blocks were selected;

apply a first level of error protection to encoded blocks within the central section; and

if any encoded blocks are located outside the central section, apply a second level of error protection, different than the first level of error protection, to the encoded blocks outside the central section.

8. The apparatus of claim 7 wherein the first level of error protection comprises forward error correction and the second level of error protection no error protection.

9. The apparatus of claim 7 wherein the processor is programmed to apply the first level of error protection by including forward error correction in an encoded bitstream using a first ratio for data related to the encoded blocks within the central section and to apply the second level of error protection by including forward error correction in the encoded bitstream using a second ratio for data related to the encoded blocks outside the central section;

and wherein the first ratio is greater than the second ratio.

10. The apparatus of claim 7 wherein the processor is programmed to encode the blocks by:

for blocks within the central section, performing at least one of motion search and motion compensation based on data from blocks only within the central section; and

for blocks outside the central section, performing at least one of motion search and motion compensation based on data from both blocks within the central section and blocks outside the central section.

11. The apparatus of claim 7 wherein the processor, responsive to a signal indicating loss of data, is programmed to:

apply the first level of error protection by retransmitting the data when the data is related to the encoded blocks within the central section; and

apply the second level of error protection by ignoring the signal when the data is related to the encoded blocks outside the central section.

12. A method for decoding a video stream including a plurality of video frames, each frame comprising a plurality of blocks, the method comprising:

receiving a bitstream of encoded blocks associated with a frame;

decoding the encoded blocks wherein the encoded blocks are encoded in an order corresponding to a first block substantially central to the frame, a second block adjacent to the first block, and subsequent blocks adjacent to a previously selected block in a spiral manner;

responsive to data errors affecting decoding of a block of the encoded blocks, using a processor to perform data recovery operations by;

attempting recovery of data related to the block using a first level of error protection when the block resides within a central section of the frame, the central section of the frame including at least a contiguous set of blocks about a center of the frame; and

if the block resides outside the central section, performing one of;

attempting recovery of data related to the block using a second level of error protection, wherein the second level of protection is different than the first level of error protection;

or

skipping recovery of the data related to the block.

13. The method of claim 12 wherein attempting recovery of data related to the block using the first level of error protection comprises:

using a forward error correction packet to reconstruct the data, the forward error correction packet generated using a first ratio; and

wherein attempting recovery of data related to the block using the second level of error protection comprises;

using a forward error correction packet to reconstruct the data, the forward error correction packet generated using a second ratio, the first ratio being greater than the second ratio.

14. The method of claim 12 wherein the bitstream comprises:

motion vectors generated by performing at least one of motion search and motion compensation based on data from blocks only within the central section; and

motion vectors generated by performing at least one of motion search and motion compensation based on data from both within the central section and blocks outside the central section.

15. The method of claim 12 wherein attempting recovery of data related to the block using the first level of error protection comprises:

transmitting a signal requesting retransmission of the data.

16. The method of claim 15 wherein the signal includes a packet ID for a lost data packet.

17. An apparatus for decoding a video stream having a plurality of video frames, each frame having a plurality of blocks, the apparatus comprising:

memory; and

at least one processor programmed to execute instructions contained in the memory to;

receive a bitstream of encoded blocks associated with a frame;

decode the encoded blocks wherein the encoded blocks are encoded in an order corresponding to a first block substantially central to the frame, a second block adjacent to the first block, and subsequent blocks adjacent to a previously selected block in a spiral manner;

responsive to data errors affecting decoding of a block of the encoded blocks, perform data recovery by;

attempting recovery of data related to the block using a first level of error protection when the block resides within a central section of the frame, the central section of the frame including at least a contiguous set of blocks about a center of the frame; and

if the block resides outside the central section, perform one of;

attempting recovery of data related to the block using a second level of error protection, wherein the second level of protection is different than the first level of error protection;

or

skipping recovery of the data related to the block.

18. The apparatus of claim 17 where the processor is programmed to attempt recovery of data related to the block using the first level of error protection by:

using a forward error correction packet to reconstruct the data, the forward error correction packet generated using a first ratio; and

to attempt recovery of data related to the block using the second level of error protection by;

using a forward error correction packet to reconstruct the data, the forward error correction packet generated using a second ratio, the first ratio being greater than the second ratio.

19. The apparatus of claim 17 wherein the processor is programmed to attempt recovery of data related to the block using the first level of error protection by requesting retransmission of the data.

20. The apparatus of claim 17 wherein the processor is configured to determine whether the block resides within the central section of the frame or outside the central section based on a position of the block within the bitstream relative to other blocks of the frame.