Video compression across continuous frame edges

• US 10,609,379 B1
• Filed: 09/01/2015
• Issued: 03/31/2020
• Est. Priority Date: 09/01/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wang, Qi Keith, Brailovskiy, Ilya Vladimirovich, Patuck, Naushirwan Navroze
• Primary Examiner(s): Volentine, Rebecca A
• Assistant Examiner(s): Lee, Jimmy S
• Application Number:
  US14/842,558
• Time in Patent Office: 1,673 Days
• Field of Search: 375 2416
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Various examples are directed to methods and systems for compressing panoramic video. An image processor may receive a frame comprising a plurality of pixel values arranged according to a two-dimensional grid and divide the frame into a plurality of pixel blocks comprising a first pixel block and a second pixel block. The first pixel block may be positioned in a first edge column adjacent a first edge of the frame and the second pixel block may be positioned in a second edge column adjacent a second edge of the frame opposite the first edge. The image processor may identify a spatial continuity between the first pixel block and the second pixel block and replace the first pixel block with a reference to the second pixel block.

Claims

1. A method for compressing video, the method comprising:

receiving, by an image processor, a panoramic video file, wherein receiving the panoramic video file comprises receiving frame data describing a plurality of frames;

selecting a subject frame from the panoramic video file, wherein the subject frame comprises a first plurality of pixel values corresponding to positions on a two-dimensional grid, wherein the first plurality of pixel values depict a cylindrical projection of a 360°

field-of-view;

dividing, by the image processor, the subject frame into a first plurality of macroblocks, wherein each macroblock of the first plurality of macroblocks comprises a set of pixel values that are selected from the first plurality of pixel values and that correspond to contiguous positions in the two-dimensional grid, and wherein the first plurality of macroblocks comprises a subject frame left column of macroblocks positioned adjacent to a left edge of the subject frame and a subject frame right column of macroblocks positioned adjacent a right edge of the subject frame;

identifying, by the image processor, cross-edge spatial continuity between a first macroblock in the subject frame left column of macroblocks and a second macroblock in the subject frame right column of macroblocks, wherein identifying the cross-edge spatial continuity comprises identifying a similarity between first pixel values of the first macroblock and second pixel values of the second macroblock;

receiving a subsequent frame of the panoramic video file, wherein the subsequent frame comprises a second plurality of pixel values corresponding to positions on the two-dimensional grid, and wherein the subsequent frame is positioned immediately after the subject frame according to a temporal sequence of frames of the panoramic video file;

dividing, by the image processor, the subsequent frame into a second plurality of macroblocks, wherein the second plurality of macroblocks comprises a subsequent frame left column of macroblocks positioned on a left side of the subsequent frame and a subsequent frame right column of macroblocks positioned on a right side of the subsequent frame;

identifying, by the image processor, a first motion vector between a first initial position including a portion of a subject frame right column macroblock to a first next position including a portion of a subsequent frame left column macroblock; and

generating a compressed copy of the panoramic video file, wherein generating the compressed copy of the panoramic video file comprises;

writing the second macroblock in the subject frame right column of macroblocks to the compressed copy of the panoramic video file; and

for the first macroblock in the subject frame left column of macroblocks, writing a cross-edge reference to the second macroblock in the subject frame right column of macroblocks, wherein the cross-edge reference comprises a first pointer to the second macroblock and difference data describing a difference between the first macroblock and the second macroblock; and

replacing, by the image processor, the subject frame right column macroblock with a second reference to the subsequent frame left column macroblock, wherein the second reference comprises a second pointer to the subsequent frame left column macroblock and second difference data describing a second difference between the subject frame right column macroblock and the subsequent frame left column macroblock.

2. The method of claim 1, wherein the panoramic video file further comprises a prior frame, wherein the prior frame comprises a third plurality of pixel values corresponding to positions on the two-dimensional grid, and wherein the prior frame is positioned immediately before the subject frame according to the temporal sequence of frames of the panoramic video file, the method further comprising:

dividing, by the image processor, the prior frame into a third plurality of macroblocks, wherein the third plurality of macroblocks comprises a prior frame left column of macroblocks positioned on a left side of the prior frame and a prior frame right column of macroblocks positioned on a right side of the prior frame;

identifying, by the image processor, a second motion vector between a second initial position including a portion of a prior frame right column macroblock to a second next position including a portion of a second subject frame left column macroblock; and

replacing, by the image processor, the second subject frame left column macroblock with a third reference to the prior frame right column macroblock, wherein the third reference comprises a third pointer to the prior frame right column macroblock and third difference data describing a second difference between the second subject frame left column macroblock and the prior frame right column macroblock.

3. A system for compressing video, the system comprising:

an image processor, the image processor comprising at least one processor and operatively associated non-transitory memory, wherein the image processor is programmed to;

receive a first frame comprising a plurality of pixel values arranged according to a two-dimensional grid;

determine a first pixel block positioned in a first edge column adjacent a first edge of the first frame;

determine a second pixel block positioned in a second edge column adjacent a second edge of the first frame opposite the first edge;

identify a spatial continuity between the first pixel block and the second pixel block;

replace the first pixel block positioned in the first edge column adjacent the first edge of the first frame with a first cross-edge reference to the second pixel block positioned in the second edge column adjacent the second edge of the first frame opposite the first edge;

receive a second frame, wherein the second frame is after the first frame according to a first temporal sequence of a video comprising the first frame and the second frame;

determine a third pixel block positioned in a third edge column adjacent a first edge of the second frame, wherein the first edge of the second frame and the first edge of the first frame have equivalent positions on the two-dimensional grid;

identify a first motion vector between the third pixel block and a fourth pixel block selected from the second edge column of the first frame; and

replace the fourth pixel block with a second cross-edge reference to the third pixel block.

4. The system of claim 3, wherein the image processor is further programmed to:

receive a third frame, wherein the third frame is prior to the first frame according to a second temporal sequence of the video further comprising the third frame;

determine a fifth pixel block positioned in a fourth edge column adjacent a first edge of the third frame, wherein the first edge of the third frame and the first edge of the first frame have equivalent positions on the two-dimensional grid;

identify a second motion vector between the fifth pixel block and a sixth pixel block selected from the second edge column of the first frame; and

replace the sixth pixel block with a third cross-edge reference to the fifth pixel block.

5. The system of claim 4, wherein the third frame is adjacent the first frame according to the second temporal sequence of the video.

6. The system of claim 3, wherein the first cross-edge reference to the second pixel block comprises an indication of the first pixel block and difference data describing a difference between the first pixel block and the second pixel block.

7. The system of claim 3, wherein the image processor is further programmed to:

determine a fifth pixel block positioned in a first edge row adjacent a third edge of the first frame;

determine a sixth pixel block positioned in a second edge row adjacent a fourth edge of the first frame opposite the third edge;

identify a second spatial continuity between the fifth pixel block and the sixth pixel block; and

replace the fifth pixel block with a third cross-edge reference to the sixth pixel block.

8. The system of claim 3, wherein the first pixel block comprises a 16 by 16 square of pixel values selected from the plurality of pixel values.

9. The system of claim 3, wherein identifying the spatial continuity between the first pixel block and the second pixel block comprises determining that at least one pixel value of the first pixel block is equal to at least one pixel value of the second pixel block.

10. A method comprising:

receiving a first frame comprising a plurality of pixel values arranged according to a two-dimensional grid;

determining a first pixel block positioned in a first edge column adjacent a first edge of the first frame;

determining a second pixel block positioned in a second edge column adjacent a second edge of the first frame opposite the first edge;

identifying a spatial continuity between the first pixel block and the second pixel block;

replacing the first pixel block positioned in the first edge column adjacent the first edge of the first frame with a first cross-edge reference to the second pixel block positioned in the second edge column adjacent the second edge of the first frame opposite the first edge;

receiving a second frame, wherein the second frame is after the first frame according to a first temporal sequence of a video comprising the first frame and the second frame;

determining a third pixel block positioned in a third edge column adjacent a first edge of the second frame, wherein the first edge of the second frame and the first edge of the first frame have equivalent positions on the two-dimensional grid;

identifying a first motion vector between the third pixel block and a fourth pixel block selected from the second edge column of the first frame; and

replacing the fourth pixel block with a second cross-edge reference to the third pixel block.

11. The method of claim 10, further comprising:

receiving a third frame, wherein the third frame is prior to the first frame according to a second temporal sequence of the video further comprising the third frame;

determining a fifth pixel block positioned in a fourth edge column adjacent a first edge of the third frame, wherein the first edge of the third frame and the first edge of the first frame have equivalent positions on the two-dimensional grid;

identifying a second motion vector between the fifth pixel block and a sixth pixel block selected from the second edge column of the first frame; and

replacing the sixth pixel block with a third cross-edge reference to the fifth pixel block.

12. The method of claim 11, wherein the third frame is adjacent the first frame according to the second temporal sequence of the video.

13. The method of claim 10, wherein the second cross-edge reference to the third pixel block comprises an indication of the fourth pixel block and difference data describing a difference between the third pixel block and the fourth pixel block.

14. The method of claim 10, wherein the first cross-edge reference to the second pixel block comprises an indication of the first pixel block and difference data describing a difference between the first pixel block and the second pixel block.

15. The method of claim 10, further comprising:

determining a fifth pixel block positioned in a first edge row adjacent a third edge of the first frame;

determining a sixth pixel block positioned in a second edge row adjacent a fourth edge of the first frame opposite the third edge;

identifying a second spatial continuity between the fifth pixel block and the sixth pixel block; and

replacing the fifth pixel block with a third cross-edge reference to the sixth pixel block.

16. The method of claim 10, wherein the first pixel block comprises a 16 by 16 square of pixel values selected from the plurality of pixel values.

17. The method of claim 10, wherein identifying the spatial continuity between the first pixel block and the second pixel block comprises determining that at least one pixel value of the first pixel block is equal to at least one pixel value of the second pixel block.