Method and apparatus for encoding video by changing frame resolution

• US 9,210,420 B1
• Filed: 02/07/2014
• Issued: 12/08/2015
• Est. Priority Date: 04/28/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jia, Wei
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Pham, Nam
• Application Number:
  US14/175,127
• Time in Patent Office: 669 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques

Abstract

Encoding a video signal having a plurality of frames is described. Test frames in the sequence of test frames have an original resolution. A variance for each test frame in the sequence is calculated, as is a first peak signal-to-noise ratio (PSNR) for each test frame using the original resolution. A threshold is determined using the variances and first PSNRs. The threshold is provided to an encoder to select a frame resolution for a first frame of the plurality of frames, the frame resolution being one of the original resolution or a resolution different from the original resolution.

Claims

1. A method for encoding a video signal having a plurality of frames, each frame having a plurality of blocks, the method comprising:

calculating an inter-prediction variance for each test frame in a sequence of test frames, the test frames in the sequence of test frames having an original resolution;

calculating a first peak signal-to-noise ratio (PSNR) for each test frame using the original resolution;

determining an inter threshold using the inter-prediction variances and first PSNRs; and

providing the inter threshold to an encoder to select a frame resolution for a first frame of the plurality of frames, the frame resolution being one of the original resolution or a resolution different from the original resolution.

2. The method of claim 1, further comprising:

calculating an intra-prediction variance for each test frame in the sequence of test frames;

calculating a second PSNR for each test frame using a lower resolution that is less than the original resolution;

determining an intra threshold using the intra-prediction variances, first PSNRs, and second PSNRs; and

providing the intra threshold to the encoder to select a frame resolution for a second frame of the plurality of frames, the frame resolution being one of the original resolution or the resolution different from the original resolution.

3. The method of claim 2 wherein calculating the second PSNR for each test frame comprises:

selecting a test frame from the sequence of test frames;

determining a downsampled test frame at the lower resolution from the selected test frame;

encoding the downsampled test frame;

decoding the encoded downsampled test frame;

determining a decoded upsampled test frame at the original resolution using the decoded downsampled test frame; and

calculating the second PSNR using the selected test frame and the decoded upsampled test frame.

4. The method of claim 3 wherein encoding the downsampled test frame uses a target bitrate.

5. The method of claim 2 wherein determining the intra threshold using the intra-prediction variances, the first PSNRs and the second PSNRs comprises:

identifying a first series comprising the intra-prediction variances and the first PSNRs;

identifying a second series comprising the intra-prediction variances and the second PSNRs;

determining an intersection between the first series and the second series; and

determining the intra threshold using the intersection.

6. The method of claim 5 wherein determining the intersection between the first series and the second series comprises:

determining a first fitted curve from the first series;

determining a second fitted curve from the second series; and

determining the intersection as a value at which the first fitted curve and the second fitted curve intersect.

7. The method of claim 2 wherein the second frame is a frame of the plurality of frames to be encoded without reference to another frame, the method further comprising:

determining a variance for the second frame;

if the variance exceeds the intra threshold;

selecting a frame resolution for the second frame that is less than the original resolution;

if the variance does not exceed the intra threshold;

selecting the frame resolution for the second frame as the original resolution; and

encoding the second frame using the frame resolution.

8. The method of claim 7, further comprising:

identifying a third frame from the plurality of frames, the third frame being a frame after the second frame that is to be at least partially encoded with reference to another frame; and

encoding the third frame using the frame resolution for the second frame.

9. The method of claim 8, further comprising, upon a condition that the frame resolution is less than the original resolution:

determining a variance set for the third frame;

selecting a pre-determined resolution using the variance set; and

encoding a fourth frame subsequent to the third frame using the pre-determined resolution and without reference to another frame.

10. The method of claim 9 wherein determining the variance set for the third frame comprises:

calculating an inter-prediction variance for at least some of the blocks in the third frame;

calculating an intra-prediction variance for at least some of the blocks in the third frame;

calculating an inter-prediction variance total for the third frame;

calculating an intra-prediction variance total for the third frame; and

normalizing the inter-prediction variance total and the intra-prediction variance total.

11. The method of claim 10 wherein calculating the inter-prediction variance total comprises:

calculating the inter-prediction variance total using a smallest variance for each block, the smallest variance of each block being the smallest of the inter-prediction variance and the intra-prediction variance.

12. The method of claim 10 wherein selecting the pre-determined resolution comprises, upon a condition that the inter threshold is less than the normalized inter-prediction variance total and the intra threshold is less than the normalized intra-prediction variance total:

selecting the pre-determined resolution that is less than the original resolution.

13. The method of claim 10 wherein selecting the pre-determined resolution comprises, upon a condition that the inter threshold is less than the normalized inter-prediction variance total and the intra threshold is not less than the normalized intra-prediction variance total:

selecting the pre-determined resolution as the original resolution.

14. The method of claim 7, further comprising:

encoding each inter predicted frame within a set of inter predicted frames after the second frame, each encoded inter predicted frame having a variance;

if a majority of the variances of the set of inter predicted frames are below a variance threshold;

selecting a pre-determined resolution as the original resolution;

if the majority of the variances of the set of inter predicted frames are above the variance threshold;

selecting a pre-determined resolution that is less than the original resolution; and

encoding a next frame after a second one of the set of inter predicted frames using the pre-determined resolution and without reference to another frame.

15. The method of claim 1 wherein calculating the first PSNR comprises:

selecting a test frame from the sequence of frames;

encoding the selected test frame;

decoding the encoded test frame; and

calculating the first PSNR using the selected test frame and the decoded test frame.

16. The method of claim 15 wherein encoding the selected test frame uses a target bitrate.

17. The method of claim 1 wherein determining the inter threshold comprises:

selecting into a candidate variance set the inter-prediction variance for each test frame having a first PSNR greater than a first PSNR threshold;

selecting a largest variance from the candidate variance set; and

determining the inter threshold using the largest variance.

18. An apparatus for encoding a video signal having at least one frame, each frame having a plurality of blocks, each block having a plurality of pixels, the apparatus comprising:

a memory; and

a processor configured to execute instructions stored in the memory to;

calculate a variance for each test frame in a sequence of test frames, the test frames in the sequence of test frames having an original resolution;

calculate a first peak signal-to-noise ratio (PSNR) for each test frame using the original resolution;

determine a threshold by;

when the variances are intra-prediction variances;

calculating a second PSNR for each test frame using a lower resolution that is less than the original resolution; and

determining the threshold as an intra threshold using the variances, first PSNRs and second PSNRs;

when the variances are inter-prediction variances;

determining the threshold as an inter threshold using the variances and first PSNRs; and

provide the threshold to an encoder to select a frame resolution for a first frame of the plurality of frames, the frame resolution being one of the original resolution or a resolution different from the original resolution.

19. The apparatus of claim 18 wherein the resolution different from the original resolution is one of a plurality of resolutions having an image resolution lower than the original resolution.

20. The apparatus of claim 18 wherein the first frame is a frame of the plurality of frames to be encoded without reference to another frame and further comprising the encoder configured to:

determine a variance for the first frame;

if the variance exceeds the intra threshold;

select a frame resolution for the first frame that is less than the original resolution;

if the variance does not exceed the intra threshold;

select the frame resolution for the first frame as the original resolution; and

encode the first frame using the frame resolution.