Resampling and picture resizing operations for multi-resolution video coding and decoding

• US 9,319,729 B2
• Filed: 05/30/2014
• Issued: 04/19/2016
• Est. Priority Date: 01/06/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sullivan, Gary J.
• Primary Examiner(s): Tran, Trang U
• Application Number:
  US14/292,073
• Publication Number:
  US 20140269912A1
• Time in Patent Office: 690 Days
• Field of Search: 348/581, 348/441, 348/443, 348445-446, 348/448, 348458-459, 348714-721, 348/413, 348415-416, 348/401, 348/402, 348/409, 348/410, 345660-670, 345/555, 345/560, 382/298, 382/307, 382/297, 358/451, 358/426.26, 358/240.26, 358/240.29
• US Class Current: 1/1
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 7/0102 involving the resampling of the incoming video signal

Abstract

Techniques and tools for high accuracy position calculation for picture resizing in applications such as spatially-scalable video coding and decoding are described. In one aspect, resampling of a video picture is performed according to a resampling scale factor. The resampling comprises computation of a sample value at a position i,j in a resampled array. The computation includes computing a derived horizontal or vertical sub-sample position x or y in a manner that involves approximating a value in part by multiplying a 2ⁿ value by an inverse (approximate or exact) of the upsampling scale factor. The approximating can be a rounding or some other kind of approximating, such as a ceiling or floor function that approximates to a nearby integer. The sample value is interpolated using a filter.

Claims

1. In a computer system that implements a video encoder or video decoder, a method comprising:

determining one or more bandwidth control parameters, wherein a syntax element in a bitstream of encoded data indicates the one or more bandwidth control parameters;

based at least in part on the one or more bandwidth control parameters and phase information, computing plural filter tap values of an upsampling filter;

buffering video picture data at a first spatial resolution;

upsampling at least part of the video picture data to a second spatial resolution higher than the first spatial resolution using the upsampling filter; and

using the upsampled video picture data for prediction of sample values of a picture at the second spatial resolution.

2. The method of claim 1 wherein the one or more bandwidth control parameters comprise a blurriness control parameter.

3. The method of claim 1 wherein the one or more bandwidth control parameters are integerized.

4. The method of claim 1 wherein the upsampling filter is a Catmull-Rom filter.

5. The method of claim 1 wherein the upsampling filter is an approximation of a Lanczos-2 filter.

6. The method of claim 1 wherein the upsampling filter has a smooth impulse response and/or a unity DC response.

7. The method of claim 1 further comprising, during video encoding:

setting the one or bandwidth control parameters; and

outputting the syntax element as part of the bitstream of encoded data.

8. The method of claim 1 further comprising, during video decoding:

receiving the syntax element as part of the bitstream of encoded data; and

setting the one or bandwidth control parameters.

9. The method of claim 1 wherein the computing the plural filter tap values of the upsampling filter uses no trigonometric functions, transcendental functions or irrational-number processing.

10. One or more computer-readable media storing computer-executable instructions for causing a processor programmed thereby to perform scalable video coding or scalable video decoding, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory and magnetic disk, the computer-executable instructions comprising:

instructions for determining one or more bandwidth control parameters, wherein a syntax element in a bitstream of encoded data indicates the one or more bandwidth control parameters;

instructions for, based at least in part on the one or more bandwidth control parameters and phase information, computing plural filter tap values of an upsampling filter;

instructions for buffering video picture data at a first spatial resolution;

instructions for upsampling at least part of the video picture data to a second spatial resolution higher than the first spatial resolution using the upsampling filter; and

instructions for using the upsampled video picture data for prediction of sample values of a picture at the second spatial resolution.

11. The one or more computer-readable media of claim 10 wherein the one or more bandwidth control parameters comprise a blurriness control parameter.

12. The one or more computer-readable media of claim 10 wherein the one or more bandwidth control parameters are integerized.

13. The one or more computer-readable media of claim 10 wherein the upsampling filter is a Mitchell-Netravali-type filter.

14. The one or more computer-readable media of claim 13 wherein the upsampling filter is a Catmull-Rom filter or an approximation of a Lanczos-2 filter.

15. The one or more computer-readable media of claim 10 wherein the upsampling filter has a smooth impulse response and/or a unity DC response.

16. A computer system comprising a processor and memory, wherein the computer system implements:

a buffer configured to store video picture data at a first spatial resolution; and

a video encoder or video decoder configured to;

determine one or more bandwidth control parameters, wherein a syntax element in a bitstream of encoded data indicates the one or more bandwidth control parameters;

based at least in part on the one or more bandwidth control parameters and phase information, compute plural filter tap values of an upsampling filter;

upsample at least part of the video picture data to a second spatial resolution higher than the first spatial resolution using the upsampling filter, wherein the upsampling filter has (a) a single side lobe on each side of a main lobe, (b) a smooth impulse response, and (c) a unity gain DC response; and

use the upsampled video picture data for prediction of sample values of a picture at the second spatial resolution.

17. The computer system of claim 16 wherein the one or more parameters include one or more bandwidth control parameters.

18. The computer system of claim 17 wherein the one or more bandwidth control parameters comprise a blurriness control parameter.

19. The computer system of claim 17 wherein the one or more bandwidth control parameters are integerized.

20. The computer system of claim 16 wherein the upsampling filter is a Mitchell-Netravali-type filter.

21. The computer system of claim 20 wherein the upsampling filter is a Catmull-Rom filter or an approximation of a Lanczos-2 filter.

22. The computer system of claim 16 wherein the upsampling filter has 4 taps with integer tap values or 6 taps with integer tap values.