Resampling and picture resizing operations for multi-resolution video coding and decoding

• US 20070160153A1
• Filed: 01/05/2007
• Published: 07/12/2007
• Est. Priority Date: 01/06/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sullivan, Gary J.
• Granted Patent: US 7,956,930 B2
• Time in Patent Office: Days
• US Class Current: 375/240.290
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 7/0102 involving the resampling of the incoming video signal

Abstract

Techniques and tools for high accuracy position calculation for picture resizing in applications such as spatially-scalable video coding and decoding are described. In one aspect, resampling of a video picture is performed according to a resampling scale factor. The resampling comprises computation of a sample value at a position i,j in a resampled array. The computation includes computing a derived horizontal or vertical sub-sample position x or y in a manner that involves approximating a value in part by multiplying a 2ⁿ value by an inverse (approximate or exact) of the upsampling scale factor. The approximating can be a rounding or some other kind of approximating, such as a ceiling or floor function that approximates to a nearby integer. The sample value is interpolated using a filter.

Claims

1. A method comprising:

performing resampling of image data according to a horizontal resampling scale factor, wherein the resampling comprises computation of a sample value at horizontal position i in a resampled array, and wherein the computation comprises;

computing a derived horizontal sub-sample position x in a manner that is mathematically equivalent in result to the formula x=(i*C+D)>

>

S, wherein C is derived by approximating a value equivalent to 2^S+F multiplied by an inverse of the horizontal resampling scale factor, and wherein F, C, D, and S are integer values.

2. A method comprising:

performing resampling of image data according to a vertical resampling scale factor, wherein the resampling comprises computation of a sample value at vertical position j in an resampled array, and wherein the computation comprises;

computing a derived horizontal sub-sample position y in a manner that is mathematically equivalent in result to the formula y=(j*C+D)>

>

S, wherein C is derived by approximating a value equivalent to 2^S+F multiplied by an inverse of the vertical resampling scale factor, and wherein F, C, D, and S are integer values.

3. A method comprising:

performing upsampling of a video picture according to a horizontal upsampling scale factor and a vertical upsampling scale factor, wherein the upsampling comprises computation of an interpolated sample value at horizontal position i and vertical position j in an upsampled array, and wherein the computation comprises;

computing a derived horizontal sub-sample position x in a manner that is mathematically equivalent in result to the formula x=(i*C+D)>

>

S, wherein C is derived by approximating a value equivalent to 2^S+F multiplied by an inverse of the horizontal upsampling scale factor, and wherein F, C, D, and S are integer values;

computing a derived vertical sub-sample position y in a manner that is mathematically equivalent in result to the formula y=(j*C+D)>

>

S, wherein C is derived by approximating a value equivalent to 2^S+F multiplied by an inverse of the vertical upsampling scale factor; and

interpolating a sample value at the derived sub-sample position x, y.

4. The method of claim 3 wherein the computation further comprises:

selecting a horizontal resampling filter based on F least significant bits of the derived horizontal sub-sample position x; and

selecting lower resolution samples to be filtered based on the remaining more significant bits of the derived horizontal sub-sample position x; and

wherein interpolating a sample value at the derived sub-sample position x, y comprises;

interpolating the sample value based on the selected lower resolution samples and using the selected horizontal resampling filter.

5. The method of claim 4 wherein a horizontal resampling filter applied for at least one value of the F least significant bits of the derived horizontal sub-sample position x is a finite impulse response filter with more than two non-zero filter tap values.

6. The method of claim 5, wherein a horizontal resampling filter applied for all values other than 0 for the F least significant digits of the derived horizontal sub-sample position x is a finite impulse response filter with four non-zero filter tap values.

7. The method of claim 3 wherein the computation further comprises:

selecting a vertical resampling filter based on F least significant bits of the derived vertical sub-sample position y; and

selecting lower resolution samples to be filtered based on the remaining more significant bits of the derived vertical sub-sample position y; and

wherein interpolating a sample value at the derived sub-sample position x, y comprises;

interpolating the sample value based on the selected lower resolution samples and using the selected vertical resampling filter.

8. The method of claim 7 wherein a vertical resampling filter applied for at least one value of the F least significant bits of the derived vertical sub-sample position y is a finite impulse response filter with more than two non-zero filter tap values.

9. The method of claim 8, wherein a vertical resampling filter applied for all values other than 0 for the F least significant digits of the derived vertical sub-sample position x is a finite impulse response filter with four non-zero filter tap values.

10. The method of claim 3 wherein the upsampling is performed using one or more Mitchell-Netravalli resampling filters.

11. The method of claim 3 wherein the upsampling is performed using one or more Catmull-Rom resampling filters.

12. The method of claim 3 wherein at least one of the vertical or horizontal values of F, C, D, or S differs based at least in part on whether the sample value is a chroma sample value or a luma sample value.

13. The method of claim 3 wherein a manner that is mathematically equivalent in result to the formula x=(i*C+D)>

>

S comprises an implementation of the formula x=((i*C+D)>

>

S)+E,where E is an offset.

14. The method of claim 3 wherein the upsampling is performed using one or more resampling filters having filter tap values controlled by a bandwidth control parameter.

15. The method of claim 3 wherein the upsampling is performed in a layered spatially-scalable video decoding process.

16. The method of claim 3 wherein the upsampling is performed in a layered spatially-scalable video encoding process.

17. The method of claim 3 wherein the upsampling is performed for reference picture resampling.

18. The method of claim 3 wherein the value of F is equal to 4 and the value of S is equal to 12.

19. The method of claim 3 wherein the approximating comprises rounding.

20. The method of claim 3 wherein the inverse is an approximate inverse.

21. The method of claim 3 wherein at least one of the F, C, D, and S integer values are different for the horizontal computing than for the vertical computing.

22. A method comprising:

performing upsampling of a video picture according to an upsampling scale factor, wherein the upsampling comprises computation of an interpolated sample value at horizontal position i and vertical position j in an upsampled array, and wherein the computation comprises;

computing a derived horizontal sub-sample position x in a manner mathematically equivalent in result to the formula x=((2^J*i+Q)*C+D)>

>

S, wherein C is derived by approximating a value equivalent to 2^S+F multiplied by an inverse of the upsampling scale factor, and wherein F, C, D, S, J and Q are integers;

computing a derived vertical sub-sample position y in a manner that is mathematically equivalent in result to the formula y=((2^J*j+Q)*C+D)>

>

S; and

interpolating a sample value at the derived sub-sample position x, y.