Spatial scalability in 3D sub-band decoding of SDMCTF-encoded video

• US 20060114993A1
• Filed: 05/10/2005
• Published: 06/01/2006
• Est. Priority Date: 07/13/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Shipeng, Wu, Feng, Xu, Jizheng, Xiong, Ruiqin
• Granted Patent: US 8,374,238 B2
• Time in Patent Office: Days
• US Class Current: 375/240.110
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/615 using motion compensated temporal filtering [MCTF]
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details

Abstract

Techniques and tools are described for scalable video coding and decoding. For example, a 3D sub-band decoder receives video encoded using spatial-domain motion-compensated temporal filtering at a first spatial resolution. The decoder decodes at least part of the video for output at a second spatial resolution lower than the first spatial resolution. The decoder uses any of several techniques to improve performance by devoting extra computational resources to the decoding, devoting extra buffer resources to storing reference picture information, and/or considering spatial high-pass sub-band information.

Claims

1. A method comprising:

receiving video encoded by SDMCTF at a first spatial resolution; and

decoding at least part of the video with IBIMCTF for output at a second spatial resolution lower than the first spatial resolution, including performing interpolation for one or more reference pictures, wherein the interpolation is not in-band interpolation on buffered spatial low-pass sub-band information.

2. The method of claim 1 wherein the decoding comprises:

performing an inverse spatial sub-band transform on the buffered spatial low-pass sub-band information;

performing an overcomplete spatial sub-band transform on results of the inverse spatial sub-band transform, wherein the interpolation applies to results of the overcomplete spatial sub-band transform; and

performing motion-compensated prediction.

3. The method of claim 2 wherein the inverse spatial sub-band transform is an IDWT, and wherein the overcomplete spatial sub-band transform is an overcomplete DWT.

4. The method of claim 1 wherein the decoding comprises:

performing an inverse spatial sub-band transform on the buffered spatial low-pass sub-band information, wherein the interpolation applies to results of the inverse spatial sub-band transform;

performing a DSDWT on results of the interpolation; and

performing motion-compensated prediction.

5. The method of claim 4 wherein the inverse spatial sub-band transform is an IDWT, and wherein the DSDWT is an m-sampled overcomplete DWT.

6. The method of claim 1 wherein motion-compensated prediction in the IBIMCTF uses motion information from the SDMCTF.

7. The method of claim 1 wherein the decoding further comprises performing an inverse spatial sub-band transform on the buffered spatial low-pass sub-band information but not any spatial high-pass sub-band information, and wherein the interpolation follows the inverse spatial sub-band transform.

8. The method of claim 1 wherein the first spatial resolution is double the second spatial resolution horizontally and vertically.

9. A method comprising:

receiving video encoded by SDMCTF at a first spatial resolution; and

decoding at least part of the video with SDIMCTF for output at a second spatial resolution lower than the first spatial resolution, wherein one or more reference pictures for the SDIMCTF are reconstructed from received spatial low-pass sub-band information but not from received spatial high-pass sub-band information associated with resolution enhancement up to the first spatial resolution.

10. The method of claim 9 wherein the decoding comprises:

performing an inverse spatial sub-band transform on the received spatial low-pass sub-band information but not any spatial high-pass sub-band information;

performing spatial-domain interpolation on results of the inverse spatial sub-band transform;

performing motion-compensated prediction for the SDIMCTF using results of the spatial-domain interpolation as reference picture information; and

performing a spatial sub-band transform on results of the motion-compensated prediction.

11. The method of claim 10 wherein the inverse spatial sub-band transform is an IDWT, and wherein the spatial sub-band transform is a DWT.

12. The method of claim 10 wherein one or more buffers store the received spatial low-pass sub-band information but not spatial high-pass sub-band information.

13. The method of claim 9 wherein the decoding comprises:

performing spatial-domain interpolation on buffered spatial-domain information; and

performing motion-compensated prediction for the SDIMCTF using results of the spatial-domain interpolation as reference picture information.

14. The method of claim 12 wherein for an initial level of the SDIMCTF the buffered spatial-domain information is generated from the received spatial low-pass sub-band information but not any spatial high-pass sub-band information, and wherein for one or more subsequent levels of the SDIMCTF the buffered spatial-domain information retains spatial high-pass sub-band information generated as intermediate results of the SDIMCTF.

15. A method comprising:

receiving video encoded by SDMCTF at a first spatial resolution; and

decoding at least part of the video with SDIMCTF for output at a second spatial resolution lower than the first spatial resolution, wherein the decoding includes using spatial high-pass sub-band information associated with resolution enhancement up to the first spatial resolution to improve motion compensation performance in the SDIMCTF but not using the spatial high-pass sub-band information to enhance resolution up to the first spatial resolution.

16. The method of claim 15 wherein the decoding comprises:

performing spatial-domain interpolation on buffered spatial-domain information; and

performing motion-compensated prediction for the SDIMCTF using results of the spatial-domain interpolation as reference picture information.

17. The method of claim 16 wherein the spatial high-pass sub-band information is generated as intermediate results of the SDIMCTF and retained as the buffered spatial-domain information.

18. The method of claim 16 wherein the spatial high-pass sub-band information is part of the received video, and wherein the buffered spatial-domain information is generated from received spatial low-pass sub-band information and the received spatial high-pass sub-band information.

19. The method of claim 15 wherein using the spatial high-pass sub-band information helps control drift in the SDIMCTF.

20. The method of claim 15 wherein one or more gain factors associated with the spatial high-pass sub-band information account for how the spatial high-pass sub-band information reduces distortion in the output at the second spatial resolution.