Rebinning methods and arrangements for use in compressing image-based rendering (IBR) data

• US 7,065,260 B2
• Filed: 10/13/2004
• Issued: 06/20/2006
• Est. Priority Date: 10/27/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Yunnan, Zhang, Cha, Li, Jin
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US10/963,882
• Publication Number:
  US 20050063602A1
• Time in Patent Office: 615 Days
• Field of Search: 382/284, 382/285, 382/289, 382/293, 382/154, 382/232, 382/239, 382/240, 382/248, 345/427, 345/419, 345/656, 345/634, 345/635, 345/638, 345/630, 345/629, 345/619, 348/36, 348/38, 348/39, 348/42, 348/46
• US Class Current: 382/284
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/615 using motion compensated temporal filtering [MCTF]
• CPC Class Codes: H04N 19/619 the transform being operated outside the prediction loop
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

Rebinning methods and arrangements are provided that significantly improve the 3D wavelet compression performance of the image based rendering data, such as, e.g., concentric mosaic image data. Through what is essentially a selective cutting and pasting process the image data is divided into stripes that are then used to form a set of multi-perspective panoramas. The rebinning process greatly improves the performance of the cross shot filtering, and thus improves the transform and coding efficiency of 3D wavelet codecs. While the region of support after rebinning may cease to be rectangular in some cases, a padding scheme and an arbitrary shape wavelet coder can be implemented to encode the result data volume of the smart rebinning. With an arbitrary shape wavelet codec, the rebinning outperforms MPEG-2 by 3.7 dB, outperforms direct 3D wavelet coder by 4.3 dB, and outperforms a reference block coder (RBC) by 3.2 dB on certain tested concentric mosaic image scenes. Hence, the rebinning process nearly quadruples the compression ratio for selected scenes. Additional methods and arrangements are provided that include selectively dividing the image data into slits and rebinning the slits into a huge 2D array, which is then compressed using conventional still image codecs, such as, JPEG.

Claims

1. A method comprising:

selectively aligning at least two image based rendering (IBR) image data along a specific direction;

selectively rebinning the aligned IBR image data to form a multi-perspective panorama that includes an arbitrary region of support, wherein the rebinning includes subdividing each of the IBR image data into a plurality of portions and combining a specific portion selected from each IBR image data to form a portion of the multi-perspective panorama;

determining at least one displacement vector associated with the at least two IBR image data and subdividing at least one of the IBR image data based on the magnitude of the displacement vector; and

setting the displacement vector for each of the at least two IBR image data to be of different magnitude to support a smart rebinning process for a data volume that includes the arbitrary region of support and an unsupported region of the multi-perspective panorama.

2. The method as recited in claim 1, further comprising:

padding pixel elements of the unsupported region with a pixel value of a boundary between the unsupported region and the arbitrary region of support;

applying a 3D wavelet transform and coding algorithm to the data volume; and

compressing the data volume using a 3D wavelet codec.

3. The method as recited in claim 2, wherein the padding includes:

selecting a smallest rectangular data volume that contains the unsupported region of the multi-perspective panorama; and

padding pixel elements of said smallest rectangular data volume with the pixel value.

4. The method as recited in claim 3, further comprising applying a rectangular 3D wavelet transform and coding algorithm to the pixel elements in said smallest rectangular data volume.

5. The method as recited in claim 1, further comprising applying a low pass filter to pixel elements in the unsupported region to create a gradual transition in pixel values near the boundary.