System and method for head size equalization in 360 degree panoramic images

• US 7,184,609 B2
• Filed: 04/14/2005
• Issued: 02/27/2007
• Est. Priority Date: 06/28/2002
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Cohen, Michael, Liu, Zicheng, Cutler, Ross
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US11/106,311
• Publication Number:
  US 20050180656A1
• Time in Patent Office: 684 Days
• Field of Search: 382275-276, 382/284, 382/297, 382/299, 382/289, 382/312, 348/36, 348/629, 348/648, 348/660, 348/207.99, 359/725, 345/427, 345/629, 345647-648, 345/660
• US Class Current: 382/276
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A real-time approximately 360 degree image correction system and a method for alleviating distortion and perception problems in images captured by omni-directional cameras. In general, the real-time panoramic image correction method generates a warp table from pixel coordinates of a panoramic image and applies the warp table to the panoramic image to create a corrected panoramic image. The corrections are performed using a parametric class of warping functions that include Spatially Varying Uniform (SVU) scaling functions. The SVU scaling functions and scaling factors are used to perform vertical scaling and horizontal scaling on the panoramic image pixel coordinates. A horizontal distortion correction is performed using the SVU scaling functions at at least two different scaling factors. This processing generates a warp table that can be applied to the panoramic image to yield the corrected panoramic image. In one embodiment the warp table is concatenated with a stitching table used to create the panoramic image.

Claims

1. A method for performing real-time correction of a panoramic image, comprising:

obtaining panoramic pixel coordinates from an a approximately 360 degree panoramic image;

generating a warp table by applying a Spartially Varying Uniform (SVU) scaling function to the panoramic pixel coordinates wherein generating a warp table further comprises;

specifying source curves;

specifying a warping factor; and

computing target curves using the source curves and the warping factor wherein the source curves further comprise a bottom source curve, wherein the bottom source curve is defined by the horizontal feature geometry, and a top source curve;

vertically scaling the panoramic image pixel coordinates using the SVU scaling function to create vertically-scaled panoramic image pixel coordinates wherein vertically scaling comprises using a main scaling factor to scale the panoramic image pixel coordinates in the vertical direction and wherein the main scaling factor is computed using the source curves and target curves;

horizontally scaling the vertically-scaled panoramic image pixel coordinates using the SVU scaling function to create a preliminary warp table containing preliminary pixel coordinates; and

performing horizontal distortion correction on the preliminary warp table using the SVU scaling function to create the warp table; and

warping the panoramic image using the warp table and the geometry of horizontal features in the panoramic image to create a corrected panoramic image.

2. The method as set forth in claim 1 wherein the horizontal feature is a table and wherein the camera capturing the panoramic image is sitting in approximately the middle of the table.

3. The method as set forth in claim 2 wherein the bottom source curve is defined by the equations:

4. The method as set forth in claim 1 wherein the top source curve is based on a plane approximately touching the top of the heads of people sitting around a table with the camera capturing the panoramic image sitting in the middle.

5. The method as set forth in claim 4 wherein the top source curve is defined by the equations:

6. The method as set forth in claim 1, wherein horizontally scaling further comprises using a main scaling factor for each vertical scanline to scale the panoramic image pixel coordinates in the horizontal direction.

7. The method as set forth in claim 6, wherein the main scaling factor is computed using source curves and target curves.

8. The method as set forth in claim 1, wherein performing horizontal distortion correction further comprises:

dividing the preliminary warp table into a plurality of sections; and

scaling preliminary pixel coordinates contained in each of the sections in the vertical direction using at least two different scaling factors.

9. A method for performing real-time correction of a panoramic image, comprising:

obtaining panoramic pixel coordinates from an approximately 360 degree panoramic image;

generating a warp table by applying a Spatially Varying Uniform (SVU) scaling function to the panoramic pixel coordinates wherein the warp table is computed from user-specified table dimensions and camera parameters; and

warping the panoramic image using the warp table and the geometry of horizontal features in the panoramic image to create a corrected panoramic image.

10. A computer-readable medium having computer-executable instructions for performing the method recited in claim 9.

11. A method for correcting distortion of an approximately 360 degree panoramic image, comprising:

inputting a set of images;

inputting image pixel coordinates corresponding to pixel coordinates of an imaginary panoramic image created from the input set of images;

using Spatially Varying Uniform (SVU) scaling functions having scaling factors to generate a warp table containing corrected pixel coordinates which correct for image distortion;

concatenating a stitching table used to create the imaginary panoramic image with the warp table; and

creating a corrected panoramic image from the input set of images using the concatenated warping and stitching table.

12. The method as set forth in claim 11 wherein the corrected panoramic image is created in real time.

13. The method as set forth in claim 12, further comprising concatenating the warp table with a stitching table used to stitch together images to create the panoramic image.

14. The method as set forth in claim 12 wherein the corrected panoramic image is corrected to increase the size of objects that are further away from the camera capturing the sequence of images.

15. A computer-readable medium having computer-executable instructions for real-time correction and viewing of a panoramic image, comprising:

generating a warp table to warp the pixels in a sequence of images used to create a panoramic image, further comprising;

specifying source curves on the panoramic image;

specifying a warping factor corresponding to an amount of warping applied to tie panoramic Image pixel coordinates;

computing target curves using the source curves and the warping factor;

computing a main scaling factor using the source curves and the target curves;

using the main scaling factor to scale the panoramic image pixel coordinates to generate the warp table;

concatenating the warp table with a stitching table used to create the panoramic image and

applying the concatenated warp and stitching table while stitching the sequence of images into a corrected panoramic image.

16. The computer-readable medium of claim 15 wherein the computer-readable medium is embedded in an omni-directional camera.