CAMERA CALIBRATION WITH LENS DISTORTION FROM LOW-RANK TEXTURES

• US 20120133780A1
• Filed: 12/03/2011
• Published: 05/31/2012
• Est. Priority Date: 11/29/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Matsushita, Yasuyuki, Ma, Yi, Zhang, Zhengdong
• Granted Patent: US 8,818,132 B2
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A “Camera Calibrator” provides various techniques for recovering intrinsic camera parameters and distortion characteristics by processing a set of one or more input images. These techniques are based on extracting “Transform Invariant Low-Rank Textures” (TILT) from input images using high-dimensional convex optimization tools for matrix rank minimization and sparse signal recovery. The Camera Calibrator provides a simple, accurate, and flexible method to calibrate intrinsic parameters of a camera even with significant lens distortion, noise, errors, partial occlusions, illumination and viewpoint change, etc. Distortions caused by the camera can then be automatically corrected or removed from images. Calibration is achieved under a wide range of practical scenarios, including using multiple images of a known pattern, multiple images of an unknown pattern, single or multiple images of multiple patterns, etc. Significantly, calibration is achieved without extracting or manually identifying low-level features such as corners or edges from the calibration images.

Claims

1. A method for estimating camera characteristics, comprising using a computing device to perform steps for:

receiving one or more input images captured by the same camera;

selecting a region of each input image containing alternate views of the same arbitrary regular structure; and

processing each selected region using an iterative convex optimization process that models the selected region as a low-rank matrix and repeatedly estimates and updates camera lens distortions and intrinsic camera parameters that would cause the deformation of the arbitrary regular structure in the modeled region until a rank of the matrix is minimized.

2. The method of claim 1 further comprising performing an automated self-calibration of the camera using the estimated lens distortions and intrinsic camera parameters.

3. The method of claim 1 further comprising correcting distortions in one or more additional images captured by the camera based on the estimated lens distortions and intrinsic camera parameters.

4. The method of claim 1 wherein the intrinsic parameters of the camera include a focal length, skew, and a principal point of each input image.

5. The method of claim 1 further comprising steps for:

recovering a low-rank texture from each minimized low-rank matrix; and

estimating extrinsic parameters associated with each image from two or more of the low-rank textures, and wherein the extrinsic parameters include one or more of rotation and translation of the images relative to the camera.

6. The method of claim 5 further comprising steps for rectifying one or more of the input images based on the estimated extrinsic parameters.

7. The method of claim 5 further comprising performing an automated self-calibration for the camera using the estimated lens distortions, the intrinsic camera parameters, and the extrinsic camera parameters.

8. The method of claim 1 wherein selecting the region of each input image containing the arbitrary regular structure comprises selecting a region that approximately corresponds to a portion of the image that contains the arbitrary regular structure.

9. The method of claim 5 further comprising steps for rectifying one or more of the low-rank textures, and outputting the rectified low-rank textures as corrected versions of the corresponding selected region of each input image.

10. The method of claim 1 wherein the arbitrary regular structure is a known pattern having known deformations.

11. A system for modeling camera parameters, comprising:

a device for receiving one or more input images;

a device for selecting a region of each input image containing alternate views of the same arbitrary regular structure;

a device for processing each selected region using an iterative convex optimization process that models the selected region as a low-rank matrix and repeatedly estimates and updates camera lens distortions and intrinsic camera parameters that would cause the deformation of the arbitrary regular structure in the modeled region until a rank of the matrix is minimized;

wherein the intrinsic parameters include a focal length, skew, and a principal point of each input image;

a device for recovering a low-rank texture from each selected image region based on the corresponding minimized low-rank matrix; and

given two or more input images, a device for estimating extrinsic parameters associated with each image from two or more of the low-rank textures, said extrinsic parameters including one or more of rotation and translation of the images relative to the camera.

12. The system of claim 11 further comprising performing an automated self-calibration of the camera using the estimated lens distortions and the intrinsic camera parameters.

13. The system of claim 11 further comprising correcting distortions in one or more additional images captured by the camera based on the estimated lens distortions and intrinsic camera parameters.

14. The system of claim 11 further comprising rectifying one or more of the input images based on the estimated extrinsic parameters.

15. The system of claim 11 wherein selecting the region of each input image containing the arbitrary regular structure comprises selecting a region that approximately corresponds to a portion of the image that contains the arbitrary regular structure.

16. A computer-readable storage device having computer executable instructions stored therein for estimating camera parameters, said instructions comprising:

a program module for receiving one or more input images;

a program module for selecting a region of each input image containing alternate views of the same arbitrary regular structure;

a program module for processing each selected region using an iterative convex optimization process that models the selected region as a low-rank matrix and repeatedly estimates and updates camera lens distortions and intrinsic camera parameters that would cause the deformation of the arbitrary regular structure in the modeled region until a rank of the matrix is minimized;

wherein the intrinsic camera parameters include a focal length, skew, and a principal point of each input image;

a program module for recovering a low-rank texture from each selected image region based on the corresponding minimized low-rank matrix; and

given two or more input images, a program module for estimating extrinsic parameters associated with each image from two or more of the low-rank textures, said extrinsic parameters including one or more of rotation and translation of the images relative to the camera.

17. The computer-readable storage device of claim 16 further comprising instructions for performing an automated self-calibration of the camera using the estimated lens distortions and the intrinsic camera parameters.

18. The computer-readable storage device of claim 16 further comprising instructions for correcting distortions in one or more additional images captured by the camera based on the estimated lens distortions and intrinsic camera parameters.

19. The computer-readable storage device of claim 16 further comprising instructions for rectifying one or more of the input images based on the estimated extrinsic parameters.

20. The computer-readable storage device of claim 16 wherein selecting the region of each input image containing the arbitrary regular structure comprises selecting a region that approximately corresponds to a portion of the image that contains the arbitrary regular structure.