CALIBRATION AND REGISTRATION OF CAMERA ARRAYS USING A SINGLE CIRCULAR GRID OPTICAL TARGET

• US 20140168378A1
• Filed: 12/14/2012
• Published: 06/19/2014
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hall, Michael
• Granted Patent: US 9,210,404 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Methods for determining intrinsic parameters associated with a camera and for registering cameras within a camera array are described. In some embodiments, a camera may be calibrated using an optical target that is positioned at a depth outside the depth of focus of the camera. The optical target may include a grid of symmetrical shapes (e.g., a rectilinear grid of circles or tilted squares). The intrinsic parameters for the camera may be determined using an iterative calibration process in which a cost function is used to evaluate the straightness of lines intersecting rows and columns associated with centroids of the symmetrical shapes. In some embodiments, the registration of a color camera with a depth camera may include mapping centroids identified within a first color image captured by the color camera with corresponding centroids identified within an undistorted intensity image captured by the depth camera.

Claims

1. A method for calibrating a camera array, comprising:

capturing a first image of an environment including an optical target using a first camera, the optical target includes a plurality of symmetrical shapes;

determining a plurality of centroid locations within the first image corresponding with the plurality of symmetrical shapes;

determining one or more distortion parameters associated with the first camera based on the plurality of centroid locations;

generating an undistorted image using the first image and the one or more distortion parameters; and

outputting the undistorted image.

2. The method of claim 1, wherein:

the optical target is positioned within the environment at a depth outside a depth of focus of the first camera.

3. The method of claim 1, wherein:

the plurality of symmetrical shapes includes a plurality of circles.

4. The method of claim 1, wherein:

the determining a plurality of centroid locations includes thresholding the first image and determining a center of mass for each of the plurality of symmetrical shapes.

5. The method of claim 1, further comprising:

determining a center of distortion based on the one or more distortion parameters; and

determining a principal point based on the center of distortion.

6. The method of claim 1, further comprising:

determining an average distance between neighboring centroids within the undistorted image; and

determining a focal length associated with the first camera based on the average distance between neighboring centroids.

7. The method of claim 1, wherein:

the determining one or more distortion parameters associated with the first camera includes optimizing the one or more distortion parameters using a cost function that evaluates the straightness of lines intersecting rows and columns associated with the plurality of centroids.

8. The method of claim 1, further comprising:

capturing a second image of the optical target using a second camera different from the first camera;

determining a registration mapping for the second image onto the undistorted image;

generating a registered image based on the second image and the registration mapping; and

outputting the registered image.

9. The method of claim 8, further comprising:

determining depth information associated with the undistorted image, the first camera comprises a depth camera, the registration mapping includes one or more registration mapping coefficients; and

adjusting the one or more registration mapping coefficients based on the depth information.

10. The method of claim 1, wherein:

the outputting includes displaying the undistorted image;

the first camera comprises a depth camera; and

the second camera comprises a color camera.

11. A system for calibrating a camera array, comprising:

a depth camera, the depth camera captures a first image of an environment including an optical target, the optical target includes a plurality of symmetrical shapes; and

one or more processors, the one or more processors in communication with the depth camera, the one or more processors determine a plurality of centroid locations within the first image corresponding with the plurality of symmetrical shapes, the one or more processors determine one or more distortion parameters associated with the first camera based on the plurality of centroid locations, the one or more processors generate an undistorted image using the first image and the one or more distortion parameters.

12. The system of claim 11, wherein:

the optical target is positioned within the environment at a depth outside a depth of focus of the depth camera.

13. The system of claim 11, wherein:

the plurality of symmetrical shapes includes a plurality of circles.

14. The system of claim 11, further comprising:

a color camera in communication with the one or more processors, the color camera captures a second image of the environment including the optical target, the one or more processors determine a registration mapping for the second image onto the undistorted image, the one or more processors generate a registered image by applying the registration mapping to the second image.

15. One or more storage devices containing processor readable code for programming one or more processors to perform a method for calibrating a camera array comprising the steps of:

capturing a first image of an environment including an optical target using a depth camera, the optical target includes a plurality of symmetrical shapes;

determining a plurality of centroid locations within the first image corresponding with the plurality of symmetrical shapes;

determining one or more intrinsic parameters associated with the depth camera based on the plurality of centroid locations;

generating an undistorted image using the first image and the one or more intrinsic parameters;

capturing a second image of the optical target using a color camera;

determining a registration mapping for the second image onto the undistorted image;

generating a registered image using the second image and the registration mapping; and

outputting the undistorted image and the registered image.

16. The one or more storage devices of claim 15, wherein:

the optical target is positioned within the environment at a depth outside a depth of focus of the depth camera.

17. The one or more storage devices of claim 15, wherein:

the plurality of symmetrical shapes includes a plurality of circles.

18. The one or more storage devices of claim 15, wherein:

the determining a plurality of centroid locations includes thresholding the first image and determining a center of mass for each of the plurality of symmetrical shapes.

19. The one or more storage devices of claim 15, wherein:

the outputting the undistorted image and the registered image includes displaying the undistorted image.

20. The one or more storage devices of claim 15, further comprising:

determining an average distance between neighboring centroids within the undistorted image; and

determining a focal length associated with the first camera based on the average distance between neighboring centroids.