Interactive camera calibration tool

• US 9,282,326 B2
• Filed: 10/28/2014
• Issued: 03/08/2016
• Est. Priority Date: 10/28/2013
• Status: Active Grant

• Current Assignee: Board of Regents of the University of Michigan (University of Michigan)
• Original Assignee: Board of Regents of the University of Michigan (University of Michigan)
• Inventors: Olson, Edwin B., Strom, Johannes, Richardson, Andrew
• Primary Examiner(s): YENKE, BRIAN P
• Application Number:
  US14/525,709
• Publication Number:
  US 20150130951A1
• Time in Patent Office: 497 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

An interactive camera calibration tool is presented that provides live feedback on the state of the calibration and produces tightly-distributed calibration parameters even when used by novices. Target positions are suggested by the calibration tool. Once the target has been aligned with the target positions, image data of the target is captured and used to compute calibration parameters. This process is repeated until the computed parameters meet the accuracy requirements specified by the user. A novel calibration quality metric is also leveraged to automatically determine whether a calibration is sufficiently accurate.

Claims

1. A computer-assisted method for calibrating a camera, comprising:

(a) initializing parameters of a camera model using image data of a calibration target captured by a camera, where the camera model defines a relationship between an image captured by the camera and a corresponding image without any distortion caused by the camera;

(b) predicting parameters of the camera model for each of a plurality of candidate target positions, thereby yielding an estimated camera model for each of the plurality of candidate target positions;

(c) determining a certainty measure for each of the estimated camera models;

(d) selecting a target position from the plurality of candidate target positions, where the selected target position resulted in the estimated camera model having the smallest certainty measure;

(e) presenting the selected target position on a display device;

(f) capturing, by the camera, image data of the calibration target while the calibration target is positioned proximate to the selected target position;

(g) computing parameters of the camera model using the image data captured by the camera of the calibration target while the calibration target is positioned proximate to the selected target position; and

(h) determining a certainty measure for the camera model using the computed parameters of the camera model.

2. The computer-assisted method of claim 1 further comprises repeating steps (b)-(h) until the certainty measure for the camera model is less than a threshold.

3. The computer-assisted method of claim 2 further comprises capturing, by the camera, image data of a scene subsequent to the calibration process and correcting the image data using the computed parameters of the camera model.

4. The computer-assisted method of claim 1 wherein predicting parameters of the camera model further comprises predicting where visual fiducials on the target would appear in an image using the parameters of the camera model and when the target is oriented in a given candidate target position, and computing parameters for an estimated camera model in part based on the predicted locations of the visual fiducials.

5. The computer-assisted method of claim 4 wherein computing parameters for an estimated camera model further comprises applying a least squares optimization method to the predicted locations of the visual fiducials and locations of visual fiducials on the target from previously acquired image data.

6. The computer-assisted method of claim 1 wherein the determination of a certainty measure for a given estimated camera model further comprises

sampling from the distribution for each of the parameters in the given camera model N times, thereby generating N sets of sampled parameters and each set of sampled parameters creates a hypothetical camera model;

for each set of sampled parameters, projecting a grid of points into a pixel space using the corresponding hypothetical camera model;

computing an expected reprojection error for each point in the grid; and

selecting expected reprojection error having the largest value as the certainty measure for the given estimate camera model.

7. The computer-assisted method of claim 1 wherein presenting the selected target position further comprises rendering a template for the selected target position using the estimated camera model associated with the selected target position.

8. The computer-assisted method of claim 1 wherein the calibration target is further defined as a plurality of visual fiducials, where each visual fiducial is uniquely identifiable from the remaining visual fiducials.

9. The computer-assisted method of claim 1 wherein the camera model is further defined as a radial distortion model.

10. The computer-assisted method of claim 1 further comprises executing steps (b), (c), (d), (g) and (h) using a processor residing on the camera.

11. A computer-assisted method for calibrating a camera, comprising:

(a) initializing parameters of a camera model using image data of a calibration target captured by a camera, where the camera model defines a relationship between an image captured by the camera and a corresponding image without any distortion caused by the camera;

(b) predicting parameters of the camera model for each of a plurality of candidate target positions, thereby yielding an estimated camera model for each of the plurality of candidate target positions;

(c) determining a certainty measure for each of the estimated camera models;

(d) selecting a target position from the plurality of candidate target positions, where the selected target position resulted in the estimated camera model having the smallest certainty measure;

(e) positioning the calibration target in relation to the selected target position using a robot;

(f) capturing, by the camera, image data of the calibration target while the calibration target is positioned proximate to the selected target position;

(g) computing parameters of the camera model using the image data captured by the camera of the calibration target while the calibration target is positioned proximate to the selected target position; and

(h) determining a certainty measure for the camera model using the computed parameters of the camera model.

12. The computer-assisted method of claim 11 further comprises repeating steps (b)-(h) until the certainty measure for the camera model is less than a threshold.

13. The computer-assisted method of claim 12 further comprises capturing, by the camera, image data of a scene subsequent to the calibration process and correcting the image data using the computed parameters of the camera model.

14. The computer-assisted method of claim 11 wherein predicting parameters of the camera model further comprises predicting where visual fiducials on the target would appear in an image using the parameters of the camera model and when the target is oriented in a given candidate target position, and computing parameters for an estimated camera model in part based on the predicted locations of the visual fiducials.

15. The computer-assisted method of claim 14 wherein computing parameters for an estimated camera model further comprises applying a least squares optimization method to the predicted locations of the visual fiducials and locations of visual fiducials on the target from previously acquired image data.

16. The computer-assisted method of claim 11 wherein the determination of a certainty measure for a given estimated camera model further comprises

sampling from the distribution for each of the parameters in the given camera model N times, thereby generating N sets of sampled parameters and each set of sampled parameters creates a hypothetical camera model;

for each set of sampled parameters, projecting a grid of points into a pixel space using the corresponding hypothetical camera model;

computing an expected reprojection error for each point in the grid; and

selecting expected reprojection error having the largest value as the certainty measure for the given estimate camera model.

17. The computer-assisted method of claim 11 wherein the calibration target is further defined as a plurality of visual fiducials, where each visual fiducial is uniquely identifiable from the remaining visual fiducials.

18. The computer-assisted method of claim 11 wherein the camera model is further defined as a radial distortion model.

19. The computer-assisted method of claim 11 further comprises executing steps (b), (c), (d), (g) and (h) using a processor residing on the camera.