Self-calibration for a catadioptric camera

• US 7,548,253 B2
• Filed: 12/15/2004
• Issued: 06/16/2009
• Est. Priority Date: 06/12/2000
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kang, Sing Bing
• Primary Examiner(s): Ho; Tuan V
• Assistant Examiner(s): Trehan; Akshay
• Application Number:
  US11/015,828
• Publication Number:
  US 20050099502A1
• Time in Patent Office: 1,644 Days
• Field of Search: 348/36, 348/208.4, 348/169, 348/144, 348/222.1, 348/39, 348/187, 352/69, 382/151, 396/9
• US Class Current: 348/187
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and a system for self-calibrating a wide field-of-view camera (such as a catadioptric camera) using a sequence of omni-directional images of a scene obtained from the camera. The present invention uses the consistency of pairwise features tracked across at least a portion of the image collection and uses these tracked features to determine unknown calibration parameters based on the characteristics of catadioptric imaging. More specifically, the self-calibration method of the present invention generates a sequence of omni-directional images representing a scene and tracks features across the image sequence. An objective function is defined in terms of the tracked features and an error metric (an image-based error metric in a preferred embodiment). The catadioptric imaging characteristics are defined by calibration parameters, and determination of optimal calibration parameters is accomplished by minimizing the objective function using an optimizing technique.

Claims

1. A method for automatically calibrating a wide field-of-view camera, comprising:

obtaining a sequence of omni-directional images from the camera;

tracking a feature across the sequence of omni-directional images to obtain a feature track;

defining an objective function as an error between an actual location and a predicted location of the feature in the feature track across two images in the sequence omni-directional images;

optimizing the objective function to automatically determine an optimal set of calibration parameters for the camera that includes an aspect ratio and an image skew.

2. The method of claim 1, further comprising:

determining a consistency of the feature track.

3. The method of claim 2, wherein the feature track includes at least one of:

(a) point;

(b) line;

(C) plane.

4. The method of claim 2, further comprising defining a relationship between the feature track and an initial set of calibration parameters.

5. The method of claim 4, further comprising optimizing the relationship to obtain the optimal set of calibration parameters.

6. A computer-readable storage medium having stored thereon computer-executable instructions for causing a computing device to perform the method recited in claim 1.

7. A computer-implemented process stored in a computer memory for causing a computer to self-calibrate a catadioptric camera, comprising;

an initialization step for obtaining an initial set of calibration parameters;

a capture step for capturing a sequence of images from the catadioptric camera;

a consistency step for determining a consistency of tracked features across at least a portion of the sequence of images;

a characterization step for defining an epipolar geometry that links the tracked features across two images within the sequence of images;

a definition step for defining an objective function as a deviation from the epipolar geometry for the tracked features across two images in the sequence of images; and

a calculating step for using the tracked features to find a set of optimal calibration parameters.

8. The computer-implemented process as set forth in claim 7, further comprising an optimization step for optimizing the objective function to extract the set of optimal calibration parameters.

9. The computer-implemented process as set forth in claim 7, further comprising a minimization step for minimizing the objective function to extract the set of optimal calibration parameters.

10. A computer-readable medium having stored thereon computer-executable instructions for causing a computing device to calibrate a catadioptric camera, comprising:

capturing an image sequence using the catadioptric camera;

tracking a feature across the image sequence;

determining a consistency of the tracked features across the image sequence by defining a relationship between the tracked features and calibration parameters, further comprising defining the relationship as an objective function representing errors between actual locations and predicted locations of the tracked features within the image sequence; and

automatically calibrating the catadioptric camera using the image sequence without a need for user input during the calibration process.

11. The computer-readable medium of claim 10, further comprising:

optimizing the objective function; and

extracting a set of optimal calibration parameters from the optimized objective function that are used to automatically calibrate the catadioptric camera.

12. The computer-readable medium of claim 10, wherein the objective function further comprises an error metric defined in terms of deviations from an epipolar geometry of the image sequence.

13. The computer-readable medium of claim 12, wherein the error metric further comprises an image-based error metric that defines an image distance to an epipolar curve.

14. The computer-readable medium of claim 12, wherein the error metric further comprises an algebraic error metric.

15. The computer-readable medium of claim 11, further comprising selecting initial values for at least some unknown calibration parameters prior to optimizing the objective function.