Camera model and calibration procedure for omnidirectional paraboloidal catadioptric cameras

• US 20030004694A1
• Filed: 05/29/2002
• Published: 01/02/2003
• Est. Priority Date: 05/29/2001
• Status: Active Grant

• Current Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Aliaga, Daniel G., Carlbom, Ingrid B.
• Granted Patent: US 7,362,969 B2
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A paraboloidal catadioptric camera is calibrated by relaxing the assumption of an ideal system to account for perspective projection, radial distortion, and mirror misalignment occurring within the camera system. Calibration points, which are small and visually distinct objects, are distributed at fixed locations within an environment. Omnidirectional images are captured by the catadioptric camera at different locations of the environment. Data points are obtained by identifying the location of the calibration points in each captured image. An optimization algorithm best-fits the data points to a perspective camera model in order to derive parameters, which are used to calibrate the catadioptric camera.

Claims

1. A method of calibrating an omnidirectional camera comprising:

collecting calibration points using the omnidirectional camera; and

determining at least one correction factor based on the plurality of calibration points, each correction factor being used to correct a deviation of the omnidirectional camera from an ideal camera model.

2. The method of claim 1, wherein

the omnidirectional camera records images reflected by an attached convex paraboloidal mirror onto an image plane, and

the determining step determines at least one correction factor that includes at least one of an effective convergence distance of reflected lights rays from the mirror, a camera-to-world translation, a camera-to-world rotation, a radial distortion, and an uncertainty scale factor from a series of calibration points.

3. The method of claim 2, wherein the at least one correction factor is used to obtain an effective subset of the mirror reflecting a 360 degree by 180 degree field of view of the environment.

4. The method of claim 2, wherein the camera-to-world translation represents a lateral offset between the image plane and a reference plane of the mirror.

5. The method of claim 2, wherein the camera-to-world rotation represents a rotation of a reference plane of the mirror with respect to an assumed coordinate frame of the environment.

6. The method of claim 2, wherein the effective convergence distance of light rays reflected off the mirror represents a distance in front of the reflective side of the mirror at which a convergence point of the reflected light rays is located.

7. The method of claim 2, wherein the radial distortion includes at least one coefficient used to define a first-order or higher approximation of the radial component of lens distortion.

8. The method of claim 2, wherein the uncertainty scale factor includes at least one coefficient used to scale an effective size of pixels in the image plane.

9. The method of claim 1, wherein the step of collecting calibration points comprises:

moving the omnidirectional camera to various positions within an environment;

recording an image of the environment with the omnidirectional camera at each of the various positions; and

for each recorded image,

identifying calibration points in the recorded image, each calibration point having a fixed position within the environment, and

determining an image position of each identified calibration point.

10. The method of claim 9, wherein the calibration points comprise visually distinct features within the environment.

11. The method of claim 2, wherein the step of determining at least one correction factor comprises:

modeling the omnidirectional camera using at least one model parameter;

optimizing the at least one model parameter to best-fit the calibration points; and

calculating the at least one correction factor according to the at least one model parameter.

12. The method of claim 11, wherein the modeling steps models the omnidirectional camera according to a perspective camera model.

13. The method of claim 12, wherein the perspective camera model constitutes a pinhole camera model.

14. The method of claim 11, wherein the optimizing step includes,

making an initial estimate of the value of each of the at least one model parameter, the at least one model parameter including a distance from the focal point of the mirror from which light rays reflected from the mirror converge, a camera-to-world translation, a camera-to-world rotation, a radial distortion, and an uncertainty scale factor;

and

performing an iterative procedure including,

a) determining a fitness of the estimated values for the at least one model parameter according to predetermined criteria;

b) re-estimating the value of each the at least one model parameter according to the determined fitness when the predetermined criteria is not satisfied; and

c) repeating the above steps until the predetermined criteria is satisfied.

16. The system of claim 15, wherein

the omnidirectional camera records images reflected by an attached convex paraboloidal mirror onto an image plane, and

the processing means determines at least one calibration factor that includes at least one of an effective convergence distance of recorded light rays from the mirror, an effective subset of the mirror, a camera-to-world translation, a radial distortion, and an uncertainty factor in the horizontal direction from a series of calibration points.

17. The system of claim 16, wherein the processing means further includes,

modeling means for modeling the omnidirectional camera using at least one model parameter;

optimizing means for optimizing the at least one model parameter to best-fit the extracted calibration points; and

calculating means for calculating the at least one correction factor according to the at least one model parameter.

19. The method of claim 18, further comprising,

using the optimized at least one model parameter to estimate a pose of the omnidirectional camera based on images recorded by the omnidirectional camera.

15. A system for calibrating an omnidirectional camera comprising:

data storage device for storing images recorded using the omnidirectional camera;

extracting means for extracting calibration points from the stored images; and

processing means for determining at least one correction factor based on the plurality of calibration points, each correction factor being used to correct a deviation of the omnidirectional camera from an ideal camera model.

18. A method for modeling an omnidirectional camera, which is configured to record images reflected by an attached convex paraboloidal mirror onto an image plane, the method comprising:

defining at least one model parameter including at least one of an effective convergence distance of reflected light rays from the mirror, a camera-to-world translation, a camera-to-world rotation, a radial distortion, and an uncertainty scale factor;

optimizing the at least one model parameter to best-fit a set of calibration points obtained using the omnidirectional camera, the calibration points comprising a set of visually distinct features of fixed location within an environment of the omnidirectional camera.