Camera calibration device and method, and computer system

• US 6,985,175 B2
• Filed: 07/13/2001
• Issued: 01/10/2006
• Est. Priority Date: 07/13/2000
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yoshigahara, Takayuki, Iwai, Yoshiaki
• Primary Examiner(s): Natnael, Paulos M.
• Application Number:
  US09/905,450
• Publication Number:
  US 20020113878A1
• Time in Patent Office: 1,642 Days
• Field of Search: 348/187, 348/188, 348/189, 348180-182, 348190-191, 348/42, 348/43, 348/46, 348/47, 348/48, 348/51, 358/504, 382/154, 382/153, 382/106, 702/94, 702/95
• US Class Current: 348/187
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A camera calibration device capable of simply calibrating a stereo system consisting of a base camera and detection camera. First, distortion parameters of the two cameras necessary for distance measurement are presumed by the use of images obtained by shooting a patterned object plane with the base camera and the reference camera at three or more view points free from any spatial positional restriction, and projective transformation matrixes for projecting the images respectively onto predetermined virtual planes are calculated. Then internal parameters of the base camera are calculated on the basis of the projective transformation matrixes relative to the images obtained from the base camera. Subsequently the position of the shot plane is presumed on the basis of the internal parameter of the base camera and the images obtained therefrom, whereby projection matrixes for the detection camera are calculated on the basis of the plane position parameters and the images obtained from the detection camera. According to this device, simplified calibration can be achieved stably without the necessity of any exclusive appliance.

Claims

1. A camera calibration device for calibrating a stereo system, which includes a base camera and a detection camera, the device comprising:

an image holding device for holding images obtained by shooting a plane, where a known pattern is drawn, with the base camera and the detection camera at at least three view points free from any spatial positional restriction; and

a parameter calculating device for calculating parameters necessary for distance measurement in the stereo system based on the images held by the image holding part, wherein the parameter calculating device comprises a first part for presuming distortion parameters of the base camera and the detection camera by the use of the shot images;

a second part for calculating projective transformation matrixes to project the shot images respectively onto predetermined virtual planes;

a third part for calculating internal parameters of the base camera on the basis of the projective transformation matrixes obtained by the second part with regard to the images from the base camera;

a fourth part for presuming the position of the shot plane based on both the internal parameters of the base camera calculated by the third part and the images obtained from the base camera; and

a fifth part for calculating projection matrixes for the detection camera based on both parameters of the plane position presumed by the fourth part and the images obtained from the detection camera.

2. A camera calibration device as claimed in claim 1, further comprising:

a parameter correcting device for optimizing the plane position parameters and the projection matrixes for the detection camera based on more than two images obtained from the base camera and the detection camera and held by the image holding device.

3. A camera calibration device as claimed in claim 2, wherein the parameter correcting device comprises:

a sixth part for calculating projective transformation matrixes to project the images respectively onto predetermined virtual planes;

a seventh part for presuming the position of the shot plane based on both the internal parameters of the base camera and the images obtained from the base camera;

an eighth part for calculating projection matrixes for the detection camera based on both parameters of the plane position presumed by the seventh part and the images obtained from the detection camera; and

a ninth part for optimizing the plane position parameters and the projection matrixes for the detection camera based on the shot images.

4. A camera calibration device as claimed in claim 1, wherein at least one of the distortion parameters and the projective transformation matrixes are calculated through a process of image registration which registers the individual images in a manner to form a predetermined synthetic image.

5. A camera calibration method for calibrating a stereo system, which includes a base camera and a detection camera, by using images obtained by shooting a plane, where a known pattern is drawn, with the individual cameras at at least three view points free from any spatial positional restriction, the method comprising the steps of:

presuming distortion parameters of the base camera and the detection camera by using the images thus obtained;

calculating projective transformation matrixes to project the images respectively onto predetermined virtual planes;

calculating internal parameters of the base camera based on the projective transformation matrixes obtained during the step of calculating projective transformation matrixes with regard to the images from the base camera;

presuming the position of the shot plane based on both the internal parameters of the base camera calculated at the step of calculating internal parameters and the images obtained from the base camera; and

calculating projection matrixes for the detection camera based on both parameters of the plane position presumed at the step of presuming and the images obtained from the detection camera.

6. A camera calibration method as claimed in claim 5, further comprising the step of:

optimizing the plane position parameters and the projection matrixes for the detection camera based on more than two images obtained from the base camera and the detection camera.

7. A camera calibration method as claimed in claim 6, wherein the step of optimizing the plane position parameters comprises the steps of:

calculating further projective transformation matrixes to project the images respectively onto predetermined virtual planes;

presuming further the position of the shot plane based on both the internal parameters of the base camera and the images obtained from the base camera;

calculating further projection matrixes for the detection camera based on both parameters of the plane position presumed during the step of presuming and the images obtained from the detection camera; and

optimizing further the plane position parameters and the projection matrixes for the detection camera based on the shot images.

8. A camera calibration method as claimed in claim 5, wherein at least one of the distortion parameters and the projective transformation matrixes are calculated through a process of image registration which registers the individual images in a manner to form a predetermined synthetic image.

9. A computer system for running a processing routine of camera calibration for a stereo system, which includes a base camera and a detection camera, by using images obtained by shooting a plane, where a known pattern is drawn, with the individual cameras at at least three view points free from any spatial positional restriction, the computer system comprising:

a storage medium for storing computer software in a format readable by the computer system; and

computer software for executing, on the computer system, the processing routine of camera calibration, the computer software including a first program step of presuming distortion parameters of the base camera and the detection camera by using the images thus obtained, a second program step of calculating projective transformation matrixes to project the images respectively onto predetermined virtual planes, a third program step of calculating internal parameters of the base camera based on the projective transformation matrixes obtained at the second program step with regard to the images from the base camera, a fourth program step of presuming the position of the shot plane based on the internal parameters of the base camera calculated at the third program step, and also based on the images obtained from said base camera, and a fifth program step of calculating projection matrixes for the detection camera based on both parameters of the plane position presumed at the fourth program step, and the images obtained from the detection camera.

10. A computer system as claimed in claim 9, wherein the computer software further includes a parameter correcting program step of optimizing the plane position parameters and the projection matrixes for the detection camera based on more than two images obtained from the base camera and the detection camera.

11. A computer system as claimed in claim 10, wherein the parameter correcting program step includes a sixth program step of calculating projective transformation matrixes to project the images respectively onto predetermined virtual planes, a seventh program step of presuming the position of the shot plane based on the internal parameters of the base camera and the images obtained from the base camera, an eighth program step of calculating projection matrixes for the detection camera based on both parameters of the plane position presumed at the seventh program step, and the images obtained from the detection camera, and a ninth program step of optimizing the plane position parameters and the projection matrixes for the detection camera based on the shot images.