Method for Calibrating Cameras with Non-Overlapping Views

• US 20160012588A1
• Filed: 07/14/2014
• Published: 01/14/2016
• Est. Priority Date: 07/14/2014
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation, Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Taguchi, Yuichi, Ramalingam, Srikumar, Miki, Yohei, Cansizoglu, Esra
• Granted Patent: US 11,051,000 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A method calibrates one or more cameras, wherein the one or more cameras acquire images of a scene for different viewpoints, and wherein the images are non-overlapping, by first constructing a 3D model of the scene using a calibration camera that is independent of the one or more cameras, and a. simultaneous localization and mapping (SLAM) procedure. 2D-to-3D correspondences between each of the images and the 3D model are determined. Then, calibration parameters of each of the one or more cameras are estimated using a 2D-to-3D registration procedure.

Claims

1. A method for calibrating one or more cameras, wherein the one or more cameras acquire images of a scene for different viewpoints, and wherein the images are non-overlapping, comprising the steps of:

constructing a 3D model of the scene using a calibration camera that is independent of the one or more cameras, and a simultaneous localization and mapping (SLAM) procedure;

determining 2D-to-3D correspondences between each of the images and the 3D model; and

estimating calibration parameters of each of the one or more cameras using a to 2D-to-3D registration procedure, wherein the steps are performed in a processor.

3. The method of claim 1, wherein the calibration parameters include intrinsic calibration parameters.

4. The method of claim 1, wherein the calibration camera is a red, green, blue and depth (RGB-D) camera.

5. The method of claim 4, wherein the constructing uses point and plane features in an RGB-D image acquired with the RGB-D camera.

6. The Method of claim 5, wherein the SLAM procedure stores. images with representative poses acquired with the calibration camera as keyimages in a map.

7. The method of claim 5, further comprising:

extacting the point features using a keypoint detector; and

extracting the plane features using a RANdom Sample CONsensus (RANSAC)-based plane fitting procedure.

8. The method of claim 6, further comprising:

adding a particular image to the map when an estimated pose of the particular image is sufficiently different from any poses associated with the keyimages.

9. The method of claim 6, further comprising:

optimizing the poses associated with the keyimages, the point features and the plane features in the map using joint bundle adjustment.

10. The method of claim 6, further comprising:

representing an appearance of each key mage using a bag of visual words representation of point features.

11. The method of claim 1, wherein the 2D-to-3D registration procedure uses a Perspective-n-Point (PnP) procedure.

12. The method of claim 10, further comprising:

using appearance-based keyimage matching and geometric verification to determine the correspondences.

13. The method of claim 1, wherein the one or more cameras are displayed with the 3D model using the calibration parameters.

14. The method of claim 1, wherein the 3D model and the calibration parameters are used to browse the images.

2. The method of claim I, wherein the calibration parameters include extrinsic calibration parameters.