METHOD FOR LOCATING A CAMERA AND FOR 3D RECONSTRUCTION IN A PARTIALLY KNOWN ENVIRONMENT

• US 20140105486A1
• Filed: 05/29/2012
• Published: 04/17/2014
• Est. Priority Date: 05/30/2011
• Status: Active Grant

• Current Assignee: Commissariat a L'Energie Atomique (French Alternative Energies and Atomic Energy Commission)
• Original Assignee: Commissariat a L'Energie Atomique (French Alternative Energies and Atomic Energy Commission)
• Inventors: Bourgeois, Steve, Dhome, Michel, Naudet, Sylvie, Tamaazousti, Mohamed, Gay-Bellile, Vincent
• Granted Patent: US 9,613,420 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A method for locating a camera and 3D reconstruction of its static environment, comprising an object of interest, the 3D model of which is known, includes: calculating an initial pose of the camera in the environment and an initial reconstruction; calculating the pose of the camera for each new image by pairing 3D primitives of the environment with 2D primitives of said image and reconstructing 3D primitives of the environment by triangulation; and simultaneously optimizing the poses of the camera and 3D primitives by minimizing a reprojection error over a plurality of images. The 3D model is a geometric description of the object of interest, the reprojection error has only two types of terms, a first type associated with primitives constrained by the 3D model and a second type associated with primitives of the environment other than the object, the optimization associating the primitives with the environment or 3D model.

Claims

1. A method for locating a camera and for the 3D reconstruction of the static environment in which it is moving and of which the camera forms images, this environment comprising an object of interest of which the 3D model is known, which comprises the following steps:

a) calculation of an initial pose of the camera in the environment and of an initial reconstruction,

b) calculation of the pose of the camera for each new image by matching 3D primitives of the environment with 2D primitives of said image and reconstructing 3D primitives of the environment by triangulation,

c) simultaneous optimization of the poses of the camera and of the 3D primitives by minimizing a reprojection error over a plurality of images,

wherein the 3D model is a geometric description of the object of interest, in that the reprojection error comprises only two types of terms, a first type of terms linked to primitives constrained by the 3D model and a second type of terms linked to primitives of the environment other than the object, and in that the optimization step comprises a substep of associating primitives with the environment or with the 3D model.

2. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the optimization step is performed with a robust estimator exhibiting a rejection threshold, by adaptively controlling the influence of each type of terms as a function of the rejection threshold.

3. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein, each type of terms of the reprojection error being associated with a rejection threshold linked to said rejection threshold of the robust estimator, the two rejection thresholds (c1, c2) are equal to the rejection threshold of the estimator estimated on residues based on the 3D model.

4. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the calculation of the initial pose is performed semi-automatically, with a target of known size and positioned relative to one of the planes of the reference frame of the object, a target being a set of 3D primitives located between them and identifiable in the image, and the initial reconstruction is obtained by back-projection of the 2D primitives onto the model.

5. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the step of calculating the pose of the camera for each new image is performed on the basis of the primitives constrained by the 3D model and of the primitives of the environment other than the object.

6. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the optimization step is followed by a triangulation step in which the primitives are not constrained by the 3D model.

7. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the 3D reconstruction step and/or the optimization step are performed only for key images chosen when the number of primitives matched between the current image and the last key image is lower than a minimum value.

8. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the optimization step is performed with a local bundle adjustment.

9. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the images originate from a real time video stream and from a prerecorded video.

10. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the object of interest is not visible in certain images or is modified from one image to another.

11. The method for locating a camera and for the 3D reconstruction of the environment as claimed in claim 1, wherein the primitives constrained by the 3D model are chosen as a function of whether the object is textured and/or untextured.