3-D reconstruction and registration

• US 9,280,821 B1
• Filed: 03/18/2013
• Issued: 03/08/2016
• Est. Priority Date: 05/20/2008
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Choe, Tae Eun, Medioni, Gerard Guy
• Primary Examiner(s): Dulaney, Kathleen Y
• Application Number:
  US13/846,622
• Time in Patent Office: 1,086 Days
• Field of Search: 382/154, 382/203, 382/266, 382/285, 382/294, 345/419, 356/12
• US Class Current: 1/1
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Generating three-dimensional information can include obtaining multiple different images of an object taken by camera(s), each image having a near-planar surface depiction of the object; registering the images in two-dimensions by identifying one or more features of each image and generating a two-dimensional representation of each feature; selecting first and second images from the registered images; generating one or more correspondences between one or more features of the first and second images; estimating a camera parameter set for each of the first and second images within respective ones of the identified features; reconstructing a three-dimensional structure of the object in Euclidean space responsive to the one or more correspondences and the estimated camera parameter sets; refining the estimated camera parameter sets using the three-dimensional structure; and refining the three-dimensional structure using the refined camera parameter sets. Camera parameter sets can include a rotation matrix, translation vector, and focal length.

Claims

1. A computer-implemented method for three-dimensional reconstruction, comprising:

obtaining multiple different images of an object taken by one or more cameras, each image having a near-planar surface depiction of the object;

registering the multiple different images in two-dimensions, the registering comprising identifying one or more features of each image, and determining and refining homographies and feature matches between pairs of the images based on the identified features and iterative nearest neighbor matching to produce refined homographies;

determining shortest path costs respectively for the registered images based on the refined homographies;

selecting a first image and a second image from the registered images, the first image being selected based on the shortest path costs;

generating one or more correspondences between one or more features of the first image and one or more features of the second image;

estimating a camera parameter set for each of the first and second images within respective ones of the identified one or more features, each camera parameter set comprising a rotation matrix, a translation vector, and a focal length;

reconstructing a three-dimensional structure of the object in Euclidean space responsive to the one or more correspondences and the estimated camera parameter sets;

refining the estimated camera parameter sets using the three-dimensional structure; and

refining the three-dimensional structure using the refined camera parameter sets.

2. The method of claim 1, further comprising:

registering at least a portion of the multiple different images in three-dimensions using the refined three-dimensional structure.

3. The method of claim 1, wherein the selecting comprises:

identifying registration errors of the registered images; and

selecting an image from the registered images responsive to the registration errors.

4. A non-transitory computer readable medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

obtaining multiple different images of an object taken by one or more cameras, each image having a near-planar surface depiction of the object;

registering the multiple different images in two-dimensions, the registering comprising identifying one or more features of each image, and determining and refining homographies and feature matches between pairs of the images based on the identified features and iterative nearest neighbor matching to produce refined homographies;

determining shortest path costs respectively for the registered images based on the refined homographies;

selecting a first image and a second image from the registered images, the first image being selected based on the shortest path costs;

generating one or more correspondences between one or more features of the first image and one or more features of the second image;

estimating a camera parameter set for each of the first and second images within respective ones of the identified one or more features;

reconstructing a three-dimensional structure of the object in Euclidean space responsive to the one or more correspondences and the estimated camera parameter sets;

refining the estimated camera parameter sets using the three-dimensional structure; and

refining the three-dimensional structure using the refined camera parameter sets.

5. The computer readable medium of claim 4, the operations further comprising:

registering at least a portion of the multiple different images in three-dimensions using the refined three-dimensional structure.

6. The computer readable medium of claim 4, wherein the selecting comprises:

identifying registration errors of the registered images; and

selecting an image from the registered images responsive to the registration errors.

7. The computer readable medium of claim 4, wherein each of the camera parameter sets comprises a rotation matrix, a translation vector, and a focal length.

8. An image processing system comprising:

a memory configured to store multiple different images of an object taken by one or more cameras, each image having a near-planar surface depiction of the object; and

processor electronics in communication with the memory, configured to perform operations comprising;

accessing the multiple different images of the object;

registering the multiple different images in two-dimensions, the registering comprising identifying one or more features of each image, and determining and refining homographies and feature matches between pairs of the images based on the identified features and iterative nearest neighbor matching to produce refined homographies;

determining shortest path costs respectively for the registered images based on the refined homographies;

selecting a first image and a second image from the registered images, the first image being selected based on the shortest path costs;

generating one or more correspondences between one or more features of the first image and one or more features of the second image;

estimating a camera parameter set for each of the first and second images within respective ones of the identified one or more features;

reconstructing a three-dimensional structure of the object in Euclidean space responsive to the one or more correspondences and the estimated camera parameter sets;

refining the estimated camera parameter sets using the three-dimensional structure; and

refining the three-dimensional structure using the refined camera parameter sets.

9. The system of claim 8, the operations further comprising:

registering at least a portion of the multiple different images in three-dimensions using the refined three-dimensional structure.

10. The system of claim 8, wherein the selecting comprises:

identifying registration errors of the registered images; and

selecting an image from the registered images responsive to the registration errors.

11. The system of claim 8, wherein each of the camera parameter sets comprises a rotation matrix, a translation vector, and a focal length.