Forming 3D models using two images

• US 8,447,099 B2
• Filed: 01/11/2011
• Issued: 05/21/2013
• Est. Priority Date: 01/11/2011
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Eastman Kodak Company
• Inventors: Wang, Sen, He, Lulu
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US13/004,186
• Publication Number:
  US 20120177283A1
• Time in Patent Office: 861 Days
• Field of Search: 382/190, 382/285, 382/305, 382/312, 382/154, 382/276, 702/153, 345/647, 345/648, 345/649
• US Class Current: 382/154
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A method for determining a three-dimensional model from two images comprising: receiving first and second images captured from first and second viewpoints, respectively, each image including a two-dimensional image together with a corresponding range map; identifying a set of corresponding features in the first and second two-dimensional images; removing any extraneous corresponding features in the set of corresponding features responsive to the first and second range maps to produce a refined set of corresponding features; determining a geometrical transform for transforming three-dimensional coordinates for the first image to be consistent three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features, the three-dimensional coordinates comprising two-dimensional pixel coordinates from the corresponding two-dimensional image together with a range coordinate from the corresponding range map; and determining a three-dimensional model responsive to the first image, the second image and the geometrical transform.

Claims

1. A method for determining a three-dimensional model from two images the method implemented at least in part by a data processing system and comprising the steps of:

receiving a first image captured from a first viewpoint, the first image including a first two-dimensional image together with a corresponding first range map;

receiving a second image captured from a second viewpoint, the second image including a second two-dimensional image together with a corresponding second range map, the second viewpoint being different from the first viewpoint;

identifying a set of corresponding features in the first and second two-dimensional images;

removing any extraneous corresponding features in the set of corresponding features responsive to the first and second range maps to produce a refined set of corresponding features;

determining a geometrical transform for transforming three-dimensional coordinates for the first image to be consistent three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features, the three-dimensional coordinates comprising two-dimensional pixel coordinates from the corresponding two-dimensional image together with a range coordinate from the corresponding range map;

determining a three-dimensional model responsive to the first image, the second image and the geometrical transform; and

storing the three-dimensional model in a processor-accessible memory system.

2. The method of claim 1 wherein the corresponding features are identified using a SIFT algorithm or a SURF algorithm.

3. The method of claim 1 wherein corresponding features are identified for corner points or edge points in the first and second two-dimensional images.

4. The method of claim 1 wherein the first and second two-dimensional images are color images, and wherein the step of identifying the set of corresponding features includes evaluating spatial and color features in the first and second two-dimensional images.

5. The method of claim 1 wherein the surface topologies in neighborhoods around the corresponding features are compared, and wherein corresponding features having surface topologies that differ by more than a specified threshold are identified as extraneous corresponding features.

6. The method of claim 1 wherein the geometrical transform includes a transformation matrix.

7. The method of claim 5 wherein the transformation matrix performs rotation, translation and scaling when multiplied by a vector including a set of three-dimensional coordinates.

8. The method of claim 5 wherein the transformation matrix, is determined by:

determining an initial transformation matrix by applying a least squares fitting algorithm to the three-dimensional coordinates for the refined set of corresponding features;

using the initial transformation matrix to transform three-dimensional coordinates for the first image forming transformed three-dimensional coordinates for the first image; and

refining the transformation matrix responsive to differences between the transformed three-dimensional coordinates for the first image and the three-dimensional coordinates for the second image.

9. The method of claim 7 wherein the transformation matrix is refined using an iterative closest points algorithm.

10. The method of claim 1 wherein the three-dimensional model is determined by:

using the transformation matrix to transform three-dimensional coordinates for the first image forming a transformed first image; and

combining the transformed first image with the second image to form the three-dimensional model.

11. The method of claim 10 wherein the transformed first image and the second image are combined by forming a combined point cloud including points from the transformed first image and points from the second image.

12. The method of claim 10 wherein the transformed first image and the second image are combined to form a triangulated mesh of surface points.

13. A system comprising:

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for determining a three-dimensional model from two images, wherein the instructions comprise;

receiving a first image captured from a first viewpoint, the first image including a first two-dimensional image together with a corresponding first range map;

receiving a second image captured from a second viewpoint, the second image including a second two-dimensional image together with a corresponding second range map, the second viewpoint being different from the first viewpoint;

identifying a set of corresponding features in the first and second two-dimensional images;

removing any extraneous corresponding features in the set of corresponding features responsive to the first and second range maps to produce a refined set of corresponding features;

determining a geometrical transform for transforming three-dimensional coordinates for the first image to be consistent three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features, the three-dimensional coordinates comprising two-dimensional pixel coordinates from the two-dimensional image together with a range coordinate from the corresponding range map;

determining a three-dimensional model responsive to the first image, the second image and the geometrical transform; and

storing the three-dimensional model.

14. A computer program product embodied on a non-transitory tangible computer readable storage medium for determining a three-dimensional model from two images comprising an executable software application for causing a data processing system to perform the steps of:

receiving a first image captured from a first viewpoint, the first image including a first two-dimensional image together with a corresponding first range map;

receiving a second image captured from a second viewpoint, the second image including a second two-dimensional image together with a corresponding second range map, the second viewpoint being different from the first viewpoint;

identifying a set of corresponding features in the first and second two-dimensional images;

removing any extraneous corresponding features in the set of corresponding features responsive to the first and second range maps to produce a refined set of corresponding features;

determining a geometrical transform for transforming three-dimensional coordinates for the first image to be consistent three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features, the three-dimensional coordinates comprising two-dimensional pixel coordinates from the two-dimensional image together with a range coordinate from the corresponding range map;

determining a three-dimensional model responsive to the first image, the second image and the geometrical transform; and

storing the three-dimensional model in a processor-accessible memory system.