Detection and reconstruction of 3D objects with passive imaging sensors

• US 8,547,374 B1
• Filed: 10/09/2009
• Issued: 10/01/2013
• Est. Priority Date: 07/24/2009
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Sadjadi, Firooz A., Ribnick, Evan J.
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US12/576,687
• Time in Patent Office: 1,453 Days
• Field of Search: 345/423, 345/424, 345/426, 345/427, 345419-421, 348/294, 382/103, 382/154
• US Class Current: 345/419
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A 3D reconstruction technique using integral imaging, which does not require computing point correspondences. The technique has been demonstrated that it can effectively estimate the surface of an object, and reconstruct the surfaces of partially occluded objects. Reconstruction can be performed from either an array of imaging sensors, or a single moving imaging sensor with known positions. The technique relies on the concept that each view is allowed to vote on the contents of the scene via ray tracing. Various sharpness metrics are utilized for estimating the locations of visible surfaces in the views, and reconstruction is posed as an optimization problem.

Claims

1. A method of reconstructing object surfaces in three-dimensions from a plurality of two-dimensional input images of an object, comprising:

a) obtaining the plurality of two-dimensional input images of the object using at least one passive imaging sensor;

b) determining a reconstruction volume that contains the object surfaces to be reconstructed;

c) dividing the reconstruction volume into three-dimensional voxels of uniform size;

d) for a predetermined point in each voxel, collecting a vote for each of the input images;

e) determining the cost of the votes for each voxel;

f) detecting the object surfaces based on the determined costs of the voxels;

g) reconstructing the object surfaces based on the detected object surfaces and forming a reconstruction of the object surfaces contained in the reconstruction volume; and

h) displaying an image of the reconstruction of the object surfaces on a display device.

2. The method of claim 1, wherein obtaining the plurality of two-dimensional input images of the object using at least one passive imaging sensor comprises using a single passive imaging sensor that is movable relative to the object or using an array of passive imaging sensors that are at different positions relative to the object.

3. The method of claim 1, wherein the passive imaging sensor is a camera or an infrared sensor.

4. The method of claim 1, wherein the reconstruction volume contains the entirety of the object.

5. The method of claim 1, wherein the predetermined point in each voxel is the centerpoint of each voxel.

6. The method of claim 1, wherein the object is partially occluded, and further comprising discarding votes associated with the input images in which the predetermined points are occluded, re-determining the cost of the votes for voxels having occluded predetermined points, and thereafter repeating f)-g).

7. The method of claim 1, wherein e) comprises using a plurality of sharpness metrics.

8. The method of claim 1, wherein d) comprises using ray backprojection from each of the predetermined points to the passive imaging sensor, and for each two-dimensional input image the pixel value at the location where the ray intersects the image plane is the vote for the predetermined point in the respective voxel.

9. A system for reconstructing object surfaces in three-dimensions from a plurality of two-dimensional input images of an object, comprising:

a) at least one passive imaging sensor that is configured to obtain the plurality of two-dimensional input images of the object;

b) a reconstruction volume determination module that is configured to determine a reconstruction volume that contains the object surfaces to be reconstructed;

c) a reconstruction volume dividing module that is configured to divide the determined reconstruction volume into three-dimensional voxels of uniform size;

d) a vote collection module that is configured to, for a predetermined point in each voxel, collect a vote for each of the input images;

e) a cost determination module that is configured to determine the cost of the votes for each voxel;

f) an object surface detection module that is configured to detect the object surfaces based on the determined costs of the voxels;

g) an object surface reconstruction module that is configured to reconstruct the object surfaces based on the detected object surfaces and form a reconstruction of the object surfaces contained in the reconstruction volume; and

h) a display device that is configured to display an image of the reconstruction of the object surfaces.

10. The system of claim 9, wherein the at least one passive imaging sensor comprises a single passive imaging sensor that is movable relative to the object or an array of passive imaging sensors that are at different positions relative to the object.

11. The system of claim 9, wherein the passive imaging sensor is a camera or an infrared sensor.

12. The system of claim 9, wherein the cost determination module is configured to discard votes associated with input images in which the predetermined points are occluded, and re-determine the cost of the votes for voxels having occluded predetermined points.

13. The system of claim 9, wherein the cost determination module includes a plurality of sharpness metrics.

14. The system of claim 9, wherein the vote collection module is configured to use ray backprojection from each of the predetermined points to the at least one passive imaging sensor, and to determine the pixel value at the location where the ray intersects the image plane.