Method and apparatus for automatic registration and visualization of occluded targets using ladar data

• US 20050243323A1
• Filed: 04/16/2004
• Published: 11/03/2005
• Est. Priority Date: 04/18/2003
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Hanna, Keith J, Samarasekera, Supun, Kumar, Rakesh, Hsu, Stephen Charles, Zhao, Wen-Yi
• Granted Patent: US 7,242,460 B2
• Time in Patent Office: Days
• US Class Current: 356/450
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A method and apparatus for high-resolution 3D imaging ladar system which can penetrate foliage and camouflage to sample fragments of concealed surfaces of interest is disclosed. Samples collected while the ladar moves can be integrated into a coherent object shape. In one embodiment, a system and method for automatic data-driven registration of ladar frames, comprises a coarse search stage, a pairwise fine registration stage using an iterated closest points algorithm, and a multi-view registration strategy. After alignment and aggregation, it is often difficult for human observers to find, assess and recognize objects from a point cloud display. Basic display manipulations, surface fitting techniques, and clutter suppression to enhance visual exploitation of 3D imaging ladar data may be utilized.

Claims

1. A method of registering ladar data, comprising:

receiving a plurality of ladar frames; and

registering at least two of said plurality of ladar frames for determining a sensor pose with respect to a reference.

2. The method of claim 1, wherein said registering step uses information provided by a Global Positioning System (GPS) or Inertial Navigation System (INS).

3. The method of claim 1, wherein said registering step comprises:

performing a coarse search for determining a translation shift; and

performing a fine registration.

4. The method of claim 3, wherein performing said coarse search comprises:

constructing range images from point sets derived from said at least two ladar frames;

estimating a 2D image shift from said at least two ladar frames;

scaling said 2D image shift for generating said translation shift in cross-range directions; and

estimating said translation shift in a down-range direction from said at least two ladar frames.

5. The method of claim 3, wherein performing said coarse search comprises:

binning point sets from each frame of said at least two ladar frames into coarse 3D grids of binary voxels; and

correlating the grids for generating said translation shift.

6. The method of claim 3, wherein performing said fine registration employs an iterated closest points (ICP) method.

7. The method of claim 6, wherein said ICP method performs a bounds test for eliminating false matches.

8. The method of claim 6, wherein said ICP method is accelerated by using an extrapolated point to compute motion.

9. The method of claim 6, wherein said ICP method removes points in either ladar frame on an interior of a smooth densely sampled surface from consideration.

10. The method of claim 6, wherein said ICP method ignores closest point pairs within said at least two ladar frames with distance exceeding a limit.

11. The method of claim 6, wherein said ICP method incrementally estimates rotation and translation from point pairs of said at least two ladar frames.

12. The method of claim 11, wherein translation is estimated as Δ

T-median{q_k−

(Rp_k+T)}, a robust center of translation-corrected points c_k=R_pk+T+Δ

T is computed as μ

=median{c_k}, and rotation around each axis is estimated from median {∠

_x(c_k−

μ

), (q_k−

μ

)} where ∠

_x a,b denotes the angle between vectors projected onto the yz plane.

13. The method of claim 6, wherein said ICP method comprises;

a) creating a point cloud from said at least two ladar frames at a plurality of resolution levels; and

b) performing said ICP method at each of said plurality of resolution levels.

14. The method of claim 1, wherein said sensor pose is determined using a hierarchical approach, where groups of nearby ladar frames are first registered and then are aggregated into composite point sets.

15. The method of claim 1, wherein said sensor pose is determined using a bundle approach, where pairwise registration is performed on said plurality of ladar frames separated by different temporal distances.

16. The method of claim 15, wherein a visual representation of said at least two ladar frames is produced.

17. The method of claim 1, wherein static noise cleaning is performed before said registering step.

18. The method of claim 1, wherein dynamic noise cleaning is performed before said registering step.

19. A method of recognizing a target object, comprising:

registering ladar data having said target object; and

producing a visual representation of said registered ladar data where recognition of said target object is enhanced.

20. The method of claim 19, wherein said producing step comprises:

removing clutter from the visual representation.

21. The method of claim 19, wherein said producing step comprises:

adding depth cues in point cloud displays by modulating point markers.

22. The method of claim 21, wherein point markers are modulated by changing brightness or color as a function of height above ground, distance to a sensor, or distance to a viewer.

23. The method of claim 19, wherein said producing step comprises interactive cropping.

24. The method of claim 19, wherein said producing step comprises reducing a data set by aggregating fewer frames to perceive a densely sampled surface.

25. The method of claim 19, wherein said producing step comprises using spatial thinning to perceive a densely sampled surface.

26. The method of claim 19, wherein said producing step comprises using surface rendering in order to extract a structure from a group of sample points.

27. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method of registering ladar data, comprising of:

receiving a plurality of ladar frames; and

registering at least two of said plurality of ladar frames for determining a sensor pose with respect to a reference.

28. An apparatus for registering ladar data, comprising:

means for receiving a plurality of ladar frames; and

means for registering at least two of said plurality of ladar frames for determining a sensor pose with respect to a reference.

29. The apparatus of claim 28, wherein said means for registering comprises:

means for performing a coarse search for determining a translation shift; and

means for performing a fine registration.

30. The apparatus of claim 29, wherein said means for performing said coarse search comprises:

means for constructing range images from point sets derived from said at least two ladar frames;

means for estimating a 2D image shift from said at least two ladar frames;

means for scaling said 2D image shift for generating said translation shift in cross-range directions; and

means for estimating said translation shift in a down-range direction from said at least two ladar frames.

31. The apparatus of claim 29, wherein said means for performing said coarse search comprises:

means for binning point sets from each frame of said at least two ladar frames into coarse 3D grids of binary voxels; and

means for correlating the grids for generating said translation shift.

32. The apparatus of claim 28, wherein said device is deployed on a vehicle.

33. An apparatus for recognizing a target object, comprising:

means for registering ladar data having said target object; and

means for producing a visual representation of said registered ladar data where recognition of said target object is enhanced.