AUTOMATED GRAPH LOCAL CONSTELLATION (GLC) METHOD OF CORRESPONDENCE SEARCH FOR REGISTRATION OF 2-D AND 3-D DATA

• US 20150036870A1
• Filed: 07/30/2013
• Published: 02/05/2015
• Est. Priority Date: 07/30/2013
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Mundhenk, Terrell Nathan, Kim, Kyungnam, Owechko, Yuri, Chen, Hai-Wen
• Granted Patent: US 9,754,165 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/176 Urban or other man-made structures

Abstract

According to an embodiment, a 2-dimensional (2-D) image of a geographical region is transformed via a regional maxima transform (RMT) or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image. The filtered 2-D image is iteratively eroded and opened to produce a processed EO 2-D image, 2-D object shape morphology is extracted from the processed EO 2-D image, and 2-D shape properties are extracted from the 2-D object shape morphology. A height slice of a 3-dimensional (3-D) point cloud comprising 3-D coordinate and intensity measurements of the geographical region is generated, and slice object shape morphology is extracted from the height slice. Slice shape properties from the slice object shape morphology are extracted, and the 2-D image is constellation matched to the height slice based on the 2-D shape properties and the slice shape properties.

Claims

1. A method for registration of two and three-dimensional (3-D) images, the method comprising:

transforming a 2-dimensional (2-D) image of a geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image;

extracting 2-D shape properties from the 2-D object shape morphology;

generating a height slice of a 3-dimensional point cloud comprising 3-D coordinate and intensity measurements of the geographical region;

extracting slice object shape morphology from the height slice;

extracting slice shape properties from the slice object shape morphology; and

constellation matching the 2-D image to the height slice based on the 2-D shape properties and the slice shape properties.

2. The method of claim 1, wherein the 3-D coordinate and intensity measurements comprise Light Detection and Ranging (LIDAR) measurements.

3. The method of claim 1, further comprising capturing the 2-D image comprising 2-D image points of the geographical region by a camera attached to a first vehicle.

4. The method of claim 3, further comprising scanning the geographical region by a LIDAR system attached to a second vehicle to produce the 3-dimensional point cloud comprising 3-D cloud image points thereof.

5. The method of claim 4, further comprising:

determining matching points between the 2-D image points and the 3-D cloud image points; and

estimating a pose position of the camera based on the matching points to provide an estimated pose position.

6. The method of claim 5, further comprising fusing the 2-D image points and the 3-D cloud image points based on the estimated pose position.

7. The method of claim 5, further comprising determining a geo-location of the second vehicle based on the estimated pose position.

8. The method of claim 1, further comprising:

finding a plurality of shapes for a plurality of objects;

identifying the shapes for further extraction to produce identified shapes;

extracting properties of the identified shapes comprising shape corners and shape dimensions; and

matching a constellation of objects via matching LIDAR object to EO objects based on the properties of the identified shapes.

9. A system for registration of two and three-dimensional images, the system comprising:

a regional maxima transform module configured to transform a 2-dimensional (2-D) image of a geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

a filter module configured to iteratively erode and open the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

a 2-D morphology module configured to extract 2-D object shape morphology from the processed EO 2-D image;

a 2-D shape extraction module configured to extract 2-D shape properties from the 2-D object shape morphology;

a height slice module configured to generate a height slice of a 3-dimensional (3-D) point cloud comprising Light Detection and Ranging (LIDAR) measurements of the geographical region;

a 3-D morphology module configured to extract slice object shape morphology from the height slice;

a 3-D shape extraction module configured to extract slice shape properties from the slice object shape morphology; and

a constellation matching module configured to constellation match the 2-D image to the height slice based on the 2-D shape properties and the slice shape properties.

10. The system of claim 9, further comprising a first vehicle system configured to image the 2-D image comprising 2-D image points of the geographical region by a camera attached to a first vehicle.

11. The system of claim 10, further comprising a first vehicle system configured to scan the 3-dimensional point cloud comprising 3-D cloud image points of the geographical region by a LIDAR system attached to a second vehicle.

12. The system of claim 11, further comprising a camera pose and location estimator configured to:

match points of the 2-D image and the 3-dimensional point cloud based on the 2-D shape properties and the slice shape properties to produce a plurality of matched points; and

estimate a pose position of the camera based on the matched points to provide an estimated pose position.

13. The system of claim 12, wherein the camera pose and location estimator (CPLE) is further configured to determine a geo-location of the second vehicle based on the estimated pose position.

14. A non-transitory computer readable storage medium comprising computer-executable instructions for registration of two and three-dimensional (3-D) images, the computer-executable instructions comprising:

transforming a 2-dimensional (2-D) image of a geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image;

extracting 2-D shape properties from the 2-D object shape morphology;

generating a height slice of a 3-dimensional point cloud comprising Light Detection and Ranging (LIDAR) measurements of the geographical region;

extracting slice object shape morphology from the height slice;

extracting slice shape properties from the slice object shape morphology; and

constellation matching the 2-dimensional (2-D) image to the height slice based on the 2-D shape properties and the slice shape properties.

15. The non-transitory computer readable storage medium of claim 14, further comprising computer-executable instructions for:

determining matching points between 2-D image points and 3-D cloud image points to produce a plurality of matched points; and

estimating a pose position of a camera based on the matched points to provide an estimated pose position.

16. The non-transitory computer readable storage medium of claim 15, further comprising computer-executable instructions for:

fusing the 2-D image points and the 3-D cloud image points based on the estimated pose position.

17. The non-transitory computer readable storage medium of claim 15, further comprising computer-executable instructions for:

determining a geo-location of a vehicle based on the estimated pose position.

18. The non-transitory computer readable storage medium of claim 14, further comprising computer-executable instructions for:

capturing the 2-D image comprising 2-D image points of the geographical region by a camera attached to a first vehicle.

19. The non-transitory computer readable storage medium of claim 18, further comprising computer-executable instructions for:

scanning the geographical region by a LIDAR system attached to a second vehicle to produce the 3-dimensional point cloud comprising 3-D cloud image points thereof.

20. The non-transitory computer readable storage medium of claim 14, further comprising computer-executable instructions for:

finding a plurality of shapes for a plurality of objects;

identifying the shapes for further extraction to produce identified shapes;

extracting properties of the identified shapes comprising shape corners and shape dimensions; and

matching a constellation of objects via matching LIDAR object to EO objects based on the properties of the identified shapes.