Augmented three dimensional point collection of vertical structures

• US 10,338,222 B2
• Filed: 11/30/2016
• Issued: 07/02/2019
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Original Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Inventors: Schultz, Stephen L., Nilosek, David R., Petterson, David S., Harrington, Timothy S.
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US15/364,812
• Publication Number:
  US 20170082750A1
• Time in Patent Office: 944 Days
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/66 of image moments or centre of gravity
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals

Abstract

Automated methods and systems are disclosed, including a method comprising: capturing images and three-dimensional LIDAR data of a geographic area with an image capturing device and a LIDAR system, as well as location and orientation data for each of the images corresponding to the location and orientation of the image capturing device capturing the images, the images depicting an object of interest and the three-dimensional LIDAR data including the object of interest; storing the three-dimensional LIDAR data on a non-transitory computer readable medium; analyzing the images with a computer system to determine three dimensional locations of points on the object of interest; and updating the three-dimensional LIDAR data with the three dimensional locations of points on the object of interest determined by analyzing the images to create a 3D point cloud.

Claims

1. An automated method, comprising:

capturing images and three-dimensional LIDAR data of a geographic area with an image capturing device and a LIDAR system, as well as location and orientation data for each of the images corresponding to the location and orientation of the image capturing device capturing the images, the images depicting an object of interest and the three-dimensional LIDAR data including the object of interest, the image capturing device capturing the images at a first resolution, and the LIDAR system capturing the three-dimensional LIDAR data at a second resolution less than the first resolution;

storing the three-dimensional LIDAR data on a non-transitory computer readable medium;

analyzing the images with a computer system to determine three dimensional locations of points on the object of interest; and

updating the three-dimensional LIDAR data with the three dimensional locations of points on the object of interest determined by analyzing the images to create a 3D point cloud having a resolution greater than the resolution of the three-dimensional LIDAR data.

2. The automated method of claim 1, wherein the image capturing device and the LIDAR system are mounted to a platform, and wherein capturing images and three-dimensional LIDAR data are defined further as flying the platform over the geographic area.

3. The automated method of claim 2, wherein the platform is unmanned.

4. The automated method of claim 2, wherein the platform is manned.

5. The automated method of claim 1, wherein the image capturing device is mounted to a platform to capture images from a first perspective, and the LIDAR system is mounted to the platform to capture the three-dimensional LIDAR data from a second perspective different from the first perspective.

6. The automated method of claim 5, wherein the first perspective is an oblique perspective, and the second perspective is a nadir perspective.

7. The automated method of claim 1, wherein the image capturing device and the LIDAR system are mounted to an airplane, and wherein capturing images and three-dimensional LIDAR data are defined further as flying over the geographic area with the airplane.

8. The automated method of claim 1, wherein the object of interest includes a utility tower, and wherein the step of analyzing the images is defined further as utilizing GIS data of a utility network to assist in locating the object of interest within one or more images.

9. The automated method of claim 1, wherein the object of interest includes a utility tower, and wherein the step of analyzing the images is defined further as scanning the images with an edge detection algorithm to locate utility wires depicted within the images, prior to determining three dimensional location of points on the utility tower.

10. A system, comprising:

an image capturing device capturing images of a geographic area, the images depicting an object of interest, the images having a first resolution;

one or more geo-locating devices capturing location and orientation data for each of the images corresponding to the location and orientation of the one or more image capturing devices capturing the images;

one or more LIDAR system capturing three-dimensional LIDAR data of the geographic area such that the three-dimensional LIDAR data includes the object of interest, the three-dimensional LIDAR data having a second resolution less than the first resolution;

one or more non-transitory computer readable medium storing the three-dimensional LIDAR data; and

a computer system analyzing the images to determine three dimensional locations of points on the object of interest and updating the three-dimensional LIDAR data with the three dimensional locations of points on the object of interest determined by analyzing the images to create a 3D point cloud having a resolution greater than the second resolution of the three-dimensional LIDAR data.

11. The system of claim 10, further comprising a platform upon which the one or more image capturing devices and the one or more LIDAR system are mounted.

12. The system of claim 11, wherein the platform is unmanned.

13. The system of claim 11, wherein the platform is manned.

14. The system of claim 10, further comprising an airplane upon which the one or more image capturing devices and the one or more LIDAR system are mounted such that the images and three-dimensional LIDAR data are captured by flying over the geographic area with the airplane.

15. The system of claim 10, wherein the object of interest includes a utility tower, and wherein analyzing the images is defined further as utilizing GIS data of a utility network to assist in locating the object of interest within one or more images.

16. The system of claim 10, wherein the object of interest includes a utility tower, and wherein analyzing the images is defined further as scanning the images with an edge detection algorithm to locate utility wires depicted within the images, prior to determining three dimensional location of points on the utility tower.

17. A system, comprising:

a computer system analyzing a 3D LIDAR point cloud having information indicative of a side of an object of interest, a side of a background object behind the object of interest, and a ground surface, the 3D LIDAR point cloud having points contributed by a LIDAR scanner having a first resolution, and images having a second resolulution greater than the first resolution, the 3D LIDAR point cloud having a resolution greater than the first resolution, the points having X, Y and Z location information and wherein the computer system uses predetermined algorithms to analyze the points within the 3D LIDAR point cloud to locate the object of interest, the background object, and the ground surface within the 3D LIDAR point cloud, and classify the points into at least three categories including the points representing the object of interest in an objects of interest category, the points representing the background object into a background structure category, and the points representing the ground surface into a surface point category.

18. The system of claim 17, wherein the classification is accomplished using an elevation threshold in which points having a Z value below the elevation threshold are classified as at least one of a background structure category and the surface point category.

19. The system of claim 17, wherein the classification is accomplished using an algorithm that looks for an increased number of points at a predetermined radius from a search point having unclassified points.