System and method for detecting features in aerial images using disparity mapping and segmentation techniques

• US 10,540,577 B2
• Filed: 06/13/2017
• Issued: 01/21/2020
• Est. Priority Date: 08/02/2013
• Status: Active Grant

• Current Assignee: Xactware Solutions, Inc. (Verisk Analytics Incorporated)
• Original Assignee: Xactware Solutions, Inc. (Verisk Analytics Incorporated)
• Inventors: Taylor, Jeffrey, Webecke, Edmund
• Primary Examiner(s): Chawan, Sheela C
• Application Number:
  US15/621,458
• Publication Number:
  US 20170277980A1
• Time in Patent Office: 952 Days
• Field of Search: 382100, 382113, 382224, 382190, 382103, 382155, 382154, 382294, 382278, 382181, 382173
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

A system for aerial image detection and classification is provided herein. The system comprising an aerial image database storing one or more aerial images electronically received from one or more image providers, and an object detection pre-processing engine in electronic communication with the aerial image database, the object detection pre-processing engine detecting and classifying objects using a disparity mapping generation sub-process to automatically process the one or more aerial images to generate a disparity map providing elevation information, a segmentation sub-process to automatically apply a pre-defined elevation threshold to the disparity map, the pre-defined elevation threshold adjustable by a user, and a classification sub-process to automatically detect and classify objects in the one or more stereoscopic pairs of aerial images by applying one or more automated detectors based on classification parameters and the pre-defined elevation threshold.

Claims

1. A system for aerial image detection and classification, comprising:

an aerial image database storing one or more aerial images electronically received from one or more image providers; and

a computer system in communication with the aerial image database, the computer system including an object detection pre-processing engine in electronic communication with the aerial image database, the object detection pre-processing engine detecting and classifying objects using;

a disparity mapping generation module for automatically processing an aerial image of the one or more aerial images to generate a disparity map providing elevation information;

a segmentation module for automatically detecting changes in different parts of the aerial image; and

a classification module for automatically detecting and classifying objects in the aerial image by applying one or more automated detectors based on the elevation information in the disparity map and the detected changes in aerial image detected by the segmentation module.

2. The system of claim 1, wherein the classification module relies on a plurality of classification parameters from at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

3. The system of claim 1, further comprising a mass production engine in electronic communication with the object detection pre-processing engine, the mass production engine automatically producing drawings, sketches, and models using the elevation information generated by the object detection pre-processing engine.

4. The system of claim 1, further comprising a project and task management system in electronic communication with the object detection pre-processing engine, the project and task management system guiding a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine.

5. The system of claim 1, wherein the disparity mapping generation module of the object detection pre-processing engine generates a point cloud.

6. The system of claim 5, wherein the object detection pre-processing engine generates the point cloud by computing point heights using an eye-ray method.

7. The system of claim 1, wherein the segmentation module detects changes by determining whether the aerial image is uniform or non-uniform.

8. The system of claim 7, wherein if the aerial image is non-uniform, the aerial image is split into a plurality of smaller portions.

9. The system of claim 8, wherein if any of the plurality of smaller portions are non-uniform, continuously split the plurality of smaller portions that are non-uniform until all portions of the aerial image are uniform.

10. A method for aerial image detection and classification, comprising:

electronically receiving at a computer system including an object detection pre-processing engine one or more aerial images from an aerial image database, the one or more aerial images electronically received at the aerial image database from one or more image providers;

automatically processing, by the object detection pre-processing engine, an aerial image of the one or more aerial images to generate a disparity map providing elevation information;

automatically applying, by the object detection pre-processing engine, a segmentation module for detecting changes in different parts of the aerial image; and

automatically detecting and classifying, by the object detection pre-processing engine, objects in the aerial image by applying one or more automated detectors based on the elevation information in the disparity map and the detected changes in the aerial image detected by the segmentation module.

11. The method of claim 10, further comprising the step of classifying objects in the aerial image based on a plurality of classification parameters from at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

12. The method of claim 10, further comprising the step of automatically producing, by a mass production engine, drawings, sketches, and models by using the elevation information generated by the object detection pre-processing engine, the mass production engine in electronic communication with the object detection pre-processing engine.

13. The method of claim 10, further comprising the step of guiding, by a project and task management system, a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine, the project and task management system in electronic communication with the object detection pre-processing engine.

14. The method of claim 10, further comprising the step of generating a point cloud by the object detection pre-processing engine.

15. The method of claim 14, further comprising the step of generating the point cloud by computing point heights using an eye-ray method.

16. The method of claim 10, further comprising the step of detecting changes by determining whether the aerial image is uniform or non-uniform.

17. The method of claim 16, further comprising the step of splitting the aerial image into a plurality of smaller portions if the aerial image is non-uniform.

18. The method of claim 17, further comprising the step of splitting the plurality of smaller portions that are non-uniform until all portions of the aerial image are uniform.

19. A non-transitory computer-readable medium having computer-readable instructions stored thereon which, when executed by a computer system having an object detection pre-processing engine, cause the computer system to perform the steps of:

electronically receiving one or more aerial images from an aerial image database, the one or more aerial images electronically received at the aerial image database from one or more image providers;

automatically processing, by the object detection pre-processing engine, an aerial image of the one or more aerial images to generate a disparity map providing elevation information;

automatically applying, by the object detection pre-processing engine, a segmentation module for detecting changes in different parts of the aerial image; and

automatically detecting and classifying, by the object detection pre-processing engine, objects in the aerial image by applying one or more automated detectors based on the elevation information in the disparity map and the detected changes in the aerial image detected by the segmentation module.

20. The non-transitory computer-readable medium of claim 19, further comprising the step of classifying objects in the aerial image based on a plurality of classification parameters from at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

21. The non-transitory computer-readable medium of claim 19, further comprising the step of producing, automatically, by a mass production engine, drawings, sketches, and models by using the elevation information generated by the object detection pre-processing engine, the mass production engine in electronic communication with the object detection pre-processing engine.

22. The non-transitory computer-readable medium of claim 19, further comprising the step of guiding, by a project and task management system, a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine, the project and task management system in electronic communication with the object detection pre-processing engine.

23. The non-transitory computer-readable medium of claim 19, further comprising the step of generating a point cloud by the object detection pre-processing engine.

24. The non-transitory computer-readable medium of claim 19, further comprising the step of generating the point cloud by computing point heights using an eye-ray method.