System and method for detecting features in aerial images using disparity mapping and segmentation techniques

• US 9,679,227 B2
• Filed: 08/04/2014
• Issued: 06/13/2017
• Est. Priority Date: 08/02/2013
• Status: Active Grant

• Current Assignee: Xactware Solutions, Inc. (Verisk Analytics Incorporated)
• Original Assignee: Xactware Solutions, Inc. (Verisk Analytics Incorporated)
• Inventors: Taylor, Jeffrey, Webecke, Edmund
• Primary Examiner(s): Chawan, Sheela C
• Application Number:
  US14/450,861
• Publication Number:
  US 20150347872A1
• Time in Patent Office: 1,044 Days
• Field of Search: 382100, 382113, 382224, 382190, 382103, 382155, 382294, 382278, 382181, 382173, 382154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

A system for aerial image detection and classification is provided herein. The system comprising an aerial image database storing one or more aerial images electronically received from one or more image providers, and an object detection pre-processing engine in electronic communication with the aerial image database, the object detection pre-processing engine detecting and classifying objects using a disparity mapping generation sub-process to automatically process the one or more aerial images to generate a disparity map providing elevation information, a segmentation sub-process to automatically apply a pre-defined elevation threshold to the disparity map, the pre-defined elevation threshold adjustable by a user, and a classification sub-process to automatically detect and classify objects in the one or more stereoscopic pairs of aerial images by applying one or more automated detectors based on classification parameters and the pre-defined elevation threshold.

Claims

1. A system for aerial image detection and classification, comprising:

an aerial image database storing one or more aerial images electronically received from one or more image providers; and

a computer system in communication with the aerial image database, the computer system including an object detection pre-processing engine in electronic communication with the aerial image database, the object detection pre-processing engine detecting and classifying objects using;

a disparity mapping generation module for automatically processing the one or more aerial images to generate a disparity map providing elevation information;

a segmentation module for automatically applying a pre-defined elevation threshold to the disparity map, the pre-defined elevation threshold adjustable by a user; and

a classification module for automatically detecting and classifying objects in the one or more stereoscopic pairs of aerial images by applying one or more automated detectors based on classification parameters and the pre-defined elevation threshold;

wherein, if the height of an object is above the pre-defined elevation threshold, then the classification module automatically classifies the object as a building, and if the height of the object is below the pre-defined elevation threshold, then the classification module applies the one or more automated detectors to the object in order to classify the object.

2. The system of claim 1, wherein the classification parameters include at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

3. The system of claim 1, further comprising a mass production engine in electronic communication with the object detection pre-processing engine, the mass production engine automatically producing drawings, sketches, and models using the elevation information generated by the object detection pre-processing engine.

4. The system of claim 1, further comprising a project and task management system in electronic communication with the object detection pre-processing engine, the project and task management system guiding a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine.

5. The system of claim 1, wherein the one or more automated detects includes at least one of a region growing algorithm or a split-and-merge segmentation algorithm.

6. The system of claim 1, wherein the disparity mapping generation module of the object detection pre-processing engine generates a point cloud.

7. The system of claim 6, wherein the object detection pre-processing engine generates the point cloud by computing point heights using an eye-ray method.

8. A method for aerial image detection and classification, comprising:

electronically receiving at a computer system including an object detection pre-processing engine one or more aerial images from an aerial image database, the one or more aerial images electronically received at the aerial image database from one or more image providers;

automatically processing, by the object detection pre-processing engine, the one or more aerial images to generate a disparity map providing elevation information;

automatically applying, by the object detection pre-processing engine, a pre-defined elevation threshold to the disparity map, the pre-defined elevation threshold adjustable by a user; and

automatically detecting and classifying, by the object detection pre-processing engine, objects in the one or more stereoscopic pairs of aerial images by applying one or more automated detectors based on classification parameters and the pre-defined elevation threshold

wherein the step of automatically detecting and classifying the object comprises automatically classifying an object as a building if a height of the object is above the pre-defined elevation threshold, and applying the one or more automated detectors to classify the object if the height of the object is below the pre-defined elevation threshold.

9. The method of claim 8, wherein the classification parameters include at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

10. The method of claim 8, further comprising automatically producing, by a mass production engine, drawings, sketches, and models by using the elevation information generated by the object detection pre-processing engine, the mass production engine in electronic communication with the object detection pre-processing engine.

11. The method of claim 8, further comprising guiding, by a project and task management system, a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine, the project and task management system in electronic communication with the object detection pre-processing engine.

12. The method of claim 8, wherein the engine executes at least one of a region growing algorithm or a split-and-merge segmentation algorithm.

13. The method of claim 8, further comprising generating a point cloud by the object detection pre-processing engine.

14. The method of claim 13, wherein the object detection pre-processing engine generates the point cloud by computing point heights using an eye-ray method.

15. A non-transitory computer-readable medium having computer-readable instructions stored thereon which, when executed by a computer system having an object detection pre-processing engine, cause the computer system to perform the steps of:

electronically receiving one or more aerial images from an aerial image database, the one or more aerial images electronically received at the aerial image database from one or more image providers;

automatically processing, by the object detection pre-processing engine, the one or more aerial images to generate a disparity map providing elevation information;

automatically applying, by the object detection pre-processing engine, a pre-defined elevation threshold to the disparity map, the pre-defined elevation threshold adjustable by a user; and

automatically detecting and classifying, by the object detection pre-processing engine, objects in the one or more stereoscopic pairs of aerial images by applying one or more automated detectors based on classification parameters and the pre-defined elevation threshold;

wherein the step of automatically detecting and classifying the object comprises automatically classifying an object as a building if a height of the object is above the pre-defined elevation threshold, and applying the one or more automated detectors to classify the object if the height of the object is below the pre-defined elevation threshold.

16. The non-transitory computer-readable medium of claim 15, wherein the classification parameters include at least one of aspect parameters, disparity parameters, color parameters, or texture parameters.

17. The non-transitory computer-readable medium of claim 15, further comprising automatically producing, by a mass production engine, drawings, sketches, and models by using the elevation information generated by the object detection pre-processing engine, the mass production engine in electronic communication with the object detection pre-processing engine.

18. The non-transitory computer-readable medium of claim 15, further comprising guiding, by a project and task management system, a user in creating and publishing one or more projects using the elevation information generated by the object detection pre-processing engine, the project and task management system in electronic communication with the object detection pre-processing engine.

19. The non-transitory computer-readable medium of claim 15, wherein the engine executes at least one of a region growing algorithm or a split-and-merge segmentation algorithm.

20. The non-transitory computer-readable medium of claim 15, further comprising generating a point cloud by the object detection pre-processing engine.

21. The non-transitory computer-readable medium of claim 20, wherein the object detection pre-processing engine generates the point cloud by computing point heights using an eye-ray method.