Methods and systems for automatic object detection from aerial imagery

• US 10,546,195 B2
• Filed: 12/02/2016
• Issued: 01/28/2020
• Est. Priority Date: 12/02/2016
• Status: Active Grant

• Current Assignee: GEOSAT Aerospace & Technology, Co., GEOSAT Aerospace & Technology Inc.
• Original Assignee: GEOSAT Aerospace & Technology Inc.
• Inventors: Lo, Cheng-Fang, Chen, Zih-Siou, Ko, Chang-Rong, Wu, Chun-Yi
• Primary Examiner(s): Safaipour, Bobbak
• Application Number:
  US15/367,975
• Publication Number:
  US 20180157911A1
• Time in Patent Office: 1,152 Days
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30188 Vegetation; Agriculture
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/188 Vegetation

Abstract

Methods and systems for detecting objects from aerial imagery are disclosed. According to certain embodiments, the method may include obtaining a Digital Surface Model (DSM) image of an area. The method may also include obtaining a DSM image of one or more target objects. The method may further include detecting the target object in the area based on the DSM images of the area and the one or more target objects. The method may further include recognizing the detected target objects by artificial intelligence. The method may further include acquiring the positions of the recognized target objects. The method may further include calculating the number of the recognized target objects.

Claims

1. A non-transitory computer-readable medium storing instructions which, when executed, cause one or more processors to perform operations for detecting objects from aerial imagery, the operations comprising:

obtaining a Digital Surface Model (DSM) image of an area;

obtaining one or more DSM images representing one kind of target objects; and

detecting at least one of the kind of target objects in the area based on one or more matches between the one or more DSM images and the DSM image of the area.

2. The non-transitory computer-readable medium of claim 1,

wherein the one or more DSM images are obtained based on at least one of

shape of target objects in the one or more DSM images, or

contrast of the one or more DSM images.

3. The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:

reducing resolutions of the DSM image of the area and the one or more DSM images before detecting at least one of the kind of target objects.

4. The non-transitory computer-readable medium of claim 1,

wherein obtaining the DSM image of the area further includes:

identifying one or more target subareas on the DSM image of the area;

enhancing contrast of the one or more target subareas; and

wherein detecting at least one of the kind of target objects includes;

detecting at least one of the kind of target objects based on the enhanced DSM image of the area and the one or more DSM images.

5. The non-transitory computer-readable medium of claim 4, wherein identifying the one or more target subareas includes:

obtaining a color aerial image of the area corresponding to the DSM image of the area and one or more color aerial images corresponding to the one or more DSM images;

identifying the one or more target subareas based on the color aerial image of the area and the one or more color images.

6. The non-transitory computer-readable medium of claim 1, wherein detecting at least one of the kind of target objects in the area further includes:

calculating match rates between the one or more DSM images and one or more DSM sub-images of the area; and

determining whether the one or more DSM sub-images of the area are one or more target objects based on the match rates.

7. The non-transitory computer-readable medium of claim 6, wherein determining whether the one or more DSM sub-images of the area are one or more target objects includes determining based on at least one of:

a match rate of a DSM sub-image is a maximum among other DSM sub-images within a first distance;

or

a height of a DSM sub-image is higher than a height of a lowest position within a second distance by a height threshold.

8. The non-transitory computer-readable medium of claim 1, wherein obtaining the DSM image of the area includes:

collecting the DSM of the area by an Unmanned Aerial Vehicle, a drone, an aircraft, a helicopter, a balloon, or a satellite; and

receiving the DSM of the area from the Unmanned Aerial Vehicle, the drone, the aircraft, the helicopter, the balloon, or the satellite.

9. The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:

acquiring one or more positions of the detected at least one of the kind of target objects.

10. The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:

calculating a number of the detected at least one of the kind of target objects.

11. The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:

acquiring one or more regional aerial images at the detected at least one of the kind of target objects;

obtaining a classifier;

classifying the one or more regional aerial images by the classifier; and

recognizing the kind of target objects among the one or more regional aerial images based on the classified results.

12. The non-transitory computer-readable medium of claim 11, wherein obtaining a classifier further includes:

obtaining a plurality of training data; and

training the classifier based thereon.

13. The non-transitory computer-readable medium of claim 12, wherein the plurality of training data includes:

one or more aerial images representing the kind of target objects; and

one or more aerial images representing non-target objects.

14. The non-transitory computer-readable medium of claim 13, wherein classifying the one or more regional aerial images further includes:

extracting one or more feature vectors from the one or more regional aerial images; and

classifying the one or more regional aerial images in accordance with the one or more feature vectors.

15. The non-transitory computer-readable medium of claim 14, wherein extracting the one or more feature vectors includes:

extracting based on Gabor filter, Gray-Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), Histograms of Oriented Gradients (HOG), first-order feature description, second-order feature description, or any combination thereof.

16. The non-transitory computer-readable medium of claim 11,

wherein the one or more regional aerial images are acquired in a first color space and transferred into a second color space.

17. The non-transitory computer-readable medium of claim 14,

wherein acquiring the one or more regional aerial images includes:

acquiring the one or more regional aerial images in a first color space; and

transferring the one or more regional aerial images in the first color space to one or more regional aerial images in a second color space, and

wherein extracting the one or more feature vectors includes extracting from at least one of;

the one or more regional aerial images in the first color space;

or

the one or more regional aerial images in the second color space.

18. The non-transitory computer-readable medium of claim 17,

wherein the first and the second color spaces include RGB, grayscale, HSI, L*a*b, multi-spectral space, or any combination thereof.

19. The non-transitory computer-readable medium of claim 11, wherein acquiring the one or more regional aerial images includes:

obtaining a plurality of aerial images of parts of the area;

stitching the plurality of aerial images of parts of the area to obtain the aerial image of the area; and

acquiring the one or more regional aerial images from the aerial image of the area.

20. The non-transitory computer-readable medium of claim 11,

wherein the classifier includes a Support Vector Machine (SVM) classifier, an Artificial Neural Network (ANN) classifier, a Decision Tree classifier, a Bayes classifier, or any combination thereof.

21. A method for detecting objects from aerial imagery performed by one or more integrated circuits, one or more field programmable gate arrays, one or more processors or controllers executing instructions that implement the method, or any combination thereof, the method comprising:

obtaining a Digital Surface Model (DSM) image of an area;

obtaining one or more DSM images representing one kind of target objects; and

detecting at least one of the kind of target objects in the area based on one or more matches between the one or more DSM images and the DSM image of the area.

22. The method of claim 21,

wherein the one or more DSM images are obtained based on at least one of

shape of target objects in the one or more DSM images, or

contrast of the one or more DSM images.

23. The method of claim 21, further comprising:

reducing resolutions of the DSM image of the area and the one or more DSM images before detecting at least one of the kind of target objects.

24. The method of claim 21,

wherein obtaining the DSM image of the area further includes:

identifying one or more target subareas on the DSM image of the area;

enhancing contrast of the one or more target subareas; and

wherein detecting at least one of the kind of target objects includes;

detecting at least one of the kind of target objects based on the enhanced DSM image of the area and the one or more DSM images.

25. The method of claim 24, wherein identifying the one or more target subareas includes:

obtaining a color aerial image of the area corresponding to the DSM image of the area and one or more color aerial images corresponding to the one or more DSM images;

identifying the one or more target subareas based on the color aerial image of the area and the one or more color images.

26. The method of claim 21, wherein detecting at least one of the kind of target objects in the area further includes:

calculating match rates between the one or more DSM images and one or more DSM sub-images of the area; and

determining whether the one or more DSM sub-images of the area are one or more target objects based on the match rates.

27. The method of claim 26, wherein determining whether the one or more DSM sub-images of the area are one or more target objects includes determining based on at least one of:

a match rate of a DSM sub-image is a maximum among other DSM sub-images within a first distance;

or

a height of a DSM sub-image is higher than a height of a lowest position within a second distance by a height threshold.

28. The method of claim 21, wherein obtaining the DSM image of the area includes:

collecting the DSM of the area by an Unmanned Aerial Vehicle, a drone, an aircraft, a helicopter, a balloon, or a satellite; and

receiving the DSM of the area from the Unmanned Aerial Vehicle, the drone, the aircraft, the helicopter, the balloon, or the satellite.

29. The method of claim 21, further comprising:

acquiring one or more positions of the detected at least one of the kind of target objects.

30. The method of claim 21, further comprising:

calculating a number of the detected at least one of the kind of target objects.

31. The method of claim 21, further comprising:

acquiring one or more regional aerial images at the detected at least one of the kind of target objects;

obtaining a classifier;

classifying the one or more regional aerial images by the classifier; and

recognizing the kind of target objects among the one or more regional aerial images based on the classified results.

32. The method of claim 31, wherein obtaining a classifier further includes:

obtaining a plurality of training data; and

training the classifier based thereon.

33. The method of claim 32, wherein the plurality of training data includes:

one or more aerial images representing the kind of target objects; and

one or more aerial images representing non-target objects.

34. The method of claim 33, wherein classifying the one or more regional aerial images further includes:

extracting one or more feature vectors from the one or more regional aerial images; and

classifying the one or more regional aerial images in accordance with the one or more feature vectors.

35. The method of claim 34, wherein extracting the one or more feature vectors includes:

extracting based on Gabor filter, Gray-Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), Histograms of Oriented Gradients (HOG), first-order feature description, second-order feature description, or any combination thereof.

36. The method of claim 31,

wherein the one or more regional aerial images are acquired in a first color space and transferred into a second color space.

37. The method of claim 34,

wherein acquiring the one or more regional aerial images includes:

acquiring the one or more regional aerial images in a first color space; and

transferring the one or more regional aerial images in the first color space to one or more regional aerial images in a second color space, and

wherein extracting the one or more feature vectors includes extracting from at least one of;

the one or more regional aerial images in the first color space;

or

the one or more regional aerial images in the second color space.

38. The method of claim 37,

wherein the first and the second color spaces include RGB, grayscale, HSI, L*a*b, multi-spectral space, or any combination thereof.

39. The method of claim 31, wherein acquiring the one or more regional aerial images includes:

obtaining a plurality of aerial images of parts of the area;

stitching the plurality of aerial images of parts of the area to obtain the aerial image of the area; and

acquiring the one or more regional aerial images from the aerial image of the area.

40. The method of claim 31,

wherein the classifier includes a Support Vector Machine (SVM) classifier, an Artificial Neural Network (ANN) classifier, a Decision Tree classifier, a Bayes classifier, or any combination thereof.

41. A system for detecting objects from aerial imagery, the system comprising:

an aerial image unit, implemented by one or more processor or one or more circuits, that obtains a DSM image of an area;

a target image unit, implemented by the one or more processor or the one or more circuits, that obtains one or more DSM images representing one kind of target objects; and

a detection unit, implemented by the one or more processor or the one or more circuits, that detects at least one of the kind of target objects in the area based on one or more matches between the one or more DSM images and the DSM image of the area.

42. The system of claim 41, further comprising:

a regional aerial-image acquisition unit, implemented by one or more processor or one or more circuits, that acquires one or more regional aerial images at the detected at least one of the kind of target objects;

a classification and recognition unit, implemented by the one or more processor or the one or more circuits, that;

obtains a classifier;

classifies the one or more regional aerial images by the classifier; and

recognizes the kind of target objects among the one or more regional aerial images based on the classified results.