Precision aware drone-based object mapping based on spatial pattern recognition

• US 10,769,466 B2
• Filed: 02/20/2018
• Issued: 09/08/2020
• Est. Priority Date: 02/20/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Britto Mattos Lima, Andrea, Cardonha, Carlos Henrique, Viana, Matheus Palhares, Zortea, Maciel
• Primary Examiner(s): Dager, Jonathan M
• Application Number:
  US15/899,919
• Publication Number:
  US 20190258883A1
• Time in Patent Office: 931 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/042 specially adapted for aircraft
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers

Abstract

An image of a region captured by an unmanned aerial vehicle flying at an altitude may be received. A computer vision algorithm may be executed with the image as an input to compute an overall confidence score associated with detecting one or more candidate objects in the image. Responsive to determining that the confidence score is below a predefined minimum threshold or above a predefined maximum threshold, the unmanned aerial vehicle may be controlled to change its altitude and recapture the image of the region at a new position. Responsive to determining that the overall confidence score is not below the predefined minimum threshold, information associated with the image may be stored on a storage device.

Claims

1. A method of controlling an unmanned aerial vehicle, the method executed by at least one hardware processor coupled to the unmanned aerial vehicle and a storage device, the method comprising:

receiving an image of a region captured by the unmanned aerial vehicle flying at a current altitude;

executing a computer vision algorithm with the image as an input to the computer vision algorithm, the computer vision algorithm computing an overall confidence score associated with detecting one or more candidate objects in the image;

responsive to determining that the overall confidence score is below a predefined minimum threshold, controlling the unmanned aerial vehicle to reduce the current altitude and recapture the image of the region at the reduced altitude;

responsive to determining that the overall confidence score is not below the predefined minimum threshold, storing on a storage device the image, the one or more objects detected in the image, the confidence score, location coordinates of the unmanned aerial vehicle, and the current altitude;

repeating the receiving, the executing, the triggering and the storing while the region is not fully mapped with images and the unmanned aerial vehicle has energy capacity above a level needed to return to a designated base location.

2. The method of claim 1, further comprising:

responsive to determining that the overall confidence score is above a predefined maximum threshold, controlling the unmanned aerial vehicle to increase the current altitude.

3. The method of claim 1, wherein responsive to determining that the region is fully mapped with images, but the unmanned aerial vehicle has energy capacity above the level needed to return to the designated base location, computing a route for the unmanned aerial vehicle to revisit the region to recapture images at higher resolution.

4. The method of claim 1, wherein the computer vision algorithm processes the image in real-time to detect spatial organization patterns in the image, and locates the one or more candidate objects in the image based on the spatial organization patterns.

5. The method of claim 1, wherein the computer vision algorithm detects a plurality of candidate objects in the image and the overall confidence score represents an aggregated confidence score associated with detecting the plurality of candidate objects.

6. The method of claim 1, wherein the computer vision algorithm comprises detecting in the image, user-defined patterns comprising geometric shapes comprising at least one of lines, ellipses, and squares.

7. A computer readable storage medium storing a program of instructions executable by a machine to perform a method of controlling an unmanned aerial vehicle, the method comprising:

receiving an image of a region captured by the unmanned aerial vehicle flying at a current altitude;

executing a computer vision algorithm with the image as an input to the computer vision algorithm, the computer vision algorithm computing an overall confidence score associated with detecting one or more candidate objects in the image;

responsive to determining that the overall confidence score is below a predefined minimum threshold, controlling the unmanned aerial vehicle to reduce the current altitude and recapture the image of the region at the reduced altitude;

responsive to determining that the overall confidence score is not below the predefined minimum threshold, storing on a storage device the image, the object detected in the image, the confidence score, location coordinates of the unmanned aerial vehicle, and the current altitude;

repeating the receiving, the executing, the triggering and the storing while the region is not fully mapped with images and the unmanned aerial vehicle has energy capacity above a level needed to return to a designated base location.

8. The computer readable storage medium of claim 7, further comprising:

responsive to determining that the overall confidence score is above a predefined maximum threshold, controlling the unmanned aerial vehicle to increase the current altitude.

9. The computer readable storage medium of claim 7, wherein responsive to determining that the region is fully mapped with images, but the unmanned aerial vehicle has energy capacity above the level needed to return to the designated base location, computing a route for the unmanned aerial vehicle to revisit the region to recapture images at higher resolution.

10. The computer readable storage medium of claim 7, wherein the computer vision algorithm processes the image in real-time to detect spatial organization patterns in the image, and locates the candidate object in the image based on the spatial organization patterns.

11. The computer readable storage medium of claim 7, wherein the computer vision algorithm detects a plurality of candidate objects in the image and the overall confidence score represents an aggregated confidence score associated with detecting the plurality of candidate objects.

12. The computer readable storage medium of claim 7, wherein the computer vision algorithm comprises detecting in the image, user-defined patterns comprising geometric shapes comprising at least one of lines, ellipses, and squares.

13. A system of controlling an unmanned aerial vehicle, comprising:

at least one hardware processor coupled to an unmanned aerial vehicle; and

a storage device coupled to the at least one hardware processor;

the at least one hardware processor operable to perform;

receiving an image of a region captured by the unmanned aerial vehicle flying at a current altitude;

executing a computer vision algorithm with the image as an input to the computer vision algorithm, the computer vision algorithm computing an overall confidence score associated with detecting one or more candidate objects in the image;

responsive to determining that the overall confidence score is below a predefined minimum threshold, controlling the unmanned aerial vehicle to reduce the current altitude and recapture the image of the region at the reduced altitude;

responsive to determining that the overall confidence score is not below the predefined minimum threshold, storing on a storage device the image, the object detected in the image, the confidence score, location coordinates of the unmanned aerial vehicle, and the current altitude;

repeating the receiving, the executing, the triggering and the storing while the region is not fully mapped with images and the unmanned aerial vehicle has energy capacity above a level needed to return to a designated base location.

14. The system of claim 13, wherein the hardware processor is further operable to, responsive to determining that the overall confidence score is above a predefined maximum threshold, control the unmanned aerial vehicle to increase the current altitude.

15. The system of claim 13, wherein the hardware processor is further operable to, responsive to determining that the region is fully mapped with images, but the unmanned aerial vehicle has energy capacity above the level needed to return to the designated base location, compute a route for the unmanned aerial vehicle to revisit the region to recapture images at higher resolution.

16. The system of claim 13, wherein the computer vision algorithm processes the image in real-time to detect spatial organization patterns in the image, and locates the one or more candidate objects in the image based on the spatial organization patterns.

17. The system of claim 13, wherein the computer vision algorithm detects a plurality of candidate objects in the image and the overall confidence score represents an aggregated confidence score associated with detecting the plurality of candidate objects.

18. The system of claim 13, wherein the computer vision algorithm comprises detecting in the image, user-defined patterns comprising geometric shapes comprising at least one of lines, ellipses, and squares.