Generating a mission plan for capturing aerial images with an unmanned aerial vehicle

• US 10,008,123 B2
• Filed: 07/25/2017
• Issued: 06/26/2018
• Est. Priority Date: 10/20/2015
• Status: Active Grant

• Current Assignee: Skycatch, Inc.
• Original Assignee: Skycatch, Inc.
• Inventors: Chen, David
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/659,133
• Publication Number:
  US 20170337824A1
• Time in Patent Office: 336 Days
• Field of Search: 701 2, 701 3, 701 8, 701 9, 701 25, 701 26, 701400, 701408, 701409, 701410, 701528, 701533, 348144, 348153, 367124
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G08G 5/0034 Assembly of a flight plan
• CPC Class Codes: G08G 5/006 in accordance with predefined flight zones, e.g. to avoid prohibited zones
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935

Abstract

Systems and methods are disclosed for generating a digital flight path within complex mission boundaries. In particular, in one or more embodiments, systems and methods generate flight legs that traverse a target site within mission boundaries. Moreover, one or more embodiments include systems and methods that utilize linking algorithms to connect the generated flight legs into a flight path. Moreover, one or more embodiments include systems and methods that generate a mission plan based on the flight path. In one or more embodiments, the generated mission plan enables a UAV to traverse a flight area within mission boundaries and capture aerial images with regard to the target site. Furthermore, in one or more embodiments, systems and methods capture digital aerial images of vertical surfaces of a structure by generating a reference surface and flight legs corresponding to the reference surface.

Claims

1. A computer-implemented method comprising:

generating, by at least one processor, a three-dimensional representation of a target site based on a plurality of digital aerial images of the target site captured by a UAV;

identifying, by the at least one processor, a structure on the target site based on the three-dimensional representation;

generating, by the at least one processor, a mission plan for capturing digital aerial images of the structure from different positions and different elevations by;

identifying a vertical surface of the structure from the three-dimensional representation;

generating a reference surface from the vertical surface by offsetting the vertical surface of the structure based on a camera resolution; and

generating a plurality of flight legs that traverse the different elevations based on the reference surface;

capturing a second plurality of digital aerial images of the structure at the different elevations by causing the UAV to fly the mission plan; and

generating, by the at least one processor, a second three-dimensional representation of the structure based on the second plurality of digital aerial images of the structure.

2. The method of claim 1, wherein generating the three-dimensional representation comprises:

generating a plurality of low-resolution digital aerial images from the plurality of digital aerial images by scaling down the plurality of digital aerial images of the target site captured by the UAV; and

utilizing the low-resolution digital aerial images to generate a low-resolution three-dimensional representation.

3. The method of claim 1, wherein identifying the structure comprises:

providing the three-dimensional representation for display to a computing device of a user;

receiving a user interaction with the three-dimensional representation; and

identifying the structure on the target site based on the user interaction with the three-dimensional representation.

4. The method of claim 1, wherein:

identifying the vertical surface of the structure comprises identifying a plurality of vertical surfaces of the structure from the three-dimensional representation; and

generating the reference surface comprises offsetting the plurality of vertical surfaces of the structure based on the camera resolution to generate a three-dimensional shape.

5. The method of claim 4, wherein generating the plurality of flight legs corresponding to the reference surface comprises:

projecting the three-dimensional shape to a two-dimensional space;

determining a leg spacing based on an amount of overlap between digital aerial images; and

utilizing the leg spacing, generating parallel flight legs on the three-dimensional shape projected on the two-dimensional space.

6. The method of claim 1, wherein generating the mission plan comprises:

identifying a portion of the vertical surface below a minimum flight altitude;

determining a camera angle to capture the portion of the vertical surface below the minimum flight altitude from a position above the minimum flight altitude;

determining an offset of the vertical surface based on the camera resolution and the camera angle; and

generating a flight leg based on the offset.

7. The method of claim 4, wherein generating the reference surface comprises:

detecting a variation in width of the structure with respect to elevation;

comparing the variation in width of the structure with a variation threshold;

if the variation satisfies the variation threshold, generating the three-dimensional shape such that the three-dimensional shape comprises a change in width corresponding to the variation in width of the structure; and

if the variation does not satisfy the variation threshold, generating the three-dimensional shape such that the three-dimensional shape does not comprise a change in width corresponding to the variation in width of the structure.

8. A system comprising:

at least one processor; and

at least one non-transitory computer readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to;

generate a three-dimensional model of a target site based on a plurality of digital aerial images of the target site captured by a UAV;

identify a structure on the target site based on the three-dimensional model of the target site;

generate a mission plan for capturing digital aerial images of the structure from different positions and different elevations by;

identifying a vertical surface of the structure from the three-dimensional model;

generating a reference surface from the vertical surface by offsetting the vertical surface of the structure based on a camera resolution; and

generating a plurality of flight legs that traverse the different elevations based on the reference surface;

capturing a second plurality of digital aerial images of the structure at the different elevations by causing the UAV to fly the mission plan; and

generating a second three-dimensional model of the structure based on the second plurality of digital aerial images of the structure.

9. The system of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to generate the three-dimensional model by:

generating a plurality of low-resolution digital aerial images from the plurality of digital aerial images by scaling down the plurality of digital aerial images of the target site captured by the UAV; and

utilizing the low-resolution digital aerial images to generate a low-resolution three-dimensional model.

10. The system of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to identify the structure by:

providing the three-dimensional model for display to a computing device of a user;

receiving a user interaction with the three-dimensional model; and

identifying the structure on the target site based on the user interaction with the three-dimensional model.

11. The system of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to:

identify the vertical surface of the structure by identifying a plurality of vertical surfaces of the structure from the three-dimensional model; and

generate the reference surface by offsetting the plurality of vertical surfaces of the structure based on the camera resolution to generate a three-dimensional shape.

12. The system of claim 11, further comprising instructions that, when executed by the at least one processor, cause the system to generate the plurality of flight legs by:

projecting the three-dimensional shape to a two-dimensional space; and

generating parallel flight legs on the three-dimensional shape projected on the two-dimensional space at a leg spacing based on an amount of overlap between digital aerial images.

13. The system of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to generate the mission plan by:

determining a first offset corresponding to a portion of the vertical surface above a minimum flight altitude;

generating a first flight leg based on the first offset;

identifying a portion of the vertical surface below the minimum flight altitude;

determining a camera angle to capture a digital aerial image of the portion of the vertical surface below the minimum flight altitude from a position above the minimum flight altitude;

determining a second offset of the vertical surface based on the camera resolution and the camera angle; and

generating a second flight leg based on the second offset.

14. The system of claim 11, further comprising instructions that, when executed by the at least one processor, cause the system to generate the reference surface by:

detecting a variation in width of the structure with respect to elevation;

comparing the variation in width of the structure with a variation threshold;

if the variation satisfies the variation threshold, generating the three-dimensional shape such that the three-dimensional shape comprises a change in width corresponding to the variation in width of the structure; and

if the variation does not satisfy the variation threshold, generating the three-dimensional shape such that the three-dimensional shape does not comprise a change in width corresponding to the variation in width of the structure.

15. A computer-implemented method comprising:

while a UAV is flying a mission plan of a target site, generating, by at least one processor, a three-dimensional model of the target site based on a plurality of digital aerial images of the target site captured by the UAV during the mission plan;

identifying, by the at least one processor, a structure on the target site based on the three-dimensional model;

while the UAV is flying the mission plan, modifying, by the at least one processor, the mission plan to capture digital aerial images of the structure from different positions and different elevations by;

identifying a vertical surface of the structure from the three-dimensional model;

generating a reference surface by offsetting the vertical surface of the structure; and

generating a plurality of flight legs that traverse the different elevations based on the reference surface; and

capturing a second plurality of digital aerial images of the structure at the different elevations by causing the UAV to fly the modified mission plan; and

generating a second three-dimensional model of the structure based on the second plurality of digital aerial images of the structure.

16. The method of claim 15, wherein generating the three-dimensional model comprises:

generating a plurality of low-resolution digital aerial images from the plurality of digital aerial images by scaling down the plurality of digital aerial images of the target site captured by the UAV; and

utilizing the low-resolution digital aerial images to generate a low-resolution three-dimensional model while the UAV is flying the mission plan.

17. The method of claim 15, wherein identifying the structure comprises:

providing the three-dimensional model for display to a computing device of a user while the UAV is flying the mission plan;

receiving a user interaction with the three-dimensional model while the UAV is flying the mission plan; and

identifying the structure on the target site based on the user interaction with the three-dimensional model.

18. The method of claim 15, wherein generating the plurality of flight legs comprises:

projecting the reference surface from a three-dimensional space to a two-dimensional space;

generating parallel flight legs on the reference surface projected on the two-dimensional space at a leg spacing based on an amount of overlap between digital aerial images; and

projecting the parallel flight legs to the three-dimensional space.

19. The method of claim 15, wherein generating the reference surface comprises:

determining a first offset corresponding to a portion of the vertical surface above a minimum flight altitude;

generating a first flight leg based on the first offset;

identifying a portion of the vertical surface below the minimum flight altitude;

determining a camera angle to capture a digital aerial image of the portion of the vertical surface below the minimum flight altitude from a position above the minimum flight altitude;

determining a second offset of the vertical surface based on a camera resolution and the camera angle; and

generating a second flight leg based on the second offset.

20. The method of claim 15, further comprising:

detecting a variation in width of the structure with respect to elevation;

comparing the variation in width of the structure with a variation threshold;

if the variation satisfies the variation threshold, generating the reference surface such that the reference surface comprises a change in width corresponding to the variation in width of the structure; and

if the variation does not satisfy the variation threshold, generating the reference surface such that the reference surface does not comprise a change in width corresponding to the variation in width of the structure.