Automatic flight control for UAV based solid modeling

• US 9,352,833 B2
• Filed: 12/17/2014
• Issued: 05/31/2016
• Est. Priority Date: 09/30/2010
• Status: Expired due to Fees

• Current Assignee: Fury Technologies LLC (IP Edge LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Kruglick, Ezekiel
• Primary Examiner(s): ARTHUR JEANGLAUD, GERTRUDE
• Application Number:
  US14/574,329
• Publication Number:
  US 20150112516A1
• Time in Patent Office: 531 Days
• Field of Search: 701/2, 701/3, 701/11, 244/190
• US Class Current: 1/1
• CPC Class Codes: B64C 19/00 Aircraft control not otherwise provided for
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 2210/61 Scene description

Abstract

Technologies are generally described for controlling a flight path of a UAV based image capture system for solid modeling. Upon determining an initial movement path based on an estimate of a structure to be modeled, images of the structure to be modeled may be captured and surface hypotheses formed for unobserved surfaces based on the captured images. A normal vector and a viewing cone may be computed for each hypothesized surface. A set of desired locations may be determined based on the viewing cones for the entire structure to be modeled and a least impact path for the UAV determined based on the desired locations and desired flight parameters.

Claims

1. A computing device configured to control a flight path for an unmanned aerial vehicle (UAV) adapted to capture surface images for three-dimensional (3D) modeling, the computing device comprising:

a communication module configured to enable communication between the computing device and one or more UAVs;

a memory configured to store one or more instructions;

a processor coupled to the memory, the processor configured to execute a UAV control application in conjunction with the one or more instructions, wherein the UAV control application is configured to;

determine an initial movement path for the one or more UAVs based on an estimate of a structure to be modeled;

receive, through the communication module, one or more images of the structure to be modeled, wherein the one or more images are captured by one or more cameras onboard the one or more UAVs;

form surface hypotheses for unobserved surfaces based on the received one or more images;

determine missing surface information from the surface hypotheses; and

determine one or more least impact flight paths for the one or more UAVs based on the missing surface information and one or more flight parameters.

2. The computing device of claim 1, wherein the UAV control application is configured to determine the initial movement path for the one or more UAVs based on one of user definition and automatic estimation of an outline of the structure to be modeled.

3. The computing device of claim 1, wherein the UAV control application is further configured to compute a normal vector and a viewing cone for each of the surface hypotheses.

4. The computing device of claim 3, wherein, to compute the normal vector, the UAV control application is configured to use an average normal from lowest quality data gathered from the received one or more images.

5. The computing device of claim 3, wherein, to compute the normal vector for non-planar surfaces, the UAV control application is configured to:

break down the non-planar surfaces into polygonal cells; and

assign a similar score to each cell.

6. The computing device of claim 5, wherein the break down of the non-planar surfaces is a Voronoi tessellation, and the UAV control application is further configured to:

employ a largest area in a resulting Voronoi diagram with a score below a particular threshold as a surface for normal vector generation, and

extend the generated normal vector to define the viewing cone based on optical capabilities of the one or more cameras onboard the UAV, a distance of the one or more UAVs to the surface, and lighting conditions.

7. The computing device of claim 3, wherein the UAV control application is further configured to determine one or more locations to be included in one or more flight paths of the one or more UAVs based on the computed normal vector and viewing cone for each of the surface hypotheses.

8. The computing device of claim 7, wherein the UAV control application is further configured to adjust the one or more least impact paths for the one or more UAVs based on a comparison of the one or more flight parameters and the determined one or more locations.

9. The computing device of claim 3, wherein the UAV control application is further configured to determine a confidence interval associated with the viewing cone for each of the surface hypotheses, the confidence interval defining one or more of a priority and a value of the one or more UAVs visiting the viewing cone.

10. The computing device of claim 1, wherein the UAV control application is further configured to employ a Random Sample Consensus Surface Extraction (RANSAC) algorithm to determine the surface hypotheses.

11. An unmanned aerial vehicle (UAV) system for 3D modeling, the system comprising:

a UAV capable of communication with a controller, the UAV configured to;

receive an initial movement path from the controller, wherein the initial movement path is based on an estimate of a structure to be modeled;

capture one or more images of the structure to be modeled, wherein the one or more images are captured by one or more cameras onboard the UAV;

transmit the captured one or more images to the controller, wherein the captured one or more images are used to form a surface hypotheses for unobserved surfaces, and wherein missing surface information is determined from the surface hypotheses; and

receive a least impact path for the UAV from the controller, wherein the least impact path is based on the missing surface information and desired flight parameters.

12. The system of claim 11, wherein the controller is part of a remote computing device and the UAV is further configured to communicate with the controller via wireless means.

13. The system of claim 11, wherein the one or more flight parameters include one or more of a total flight time, available UAV power, a noise limitation, and a user defined restriction.

14. An unmanned aerial vehicle (UAV) flight control system for three-dimensional (3D) modeling, the system comprising:

a UAV with one or more onboard cameras;

a controller capable of communication with the UAV, the controller configured to;

determine an initial movement path based on an estimate of a structure to be modeled;

receive one or more images of the structure to be modeled, wherein the one or more images are captured by the one or more onboard cameras of the UAV;

form surface hypotheses for unobserved surfaces or for enhanced observation of surfaces based on the received one or more images;

compute a normal vector and a viewing cone for each of the surface hypotheses;

determine, based on the computed normal vector and viewing cone for each of the surface hypotheses, one or more locations to be included in a flight path of the UAV; and

determine a least impact path for the UAV based on the one or more locations and one or more flight parameters.

15. The system of claim 14, wherein the controller is further configured to compute the viewing cone for each of the surface hypotheses using the normal vector and a view angle.

16. The system of claim 15, wherein the controller is further configured to dynamically determine the view angle as a goodness gradient based on optical capabilities of the one or more onboard cameras, a distance of the UAV to the surface, and lighting conditions such that a goodness parameter for the flight path is optimized.

17. The system of claim 16, wherein the goodness parameter is derived from a quality of the received one or more images of the structure to be modeled and the one or more flight parameters.

18. The system of claim 16, wherein the controller is configured to employ multidimensional optimization on the goodness gradient and a cost function representing the one or more flight parameters to select the goodness parameter.

19. The system of claim 18, wherein the multidimensional optimization is one of a lazy optimization and a simulated annealing optimization.

20. The system of claim 16, wherein the goodness gradient is computed once per travel path for each portion of the structure to be modeled.