Structure from motion (SfM) processing for unmanned aerial vehicle (UAV)

• US 10,203,692 B2
• Filed: 07/10/2015
• Issued: 02/12/2019
• Est. Priority Date: 07/21/2014
• Status: Active Grant

• Current Assignee: King Abdullah University of Science and Technology
• Original Assignee: King Abdullah University of Science and Technology
• Inventors: Smith, Neil G., Shalaby, Mohamed, Passone, Luca
• Primary Examiner(s): Heinle, Courtney D
• Application Number:
  US15/327,917
• Publication Number:
  US 20170205826A1
• Time in Patent Office: 1,313 Days
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target

Abstract

A method of imaging an area using an unmanned aerial vehicle (UAV) collects a plurality of images from a sensor mounted to the UAV. The plurality of images are processed to detect regions that require additional imaging and an updated flight plan and sensor gimbal position plan is created to capture portions of the area identified as requiring additional imaging.

Claims

1. A method of imaging an area using an unmanned aerial vehicle (UAV), the method comprising:

collecting a plurality of images from a sensor mounted to the UAV;

processing the plurality of images to detect regions that require additional imaging, wherein processing the plurality of images includes generating a point-cloud reconstruction of the area being imaged and comparing point density of areas to a threshold value wherein a point density of an area less than the threshold value indicates further imaging is required; and

creating a flight plan and sensor gimbal position plan to capture portions of the area identified as requiring additional imaging.

2. The method of claim 1, further including:

receiving from the UAV positional and orientation information that describes the position and orientation of the UAV; and

associating the UAV positional and orientation information with each image collected by the sensor mounted to the UAV.

3. The method of claim 2, further including:

receiving from the UAV gimbal orientation information that describes the orientation of the camera gimbal assembly; and

associating the UAV gimbal orientation information with each image collected by the sensor mounted to the UAV.

4. The method of claim 3, wherein processing the plurality of images includes utilizing UAV positional and orientation information and gimbal orientation information associated with each image to detect images that overlap with one another, wherein overlapping images are grouped together.

5. The method of claim 4, wherein processing the plurality of images further includes only processing images determined to overlap with other images.

6. The method of claim 5, wherein processing the plurality of images further includes processing in parallel groups of images determined to overlap with one another.