Systems and methods for autonomous perpendicular imaging with a target field of view

• US 9,996,746 B1
• Filed: 09/19/2017
• Issued: 06/12/2018
• Est. Priority Date: 11/04/2016
• Status: Active Grant

• Current Assignee: Loveland Innovations, Inc.
• Original Assignee: Loveland Innovations, LLC
• Inventors: Larson, Leif, Loveland, Jim, Christiansen, Dan, Christiansen, Tad, Christiansen, Cam
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US15/708,471
• Publication Number:
  US 20180165503A1
• Time in Patent Office: 266 Days
• Field of Search: 382103, 382104, 701 23
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

An unmanned aerial vehicle (UAV) assessment and reporting system may utilize one or more scanning techniques to provide useful assessments and/or reports for structures and other objects. The scanning techniques may be performed in sequence and optionally used to further fine tune each subsequent scan. The system may include shadow elimination, annotation, and/or reduction for the UAV itself and/or other objects. A UAV may be used to determine a pitch of roof of a structure. The pitch of the roof may be used to fine tune subsequent scanning and data capture to capture perpendicular images of target field of views and/or target distances.

Claims

1. An unmanned aerial vehicle (UAV) assessment system for imaging a structure and determining a pitch of a roof of a structure, comprising:

a site selection user interface to receive an electronic input from a user identifying a geographic location of a structure based on at least one of a street address, a coordinate, and a satellite image;

a UAV to receive the geographic location of the structure from the site selection interface, the UAV comprising;

a camera to capture images of the structure at a target field of view;

a pitch determination system to determine a pitch of a planar surface of the roof that is at an angle relative to a downward direction based on at least two distance measurements; and

an imaging system to;

adjust a tilt angle of the camera of the UAV to a non-zero angle relative to a downward direction to align an optical axis of the camera perpendicular to the planar surface of the roof of the structure, and

capture an image of at least a portion of the roof of the structure at the target field of view with the optical axis of the camera aligned perpendicular to the planar surface by adjusting a tilt angle of the camera of the UAV based on the determined pitch of the planar surface of the roof,

wherein the camera is configured to capture images of the structure at a target distance, such that the imaging system is configured to capture the image of the at least a portion of the roof of the structure at the target distance and the target field of view.

2. An unmanned aerial vehicle (UAV) assessment system for imaging a structure and determining a pitch of a roof of a structure, comprising:

a UAV to receive the geographic location of a structure from a site selection interface, the UAV comprising;

a camera to capture images of a roof of the structure;

a pitch determination system to determine a pitch of a planar surface of the roof that is at an angle relative to a downward direction; and

an imaging system to;

adjust a tilt angle of the camera of the UAV to a non-zero angle relative to a downward direction to align an optical axis of the camera substantially perpendicular to the planar surface of the roof of the structure, and

capture an image of at least a portion of the roof of the structure at a target distance and field of view with the optical axis of the camera aligned perpendicular to the planar surface of the roof based on the determined pitch of the planar surface of the roof,

wherein the target distance and field of view are user-defined.

3. The UAV assessment system of claim 2, wherein the pitch determination system is configured to determine the pitch of the planar surface of the roof based on a rise over run determination in which a rise is equal to a vertical distance downward between a location of the UAV and the roof of the structure and in which the run is equal to a horizontal distance to the roof of the structure, such that the roof represents a hypotenuse of a right triangle with the UAV positioned at a 90 degree corner of the right triangle.

4. The UAV assessment system of claim 2, further comprising a boundary identification interface to receive electronic input identifying geographic boundaries associated with the location of the structure.

5. The UAV assessment system of claim 2, further comprising:

a processor in communication with the camera; and

a non-transitory computer-readable medium for receiving and storing instructions that, when executed by the processor, cause the UAV to conduct a structural assessment including;

a boustrophedonic scan of the structure that includes image capture during a boustrophedonic flight pattern within a first altitude range,

a loop scan of the structure that includes image capture during a second flight pattern for the UAV to travel around the perimeter of the structure at a second altitude range lower than the first altitude range, and

a micro scan of the structure that includes image capture in a third flight pattern that includes vertical approaches proximate the structure to capture detail images of the structure.

6. The UAV assessment system of claim 2, further comprising a hazard identification interface to receive an input from a user identifying at least one obstacle proximate the structure that is within the geographic boundaries.

7. The UAV assessment system of claim 2, further comprising an interface for a user to identify a portion of interest on the roof of the structure, and wherein the captured image with the target field of view includes at least one detail image of the portion of interest in which the optical axis of the camera is aligned perpendicular to the planar surface of the roof of the structure.

8. The UAV assessment system of claim 2, wherein the imaging system is configured to capture the image of the roof of the structure with the optical axis of the camera aligned perpendicular to the planar surface of the roof of the structure by further adjusting the location of the UAV relative to the planar surface of the roof.

9. The UAV assessment system of claim 2, further comprising:

a processor in communication with the camera; and

a non-transitory computer-readable medium for receiving and storing instructions that, when executed by the processor, cause the UAV to conduct a structural assessment including;

a loop scan of the structure that includes image capture during a flight pattern for the UAV to travel around the perimeter of the structure, and

a micro scan of the structure that includes image capture in a flight pattern that includes vertical approaches proximate the structure to capture detail images of the structure.

10. The UAV assessment system of claim 2, wherein the target distance and field of view are user-defined to correspond to target dimensions of the captured image of the at least a portion of the roof of the structure.

11. The UAV assessment system of claim 2, wherein the target distance and field of view are user-defined settings.

12. The UAV assessment system of claim 2, wherein at least one of the target distance and target field of view is user-defined based on a selection of the camera.

13. An unmanned aerial vehicle (UAV) assessment system for imaging a structure and determining a pitch of a roof of a structure, comprising:

a UAV to receive the geographic location of a structure from a site selection interface, the UAV comprising;

a camera to capture images of a roof of the structure;

a pitch determination system to determine a pitch of a planar surface of the roof that is at an angle relative to a downward direction; and

an imaging system to;

adjust a tilt angle of the camera of the UAV to a non-zero angle relative to a downward direction to align an optical axis of the camera substantially perpendicular to the planar surface of the roof of the structure, and

capture an image of at least a portion of the roof of the structure at a target distance and field of view with the optical axis of the camera aligned perpendicular to the planar surface of the roof based on the determined pitch of the planar surface of the roof, wherein the target distance and field of view are inversely related such that increases in distance can be compensated for by decreases in field of view.

14. An unmanned aerial vehicle (UAV) assessment system for imaging a structure and determining a pitch of a roof of a structure, comprising:

a UAV to receive the geographic location of a structure from a site selection interface, the UAV comprising;

a camera to capture images of a roof of the structure;

a pitch determination system to determine a pitch of a planar surface of the roof that is at an angle relative to a downward direction; and

an imaging system to;

adjust a tilt angle of the camera of the UAV to a non-zero angle relative to a downward direction to align an optical axis of the camera substantially perpendicular to the planar surface of the roof of the structure, and

capture an image of at least a portion of the roof of the structure at a target distance and field of view with the optical axis of the camera aligned perpendicular to the planar surface of the roof based on the determined pitch of the planar surface of the roof, wherein the target distance and field of view of the captured image corresponds to target dimensions on the roof of the structure.

15. An unmanned aerial vehicle (UAV) for imaging a roof of a structure, comprising:

a camera to capture images of the structure;

a pitch determination system to determine a pitch of a planar surface of the roof that is at an angle relative to a downward direction in real time during a flight based on at least two distance measurements; and

an imaging system to capture an image with a target field of view of at least a portion of the roof of the structure with an optical axis of the camera aligned perpendicular to the planar surface of the roof of the structure by adjusting a tilt angle of the camera of the UAV based on the determined pitch of the planar surface of the roof, wherein the target field of view is customer-defined.

16. The UAV of claim 15, wherein at least one additional image of the roof captured with the optical axis misaligned relative to the planar surface of the roof is de skewed using the determined pitch after image capture.

17. The UAV of claim 15, wherein the pitch determination system is configured to determine the pitch of the planar surface of the roof based on a rise over run determination in which a rise is equal to a vertical distance downward between a location of the UAV and the roof of the structure and in which the run is equal to a horizontal distance to the roof of the structure, such that the roof represents a hypotenuse of a right triangle with the UAV positioned at a 90-degree corner of the right triangle.

18. The UAV of claim 15, further comprising:

a processor in communication with the camera; and

a non-transitory computer-readable medium for receiving and storing instructions that, when executed by the processor, cause the UAV to conduct a structural assessment including;

a boustrophedonic scan of the structure that includes image capture during a boustrophedonic flight pattern within a first altitude range,

a loop scan of the structure that includes image capture during a second flight pattern for the UAV to travel around the perimeter of the structure at a second altitude range lower than the first altitude range, and

a micro scan of the structure that includes image capture in a third flight pattern that includes vertical approaches proximate the structure to capture detail images of the structure.

19. The UAV of claim 15, wherein the imaging system is configured to capture the image of the roof of the structure with the optical axis of the camera aligned perpendicular to the planar surface of the roof of the structure by further adjusting the location of the UAV relative to the planar surface of the roof.

20. The UAV of claim 15, further comprising a boundary identification interface to receive electronic input identifying geographic boundaries associated with the location of the structure.

21. The UAV of claim 15, wherein the target field of view is customer-defined by selecting a zoom setting on the camera.

22. The UAV of claim 15, wherein the target field of view is customer-defined by manually moving the UAV relative to the structure.

23. The UAV of claim 15, wherein the target field of view is customer-defined based on a selection of the camera of the UAV.