Unmanned aircraft structure evaluation system and method

• US 10,181,080 B2
• Filed: 07/30/2018
• Issued: 01/15/2019
• Est. Priority Date: 01/10/2014
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Original Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Inventors: Schultz, Stephen L., Monaco, John
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Huynh, Luat T
• Application Number:
  US16/048,537
• Publication Number:
  US 20180357477A1
• Time in Patent Office: 169 Days
• Field of Search: 701 3
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01S 19/39 the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 : for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 : End-user applications

Abstract

Methods and systems are disclosed including a computer storage medium, comprising instructions that when executed by one or more processors included in an Unmanned Aerial Vehicle (UAV), cause the UAV to perform operations, comprising: receiving, by the UAV, a flight plan configured to direct the UAV to fly a flight path having a plurality of waypoints adjacent to and above a structure and to capture sensor data of the structure from a camera on the UAV while the UAV is flying the flight path; adjusting an angle of an optical axis of the camera mounted to a gimbal to a predetermined angle within a range of 25 degrees to 75 degrees relative to a downward direction, and capturing sensor data of at least a portion of a roof of the structure with the optical axis of the camera aligned with at least one predetermined location on the structure.

Claims

1. A non-transitory computer storage medium comprising instructions that when executed by a processor included in an Unmanned Aerial Vehicle (UAV) having a computer controlled gimbal mount connected to a camera, cause the UAV to perform operations, comprising:

directing the UAV to fly a target path having a plurality of flight capture points laterally and/or vertically offset from geographic positions of an outline of the structure of interest, a first group of flight capture points adjacent to a first side of the outline, and a second group of flight capture points adjacent to a second side of the outline;

capturing images of the structure of interest from the camera on the UAV while the UAV is flying the target path at flight capture points spaced along the target path in such a manner as to ensure full coverage of the structure of interest in the images captured by the camera; and

adjusting a direction of the camera relative to the structure of interest in three dimensions as the UAV is flying the target path to point the camera at a target capture point on the structure of interest at each flight capture point.

2. The non-transitory computer storage medium of claim 1, wherein adjusting a direction of the camera relative to the structure of interest is defined further as adjusting a yaw of the UAV to assist in pointing the camera at the target capture point.

3. The non-transitory computer storage medium of claim 1, wherein adjusting the direction of the camera relative to the structure of interest is defined further as adjusting the direction of the camera relative to the structure of interest to a predetermined angle within a range of 25-75 degrees relative to a downward direction.

4. The non-transitory computer storage medium of claim 1, wherein a first one of the plurality of flight capture points direct the UAV to capture sensor data of the structure of interest from a first angle relative to the structure of interest and a second one of the plurality of flight capture points directs the UAV to capture sensor data of the structure of interest from a second angle relative to the structure.

5. The non-transitory computer storage medium of claim 1, wherein the flight capture points are spaced along the target path in such a manner as to maintain at least a minimum overlap between adjacent images to ensure full coverage of the structure of interest in the images captured by the camera.

6. The non-transitory computer storage medium of claim 1, wherein the structure of interest is a roof of a building.

7. A computerized system, comprising:

a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing software to cause the one or more processors to;

determining a first location of a structure to be inspected, the first location having first geographic coordinates;

displaying, via the display unit, a first graphical representation of the structure of interest, the first graphical representation comprising a first image describing an aerial view of the structure of interest;

receiving an alteration of the first graphical representation to indicate a change of the first geographic coordinates to second geographic coordinates;

subsequent to receiving the alteration of the first graphical representation,

generate unmanned aircraft information with information pertaining to the structure of interest identified by the second geographic coordinates for a UAV having a camera mounted to a computer controlled gimbal mount to capture images of the structure of interest, the unmanned aircraft information including flight path information, camera control information, and gimbal control information, the flight path information configured to direct an unmanned aircraft to fly a flight path above the structure of interest, the camera control information configured to control the camera to capture aerial images from the camera on the unmanned aircraft while the unmanned aircraft is flying the flight path, the gimbal control information configured to control the computer controlled gimbal mount to align the camera with the structure of interest during capture of the aerial images;

receive the aerial images of the structure of interest from the unmanned aircraft; and

generate a structure report based at least in part on the aerial images of the structure of interest.

8. The computerized system of claim 7, wherein the at least one predetermined location on the structure of interest is a roof surface and the optical axis of the camera is aligned orthogonal to the roof surface.

9. The computerized system of claim 7, wherein the camera control information includes a plurality of flight capture points adjacent to the structure to cause the camera of the unmanned aerial vehicle to capture the aerial images at the plurality of flight capture points.

10. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a footprint of the structure of interest, the footprint being a two-dimensional boundary of the structure of interest.

11. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a footprint of the structure of interest, the footprint being an outline of the structure of interest.

12. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a height of the structure of interest, the information pertaining to the height of the structure being used by the software to aid in determining altitude for the unmanned aircraft information.

13. The computerized system of claim 12, wherein the information pertaining to the height of the structure of interest is received from a user via the input unit.

14. The computerized system of claim 7, wherein the information pertaining to the structure of interest include geographic locations of obstructions in potential flight paths of the UAV, and wherein the flight path information is generated to direct the unmanned aircraft to fly the flight path around the structure of interest, and avoid the obstructions.

15. The computerized system of claim 14, wherein the one or more processors execute software to cause the one or more processors to detect the geographic location of obstructions in potential flight paths of the UAV through the use of imagery.

16. The computerized system of claim 7, wherein the second geographic coordinates are indicative of a validated location of the structure of interest.

17. The computerized system of claim 7, wherein the structure of interest is a roof of a building.

18. A computerized system, comprising:

a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing software to cause the one or more processors to;

display, via the display unit, a first flight path on a graphical representation of an external form of a structure of interest, in which an unmanned aircraft is to navigate about the structure of interest;

receive data indicative of an outline of the structure of interest, via the input unit, and using the data to generate unmanned aircraft information having a second flight path and gimbal control information, the second flight path configured to direct the unmanned aircraft through autonomous flight above the structure of interest, the gimbal control information configured to align a camera supported by the unmanned aircraft with the structure of interest during capture of the aerial images;

display, via the display unit, the second flight path, on the graphical representation of the external form of the structure of interest;

receive aerial images of the structure of interest from the unmanned aircraft, the aerial images obtained by the camera when aligned with the structure of interest during capture of the aerial images while the unmanned aircraft is being directed through the second flight path during autonomous flight; and

generate a structure report based at least in part on the aerial images of the structure of interest obtained by the camera carried by the unmanned aircraft while the aircraft is being directed through the second flight path.

19. The computerized system of claim 18, wherein the second flight path includes a plurality of spatially disposed flight capture points, the spacing of the flight capture points maintaining at least a minimum overlap between adjacent aerial images to ensure full coverage of the structure of interest in the aerial images captured by the camera.

20. The computerized system of claim 18, wherein the structure of interest has an outline, and wherein the second flight path extends outside the outline of the structure of interest.

21. The computerized system of claim 19, wherein the second flight path extends entirely beyond the outline of the structure of interest.

22. The computerized system of claim 18, wherein the first flight path is determined automatically by analyzing and using geo-referenced images.

23. The computerized system of claim 18, wherein the structure of interest is a roof of a building.

24. An Unmanned Aerial Vehicle (UAV) having a computer controlled gimbal mount connected to a camera, a processor, and a non-transitory computer storage medium comprising instructions that when executed by the processor cause the UAV to perform operations, comprising:

directing the UAV to fly a flight path having a plurality of flight capture points laterally and/or vertically offset from geographic positions of an outline of the structure of interest, a first group of flight capture points adjacent to a first side of the outline, and a second group of flight capture points adjacent to a second side of the outline;

detecting an obstruction in the flight path,

steering the UAV to avoid the obstruction, and adjusting a down-look angle of the camera with the computer controlled gimbal mount as the UAV is avoiding the obstruction; and

capturing first images of the structure of interest with the camera on the UAV at the flight capture points while the UAV is flying the flight path, and second images while the UAV is avoiding the obstruction, the first images being overlapping, and the second images being overlapping.

25. The UAV of claim 24, wherein the flight capture points are spaced along the flight path in such a manner as to ensure full coverage of the structure of interest.

26. The UAV of claim 24, wherein an angle of the camera relative to the structure of interest is adjusted to a first angle using the computer controlled gimbal mount while capturing the first images and a second angle relative to the structure of interest while capturing the second images.

27. The UAV of claim 24, wherein the UAV comprises a collision detection and avoidance system to detect the obstruction in the flight path, and to steer the UAV to avoid the obstruction.

28. The UAV of claim 27, wherein the collision detection and avoidance system is imaging based.

29. The UAV of claim 27, wherein the collision detection and avoidance system is active sensor based.

30. The UAV of claim 24, wherein the instructions that when executed by the processor including camera control instructions configured to capture images on a schedule.