Unmanned aircraft structure evaluation system and method

• US 10,037,463 B2
• Filed: 11/03/2017
• Issued: 07/31/2018
• Est. Priority Date: 01/10/2014
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Original Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Inventors: Schultz, Stephen L., Monaco, John
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Huynh, Luat T
• Application Number:
  US15/802,950
• Publication Number:
  US 20180068185A1
• Time in Patent Office: 270 Days
• Field of Search: 701 3
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01S 19/39 the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 : for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 : End-user applications

Abstract

A computerized method performed by an unmanned aerial vehicle (UAV), comprising: receiving, by the UAV, a flight plan comprising a plurality of inspection locations for a structure, wherein the plurality of inspection locations each comprise a waypoint having a geospatial reference; navigating to ascend to a first altitude above the structure; conducting an inspection for an object of interest in at least one of the plurality of inspection locations according to the flight plan, the inspection comprising: navigating to a position above a surface of the structure associated with the object of interest based on monitoring an active sensor, and obtaining, while within a particular distance from the surface of the structure, information from one or more sensors describing the structure such that obtained information includes at least a particular level of detail; navigating to another inspection location of the plurality of inspection locations; and navigating to a landing location.

Claims

1. A computerized method performed by an unmanned aerial vehicle (UAV) comprising one or more processors, the method comprising:

receiving, by the UAV, a flight plan comprising a plurality of inspection locations for a structure, wherein the plurality of inspection locations each comprise a waypoint having a geospatial reference;

navigating the UAV to ascend to a first altitude above the structure;

conducting an inspection for an object of interest in at least one of the plurality of inspection locations according to the flight plan, the inspection comprising;

navigating the UAV to a position above a surface of the structure associated with the object of interest based on monitoring an active sensor, and

obtaining, while the UAV is within a particular distance from the surface of the structure, information from one or more sensors describing the structure such that obtained information includes at least a particular level of detail;

navigating the UAV to another inspection location of the plurality of inspection locations; and

navigating the UAV to a landing location.

2. The method of claim 1, wherein navigating the UAV to a position within a particular distance above a surface of the structure comprises:

accessing one or more images obtained during a prior inspection of the structure, the one or more images including less than the particular level of detail;

performing, based on portions of the accessed images associated with the inspection location, visual odometry using sensor information obtained while descending; and

correcting a position of the UAV, along a horizontal direction, while descending based on the performed visual odometry, such that the UAV descends along a vertical direction towards the surface of the structure associated with the inspection location.

3. The method of claim 1, wherein conducting an inspection further comprises triggering the one or more sensors to obtain sensor information, and wherein triggering comprises one or more of:

periodically triggering the sensors while the UAV is within the particular distance from the surface of the structure, triggering the sensors in response to an event.

4. The method of claim 3, wherein triggering the sensors in response to an event comprises:

determining a value from accelerometers or rotational rate gyros of the UAV while the UAV is obtaining sensor information; and

determining whether the value exceeds a threshold value, and in response to a positive determination, triggering the sensors to again obtain the sensor information.

5. The method of claim 3, wherein triggering the sensors in response to an event comprises:

evaluating obtained sensor information to determine whether the sensor information passes a quality criteria; and

determining whether the sensor information passes the quality criteria, and in response to a negative determination, triggering the sensor again to obtain the sensor information.

6. The method of claim 1, further comprising:

obtaining, from a user device, a 3D model of the structure;

while navigating, obtaining geospatial location information of the UAV;

determining based on the 3D model, a position of the UAV in relation to the 3D model; and

adjusting a location of the UAV based on a determined position.

7. The method of claim 1, further comprising:

transmitting the obtained sensor information to a user device, the sensor information comprising one or more digital images, the one or more digital images having associated geospatial references.

8. The method of claim 1, wherein conducting an inspection further comprises:

allowing limited control of the UAV by a user device, the limited control comprising one or more of;

controlling a vertical rate of ascent or descent of the UAV, controlling rotation of the UAV, triggering of one or more sensors, controlling a direction of one or more sensors, nudging of the UAV or resuming auto-piloted navigation of the UAV.

9. A non-transitory computer storage medium comprising instructions that when executed by one or more processors included in an Unmanned Aerial Vehicle (UAV), cause the UAV to perform operations, comprising:

receiving, by the UAV, a flight plan comprising a plurality of inspection locations for a structure, wherein the plurality of inspection locations each comprise a waypoint having a geospatial reference;

navigating the UAV to ascend to a first altitude above a height of the structure;

conducting an inspection for an object of interest in the plurality of inspection locations, the inspection comprising;

navigating the UAV to a position above a surface of the structure associated with the inspection location, and descending, from the first altitude to a second altitude a particular distance from the surface of the structure,

the particular distance above the surface being determined to obtain at least a particular level of detail in sensor information; and

obtaining, while the UAV is within the particular distance from the surface of the structure, information from one or more sensors describing the structure, such that obtained sensor information includes at least the particular level of detail;

navigating the UAV to another inspection location of the plurality of inspection locations; and

navigating the UAV to a landing location.

10. The computer storage medium of claim 9, wherein navigating the UAV to a position within a particular distance above a surface of the structure comprises:

accessing one or more images obtained during a prior inspection of the structure, the one or more images including less than the particular level of detail;

performing, based on portions of the accessed images associated with the inspection location, visual odometry using sensor information obtained while descending; and

correcting a position of the UAV, along a horizontal direction, while descending based on the performed visual odometry, such that the UAV descends along a vertical direction towards the surface of the structure associated with the inspection location.

11. The computer storage medium of claim 9, wherein conducting an inspection further comprises triggering the one or more sensors to obtain sensor information, and wherein triggering comprises one or more of:

periodically triggering the sensors while the UAV is within the particular distance from the surface of the structure, triggering the sensors in response to an event.

12. The computer storage medium of claim 11, wherein triggering the sensors in response to an event comprises:

determining a value from accelerometers or rotational rate gyros of the UAV while the UAV is obtaining sensor information; and

determining whether the value exceeds a threshold value, and in response to a positive determination, triggering the sensors to again obtain the sensor information.

13. The computer storage medium of claim 11, wherein triggering the sensors in response to an event comprises:

evaluating obtained sensor information to determine whether the sensor information passes a quality criteria; and

determining whether the sensor information passes the quality criteria, and in response to a negative determination, triggering the sensor again to obtain the sensor information.

14. The computer storage medium of claim 9, wherein the particular level of detail indicates an acceptable image pixel resolution of digital images included in the obtained sensor information.

15. The computer storage medium of claim 9, wherein the operations further comprise:

obtaining, from a user device, a 3D model of the structure;

while navigating, obtaining geospatial location information of the UAV;

determining based on the 3D model, a position of the UAV in relation to the 3D model; and

adjusting a location of the UAV based on a determined position.

16. The computer storage medium of claim 9, wherein navigating the UAV to a position comprises:

navigating the UAV at a speed to the inspection location based on a waypoint transition speed value, wherein the waypoint transition speed value is set at different values for travel to different inspection locations.

17. The computer storage medium of claim 9, wherein the operations further comprise:

transmitting the obtained sensor information to a user device, the sensor information comprising one or more digital images, the one or more digital images having associated geospatial references.

18. The computer storage medium of claim 9, wherein conducting an inspection further comprises:

allowing limited control of the UAV by a user device, the limited control comprising one or more of;

controlling a vertical rate of ascent or descent of the UAV, controlling rotation of the UAV, triggering of one or more sensors, controlling a direction of one or more sensors, nudging of the UAV or resuming auto-piloted navigation of the UAV.

19. A computerized system, comprising:

a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing image display and analysis software to cause the one or more processors to;

receive an identification of a structure from the input device, the structure having multiple sides, an outline, and a height;

obtain characteristics of a camera mounted onto an unmanned aircraft;

generate unmanned aircraft information including;

flight path information configured to direct the unmanned aircraft to fly a flight path around the structure that is laterally and vertically offset from the structure, the lateral and vertical offset being dependent upon the height of the structure, an orientation of the camera relative to the unmanned aircraft, and the characteristics of the camera; and

,

storing the unmanned aircraft information on the one or more non-transitory computer readable medium.

20. The computerized system of claim 19, wherein the lateral and vertical offsets are determined to provide an effective image resolution due to the characteristics of the camera.

21. A computerized system, comprising:

a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing image display and analysis software to cause the one or more processors to;

receive an identification of a structure from the input device, the structure having multiple sides, an outline, and a height;

obtain characteristics of a camera mounted onto a gimbal of an unmanned aircraft;

generate unmanned aircraft information including;

flight path information, and gimbal control information, the flight path information configured to direct the unmanned aircraft to fly a flight path around the structure that is laterally and vertically offset from the structure, the gimbal control information configured to control an orientation of an optical axis of the camera so that the optical axis of the camera is aimed at the structure; and

,

storing the unmanned aircraft information on the one or more non-transitory computer readable medium.

22. The computerized system of claim 21, wherein the unmanned aircraft information further includes camera control information configured to cause the camera on the unmanned aircraft to capture at least one of oblique and nadir images of the structure.