UNMANNED AERIAL VEHICLE ROOFTOP INSPECTION SYSTEM

• US 20170199647A1
• Filed: 03/27/2017
• Published: 07/13/2017
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: Skydio, Inc.
• Original Assignee: Unmanned Innovation Inc (Delair SAS)
• Inventors: Michini, Bernard J., Poole, Alan Jay, Richman, Brian, Bauer, Mark Patrick
• Granted Patent: US 10,061,470 B2
• Time in Patent Office: Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/26 for manufacturing, inspections or repairs
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 2201/202 using tethers for connecting to ground station
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/042 specially adapted for aircraft
• CPC Class Codes: G05D 1/0653 during a phase of take-off or landing
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06Q 10/063114 : Status monitoring or status determination for a person or group
• CPC Class Codes: G06Q 10/1097 : Task assignment
• CPC Class Codes: G06Q 50/16 : Real estate
• CPC Class Codes: G06T 17/05 : Geographic models
• CPC Class Codes: G06T 2215/16 : Using real world measurements to influence rendering
• CPC Class Codes: G06V 20/17 : taken from planes or by drones
• CPC Class Codes: G06V 20/176 : Urban or other man-made structures
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/06 : Recognition of objects for industrial automation
• CPC Class Codes: G08G 5/0013 : with a ground station
• CPC Class Codes: G08G 5/0034 : Assembly of a flight plan
• CPC Class Codes: G08G 5/0065 : for taking-off
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/025 : Navigation or guidance aids radar or analogous systems specially adapted for landing purposes G01S13/913
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for an unmanned aerial system inspection system. One of the methods is performed by a UAV and includes receiving, by the UAV, flight information describing a job to perform an inspection of a rooftop. A particular altitude is ascended to, and an inspection of the rooftop is performed including obtaining sensor information describing the rooftop. Location information identifying a damaged area of the rooftop is received. The damaged area of the rooftop is traveled to. An inspection of the damaged area of the rooftop is performed including obtaining detailed sensor information describing the damaged area. A safe landing location is traveled to.

Claims

1. A system comprising one or more processors, and a computer storage media storing instructions, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

displaying, via an interface, a first graphical representation of an object, the first graphical representation comprising one or more first digital images obtained by an unmanned aerial vehicle (UAV), the first digital images describing an aerial view of the object, the one or more first digital images associated with a first level of detail;

receiving, via the interface, input specifying selections of one or more portions of the first graphical representation, the portions being associated with inspection locations indicating, at least, positions over a surface of the object for which the UAV is to navigate; and

conducting an inspection for the respective inspection locations, the inspection comprising;

navigating the UAV to an inspection location; and

causing the UAV to obtain one or more second digital images describing the surface of the object, the second digital images associated with a second level of detail greater than the first level of detail of the first digital images.

2. The system of claim 1, wherein the input comprises a selection of a variable-sized user interface control, the level of detail of the one or more second digital images to be obtained for a specified inspection location being based on the size of the control.

3. The system of claim 1, wherein receiving input further comprises:

receiving, via the user interface, a selection for a level of image detail of the one or more second digital images to be obtained for a specified inspection location, wherein a distance from the surface to obtain the second digital images is set based on the specified level of image detail.

4. The system of claim 1, conducting an inspection further comprising the operations of:

while at the inspection location causing the UAV to descend toward the surface of the object; and

providing a command to the UAV to move the UAV a certain distance in a particular direction.

5. The system of claim 4, wherein the distance to move the UAV in the particular direction is a predetermined distance from 0.1 to 3 meters.

6. The system of claim 4, further comprising the operations of:

determining that the UAV is within a threshold distance of the surface of the object;

wherein the distance to move the UAV in the particular direction, is variable based on the proximity of the UAV to the surface of the object.

7. The system of claim 1, further comprising the operations of:

receiving, via the user interface, an input to move to the UAV to the next inspection location; and

instructing the UAV to navigate to the next inspection location.

8. The system of claim 1, further comprising the operations of:

while the UAV conducts the inspection, transmitting, by the system, commands to the UAV to control a vertical rate of ascent or descent of the UAV, rotation of the UAV, triggering of a UAV sensor, a direction of a sensor, nudging of the UAV or resuming auto-piloted navigation of the UAV.

9. The system of claim 1, conducting an inspection further comprising the operations of:

instructing the UAV to hold in a position after obtaining a digital image;

determining whether the obtained one or more second digital images meets a required image quality; and

instructing the UAV to retake a digital image if the image does not meet the required image quality.

10. The system of claim 1, wherein the level of detail of a first view of an area of the surface of the object in the one or more first digital images is at a lower pixel resolution than a second view of the area of the surface of the object in the one or more second digital images.

11. A non-transitory computer storage medium comprising instructions that when executed by a system of one or more processors, cause the system to perform operations comprising:

displaying, via an interface, a first graphical representation of an object, the first graphical representation comprising one or more first digital images obtained by an unmanned aerial vehicle (UAV), the first digital images describing an aerial view of the object, the one or more first digital images associated with a first level of detail;

receiving, via the interface, input specifying selections of one or more portions of the first graphical representation, the portions being associated with inspection locations indicating, at least, positions over a surface of the object for which the UAV is to navigate; and

conducting an inspection for the respective inspection locations, the inspection comprising;

navigating the UAV to an inspection location; and

causing the UAV to obtain one or more second digital images describing the surface of the object, the second digital images associated with a second level of detail greater than the first level of detail of the first digital images.

12. The computer storage medium of claim 11, wherein the input comprises a selection of a variable-sized user interface control, the level of detail of the one or more second digital images to be obtained for a specified inspection location being based on the size of the control.

13. The computer storage medium of claim 11, wherein receiving input further comprises:

receiving, via the user interface, a selection for a level of image detail of the second digital images to be obtained for a specified inspection location, wherein a distance from the surface to obtain the second digital images is set based on the specified level of image detail.

14. The computer storage medium of claim 11, conducting an inspection further comprising the operations of:

while at the inspection location causing the UAV to descend toward the surface of the object; and

providing a command to the UAV to move the UAV a certain distance in a particular direction.

15. The computer storage medium of claim 14, wherein the distance to move the UAV in the particular direction is a predetermined distance from 0.1 to 3 meters.

16. The computer storage medium of claim 14, further comprising the operations of:

determining that the UAV is within a threshold distance of the surface of the object;

wherein the distance to move the UAV in the particular direction, is variable based on the proximity of the UAV to the surface of the object.

17. The computer storage medium of claim 11, further comprising the operations of:

receiving, via the user interface, an input to move to the UAV to the next inspection location; and

instructing the UAV to navigate to the next inspection location.

18. The computer storage medium of claim 11, further comprising the operations of:

while the UAV conducts the inspection, transmitting, by the system, commands to the UAV to control a vertical rate of ascent or descent of the UAV, rotation of the UAV, triggering of a UAV sensor, a direction of a sensor, nudging of the UAV or resuming auto-piloted navigation of the UAV.

19. The computer storage medium of claim 11, conducting an inspection further comprising the operations of:

instructing the UAV to hold in a position after obtaining a digital image;

determining whether the obtained one or more second digital images meets a required image quality; and

instructing the UAV to retake a digital image if the obtained one or more second digital images does not meet the required image quality.

20. The computer storage medium of claim 11, wherein the level of detail of a first view of an area of the surface of the object in the one or more first digital images is at a lower pixel resolution than a second view of the area of the surface of the object in the one or more second digital images.

21. A computerized method performed by a system of one or more processors, the method comprising:

displaying, via an interface, a first graphical representation of an object, the first graphical representation comprising one or more first digital images obtained by an unmanned aerial vehicle (UAV), the first digital images describing an aerial view of the object, the one or more first digital images associated with a first level of detail;

receiving, via the interface, input specifying selections of one or more portions of the first graphical representation, the portions being associated with inspection locations indicating, at least, positions over a surface of the object for which the UAV is to navigate; and

conducting an inspection for the respective inspection locations, the inspection comprising;

navigating the UAV to an inspection location; and

causing the UAV to obtain one or more second digital images describing the surface of the object, the second digital images associated with a second level of detail greater than the first level of detail of the first digital images.

22. The computerized method of claim 21, wherein the input comprises a selection of a variable-sized user interface control, the level of detail of the one or more second digital images to be obtained for a specified inspection location being based on the size of the control.

23. The computerized method of claim 21, wherein receiving input further comprises:

receiving, via the user interface, a selection for a level of image detail of the second digital images to be obtained for a specified inspection location, wherein a distance from the surface to obtain the second digital images is set based on the specified level of image detail.

24. The computerized method of claim 21, wherein conducting an inspection further comprises:

while at the inspection location causing the UAV to descend toward the surface of the object; and

providing a command to the UAV to move the UAV a certain distance in a particular direction.

25. The computerized method of claim 24, wherein the distance to move the UAV in the particular direction is a predetermined distance from 0.1 to 3 meters.

26. The computerized method of claim 24, further comprising:

determining that the UAV is within a threshold distance of the surface of the object;

wherein the distance to move the UAV in the particular direction, is variable based on the proximity of the UAV to the surface of the object.

27. The computerized method of claim 21, further comprising:

receiving, via the user interface, an input to move to the UAV to the next inspection location; and

instructing the UAV to navigate to the next inspection location.

28. The computerized method of claim 21, further comprising:

while the UAV conducts the inspection, transmitting, by the system, commands to the UAV to control a vertical rate of ascent or descent of the UAV, rotation of the UAV, triggering of a UAV sensor, a direction of a sensor, nudging of the UAV or resuming auto-piloted navigation of the UAV.

29. The computerized method of claim 21, wherein conducting an inspection further comprises:

instructing the UAV to hold in a position after obtaining a digital image;

determining whether the obtained one or more second digital images meets a required image quality; and

instructing the UAV to retake a digital image if the image does not meet the required image quality.

30. The computer storage medium of claim 21, wherein the level of detail of a first view of an area of the surface of the object in the one or more first digital images is at a lower pixel resolution than a second view of the area of the surface of the object in the one or more second digital images.