Unmanned aerial vehicle rooftop inspection system

• US 9,915,946 B2
• Filed: 03/17/2017
• Issued: 03/13/2018
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: Skydio, Inc.
• Original Assignee: Unmanned Innovation Inc (Delair SAS)
• Inventors: Poole, Alan Jay, Bauer, Mark Patrick, Gurel, Volkan, Michini, Bernard J., Kuehn, Justin Eugene
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US15/462,245
• Publication Number:
  US 20170192424A1
• Time in Patent Office: 361 Days
• Field of Search: 701 23, 701428
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/26 for manufacturing, inspections or repairs
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 2201/202 using tethers for connecting to ground station
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/042 specially adapted for aircraft
• CPC Class Codes: G05D 1/0653 during a phase of take-off or landing
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06Q 10/063114 : Status monitoring or status determination for a person or group
• CPC Class Codes: G06Q 10/1097 : Task assignment
• CPC Class Codes: G06Q 50/16 : Real estate
• CPC Class Codes: G06T 17/05 : Geographic models
• CPC Class Codes: G06T 2215/16 : Using real world measurements to influence rendering
• CPC Class Codes: G06V 20/17 : taken from planes or by drones
• CPC Class Codes: G06V 20/176 : Urban or other man-made structures
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/06 : Recognition of objects for industrial automation
• CPC Class Codes: G08G 5/0013 : with a ground station
• CPC Class Codes: G08G 5/0034 : Assembly of a flight plan
• CPC Class Codes: G08G 5/0065 : for taking-off
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/025 : Navigation or guidance aids radar or analogous systems specially adapted for landing purposes G01S13/913
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for an unmanned aerial system inspection system. One of the methods is performed by a UAV and includes receiving, by the UAV, flight information describing a job to perform an inspection of a rooftop. A particular altitude is ascended to, and an inspection of the rooftop is performed including obtaining sensor information describing the rooftop. Location information identifying a damaged area of the rooftop is received. The damaged area of the rooftop is traveled to. An inspection of the damaged area of the rooftop is performed including obtaining detailed sensor information describing the damaged area. A safe landing location is traveled to.

Claims

1. A flight planning and analysis system comprising one or more processors, and a computer storage media storing instructions, that when executed by the one or more processors, cause the one or more processors to perform pre-flight and post-flight operations for an unmanned aerial vehicle (UAV):

the pre-flight operations comprising;

creating a job for an aerial inspection of a property, comprising;

receiving location information of the property to be inspected, wherein the location information is an address of the property;

based on the location information, displaying, via a first user interface, imagery of the property;

determining an inspection area for the property, the inspection area having associated geo-spatial information describing a boundary of the inspection area; and

displaying, via the first user interface, the determined inspection area over the imagery of the property; and

assigning a particular operator to perform the job from a selection of operators; and

providing job information including the inspection area and the location information to a ground control station associated with the assigned operator,

wherein the ground control station is configured to present a user interface to the assigned operator, and enable, via the user interface, (1) modifications to the provided job information and/or (2) additions to the provided job information, and wherein the ground control station is configured to determine a flight pattern, based at least in part, on the job information,

and wherein the ground control station is configured for flight operations of the UAV for aerial inspection of the property based on the job information, and

obtain, based on the flight pattern, sensor data describing the property; and

the post-flight operations comprising;

receiving sensor data obtained by the UAV from the aerial inspection of the property;

generating at least one of a georectified imagery or a three-dimensional model of the property based on the received sensor data; and

displaying, via a second user interface, the georectified imagery or the three-dimensional model of the property.

2. The flight planning system of claim 1, wherein determining an operator comprises:

determining the particular operator based on the availability of operators for a selected time period.

3. The flight planning system of claim 1, wherein determining an operator comprises:

determining the particular operator based on the location of the job in relation to other jobs assigned to the operator.

4. The flight planning system of claim 1, further comprising the pre-flight operations of:

determining one or more times to perform the job from a selection of recommended times to perform the job.

5. The flight planning system of claim 1, further comprising the pre-flight operations of:

receiving, from the ground control station, confirmation of receipt of the job information.

6. The flight planning system of claim 1, wherein enabling modifications to the job information comprises:

displaying, via the user interface of the ground control station, a representation of the inspection area;

receiving, via the user interface of the ground control station, modifications to the inspection area;

determining, by the ground control station, the flight pattern for the UAV based on the modified inspection area; and

instructing the UAV to perform the aerial inspection based on the flight pattern.

7. The flight planning system of claim 1, wherein the one or more processors also perform flight operations comprising:

receiving, by the ground control station, geo-spatial information identifying a safe take-off and/or a safe landing location for the UAV.

8. The flight planning system of claim 1, wherein the one or more processors also perform flight operations comprising:

receiving, by the ground control station, a value for a safe altitude above which the UAV can travel freely without encountering obstructions within the inspection area; and

instructing the UAV to perform the aerial inspection using the safe altitude to fly above obstructions.

9. The flight planning system of claim 1, wherein the one or more processors also perform flight operations comprising:

determining, by the ground control station, a flight pattern for the UAV to follow based on the inspection area; and

instructing the UAV to perform the aerial inspection according to the flight pattern.

10. The flight planning system of claim 9, further comprising the flight operations of:

determining, by the ground control station, a flight pattern for the UAV to follow based on a desired ground sampling distance for images to be obtained by the UAV during the aerial survey.

11. A non-transitory computer storage medium comprising instructions that when executed by a system of one or more computers, cause the system to perform pre-flight and post-flight operations for an unmanned aerial vehicle (UAV):

the pre-flight operations comprising;

creating a job for an aerial inspection of a property, comprising;

receiving location information of the property to be inspected, wherein the location information is an address of the property;

based on the location information, displaying, via a first user interface, imagery of the property;

determining an inspection area for the property, the inspection area having associated geo-spatial information describing a boundary of the inspection area; and

displaying, via the first user interface, the determined inspection area over the imagery of the property; and

assigning a particular operator to perform the job from a selection of operators; and

providing job information including the inspection area and the location information to a ground control station associated with the assigned operator,

wherein the ground control station is configured to present a user interface, and enable, via the user interface, adjustments to the job information, and wherein the ground control station is configured to determine a flight pattern, based at least in part, on the adjusted job information,

and wherein the ground control station is configured for flight operations of the UAV for aerial inspection of the property based on the job information, and

obtain, based on the flight pattern, sensor data describing the property; and

the post-flight operations comprising;

receiving sensor data obtained by the UAV from the aerial inspection of the property;

generating at least one of a georectified imagery or a three-dimensional model of the property based on the received sensor data; and

displaying, via a second user interface, the georectified imagery or the three-dimensional model of the property.

12. The non-transitory computer storage medium of claim 11, wherein determining an operator comprises:

determining the particular operator based on the availability of operators for a selected time period.

13. The non-transitory computer storage medium of claim 11, further comprising the pre-flight operations of:

determining the particular operator based on the location of the job in relation to other jobs assigned to the operator.

14. The non-transitory computer storage medium of claim 11, further comprising the pre-flight operations of:

determining one or more times to perform the job from a selection of recommended times to perform the job.

15. The non-transitory computer storage medium of claim 11, further comprising the pre-flight operations of:

receiving, from the ground control station, confirmation of receipt of the job information.

16. The non-transitory computer storage medium of claim 11, wherein enabling adjustments to the job information comprises:

displaying, via the user interface of the ground control station, a representation of the inspection area;

receiving, via the user interface of the ground control station, modifications to the inspection area;

determining, by the ground control station, the flight pattern for the UAV based on the modified inspection area; and

instructing the UAV to perform the aerial inspection based on the flight pattern.

17. The non-transitory computer storage medium of claim 11, comprising instructions that when executed by the system of the one or more computers, cause the system to perform flight operations for the UAV, the flight operations comprising:

receiving, by the ground control station, geo-spatial information identifying a safe take-off and/or a safe landing location for the UAV.

18. The non-transitory computer storage medium of claim 11, comprising instructions that when executed by the system of the one or more computers, cause the system to perform flight operations for the UAV, the flight operations comprising:

receiving, by the ground control station, a value for a safe altitude above which the UAV can travel freely without encountering obstructions within the inspection area; and

instructing the UAV to perform the aerial inspection using the safe altitude to fly above obstructions.

19. The non-transitory computer storage medium of claim 11, comprising instructions that when executed by the system of the one or more computers, cause the system to perform flight operations for the UAV, the flight operations comprising:

determining, by the ground control station, a flight pattern for the UAV to follow based on the inspection area; and

instructing the UAV to perform the aerial inspection according to the flight pattern.

20. The non-transitory computer storage medium of claim 19, further comprising the flight operations of:

determining, by the ground control station, a flight pattern for the UAV to follow based on a desired ground sampling distance for images to be obtained by the UAV during the aerial survey.

21. The non-transitory computer storage medium of claim 11, wherein adjustments to the job information comprise (1) modifications to the job information and/or (2) additions to the job information.

22. A computerized method performed by a system of one or more computers, the method being performed in pre-flight and post-flight operations of an unmanned aerial vehicle (UAV):

the pre-flight operations comprising;

creating a job for an aerial inspection of a property, comprising;

receiving location information of the property to be inspected, wherein the location information is an address of the property;

based on the location information, displaying, via a first user interface, imagery of the property;

determining an inspection area for the property, the inspection area having associated geo-spatial information describing a boundary of the inspection area; and

displaying, via the first user interface, the determined inspection area over the imagery of the property; and

assigning a particular operator to perform the job from a selection of operators; and

providing job information including the inspection area and the location information to a ground control station associated with the assigned operator,

wherein the ground control station is configured to present a user interface, and enable, via the user interface, adjustments to the job information, and wherein the ground control station is configured to determine a flight pattern, based at least in part, on the job information,

and wherein the ground control station is configured for flight operations of the UAV for aerial inspection of the property based on the job information, and

obtain, based on the flight pattern, sensor data describing the property; and

the post-flight operations comprising;

receiving sensor data obtained by the UAV from the aerial inspection of the property;

generating at least one of a georectified imagery or a three-dimensional model of the property based on the received sensor data; and

displaying, via a second user interface, the georectified imagery or the three-dimensional model of the property.

23. The method of claim 22, wherein determining an operator comprises:

determining the particular operator based on the availability of operators for a selected time period.

24. The method of claim 22, further comprising the pre-flight operations of:

determining the particular operator based on the location of the job in relation to other jobs assigned to the operator.

25. The method of claim 22, further comprising the pre-flight operations of:

determining one or more times to perform the job from a selection of recommended times to perform the job.

26. The method of claim 22, further comprising the pre-flight operations of:

receiving, from the ground control station, confirmation of receipt of the job information.

27. The method of claim 22, wherein enabling adjustments to the job information comprises:

displaying, via the user interface of the ground control station, a representation of the inspection area;

receiving, via the user interface of the ground control station, modifications to the inspection area;

determining, by the ground control station, the flight pattern for the UAV based on the modified inspection area; and

instructing the UAV to perform the aerial inspection based on the flight pattern.

28. The method of claim 22 performed by the system of the one or more computers, further performing flight operations comprising:

receiving, by the ground control station, geo-spatial information identifying a safe take-off and/or a safe landing location for the UAV.

29. The method of claim 22 performed by the system of the one or more computers, further performing flight operations comprising:

receiving, by the ground control station, a value for a safe altitude above which the UAV can travel freely without encountering obstructions within the inspection area; and

instructing the UAV to perform the aerial inspection using the safe altitude to fly above obstructions.

30. The method of claim 22 performed by the system of the one or more computers, further performing flight operations comprising:

determining, by the ground control station, a flight pattern for the UAV to follow based on the inspection area; and

instructing the UAV to perform the aerial inspection according to the flight pattern.

31. The method of claim 30, further comprising the flight operations of:

determining, by the ground control station, a flight pattern for the UAV to follow based on a desired ground sampling distance for images to be obtained by the UAV during the aerial survey.

32. The method of claim 22, wherein adjustments to the job information comprise (1) modifications to the job information and/or (2) additions to the job information.