Ground control point assignment and determination system

• US 9,592,912 B1
• Filed: 04/07/2016
• Issued: 03/14/2017
• Est. Priority Date: 03/08/2016
• Status: Active Grant

• Current Assignee: Skydio, Inc.
• Original Assignee: Unmanned Innovation Inc (Delair SAS)
• Inventors: Blanc-Paques, Fabien, Michini, Bernard J., Bethke, Brett Michael
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US15/093,511
• Time in Patent Office: 341 Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/26 for manufacturing, inspections or repairs
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 15/02 Means for marking measuring points
• CPC Class Codes: G01C 15/06 Surveyors' staffs; Movable markers
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/223 Command input arrangements on the remote controller, e.g. joysticks or touch screens
• CPC Class Codes: G05D 1/689 Pointing payloads towards fixed or moving targets positioning towed, pushed or suspended implements G05D1/672
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/003 Flight plan management
• CPC Class Codes: G08G 5/0034 Assembly of a flight plan
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for ground control point assignment and determination. One of the methods includes receiving information describing a flight plan for the UAV to implement, the flight plan identifying one or more waypoints associated with geographic locations assigned as ground control points. A first waypoint identified in the flight plan is traveled to, and an action to designate a surface at the associated geographic location is designated as a ground control point. Location information associated with the designated surface is stored. The stored location information is provided to an outside system for storage.

Claims

1. A method implemented by an unmanned aerial vehicle (UAV) comprising one or more processors, the method comprising:

receiving information describing a flight plan for implementation, the flight plan specifying geographic locations in a real-world area assigned as ground control points, each geographic location being associated with a waypoint to which the UAV is to navigate and perform an operation to designate a surface of the geographic location as being a ground control point;

performing, for a particular waypoint, an operation to designate a surface associated with the particular waypoint as being a ground control point, the operation comprising;

navigating the UAV at a first altitude to the particular waypoint,

descending towards the surface, and placing a physical marker carried by the UAV onto the surface thereby creating an identifiable mark on the surface, such that the physical marker is aerially identifiable as a ground control point in one or more images obtained during a subsequent flight plan to perform a job, and

determining location information of the identifiable mark; and

providing the determined location information to an outside system for storage as being associated with a ground control point.

2. The method of claim 1, wherein determining location information of the identifiable mark comprises:

ascending to a second altitude above the identifiable mark; and

determining, using imagery of the identifiable mark obtained by the UAV at the second altitude, location information of the identifiable mark.

3. The method of claim 2, wherein determining location information of the identifiable mark using imagery of the identifiable mark comprises:

obtaining information identifying an offset from a center of a camera of the UAV to a location receiver of the UAV;

navigating the UAV and/or a gimbal associated with the camera such that a center of the camera is over a centroid of the physical marker; and

determining location information of the UAV, and modifying the location information based on the offset.

4. The method of claim 1, wherein determining location information of the identifiable mark comprises:

obtaining information identifying an offset from a centroid of the physical marker as carried by the UAV to a location receiver of the UAV; and

determining location information of the UAV, and modifying the location information based on the offset.

5. The method of claim 1, further comprising:

determining that the physical marker is level, wherein determining that the physical marker is level comprises one or more of;

prior to placing the physical marker, determining that the surface is flat such that distances from the UAV to the surface are within a threshold variance, or

after placing the physical marker, determining that distances from the physical marker to the UAV are within a threshold variance, or

after placing the physical marker, receiving confirmation, from the physical marker in communication with the UAV, that the physical marker is level.

6. The method of claim 1, wherein the physical marker is configured to display via electronically controlled lighting, a pattern comprising one or more of a shape, a pattern, a number, or a letter.

7. The method of claim 1, further comprising:

retrieving, via a magnetic or grabbing device of the UAV, the physical marker previously placed by the UAV.

8. The method of claim 1, wherein the physical marker transmits a geospatial position of the physical marker.

9. The method of claim 1, further comprising:

obtaining information describing a flight plan to perform an inspection of a structure;

performing the inspection of the structure, including capturing imagery that includes one or more of the designated surfaces of the specified geographic locations; and

providing the captured imagery and determined location information associated with the designated surfaces to a photogrammetry system for processing.

10. The method of claim 1, further comprising:

prior to descending towards the surface, hovering at the first altitude;

receiving, from a user device in communication with the UAV, confirmation that the UAV is to descend; and

in response to receiving the confirmation, descending towards to the surface.

11. An unmanned aerial vehicle (UAV) system comprising one or more processors, and a computer storage media storing instructions, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving information describing a flight plan for implementation, the flight plan specifying geographic locations in a real-world area assigned as ground control points, each geographic location being associated with a waypoint to which the UAV is to navigate and perform an operation to designate a surface of the geographic location as being a ground control point;

performing, for a particular waypoint, an operation to designate a surface associated with the particular waypoint as being a ground control point, the operation comprising;

navigating the UAV at a first altitude to the particular waypoint,

descending towards the surface, and placing a physical marker carried by the UAV onto the surface thereby creating an identifiable mark on the surface, such that the physical marker is aerially identifiable as a ground control point in one or more images obtained during a subsequent flight plan to perform a job, and

determining location information of the identifiable mark; and

providing the determined location information to an outside system for storage as being associated with a ground control point.

12. The system of claim 11, wherein determining location information of the identifiable mark comprises:

ascending to a second altitude above the identifiable mark; and

determining, using imagery of the identifiable mark obtained by the UAV at the second altitude, location information of the identifiable mark.

13. The system of claim 12, wherein determining location information of the identifiable mark using imagery of the identifiable mark comprises:

obtaining information identifying an offset from a center of a camera of the UAV to a location receiver of the UAV;

navigating the UAV and/or a gimbal associated with the camera such that a center of the camera is over a centroid of the physical marker; and

determining location information of the UAV, and modifying the location information based on the offset.

14. The system of claim 11, wherein determining location information of the identifiable mark comprises:

obtaining information identifying an offset from a centroid of the physical marker as carried by the UAV to a location receiver of the UAV; and

determining location information of the UAV, and modifying the location information based on the offset.

15. The system of claim 11, wherein the operations further comprise:

determining that the physical marker is level that the physical marker is level, wherein determining comprises one or more of;

prior to placing the physical marker, determining that the surface is flat such that distances from the UAV to the surface are within a threshold variance, or

after placing the physical marker, determining that distances from the physical marker to the UAV are within a threshold variance, or

after placing the physical marker, receiving confirmation, from the physical marker in communication with the UAV, that the physical marker is level.

16. The system of claim 11, wherein the physical marker is configured to display via electronically controlled lighting, a pattern comprising one or more of a shape, a pattern, a number, or a letter.

17. The system of claim 11, wherein the operations further comprise:

retrieving, via a magnetic or grabbing device of the UAV, the physical marker previously placed by the UAV.

18. The system of claim 11, wherein the UAV is in communication with the physical marker via the physical marker transmitting a geospatial position of the physical marker.

19. The system of claim 11, wherein the operations further comprise:

obtaining information describing a flight plan to perform an inspection of a structure;

performing the inspection of the structure, including capturing imagery that includes one or more of the designated surfaces of the specified geographic locations; and

providing the captured imagery and determined location information associated with the designated surfaces to a photogrammetry system for processing.

20. The system of claim 11, wherein the operations further comprise:

prior to descending towards the surface, hovering at the first altitude;

receiving, from a user device in communication with the UAV, confirmation that the UAV is to descend; and

in response to receiving the confirmation, descending towards to the surface.

21. A non-transitory computer storage media storing instructions that when executed by an unmanned aerial vehicle (UAV) comprising one or more processors, causes the one or more processors to perform operations comprising:

receiving information describing a flight plan for implementation, the flight plan specifying geographic locations in a real-world area assigned as ground control points, each geographic location being associated with a waypoint to which the UAV is to navigate and perform an operation to designate a surface of the geographic location as being a ground control point;

performing, for a particular waypoint, an operation to designate a surface associated with the particular waypoint as being a ground control point, the operation comprising;

navigating the UAV at a first altitude to the particular waypoint,

descending towards the surface, and placing a physical marker carried by the UAV onto the surface thereby creating an identifiable mark on the surface, such that the physical marker is aerially identifiable as a ground control point in one or more images obtained during a subsequent flight plan to perform a job, and

determining location information of the identifiable mark; and

providing the determined location information to an outside system for storage as being associated with a ground control point.

22. The non-transitory computer storage media of claim 21, wherein determining location information of the identifiable mark comprises:

ascending to a second altitude above the identifiable mark; and

determining, using imagery of the identifiable mark obtained by the UAV at the second altitude, location information of the identifiable mark.

23. The non-transitory computer storage media of claim 22, wherein determining location information of the identifiable mark using imagery of the identifiable mark comprises:

obtaining information identifying an offset from a center of a camera of the UAV to a location receiver of the UAV;

navigating the UAV and/or a gimbal associated with the camera such that a center of the camera is over a centroid of the physical marker; and

determining location information of the UAV, and modifying the location information based on the offset.

24. The non-transitory computer storage media of claim 21, wherein determining location information of the identifiable mark comprises:

obtaining information identifying an offset from a centroid of the physical marker as carried by the UAV to a location receiver of the UAV; and

determining location information of the UAV, and modifying the location information based on the offset.

25. The non-transitory computer storage media of claim 21, wherein the operations further comprise:

determining that the physical marker is level that the physical marker is level, wherein determining comprises one or more of;

prior to placing the physical marker, determining that the surface is flat such that distances from the UAV to the surface are within a threshold variance, or

after placing the physical marker, determining that distances from the physical marker to the UAV are within a threshold variance, or

after placing the physical marker, receiving confirmation, from the physical marker in communication with the UAV, that the physical marker is level.

26. The non-transitory computer storage media of claim 21, wherein the physical marker is configured to display via electronically controlled lighting, a pattern comprising one or more of a shape, a pattern, a number, or a letter.

27. The non-transitory computer storage media of claim 21, wherein the operations further comprise:

retrieving, via a magnetic or grabbing device of the UAV, the physical marker previously placed by the UAV.

28. The non-transitory computer storage media of claim 21, wherein the UAV is in communication with the physical marker via the physical marker transmitting a geospatial position of the physical marker.

29. The non-transitory computer storage media of claim 21, wherein the operations further comprise:

obtaining information describing a flight plan to perform an inspection of a structure;

performing the inspection of the structure, including capturing imagery that includes one or more of the designated surfaces of the specified geographic locations; and

providing the captured imagery and determined location information associated with the designated surfaces to a photogrammetry system for processing.

30. The non-transitory computer storage media of claim 21, wherein the operations further comprise:

prior to descending towards the surface, hovering at the first altitude;

receiving, from a user device in communication with the UAV, confirmation that the UAV is to descend; and

in response to receiving the confirmation, descending towards to the surface.