SCANNING ENVIRONMENTS AND TRACKING UNMANNED AERIAL VEHICLES

• US 20160292872A1
• Filed: 03/02/2016
• Published: 10/06/2016
• Est. Priority Date: 03/03/2015
• Status: Active Grant

• Current Assignee: Prenav, Inc.
• Original Assignee: Prenav, Inc.
• Inventors: Hammond, Asa, Schuett, Nathan, Busek, Naimisaranya Das
• Granted Patent: US 10,671,066 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: F03D 17/00 Monitoring or testing of wind motors, e.g. diagnostics testing during commissioning of wind motors F03D13/30
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/933 of aircraft or spacecraft
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20221 : Image fusion; Image merging
• CPC Class Codes: G06T 2207/30181 : Earth observation
• CPC Class Codes: G06T 2207/30241 : Trajectory
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0004 : Industrial image inspection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes
• CPC Class Codes: G06V 20/653 : by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 13/128 : Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals

Abstract

Systems and methods for scanning environments and tracking unmanned aerial vehicles within the scanned environments are disclosed. A method in accordance with a particular embodiment includes using a rangefinder off-board an unmanned air vehicle (UAV) to identify points in a region. The method can further include forming a computer-based map of the region with the points and using the rangefinder and a camera to locate the UAV as it moves in the region. The location of the UAV can be compared with locations on the computer-based map and, based upon the comparison, the method can include transmitting guidance information to the UAV. In a further particular embodiment, two-dimensional imaging data is used in addition to the rangefinder data to provide color information to points in the region.

Claims

1. A system, having a processor, for scanning and mapping an environment in which an unmanned aerial vehicle is expected to operate, the system comprising:

a camera configured to capture two-dimensional optical images of the environment;

a first component coupled to the camera to identify features within one or more of the captured two-dimensional optical images of the environment;

a rangefinder configured to generate depth data at least in part by determining or estimating the distance from the rangefinder to each of a plurality of the identified features;

a second component coupled to the rangefinder to transform the generated depth data from spherical coordinates to Cartesian coordinates; and

a third component coupled to the first and second components to generate a three-dimensional point cloud model of the environment based at least in part on the transformed depth data.

2. The system of claim 1, wherein the camera and rangefinder each have independent optical paths to the environment.

3. The system of claim 1, wherein a beam splitter provides identical optical information to both the camera and the rangefinder.

4. The system of claim 1, further comprising:

a tilt encoder configured to track the motion of the camera and the rangefinder relative to a horizontal axis,

wherein the transformation of the generated depth data from spherical coordinates to Cartesian coordinates is based at least in part on information collected from the tilt encoder.

5. The system of claim 1, further comprising:

a pan encoder configured to track the rotational position of the camera and the rangefinder around a vertical axis,

wherein the transformation of the generated depth data from spherical coordinates to Cartesian coordinates is based at least in part on information collected from the pan encoder.

6. The system of claim 1, further comprising:

a fourth component coupled to the third component to plan a flight for the unmanned aerial vehicle through the environment.

7. The system of claim 6, further comprising:

a fifth component coupled to the fourth component to determine that the unmanned aerial vehicle has deviated from the planned flight; and

a sixth component coupled to the fifth component to provide feedback to the unmanned aerial vehicle in response to determining that the unmanned aerial vehicle has deviated from the planned flight.

8. The system of claim 1, further comprising:

a fourth component coupled to the third component to provide positional feedback to the unmanned aerial vehicle as it operates in the environment at least in part by;

comparing position information of the unmanned aerial vehicle to the generated three-dimensional point cloud model of the environment.

9. The system of claim 8, wherein the position information of the unmanned aerial vehicle is received from the unmanned aerial vehicle.

10. The system of claim 8, wherein the position information of the unmanned aerial vehicle is determined at least in part by analyzing one or more two-dimensional optical images of the unmanned aerial vehicle captured by the camera.

11. The system of claim 1, wherein the camera and the rangefinder are attached to the unmanned aerial vehicle.

12. The system of claim 1, wherein the camera and the rangefinder are located on the ground.

13. The system of claim 1, wherein the camera is configured to receive radiation from the environment in the visible spectrum.

14. The system of claim 1, wherein the camera is configured to receive radiation from the environment in the infrared, ultraviolet, or other ranges of the electromagnetic spectrum range.

15. The system of claim 1, wherein the rangefinder comprises a lidar detector.

16. The system of claim 1, further comprising:

a fourth component coupled to the third component to supplement the generated three-dimensional point cloud model with two-dimensional optical images captured by the camera.

17. The system of claim 1, further comprising:

a fourth component coupled to the third component to receive an indication of a plurality of target features of the environment,

wherein the camera captures only two-dimensional optical images of the environment that include one or more of the target features.

18. A method, performed by a computing system having a memory and a processor, for scanning and mapping an environment, the method comprising:

capturing, with a camera, two-dimensional optical images of the environment;

identifying features within one or more of the captured two-dimensional optical images of the environment;

generating, with a rangefinder, depth data at least in part by determining the distance from the rangefinder to each of a plurality of the identified features; and

generating, with the processor, a three-dimensional point cloud model of the environment based at least in part on the transformed depth data.

19. The method of claim 18, further comprising:

retrieving a previously-scanned three-dimensional point cloud model of the environment; and

comparing the retrieved previously-scanned three-dimensional point cloud model of the environment to the generated three-dimensional point cloud model of the environment to identify new objects within the environment.

20. A computer-readable storage medium storing instructions that, if executed by a computing system having a memory and a processor, cause the computing system to perform a method for scanning and mapping an environment, the method comprising:

receiving two-dimensional optical images of the environment captured by a camera;

identifying features within one or more of the captured two-dimensional optical images of the environment;

receiving depth data generated, by a rangefinder, at least in part by estimating the distance from the rangefinder to each of a plurality of the identified features; and

generating a three-dimensional point cloud model of the environment based at least in part on the transformed depth data.

21. A method, performed by a computing system having a processor, for scanning and mapping an environment, the system comprising:

receiving an indication of a plurality of target features of the environment;

capturing, with a camera, two-dimensional optical images of the environment, wherein the capturing comprises capturing two-dimensional optical images of the environment that include one or more of the target features of the environment;

generating, with a rangefinder, depth data only for one or more of the target features of the environment; and

generating a three-dimensional point cloud model of the environment based at least in part on the depth data.

22. The method of claim 21, further comprising:

grouping the target features into regions; and

classifying each of the target regions.