Determination of object data by template-based UAV control

• US 9,898,821 B2
• Filed: 02/25/2016
• Issued: 02/20/2018
• Est. Priority Date: 02/26/2015
• Status: Active Grant

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: Metzler, Bernhard, Siercks, Knut, Van Der Zwan, Elmar Vincent
• Primary Examiner(s): Mistry, Oneal R
• Application Number:
  US15/053,935
• Publication Number:
  US 20160253808A1
• Time in Patent Office: 726 Days
• CPC Class Codes: B64C 39/02 characterised by special use
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/75 involving models

Abstract

A Method for providing information about an object using an unmanned aerial vehicle with a data acquisition unit is disclosed according to some embodiments of the invention. The method may include determining positional data with reference to the object, the positional data being referenced to a measuring coordinate system, providing a digital template regarding the object, the template at least partly representing the object in coarse manner, and referencing the template with the positional data so that the template corresponds as to its spatial parameters to the object in the measuring coordinate system.

Claims

1. Method for providing information about an object using an unmanned aerial vehicle with a data acquisition unit, the method comprising:

determining on site with a geodetic surveying instrument positional data with reference to the object, the positional data being referenced to a measuring coordinate system defined by the surveying instrument and/or in an absolute global coordinate system when the surveying instrument is already referenced thereto;

selecting a digital template regarding the object,

the digital template at least partly representing the object in coarse manner, wherein the digital template is embodied at least as one out of a plane, a cylinder, a sphere, a specific geometric figure, or a digital model for the object,

and wherein for each type of object of interest a respective template, which provides an optimal flight path and positions for image acquisition for the respective object, is provided for flight planning,

referencing the digital template with the positional data so that the digital template corresponds as to its spatial parameters to the object in the measuring coordinate system;

deriving at least one first measuring path based on the digital template and a spatial position of at least one data acquisition point or section related to the object based on at least the positional data and/or the digital template;

controlling the unmanned aerial vehicle in a manner such that it moves according to the first measuring path and that it approaches the at least one data acquisition point or section; and

acquiring object information as to at least a part of the object according to the at least one data acquisition point or section by triggering the data acquisition unit depending on fulfilling a defined distance criterion, the distance criterion defining a spatial relation between an actual position of the unmanned aerial vehicle and the at least one data acquisition point or section.

2. The method according to claim 1, comprising selecting the first measuring path based on at least the referenced digital template is performed automatically.

3. The method according to claim 1, wherein the template is adaptable regarding its size and spatial orientation, wherein its spatial parameters define a spatial extension and a spatial orientation of the template, in particular height H, width W, length L, azimuthal angle and/or tilt angle.

4. The method according to claim 1, wherein the template provides object parameters being related to the type of object, wherein the object parameters provide information about a no-flight zone and/or an area of the object of particular interest.

5. The method according to claim 1, wherein referencing the template with the positional data further comprises digitally adapting a scaling and/or a spatial orientation of the template so that size and/or orientation of the template corresponds to size and/or orientation of the object in the measuring coordinate system.

6. The method according to claim 1, wherein referencing the template with the positional data further comprises fitting template positioning data with the positional data.

7. The method according to claim 6, wherein fitting the template positioning data with the positional data further comprises fitting a set of template reference points with the positional data.

8. The method according to claim 1, wherein deriving the template is based on:

a digital model for the object, and/or

a measurement of the object, and/or

determining the positional data by;

measuring positions of a set of object representing points related to the object by use of a geodetic surveying instrument, and/or

at least partly coarsely scanning the object with the geodetic surveying instrument, wherein the geodetic surveying instrument is embodied as a laser scanner.

9. The method according to claim 1, wherein:

the data acquisition unit is embodied as an image capturing unit or as scanning unit, wherein acquisition of the object information is provided by;

capturing at least one image according to the at least one data acquisition point;

or

capturing at least one image sequence according to the at least one data acquisition section;

or

scanning the object according to the at least one data acquisition section.

10. The method according to claim 1, wherein:

a series of data acquisition points is derived based on the referenced template, wherein a series of images is captured by means of the data acquisition unit with respect to the series of data acquisition points, wherein

stereoscopic images are derived based on pairs of successive images, providing positional information of measuring points at the object, and/or

the first measuring path comprises the series of data acquisition points including the at least one data acquisition point,

wherein a point cloud is processed based on the series of images captured according to the series of data acquisition points, the point cloud representing the regions of the object covered by the images.

11. The method according to claim 1, wherein:

coarsely scanning and/or capturing a surrounding of the object and thereby providing surrounding data, the surrounding data providing information about obstacles, and

defining the at least one data acquisition point or section depending on the surrounding data.

12. The method according to claim 1, comprising:

deriving a second measuring path based on at least the referenced template, the second measuring path;

being of basically identical shape and extension compared to the first measuring path, wherein the course of the second measuring path corresponds to the course of the first measuring path,

being offset from the first measuring path in a defined manner, and

comprising further data acquisition points each of which is assigned to a respective data acquisition point of the first measuring path,

wherein a further series of images is captured by the data acquisition unit or a further image capturing unit as to the further data acquisition points of the second measuring path, wherein,

the further image capturing unit is carried by a second unmanned aerial vehicle and the second unmanned aerial vehicle being controlled in a manner such that it moves along the second measuring path, and/or

stereoscopic images are derived based on pairs of images, each pair of images is represented by a first image according to a data acquisition point of the first measuring path and a second image according to the assigned data acquisition point of the second measuring path, wherein a constant stereoscopic baseline for each pair of images is provided.

13. The method according to claim 1, wherein:

the unmanned aerial vehicle is controlled based on image processing of image data provided by an actual image of the data acquisition unit and an overview image of the object, wherein homologous features in the overview image and the actual image are derived, wherein the data acquisition unit provides real time image data and controlling of the unmanned aerial vehicle is based on the real time image data.

14. The method according to claim 1,

wherein an optical marking is generated at the object by directing a laser beam emitted from the surveying instrument to at the object.

15. The method according to claim 14,

wherein the unmanned aerial vehicle is controlled such that it approaches a defined spatial relation to the optical marking based on capturing an image with the data acquisition unit which covers the optical marking and on deriving a position of the optical marking in the captured image, wherein the optical marking provides a basis for deriving the data acquisition point or section.

16. The method according to claim 15, comprising capturing a first marking image covering the optical marking and a surrounding of the optical marking with the image capturing unit, wherein a second marking image is captured covering the optical marking and a surrounding of the optical marking, wherein a spatial position of the point at the object, which is marked by the optical marking, is determined based on the first and the second marking image.

17. The method according to claim 1, comprising continuously tracking the unmanned aerial vehicle by locking a laser beam emitted from a surveying instrument to the unmanned aerial vehicle and thereby continuously providing the actual position of the unmanned aerial vehicle.

18. A geodetic surveying system for providing information about an object, comprising:

on site in line-of-sight of the object a geodetic surveying instrument adapted to receive and provide positional data with reference to the object, the positional data being referenced to a measuring coordinate system defined by the surveying instrument and/or in an absolute global coordinate system when the surveying instrument is already referenced thereto,

a data acquisition unit for acquiring object information, and

a controlling and processing unit adapted;

to derive the at least one data acquisition point or section, and

to control acquisition of the object information, wherein;

an unmanned aerial vehicle which comprises the data acquisition unit,

a database providing a digital template regarding the object, the template at least partly representing the object in coarse manner, wherein the digital template is embodied at least as one out of a plane, a cylinder, a sphere, a specific geometric figure, or a digital model for the object,

the controlling and processing unit being adapted;

to control a movement of the unmanned aerial vehicle,

to execute a data acquisition functionality, the data acquisition functionality being at least defined by;

referencing the digital template with the positional data so that the template corresponds as to its spatial parameters to the object in the measuring coordinate system,

deriving a spatial position of at least one data acquisition point or section related to the object based on at least the positional data and/or the digital template,

controlling the unmanned aerial vehicle in a manner such that it approaches the at least one data acquisition point or section, and

acquiring object information as to at least a part of the object according to the at least one data acquisition point or section by triggering the data acquisition unit depending on fulfilling a defined distance criterion, the distance criterion defining a spatial relation between an actual position of the unmanned aerial vehicle and the at least one data acquisition point or section.

19. The geodetic surveying system according to claim 18, the unmanned aerial vehicle further comprising an inertial measuring unit.

20. The geodetic surveying system according to claim 18, the unmanned aerial vehicle further comprising at least one device selected from the list consisting of an accelerometer, tilt sensor, magnetic compass, and a gyroscope.

21. The geodetic surveying system according to claim 18, the unmanned aerial vehicle further comprising a retro-reflecting unit for reflecting a laser beam emitted from the surveying instrument.

22. A non-transitory computer program product having computer-executable instructions for controlling and executing at least the steps of:

receiving positional data of an object from a geodetic surveying instrument located on site in line-of-sight of the object, the positional data being referenced to a measuring coordinate system defined by the surveying instrument and/or in an absolute global coordinate system when the surveying instrument is already referenced thereto, and

receiving a digital template regarding the object, wherein the digital template is embodied at least as one out of a plane, a cylinder, a sphere, or a specific geometric figure, and wherein for each type of objects of interest a respective template, which provides an optimal flight path and positions for image acquisition for the respective object is provided for flight planning,

referencing the digital template with the positional data so that the digital template corresponds as to its spatial parameters to the object in the measuring coordinate system,

deriving at least one first measuring path based on the digital template and a spatial position of at least one data acquisition point or section related to the object based on at least the positional data and/or the digital template,

controlling an unmanned aerial vehicle such that it moves according to the first measuring path and that it approaches the at least one data acquisition point or section, and

acquiring object information as to at least a part of the object, based on;

continuously receiving an actual position of the unmanned aerial vehicle and/or

receiving a referenced spatial position of the unmanned aerial vehicle, the distance criterion defining a spatial relation between an actual position of the unmanned aerial vehicle and the at least one data acquisition point or section, when run on a controlling and processing unit of a geodetic surveying system.