DETERMINATION OF OBJECT DATA BY TEMPLATE-BASED UAV CONTROL

• US 20160253808A1
• Filed: 02/25/2016
• Published: 09/01/2016
• Est. Priority Date: 02/26/2015
• Status: Active Grant

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: VAN DER ZWAN, Elmar Vincent, METZLER, Bernhard, SIERCKS, Knut
• Granted Patent: US 9,898,821 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/02 characterised by special use
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/75 involving models

Abstract

A Method for providing information about an object using an unmanned aerial vehicle with a data acquisition unit is disclosed according to some embodiments of the invention. The method may include determining positional data with reference to the object, the positional data being referenced to a measuring coordinate system, providing a digital template regarding the object, the template at least partly representing the object in coarse manner, and referencing the template with the positional data so that the template corresponds as to its spatial parameters to the object in the measuring coordinate system.

Claims

1. Method for providing information about an object using an unmanned aerial vehicle with a data acquisition unit, comprising:

determining positional data with reference to the object, the positional data being referenced to a measuring coordinate system;

providing a digital template regarding the object, the template at least partly representing the object in coarse manner,

referencing the template with the positional data so that the template corresponds as to its spatial parameters to the object in the measuring coordinate system;

deriving a spatial position of at least one data acquisition point or section related to the object based on at least the positional data and/or the template;

controlling the unmanned aerial vehicle in a manner such that it approaches the at least one data acquisition point or section; and

acquiring object information as to at least a part of the object according to the at least one data acquisition point or section by triggering the data acquisition unit depending on fulfilling a defined distance criterion, the distance criterion defining a spatial relation between an actual position of the unmanned aerial vehicle and the at least one data acquisition point or section.

2. The method according to claim 1, wherein:

deriving a first measuring path based on at least the referenced template, and

controlling the unmanned aerial vehicle in a manner such that it moves according to the first measuring path.

3. The method according to claim 1, wherein the template is adaptable regarding its size and spatial orientation, wherein its spatial parameters define a spatial extension and a spatial orientation of the template, in particular height H, width W, length L, azimuthal angle and/or tilt angle.

4. The method according to claim 1, wherein the template provides object parameters being related to the type of object, wherein the object parameters provide information about a no-flight zone and/or an area of the object of particular interest.

5. The method according to claim 1, wherein referencing the template with the positional data further comprises digitally adapting a scaling and/or a spatial orientation of the template so that size and/orientation of the template corresponds to size and/orientation of the object in the measuring coordinate system.

6. The method according to claim 1, wherein referencing the template with the positional data further comprises fitting template positioning data with the positional data.

7. The method according to claim 6, wherein the fitting template positioning data with the positional data further comprises fitting a set of template reference points with the positional data.

8. The method according to claim 1, wherein deriving the template is based on:

a digital model for the object and/or

a measurement of the object, and/or

determining the positional data by;

measuring positions of a set of object representing points related to the object by use of a position determining device, and/or

at least partly coarsely scanning the object with the position determining device, wherein the position determining device is embodied as a laser scanner,

by means of geo-referenced digital terrain data and selecting at least one object representing point according to the terrain data.

9. The method according to claim 1, wherein:

the data acquisition unit is embodied as an image capturing unit or as scanning unit, wherein acquisition of the object information is provided by

capturing at least one image according to the at least one data acquisition point or

capturing at least one image sequence according to the at least one data acquisition section or

scanning the object according to the at least one data acquisition section.

10. The method according to claim 1, wherein:

a series of data acquisition points is derived based on the referenced template, wherein a series of images is captured by means of the data acquisition unit with respect to the series of data acquisition points, wherein

stereoscopic images are derived based on pairs of successive images, providing positional information of measuring points at the object, and/or

the first measuring path comprises the series of data acquisition points including the at least one data acquisition point,

wherein a point cloud is processed based on the series of images captured according to the series of data acquisition points, the point cloud representing the regions of the object covered by the images.

11. The method according to claim 1, wherein:

coarsely scanning and/or capturing a surrounding of the object and thereby providing surrounding data, the surrounding data providing information about obstacles, and

defining the at least one data acquisition point or section depending on the surrounding data.

12. The method according to claim 1, wherein:

deriving a second measuring path based on at least the referenced template, the second measuring path

being of basically identical shape and extension compared to the first measuring path, wherein the course of the second measuring path corresponds to the course of the first measuring path,

being offset from the first measuring path in defined manner and

comprising further data acquisition points each of which is assigned to a respective data acquisition point of the first measuring path,

wherein a further series of images is captured by the data acquisition unit or a further image capturing unit as to the further data acquisition points of the second measuring path, wherein

the further image capturing unit is carried by a second unmanned aerial vehicle and the second unmanned aerial vehicle being controlled in a manner such that it moves along the second measuring path and/or

stereoscopic images are derived based on pairs of images, each pair of images is represented by a first image according to a data acquisition point of the first measuring path and a second image according to the assigned data acquisition point of the second measuring path, wherein a constant stereoscopic baseline for each pair of images is provided.

13. The method according to claim 1, wherein:

the unmanned aerial vehicle is controlled based on image processing of image data provided by an actual image of the data acquisition unit and an overview image of the object, wherein homologous features in the overview image and the actual image are derived, wherein the data acquisition unit provides real time image data and controlling of the unmanned aerial vehicle is based on the real time image data.

14. The method according to claim 1,

wherein the unmanned aerial vehicle is controlled such that it approaches a defined spatial relation to the optical marking based on capturing an image with the data acquisition unit which covers the optical marking and on deriving a position of the optical marking in the captured image, wherein the optical marking provides a basis for deriving the data acquisition point or section.

15. The method according to claim 14,

wherein generating an optical marking at the object comprises directing a laser beam at the object.

16. The method according to claim 1,

Wherein capturing a first marking image covering the optical marking and a surrounding of the optical marking with the image capturing unit, wherein a second marking image is captured covering the optical marking and a surrounding of the optical marking, wherein a spatial position of the point at the object, which is marked by the optical marking, is determined based on the first and the second marking image.

17. The method according to claim 1, wherein continuously tracking the unmanned aerial vehicle by locking a laser beam emitted from a surveying instrument to the unmanned aerial vehicle and thereby continuously providing the actual position of the unmanned aerial vehicle.

18. A geodetic surveying system for providing information about an object, comprising:

a data receiving unit adapted to receive and provide positional data with reference to the object, the positional data being referenced to a measuring coordinate system,

a data acquisition unit for acquiring object information, and

a controlling and processing unit adapted

to derive the at least one data acquisition point or section and

to control acquisition of the object information, wherein;

an unmanned aerial vehicle which comprises the data acquisition unit,

a database providing a digital template regarding the object, the template at least partly representing the object in coarse manner,

the controlling and processing unit being adapted

to control a movement of the unmanned aerial vehicle,

to execute a data acquisition functionality, the data acquisition functionality being at least defined by

referencing the template with the positional data so that the template corresponds as to its spatial parameters to the object in the measuring coordinate system,

deriving a spatial position of at least one data acquisition point or section related to the object based on at least the positional data and/or the template,

controlling the unmanned aerial vehicle in a manner such that it approaches the at least one data acquisition point or section, and

acquiring object information as to at least a part of the object according to the at least one data acquisition point or section by triggering the data acquisition unit depending on fulfilling a defined distance criterion, the distance criterion defining a spatial relation between an actual position of the unmanned aerial vehicle and the at least one data acquisition point or section.

19. The geodetic surveying system according to claim 18, further comprising an inertial measuring unit.

20. The geodetic surveying system according to claim 18, further comprising at least one device selected from the list consisting of an accelerometer, tilt sensor, magnetic compass, and a gyroscope.

21. The geodetic surveying system according to claim 18, further comprising a retro-reflecting unit for reflecting a laser beam emitted from the surveying instrument.

22. A non-transitory computer program product having computer-executable instructions for controlling and executing at least the steps of

receiving the positional data and the template regarding the object,

referencing the template with the positional data,

deriving a spatial position of at least one data acquisition point or section,

controlling the unmanned aerial vehicle and

acquiring object information as to at least a part of the object, based on

continuously receiving an actual position of the unmanned aerial vehicle and/or

receiving a referenced spatial position of the unmanned aerial vehicle,

when run on a controlling and processing unit of a geodetic surveying system.