Surveying system

• US 10,274,316 B2
• Filed: 12/08/2016
• Issued: 04/30/2019
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: Siercks, Knut, Metzler, Bernhard, Van Der Zwan, Elmar Vincent, Fidler, Thomas, Parys, Roman, Velizhev, Alexander, Scheja, Jochen
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US15/373,280
• Publication Number:
  US 20170091923A1
• Time in Patent Office: 873 Days
• Field of Search: 348135
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01C 15/006 Detectors therefor
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system is disclosed that comprises a camera module and a control and evaluation unit. The camera module is designed to be attached to the surveying pole and comprises at least one camera for capturing images. The control and evaluation unit has stored a program with program code so as to control and execute a functionality in which a series of images of the surrounding is captured with the at least one camera; a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a reference point field is built up and poses for the captured images are determined; and, based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding can be computed by forward intersection using the series of images, particularly by using dense matching algorithm.

Claims

1. A surveying subsystem comprising:

a camera module;

a control and evaluation unit, wherein the surveying subsystem is adapted to be used as part of a surveying system that is adapted to determine positions of a position measuring resource that includes a GNSS-antenna or a retro-reflector; and

a hand-carried surveying pole on which the position measuring resource and the camera module are mounted, wherein;

the camera module comprising at least one camera for capturing images;

the control and evaluation unit has stored thereon a program with program code that, when executed, controls and executes a spatial representation generation functionality in which, when moving along a path through a surrounding;

a series of images of the surrounding is captured with the at least one camera, the series comprising an amount of images captured with different poses of the camera, the poses representing respective positions and orientations of the camera,

a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of images of the series of images and, based on resection and forward intersection using the plurality of respectively corresponding image points,

a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived,

the poses for the images are determined, and

a quality indicative output is generated concerning the measurement uncertainty of a 3D-position for at least one point of the surrounding to be computable by forward intersection using the images of the series of images and the determined poses.

2. The surveying subsystem according to claim 1, wherein,

the forward intersection implies using a variance-covariance-matrix, particularly wherein the forward intersection is carried out by use of a least square method, and the quality indicative output is derived from the variance-covariance-matrix.

3. The surveying subsystem according to claim 1, wherein,

the quality indicative output represents an uncertainty regarding the precision with which the 3D-position for the at least one point is computable or computed, particularly wherein the quality indicative output is influenced at least by:

a precision with which image-positions for the image points within the images is determinable,

an amount of images which are usable out of the series of images for the computation of the 3D-position for at least one point of the surrounding; and

a length of the baselines of the poses of the usable images.

4. The surveying subsystem according to claim 1, wherein,

based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding is computed by forward intersection using the images of the series of images;

the quality indicative output is generated for at least one of the computed points of the point cloud; and

the quality indicative output is generated for a subset of points of the point cloud referring to a corresponding amount of poses which the subset is determined from.

5. The surveying subsystem according to claim 4, wherein:

a graphical reproduction is generated for the point cloud representing the quality indicative output, the graphical reproduction being displayable by display means of the surveying system, thus providing for a feedback to a user about already acquired data, so that the already acquired data can be checked regarding its quality.

6. The surveying subsystem according to claim 4, wherein,

the quality indicative output is represented by a coloured point cloud, a scaled indication or a specific sign displayable on display means of the surveying system.

7. The surveying subsystem according to claim 4, wherein:

determined positions of the position measuring resource for points that have been adopted on the path are received by the control and evaluation unit from the surveying system, and

the point cloud is scaled with help of the received determined positions.

8. The surveying subsystem according to claim 1, wherein,

the position measuring resource comprises a GNSS-antenna or a retro-reflector.

9. The surveying subsystem according to claim 1, wherein,

the camera module comprises a camera-arrangement being built and designed in such a way that panoramic images with a field of view of 360°

at least in azimuthal direction are capturable, particularly wherein the camera-arrangement comprises at least two cameras, especially at least four cameras, arranged in a common housing of the camera module.

10. The surveying subsystem according to claim 9, further comprising:

a sensor unit fixedly integrated into the common housing, the sensor unit including at least one of the following sensors;

an inertial measuring unit,

a gyroscope,

a tilt sensor; and

a magnetic compass.

11. A surveying system, comprising

a hand-carried surveying pole;

a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system; and

a surveying subsystem according to claim 1.

13. A surveying subsystem according to claim 1, wherein,

the surveying subsystem further comprises an inertial measurement unit designed to be brought into a fixed mechanical connection with the position measuring resource, by being attached to the surveying pole, and designed for gathering IMU measurement data;

and

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that the 6-dof-travelling-history is derived further based on the IMU measurement data, wherein the IMU measurement data and the SLAM-evaluation are taken into account for deriving the rotational information of the 6-dof-travelling-history, wherein the IMU measurement data is used to increase resolution of rotational and positional information derivable from the determined poses by the SLAM-evaluation, wherein, the Kalman filter is fed with the poses determined within the SLAM-evaluation, the received determined positions and the IMU measurement data.

14. The surveying subsystem according to claim 1, wherein,

the camera module, the profiler and at least parts of the control and evaluation unit are integrated in a common single housing which is designed to be brought into a fixed mechanical connection with the position measuring resource by being attachable to the surveying pole, or

each of the camera module and the profiler is integrated in its own housing, each of the housings being designed to be brought into a fixed mechanical connection with the position measuring resource, by being attachable to the surveying pole, wherein each housing is provided with mutual connection elements for data transmission and transmission of trigger-information, such that the connection elements are interlocked and a connection for data transmission and for transmission of the trigger-information, between the camera module and the profiler is provided.

15. The surveying subsystem according to claim 1, wherein:

the camera module and the profiler each comprise a clock and a circuit for assigning a time stamp to gathered measurement data like image data, IMU measurement data and profiler measurement data, and

for the camera module and the profiler, a GNSS-module is provided for gathering absolute time information from the received GNSS-signal, so that the GNSS-module is used as the clock.

16. The surveying subsystem according to claim 1, wherein:

the profiler is designed in such that the rotating laser beam defines a laser plane being substantially perpendicular to the surveying pole, or

the profiler is designed in such that the rotating laser beam defines a laser plane being inclined with respect to the pole so that they confine an angle of between about 5 and about 35 degree, or

the profiler includes a rotatable mirror, or

the profiler includes a rotatable laser emitting and receiving unit.

17. The surveying subsystem according to claim 1, wherein,

the profiler being designed in a multi-beam setup in which the profiler being adapted:

for emission of at least two rotating laser beams; and

for reception and detection of a returning part of each of the emitted beams being scattered back from points of a surrounding.

18. The surveying subsystem according to claim 6, wherein the multi-beam setup is embodied with a common rotation platform for the at least two laser beams, the profiler being configured in a three-beam setup, wherein:

the profiler is designed in such that the at least two laser beams define laser planes being;

parallel and slightly shifted with respect to each other,

slightly inclined with respect to each other, so that slightly different inclination angles with respect to the pole are formed and/or so that slightly different azimuth angles with respect to the pole are formed, or

coincident with respect to each other,

and/or

the profiler is designed in such a way that the at least two rotating laser beams form an angular offset with respect to each other in rotation direction.

19. The surveying subsystem according to claim 7, wherein,

the common rotation platform comprises a common rotatable mirror for rotatably deflecting the at least two laser beams;

or

the common rotation platform comprises at least two laser emitting and receiving units.

20. The surveying subsystem according to claim 6, wherein:

the multi-beam setup is embodied with at least two rotation platforms for the at least two laser beams; and

the profiler with the at least two rotation platforms is designed in such that the at least two laser beams define laser planes being inclined with respect to each other, such that;

different inclination angles with respect to the pole are formed, wherein one of the laser planes forming an inclination angle of between about 10 and about 30 degrees and the other one of the laser planes forming an inclination angles of between about 60 and about 80 degree; and

/or

different azimuth angles with respect to the pole are formed, wherein an angle offset in azimuth direction of between about 1 and about 40 degrees is formed between the laser planes.

21. The surveying subsystem according to claim 1, wherein,

the profiler is designed in such a way that an inclination angle of a laser plane defined by the rotating laser beam is adjustable, manually or motor-driven, so that the inclination angle is variable with respect to the surveying pole.

22. The surveying subsystem according to claim 1, wherein,

the profiler is designed in such a way that an azimuth angle of a laser plane defined by the rotating laser beam is adjustable, particularly manually or motor-driven, so that—

in a condition where the profiler is attached to the surveying pole—

the azimuth angle is variable with respect to the pole.

23. The surveying subsystem according to claim 11, wherein:

the azimuth angle is adjustable by an actuator; and

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed such that the azimuth angle is automatically continuously adjusted by the actuator in a manner that alterations of the azimuthal orientation of the surveying pole, when moving along the path, are compensated for, wherein the automatic continuous adjustment of the azimuth angle is based on the derived 6-dof-travelling-history or a separate evaluation using;

the poses determined within the SLAM-evaluation, and/or

the received determined positions, and/or,

in case an inertial measurement unit is provided for, IMU measurement data, wherein higher frequency alterations having low amplitude are compensated for, wherein a relative azimuth angle being formed between the laser plane and a moving direction on the path is kept substantially constant in each point on the path.

25. The surveying system according to claim 11, wherein,

the azimuth angle is adjusted and stabilized with respect to the surrounding according to the principle of making use of conversation of angular momentum, using a flywheel that is rotatable together with the rotation platform which causes the laser beam to rotate.

26. A surveying system comprising:

a hand-carried surveying pole;

a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system, and

a surveying subsystem according to claim 1.

12. A surveying subsystem comprising:

a camera module;

a profiler; and

a control and evaluation unit, wherein the surveying subsystem is adapted to be used as part of a surveying system that is adapted to determine positions of a position measuring resource that is mounted on a hand-carried surveying pole and that includes a GNSS-antenna or a retro-reflector, wherein;

the camera module is configured to be brought into a fixed mechanical connection with the position measuring resource by being attached to the surveying pole, and includes at least one camera for capturing images;

the profiler is configured to be brought into a fixed mechanical connection with the position measuring resource by being attached to the surveying pole, and being adapted;

for emission of a rotating laser beam; and

for reception and detection of a returning part of the emitted laser beam being scattered back from points of a surrounding,

the profiler further being provided with an electronic distance measuring functionality as well as an angle measuring functionality for the rotating laser beam so that profiler measurement data comprising distance and angle information is gatherable, and

the control and evaluation unit has stored thereon a program with program code that, when executed, controls and executes a spatial representation generation functionality in which, when moving along a path through the surrounding;

a series of images of the surrounding is captured with the camera, the series including a plurality of images captured with different poses of the camera, from different points on the path and with different orientations of the camera;

a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of images of the series of images and, based on resection and forward intersection using the plurality of respectively corresponding image points;

a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived; and

the poses for the images are determined,

determined positions of the position measuring resource for points that have been adopted on the path are received by the control and evaluation unit from the surveying system;

a 6-dof-travelling-hi story including translational and rotational information in 6 degrees of freedom is derived for the moved path based at least on the poses determined within the SLAM-evaluation and the received determined positions, by use of a Kalman filter; and

coordinates for points of the surrounding are determined as the spatial representation, based on the profiler measurement data in combination with the derived 6-dof-travelling-history.

24. The surveying subsystem according to claim 12, wherein, as the moving direction:

the actual moving direction is used;

or

a smoothed moving direction is used and is derived for each point on the path by smoothing a most recently travelled segment of the path.