Surveying system

• US 10,655,962 B2
• Filed: 12/08/2016
• Issued: 05/19/2020
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: Siercks, Knut, Metzler, Bernhard, Van Der Zwan, Elmar Vincent, Fidler, Thomas, Parys, Roman, Velizhev, Alexander, Scheja, Jochen
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US15/373,226
• Publication Number:
  US 20170091922A1
• Time in Patent Office: 1,258 Days
• Field of Search: 348135
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01C 15/006 Detectors therefor
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system is disclosed that comprises a camera module and a control and evaluation unit. The camera module is designed to be attached to the surveying pole and comprises at least one camera for capturing images. The control and evaluation unit has stored a program with program code so as to control and execute a functionality in which a series of images of the surrounding is captured with the at least one camera; a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a reference point field is built up and poses for the captured images are determined; and, based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding can be computed by forward intersection using the series of images, particularly by using dense matching algorithm.

Claims

1. A surveying subsystem adapted to be used as part of a surveying system that is adapted to determine positions of a position measuring resource that is mounted on a surveying pole and includes a GNSS-antenna or a retro-reflector, wherein said surveying pole is hand-carried, the surveying subsystem comprising:

a camera module configured to be attached to the surveying pole and having a camera and an optical element, the optical element being configured such that the camera has an angle of view of 360°

in an azimuthal direction; and

a control and evaluation unit having stored thereon a program with program code that, when executed, controls and executes a spatial representation generation functionality in which, when moving along a path through a surrounding;

a first series of panoramic images of the surrounding is captured with the camera, the first series comprising an amount of panoramic images captured with different poses of the camera, the poses representing respective positions and orientations of the camera,

a SLAM-evaluation with a defined algorithm using the series of panoramic images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of panoramic images of the series of panoramic images and, based on resection and forward intersection using the plurality of respectively corresponding image points,

a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived, and

the poses for the panoramic images are determined,

wherein a position and orientation of at least one of the surveying subsystem or the camera in six degrees of freedom are derived based on one or more of the determined poses and based on a multitude of determined positions of the position measuring resource.

2. The surveying subsystem according to claim 1, wherein:

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that the reference point field forms a sparse point cloud that serves for determination of the poses of the camera/s and thus expressly for localisation purposes and,

based on the determined poses, a dense point cloud that includes 3D-positions of points of the surrounding is computed by forward intersection using the images of the first series of images and images of a second series of images.

3. The surveying subsystem according to claim 2, wherein,

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that:

determined positions of the position measuring resource for points that have been adopted on the path are received by the control and evaluation unit from the surveying system; and

the dense point cloud is scaled, and particularly geo-referenced, with help of the received determined positions.

4. The surveying subsystem according to claim 2, wherein,

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that a graphical reproduction is generated for the dense point cloud, the graphical reproduction being displayable by display means of the surveying system.

5. The surveying subsystem according to any claim 1, wherein,

the position measuring resource comprises a GNSS-antenna or a retro-reflector.

6. The surveying subsystem according to any one of claim 1, wherein,

orientations of at least one camera in six degrees of freedom are derived based on:

one or more of the determined poses; and

data from an inertial measuring unit of the camera module.

7. A surveying system comprising:

a hand-carried surveying pole,

a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system, and

a surveying subsystem according to claim 1.

8. The surveying system according to claim 7, wherein,

the position measuring resource comprises a GNSS-antenna or a retro-reflector.

9. The surveying subsystem according to claim 1, wherein, the optical element comprises a mirroring surface in form of a cone, a sphere or a curvature.

10. The surveying subsystem according to claim 1, wherein, the camera and the optical element are arranged and designed so that a surrounding to be captured is projected onto a sensor of the camera so that an image of the surrounding is provided with substantially identically sized pieces of the image covering substantially identically sized parts of the surrounding, thus providing for an image representing the captured surrounding in homogenous reproduction.

11. A surveying subsystem adapted to be used as part of a surveying system that is adapted to determine positions of a position measuring resource that is mounted on a surveying pole and includes a GNSS-antenna or a retro-reflector, wherein said surveying pole is hand-carried, the surveying subsystem comprising:

a camera module configured to be attached to the surveying pole and having a camera and an optical element, the optical element being configured such that the camera has an angle of view of 360°

in an azimuthal direction; and

a control and evaluation unit having stored thereon a program with program code that, when executed, controls and executes a spatial representation generation functionality in which, when moving along a path through a surrounding;

a first series of panoramic images of the surrounding is captured with the camera, the first series comprising an amount of panoramic images captured with different poses of the camera, the poses representing respective positions and orientations of the camera,

a SLAM-evaluation with a defined algorithm using the series of panoramic images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of panoramic images of the series of panoramic images and, based on resection and forward intersection using the plurality of respectively corresponding image points,

a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived, and

the poses for the panoramic images are determined,

wherein a position and orientation of at least one of the surveying subsystem or the camera in six degrees of freedom are derived based on one or more of the determined poses and based on a traveling history for the moved path.

12. The surveying subsystem according to claim 11, wherein, the optical element comprises a mirroring surface in form of a cone, a sphere or a curvature.

13. The surveying subsystem according to claim 11, wherein, the camera and the optical element are arranged and designed so that a surrounding to be captured is projected onto a sensor of the camera so that an image of the surrounding is provided with substantially identically sized pieces of the image covering substantially identically sized parts of the surrounding, thus providing for an image representing the captured surrounding in homogenous reproduction.

14. The surveying subsystem according to claim 11, wherein:

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that the reference point field forms a sparse point cloud that serves for determination of the poses of the camera/s and thus expressly for localisation purposes and,

based on the determined poses, a dense point cloud that includes 3D-positions of points of the surrounding is computed by forward intersection using the images of the first series of images and images of a second series of images.

15. The surveying subsystem according to claim 14, wherein,

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that:

determined positions of the position measuring resource for points that have been adopted on the path are received by the control and evaluation unit from the surveying system; and

the dense point cloud is scaled, and particularly geo-referenced, with help of the received determined positions.

16. The surveying subsystem according to claim 14, wherein,

the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that a graphical reproduction is generated for the dense point cloud, the graphical reproduction being displayable by display means of the surveying system.

17. The surveying subsystem according to any claim 11, wherein,

the position measuring resource comprises a GNSS-antenna or a retro-reflector.

18. The surveying subsystem according to any one of claim 11, wherein,

orientations of at least one camera in six degrees of freedom are derived based on:

one or more of the determined poses; and

data from an inertial measuring unit of the camera module.

19. A surveying system comprising:

a hand-carried surveying pole,

a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system, and

a surveying subsystem according to claim 11.

20. The surveying system according to claim 19, wherein,

the position measuring resource comprises a GNSS-antenna or a retro-reflector.