Determining tilt angle and tilt direction using image processing

• US 9,134,127 B2
• Filed: 02/04/2014
• Issued: 09/15/2015
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: Trimble Inc.
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Soubra, Omar Pierre, Janky, James M., Graesser, Christian, Weisenburger, Shawn D.
• Primary Examiner(s): Desta, Elias
• Application Number:
  US14/172,549
• Publication Number:
  US 20140156219A1
• Time in Patent Office: 588 Days
• US Class Current: 1/1
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 15/06 Surveyors' staffs; Movable markers
• CPC Class Codes: G01C 9/06 Electric or photoelectric indication or reading means
• CPC Class Codes: G01S 1/725 Marker, boundary, call-sign or like beacons transmitting signals not carrying directional information
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G01S 19/53 Determining attitude
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method of determining a tilt angle and a tilt direction of a survey instrument. The survey instrument has an imaging device that captures first images at a first location. The first images include features from an environment around the survey instrument. The imaging device also captures second images at a second location. The second images include a portion of the features. A pose of the imaging device is determined at the second location using observed changes in location of a common portion of the features between the first images and the second images. The tilt angle and the tilt direction of the survey instrument at the second location are determined using the pose of the imaging device.

Claims

1. A method of determining a tilt angle and a tilt direction of a survey instrument that includes an imaging device, the method comprising:

at a first location, obtaining reference information using the imaging device, the imaging device including a plurality of cameras arranged to capture images that include features from an environment around the survey instrument, the reference information including first images that include first features;

moving the survey instrument from the first location to a second location;

at the second location, obtaining image information using the imaging device, the image information including second images that include a portion of the first features;

processing the reference information and the image information to determine a pose of the imaging device at the second location, the pose determined using observed changes in location of the portion of the first features between the first images and the second images, wherein processing the reference information and the image information includes assigning a weighting factor to each of the second images, the weighting factor associated with a relative contribution of each of the second images in determining the pose of the imaging device; and

determining the tilt angle and the tilt direction of the survey instrument at the second location using the pose of the imaging device.

2. The method of claim 1 further comprising:

at the second location, determining coordinates of the survey instrument in a real-world coordinate frame using a Global Navigation Satellite System (GNSS) receiver; and

determining coordinates of a point of interest in the real-world coordinate frame using the coordinates of the survey instrument, the tilt angle, and the tilt direction.

3. The method of claim 1 wherein processing the reference information and the image information includes determining a direction of travel of the survey instrument, the direction of travel determined by identifying a vertical plane of divergence in the second images, the vertical plane of divergence identified using the observed changes in location of the portion of the first features between the first images and the second images, wherein the weighting factor assigned to each of the second images depends on an orientation of a respective camera relative to the vertical plane of divergence.

4. The method of claim 3 wherein the vertical plane of divergence is aligned with one of the second images, and the image aligned with the vertical plane of divergence is assigned a larger weighting factor than others of the second images.

5. The method of claim 3 wherein the vertical plane of divergence is between two of the second images, and the two images nearest to the vertical plane of divergence are assigned larger weighting factors than others of the second images.

6. The method of claim 3 wherein those of the second images that are nearest the vertical plane of divergence are assigned larger weighting factors than those of the second images that are farthest from the vertical plane of divergence.

7. The method of claim 1 wherein the reference information is obtained with the survey instrument substantially parallel to a local gravity vector.

8. The method of claim 1 wherein the reference information includes features at known locations in a reference frame.

9. The method of claim 1 further comprising:

moving the survey instrument from the second location to one or more additional locations;

at each of the one or more additional locations, obtaining additional image information using the imaging device, the additional image information including additional images that include features that are common with at least the first images, the second images, or additional images obtained at others of the one or more additional locations;

for each of the one or more additional locations;

processing the additional image information obtained at a particular location of the one or more additional locations to determine a pose of the imaging device at the particular location, wherein processing the additional image information includes assigning a weighting factor to each of the additional images, the weighting factor associated with a relative contribution of each of the additional images in determining the pose of the imaging device at the particular location; and

determining the tilt angle and the tilt direction of the survey instrument at the particular location using the pose of the imaging device at the particular location.

10. The method of claim 9 wherein the additional image information is processed real-time using a sequential bundle adjustment technique.

11. A method of determining a tilt angle and a tilt direction of a survey instrument that includes an imaging device, the method comprising:

at a first location, obtaining reference information using the imaging device, the reference information including a first panoramic image that includes first features from the environment around the survey instrument;

moving the survey instrument from the first location to a second location;

at the second location, obtaining image information using the imaging device, the image information including a second panoramic image that includes a portion of the first features;

processing the reference information and the image information to determine a pose of the imaging device at the second location, the pose determined using observed changes in location of the portion of the first features between the first panoramic image and the second panoramic image; and

determining the tilt angle and the tilt direction of the survey instrument at the second location using the pose of the imaging device.

12. The method of claim 11 wherein the imaging device includes a plurality of cameras arranged to capture overlapping images that form a 360°

panoramic view of an environment around the survey instrument.

13. The method of claim 11 wherein the imaging device includes a single camera arranged to capture a 360°

panoramic view of an environment around the survey instrument.

14. The method of claim 11 further comprising:

at the second location, determining coordinates of the survey instrument in a real-world coordinate frame using a Global Navigation Satellite System (GNSS) receiver; and

determining coordinates of a point of interest in the real-world coordinate frame using the coordinates of the survey instrument, the tilt angle, and the tilt direction.

15. The method of claim 11 wherein the reference information is obtained with the survey instrument substantially parallel to a local gravity vector.

16. The method of claim 11 wherein the reference information includes features at known locations in a reference frame.

17. The method of claim 11 further comprising:

moving the survey instrument from the second location to one or more additional locations;

at each of the one or more additional locations, obtaining additional image information using the imaging device, the additional image information including additional panoramic images that include features that are common with at least the first panoramic image, the second panoramic image, or additional panoramic images obtained at others of the one or more additional locations;

for each of the one or more additional locations;

processing the additional image information obtained at a particular location of the one or more additional locations to determine a pose of the imaging device at the particular location; and

determining the tilt angle and the tilt direction of the survey instrument at the particular location using the pose of the imaging device at the particular location.

18. The method of claim 17 wherein the reference information and the image information are processed real-time using a sequential bundle adjustment technique.

19. A method of determining a tilt angle and a tilt direction of a survey instrument that includes an imaging device, the method comprising:

at a first location, obtaining reference information using the imaging device, the imaging device including a plurality of cameras arranged to capture images that include features from an environment around the survey instrument, the reference information including first images that include first features;

moving the survey instrument from the first location to a second location;

at the second location, obtaining image information using the imaging device, the image information including second images that include a portion of the first features;

processing the reference information and the image information to determine a pose of the imaging device at the second location, wherein determining the pose of the imaging device includes determining poses of the imaging device based on each of the second images and corresponding ones of the first images and averaging the poses; and

determining the tilt angle and the tilt direction of the survey instrument at the second location using the pose of the imaging device.

20. The method of claim 19 further comprising:

at the second location, determining coordinates of the survey instrument in a real-world coordinate frame using a Global Navigation Satellite System (GNSS) receiver; and

determining coordinates of a point of interest in the real-world coordinate frame using the coordinates of the survey instrument, the tilt angle, and the tilt direction.

21. The method of claim 19 wherein the reference information is obtained with the survey instrument substantially parallel to a local gravity vector.

22. The method of claim 19 wherein the reference information includes features at known locations in a reference frame.

23. The method of claim 19 wherein the reference information and the image information are processed real-time using a sequential bundle adjustment technique.

24. The method of claim 19 further comprising:

moving the survey instrument from the second location to one or more additional locations;

at each of the one or more additional locations, obtaining additional image information using the imaging device, the additional image information including additional images that include features that are common with at least the first images, the second images, or additional images obtained at others of the one or more additional locations;

for each of the one or more additional locations;

processing the additional image information obtained at a particular location of the one or more additional locations to determine a pose of the imaging device at the particular location, wherein determining the pose of the imaging device at the particular location includes determining poses of the imaging device based on each of the additional images at the particular location and averaging the poses; and

determining the tilt angle and the tilt direction of the survey instrument at the particular location using the pose of the imaging device at the particular location.

25. A method of determining a tilt angle and a tilt direction of a survey instrument, the method comprising:

at a first location, capturing first images that include features from an environment around the survey instrument;

at a second location, capturing second images that include a portion of the first features;

determine a pose of the imaging device at the second location using observed changes in location of the portion of the first features between the first images and the second images, wherein determining the pose includes assigning a weighting factor to each of the second images, the weighting factor associated with a relative contribution of each of the second images in determining the pose of the imaging device; and

determining the tilt angle and the tilt direction of the survey instrument at the second location using the pose of the imaging device.

26. The method of claim 25 further comprising:

identifying a plane of divergence in the second images;

determining an actual direction of travel of the survey instrument; and

associating the plane of divergence in the second images with the actual direction of travel.

27. A method of determining a tilt angle and a tilt direction of a survey instrument, the method comprising:

at a first location, capturing a first panoramic image that includes first features from the environment around the survey instrument;

at a second location, capturing a second panoramic image that includes a portion of the first features;

determining a pose of the imaging device at the second location using observed changes in location of the portion of the first features between the first panoramic image and the second panoramic image; and

determining the tilt angle and the tilt direction of the survey instrument at the second location using the pose of the imaging device.