Method and apparatus for image-based positioning

• US 8,754,805 B2
• Filed: 06/24/2011
• Issued: 06/17/2014
• Est. Priority Date: 12/15/2005
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Wang, Sy Bor, Teng, Hongbo, Scherzinger, Bruno M., France, Peter G., Janky, James M., Soubra, Omar P., Best, Gregory C.
• Primary Examiner(s): LIU, HARRY K
• Application Number:
  US13/167,733
• Publication Number:
  US 20120163656A1
• Time in Patent Office: 1,089 Days
• Field of Search: 342/357.17
• US Class Current: 342/52
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 15/06 Surveyors' staffs; Movable markers
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Method and apparatus are provided for image based positioning comprising capturing a first image with an image capturing device. Wherein said first image includes at least one object. Moving the platform and capturing a second image with the image capturing device. The second image including the at least one object. Capturing in the first image an image of a surface; capturing in the second image a second image of the surface. Processing the plurality of images of the object and the surface using a combined feature based process and surface tracking process to track the location of the surface. Finally, determining the location of the platform by processing the combined feature based process and surface based process.

Claims

1. An image-based method for tracking the location of a platform containing at least one image capturing device and at least one range measuring device comprising:

capturing a first image with said at least one image capturing device;

wherein said first image includes at least one object and at least one surface;

obtaining a first set of depth data of said at least one surface in said first image using said at least one range measuring device;

thereafter

moving said platform and capturing a second image with said at least one image capturing device;

wherein said second image includes said at least one object and said at least one surface;

obtaining a second set of depth data of said at least one surface in said second image using said at least one range measuring device;

processing said plurality of images of said at least one object and said at least one surface using a combined feature based process and surface tracking process, wherein said feature based process is different from said surface tracking process and includes matching optical properties of pixels, and said surface tracking process includes matching said first set of depth data and said second set of depth data; and

determining the location of said platform from the combined feature based process and surface tracking process.

2. The method of claim 1, wherein said feature based process is based upon the Simultaneous Location and Mapping (SLAM) tracking process.

3. The method of claim 1, wherein said feature based process is based upon the Match Move tracking process.

4. The method of claim 1, wherein said feature based process is based upon the photogrammetric tracking process.

5. The method of claim 1, wherein said at least one object is captured in said first image and in said second image by a first image capturing device and said at least one surface is captured in said first image and in said second image by a second imaging capturing device.

6. The method of claim 5, further comprising:

synchronizing said first image capturing device and said second image capturing device.

7. The method of claim 1, further comprising:

a GNSS receiver on said platform outputting a location for said platform; and

determining the location of the platform by combining the location output by the combined feature based process and surface tracking process with the location output by said GNSS receiver.

8. The method of claim 1 further comprising:

accessing georeferenced data correlated to said at least one object; and

determining the location of said platform based upon said georeferenced data for said at least one object.

27. The method of claim 1, wherein said range measuring device is selected from the group consisting of:

a point laser beam;

a sonar;

a radar;

a laser scanner; and

a depth camera.

9. An image-based method for tracking the location of a platform containing at least one image capturing device and at least one range measuring device comprising:

capturing a first image with said at least one image capturing device;

wherein said first image includes at least one object and at least one surface;

obtaining a first set of depth data of said at least one surface in said first image using said at least one range measuring device;

thereafter

moving said platform and capturing a second image with said at least one image capturing device;

wherein said second image includes said at least one object and said at least one surface;

obtaining a second set of depth data of said at least one surface in said second image using said at least one range measuring device;

processing said plurality of images to track the location of said at least one object using a feature based process that includes matching optical properties of pixels;

outputting first coordinates for said platform based upon said feature based process;

processing said plurality of images to track the location of said at least one surface using a surface tracking process that includes matching said first set of depth data and said second set of depth data;

outputting second coordinates for said platform based upon said surface tracking process; and

determining the location of said platform by processing said first coordinates output by said feature based process and said second coordinates output by said surface based process.

10. The method of claim 9, wherein said feature based process is based upon the Simultaneous Location and Mapping (SLAM) tracking process.

11. The method of claim 9, wherein said feature based process is based upon the Match Move tracking process.

12. The method of claim 9, wherein said feature based process is based upon the photogrammetric tracking process.

13. The method of claim 9, wherein said at least one object is captured in said first image and in said second image by a first image capturing device and said at least one surface is captured in said first image and in said second image by a second imaging capturing device.

14. The method of claim 13, further comprising:

synchronizing said first image capturing device and said second image capturing device.

15. The method of claim 9, further comprising:

a GNSS receiver on said platform outputting a location for said platform; and

determining the location of the platform by combining the locations output by said feature based process, said surface based process and said GNSS receiver.

16. The method of claim 9 further comprising:

accessing georeferenced data correlated to said at least one object; and

determining the location of said platform based upon said georeferenced data for said at least one object.

28. The method of claim 9, wherein said range measuring device is selected from the group consisting of:

a point laser beam;

a sonar;

a radar;

a laser scanner; and

a depth camera.

17. An image-based method to estimate a location of a platform containing an image capturing device and a GNSS receiver comprising:

determining the location of the platform using a GNSS receiver;

capturing at least one image at a first position of said image capturing device;

wherein said first image includes at least one object;

capturing at least one image at a second position of said image capturing device;

wherein said second image includes said at least one object;

processing said plurality of images to track a location of said at least one captured object;

wherein a set of two-dimensional position fixes for said at least one captured object is obtained from processing said plurality of images, providing a set of tracking data for said at least one object;

determining the location of said image capturing device by using said set of tracking data of said at least one object;

and

calculating a new location of said platform by combining the location of said image capturing device with the location determined by said GNSS receiver.

18. The method of claim 17, further comprising:

using an inverse projection function applied to said set of tracking data of said at least one captured object to solve for a set of three-dimensional (3D) coordinates of said position of said image capturing device.

19. The method of claim 17, further comprising:

obtaining a set of two-dimensional (2D) tracking data for at least three features, wherein said set of two-dimensional (2D) tracking data for said at least three features is used to determine three-dimensional (3D) position coordinates of said image capturing device by using a triangulation method.

20. The method of claim 17 wherein said platform is selected from the group consisting of:

a GNSS rover;

a GNSS rover RTK system; and

a GIS/Mapping handheld.

21. The method of claim 17 further comprising:

a second image capturing device on said platform;

capturing at least one image at the first position with said second image capturing device;

wherein said first image includes at least one second object;

capturing at least one image at a second position with said second image capturing device;

wherein said second image includes said at least one second object;

processing said plurality of images to track a location of said at least one second object;

wherein a set of two-dimensional position fixes for said at least one second object is obtained from processing said plurality of images, providing a set of tracking data for said at least one second object;

determining the location of said second image capturing device by using said set of tracking data of said at least one second object;

and

calculating a new location of said platform by combining the location of said second image capturing device with the location of said image capturing device and the location determined by said GNSS receiver.

22. The method of claim 21 further comprising:

determining weighted feature based coordinates of said location of said platform.

23. The method of claim 22, further comprising:

assigning a first weight to a set of tracking data of said at least one first object;

assigning a second weight to a set of tracking data of said at least one second object;

and

determining feature based coordinates of said location of said platform by using said set of weighted tracking data of said at least one first object and said set of weighted tracking data of said at least one second object.

24. The method of claim 21 further comprising:

using a Kalman filter to obtain a Kalman estimate of said second position of said platform by combining said set of tracking data of said at least one first object as a first measurement and said set of tracking data of said at least one second object as a second measurement.

25. An image based positioning apparatus comprising:

an image capturing device configured to capture a first image including at least one feature and a surface;

said image capturing device configured to capture a second image including said at least one feature and said surface;

a range measuring device configured to obtain a first set of depth data of said surface in said first image;

said range measuring device configured to obtain a second set of depth data of said surface in said second image;

and

a processor configured to process each of said images;

wherein said processor is configured to determine the location of said platform by using a combined feature based process and surface tracking process, wherein said feature based process is different from said surface tracking process and includes matching optical properties of pixels, and said surface tracking process includes matching said first set of depth data and said second set of depth data.

29. The method of claim 25, wherein said range measuring device is selected from the group consisting of:

a point laser beam;

a sonar;

a radar;

a laser scanner; and

a depth camera.

26. An image based positioning apparatus comprising:

at least two image capturing devices;

each said image capturing device mounted on a platform;

said first image capturing device configured to capture an image including at least one feature;

said first image capturing device configured to capture a second image including at least one feature;

said second image capturing device configured to capture an image including a surface;

said second image capturing device configured to capture a second image including said surface;

a range measuring device configured to obtain a set of depth data of said surface in said image;

said range measuring device configured to obtain a second set of depth data of said surface in said second image;

and

a processor configured to process each of said images;

wherein said processor is configured to determine the location of said platform by using a combined feature based process and surface tracking process, wherein said feature based process is different from said surface tracking process and includes matching optical properties of pixels, and said surface tracking process includes matching said first set of depth data and said second set of depth data.

30. The method of claim 26, wherein said range measuring device is selected from the group consisting of:

a point laser beam;

a sonar;

a radar;

a laser scanner; and

a depth camera.