Image-based surface tracking

• US 9,710,919 B2
• Filed: 11/25/2015
• Issued: 07/18/2017
• Est. Priority Date: 07/07/2009
• Status: Active Grant

• Current Assignee: Trimble Inc.
• Original Assignee: Trimble Inc.
• Inventors: Teng, Hongbo, Best, Gregory C., Wang, Sy Bor
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/952,572
• Publication Number:
  US 20160078636A1
• Time in Patent Office: 601 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method of image-tracking by using an image capturing device (12). The method comprises: performing an image-capture of a scene (54) by using an image capturing device; and tracking movement (62) of the image capturing device (12) by analyzing a set of images by using an image processing algorithm (64).

Claims

1. A method of image-based positioning comprising:

(A) performing image-recording of a scene by using an image capturing device;

(B) performing an initial image analysis to determine suitability of recorded images;

(C) obtaining a set of depth data of said scene by pre-surveying said scene;

(D) using scene understanding techniques to discard unsuitable regions of images of broadly defined features for tracking, wherein said broadly defined features are selected from the group consisting of;

points;

representations of regions; and

representations of contours;

performing object recognition;

inferring an object category from at least one cluster of neighboring features;

determining the suitability of the inferred object category for surface tracking; and

(E) tracking movement of said image capturing device by analyzing a set of suitably recorded images.

2. The method of claim 1, wherein said step (B) further comprises:

(B1) performing an initial image assessment analysis to find out whether a set of recorded images is suitable for surface tracking.

3. The method of claim 2, wherein said step (B1) further comprises:

(B1,1) extracting of at least three attributes from at least one recorded image.

4. The method of claim 2, wherein said step (B1) further comprises:

(B1,2) checking saturation quality of at least one recorded image to check if an image consists substantially of a single Red-Green-Blue value.

5. The method of claim 2, wherein said step (B1) further comprises:

(B1,3) checking the texture quality of at least one recorded image.

6. The method of claim 2, wherein said step (B1) further comprises:

(B1,4) checking an image contrast of at least one recorded image.

7. The method of claim 1, wherein said step (D) further comprises:

performing extraction of sparse features from a recorded image.

8. The method of claim 1, wherein said step (D) further comprises:

performing clustering neighboring features together.

9. A method of image-based positioning comprising:

(A) performing image-recording of a scene by using an image capturing device;

(B) performing an initial image analysis to determine suitability of recorded images

(C) obtaining a set of depth data of said scene by pre-surveying said scene;

(D) using scene understanding techniques to discard unsuitable regions of images of broadly defined features for tracking, wherein said broadly defined features are selected from the group consisting of;

points;

representations of regions; and

representations of contours; and

(E) tracking movement of said image capturing device by analyzing a set of suitably recorded images, wherein said step (E) further comprises;

performing a rigid global transformation of said set of captured suitable image data and said set of scene depth data into a set of 6-coordinate data;

wherein said set of 6-coordinate data represents movement of said image capturing device; and

wherein said set of 6-coordinate data represents a surface tracking solution.

10. The method of claim 9, wherein said step (E) further comprises:

using a set of broadly defined-features to initialize said surface tracking solution.

11. The method of claim 10, wherein said step (E) further comprises:

detecting a set of broadly defined features;

establishing correspondences between said set of broadly defined features and at least two selected frames;

estimating homography between said at least selected two frames using parameters of a RANdom SAmple Consensus (RANSAC) mathematical model;

deriving an initial pose of said image capturing device from said homography between said at least selected two frames;

wherein said pose of said image-capturing device comprises position coordinates of said image-capturing device and a set of angular coordinates of said image-capturing device; and

using said derived initial pose of said image capturing device as an initial solution to said surface tracking solution.

12. The method of claim 9, wherein said step (E) further comprises:

using a set of broadly defined-features on at least one tracking surface for a strict two-dimensional (strict_2D) surface tracking solution.

13. The method of claim 12, wherein said step (E) further comprises:

detecting a set of broadly defined features on a single tracking surface;

selecting a set of coplanar broadly defined features by using parameters of said RANSAC mathematical model;

establishing correspondences between said set of coplanar broadly defined features and at least two selected frames;

deriving an initial pose of said image capturing device from said homography between said at least selected two frames;

using said derived initial pose of said image capturing device as an initial solution to said strict two-dimensional (strict_2D) surface tracking solution; and

grouping said set of coplanar features and using an area around said group of coplanar features as an input to said strict_2D surface tracking solution.

14. The method of claim 9, wherein said step further comprises:

using a set of coplanar broadly defined-features on a plurality of two-dimensional (2D) tracking surfaces for a substantially two-dimensional (sub_2D) surface tracking.

15. The method of claim 14, wherein said step further comprises:

detecting a set of broadly defined features on said plurality of tracking surfaces;

selecting a set of coplanar broadly defined features by using parameters of said RANSAC mathematical model;

establishing correspondences between said set of coplanar broadly defined features and at least two selected frames;

deriving an initial pose of said image capturing device from said homography between said at least selected two frames;

using said derived initial pose of said image capturing device as an initial solution to a substantially two-dimensional (sub_2D) surface tracking solution; and

selecting a local area around each said selected coplanar broadly defined feature, grouping a plurality of said selected local areas into a two-dimensional (2D) global area and using said 2D global area as an input to said sub_2D surface tracking solution.

16. The method of claim 9, wherein said step further comprises:

using a set of broadly defined-features extracted from a three dimensional (3D) for a substantially three-dimensional (sub_3D) surface tracking.

17. The method of claim 16, wherein said step further comprises:

detecting a set of broadly defined features on said 3D surface;

establishing correspondences between said set of broadly defined features and at least two selected frames;

estimating homography between said at least two selected frames;

deriving an initial pose of said image capturing device from said homography between said at least selected two frames;

using said derived initial pose of said image capturing device as an initial solution to said substantially three-dimensional (sub_3D) surface tracking solution; and

selecting a local area around each said selected broadly defined feature, grouping a plurality of said selected local areas into a three-dimensional (3D) global area and using said 3D global area as an input to said sub 3D surface tracking solution.