Image-based surface tracking

• US 9,224,208 B2
• Filed: 07/06/2010
• Issued: 12/29/2015
• Est. Priority Date: 07/07/2009
• Status: Active Grant

• Current Assignee: Trimble Inc.
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Teng, Hongbo, Best, Gregory C., Wang, Sy Bor
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US13/382,913
• Publication Number:
  US 20120195466A1
• Time in Patent Office: 2,002 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method of image-tracking by using an image capturing device (12). The method comprises: performing an image-capture of a scene (54) by using an image capturing device; and tracking movement (62) of the image capturing device (12) by analyzing a set of images by using an image processing algorithm (64).

Claims

1. A method of image-based positioning comprising:

(A) performing an image-capture of a scene by using an image capturing device;

wherein a set of captured images data and a set of scene depth data is created;

(B) performing a rigid global transformation of said set of captured images data and said set of scene depth data into a set of 6-coordinate data;

wherein said set of 6-coordinate data represents movement of said image capturing device;

and

(C) determining the position coordinates of said image capturing device by using an image processing algorithm;

said image processing algorithm including a global optical flow;

said image processing algorithm including at least one step of parametrizing said global optical flow with said image capturing device.

2. The method of claim 1, wherein said step (A) further comprises:

(A1) performing said image-capture of said scene in real time by using said image capturing device;

wherein said image capturing device is selected from the group consisting of;

{a digital camera;

a digital video camera;

a digital camcorder;

a stereo digital camera;

a stereo video camera;

a motion picture camera; and

a television camera}.

3. The method of claim 1, wherein said step (A) further comprises:

(A2) performing image-recording of said scene by using said image capturing device.

4. The method of claim 1, wherein said step (A) further comprises:

(A3) obtaining a set of depth data of said scene by pre-surveying said scene.

5. The method of claim 4, wherein said step (A3) further comprises:

(A3,

     1) obtaining said set of depth data of said scene by using a surveying instrument selected from the group consisting of;

{a Global Navigation Satellite System (GNSS) surveying system;

a laser plane system; and

a theodolite}.

6. The method of claim 1, wherein said step (B) of performing said rigid global transformation of said set of captured images data and said set of scene depth data into said set of 6-coordinate data further comprises:

(B1) using a mapping function of said image capturing device to map for at least one i-th frame 3D coordinates of at least one object point in the reference system attached to said image capturing device, into 2D pixel coordinates of an image point on said image capturing device sensor plane in a 2D reference coordinate system;

i being an integer.

7. The method of claim 6, wherein said step (B1) further comprises:

(B1,

     1) selecting said mapping function of said image capturing device from the group consisting of;

{a mapping function of regular rectilinear lenses;

a mapping function of fisheye lenses; and

a mapping function of custom-calibrated lenses}.

8. The method of claim 6, wherein said step (B1) further comprises:

(B1,

     2) obtaining a reverse of said mapping function;

wherein said reverse of said mapping function is configured to map, for at least one i-th frame, 2D pixel coordinates of at least one image point on said image capturing device sensor plane in a 2D reference coordinate system, into 3D coordinates of at least one object point in the reference system attached to said image capturing device.

9. The method of claim 6, wherein said step (B1) further comprises:

(B1,

     3) extracting a 6-vector describing said movement of said image capturing device from a relationship between two sequential frames;

i-th frame and k-th frame, wherein at least one object point produces at least two pixels of substantially the same intensity in said two sequential frames.

10. The method of claim 9, wherein said step (B1, 3) further comprises:

(B1, 3,

     1) optimizing said 6-vector describing the movement of said image capturing device.

11. The method of claim 9, wherein said step (B1, 3) further comprises:

(B1, 3,

     2) optimizing said 6-vector describing the movement of said image capturing device by solving a nonlinear least-squares problem for said relationship between said two sequential frames.

12. The method of claim 9, wherein said step (B1, 3) further comprises:

(B1, 3,

     3) optimizing a (6-n)-vector describing an n-dimensionally restricted movement of said image capturing device by optimizing said relationship between said two sequential frames;

wherein integer n is less than 6.

13. The method of claim 6, wherein said step (B1) further comprises:

(B1,

     4) extracting a 6-vector describing said movement of said image capturing device from a relationship between said two sequential frames;

i-th frame and k-th frame by matching the depth data for at least two pixels;

a first said pixel and a second said pixel;

wherein said first pixel is located in said i-th frame, and wherein said second pixel is located in said k-th frame.

14. The method of claim 6, wherein said step (B1) further comprises:

(B1,

     5) extracting a 6-vector describing said movement of said image capturing device from a relationship between two sequential frames;

i-th frame and k-th frame by matching the depth data for at least two pixels;

a first pixel and a second pixel;

wherein said first pixel is located in said i-th frame, and wherein said second pixel is located in said k-th frame, and by matching at least two pixels having substantially the same intensity in said two sequential frames.