SEQUENTIAL ROLLING BUNDLE ADJUSTMENT

• US 20150036888A1
• Filed: 07/31/2013
• Published: 02/05/2015
• Est. Priority Date: 07/31/2013
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Weisenburger, Shawn D.
• Granted Patent: US 9,177,384 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A method for estimating position and orientation of an image-capturing device is proposed. The method comprises the step of obtaining a preceding set of frames by using the image-capturing device. Each frame includes a set of image data. The method of the present technology further comprises the step of estimating a previous position and orientation of the image-capturing device by using the set of image data included in at least one preceding frame, and the step of estimating a current position and orientation of the image-capturing device by replacing a set of image data included in at least one preceding frame by a set of image data included in at least one subsequent frame. At least one subsequent frame is obtained by using the image-capturing device.

Claims

1. A method of estimating position and orientation of an image-capturing device comprising:

(A) obtaining a preceding set of frames by using said image-capturing device;

each said frame including a set of image data;

(B) estimating a previous position and orientation of said image-capturing device by using said set of image data included in at least one said preceding frame;

and

(C) estimating a current position and orientation of said image-capturing device by replacing a set of image data included in said at least one preceding frame by a set of image data included in at least one subsequent frame;

wherein said at least one subsequent frame is obtained by using said image-capturing device.

2. The method of claim 1, wherein said step (A) further comprises:

(A1) selecting said image capturing device from the group consisting of;

a digital camera;

a digital video camera;

a digital camcorder;

a stereo digital camera;

a stereo video camera;

a motion picture camera; and

a television camera.

3. The method of claim 1, wherein said step (A) further comprises:

(A2) calibrating said image-capturing device to correct the images for lens distortions;

said lens distortions selected from the group consisting of;

a principal distance distortion;

a focal length;

a principal point offset;

a radial distortion; and

a tangential distortion.

4. The method of claim 1, wherein said step (B) further comprises:

(B1) initializing said previous position and orientation of said image-capturing device by analyzing a first set of image data collected by using said image-capturing device;

said first set of image data included in said at least one preceding frame.

5. The method of claim 4, wherein said step (B1) further comprises:

(B1,

     1) detecting a set of at least five key-points in said first set of image data included in said at least one preceding frame;

detecting said set of at least five key-points in a second set of image data included in at least one said subsequent frame; and

based on said set of at least five key-points being detected in said at least one preceding frame and in said at least one subsequent frame substantially accurately determining a relative change between said at least one preceding frame and said at least one subsequent frame.

6. The method of claim 1, wherein said step (B) further comprises:

(B2) estimating initial position and orientation of said image-capturing device by using an additional navigation device.

7. The method of claim 6, wherein said step (B2) further comprises:

(B2,

     1) selecting said additional navigation device from the group consisting of;

a smart phone;

a tablet, a laptop, and a personal navigation device.

8. The method of claim 1, wherein said step (C) further comprises:

(C1) estimating said current position and orientation of said image-capturing device by replacing a set of image data related to at least five key-points included in said preceding frame by a set of image data related to at least five same key-points included in said subsequent frame.

9. The method of claim 8, wherein said step (C1) further comprises:

(C1,

     1) performing said estimation process by using the method of Kalman filtering.

10. The method of claim 1 further comprising

(D) selecting a relevant coordinate system.

11. The method of claim 10, wherein said step (D) further comprises:

(D1) selecting said relevant coordinate system from the group consisting of;

a Global Coordinate System; and

a Local Coordinate System.

12. An apparatus for estimating position and orientation of an image-capturing device comprising:

(A) a means for obtaining a preceding set of frames by using said image-capturing device;

each said frame including a set of image data;

(B) a means for estimating a previous position and orientation of said image-capturing device by using said set of image data included in at least one said preceding frame;

and

(C) a means for estimating a current position and orientation of said image-capturing device by replacing a set of image data included in said at least one preceding frame by a set of image data included in at least one subsequent frame;

wherein said at least one subsequent frame is obtained by using said image-capturing device.

13. The apparatus of claim 12, wherein said means (A) further comprises:

(A1) an image capturing device selected from the group consisting of;

a digital camera;

a digital video camera;

a digital camcorder;

a stereo digital camera;

a stereo video camera;

a motion picture camera; and

a television camera.

14. The apparatus of claim 12, wherein said means (B) further comprises:

(B1) a processor configured to analyze a first set of image data obtained by using said image-capturing device;

said first set of image data included in said at least one preceding frame.

15. The apparatus of claim 12, wherein said means (B) further comprises:

(B2) a processor configured to select a set of at least five key-points from said first set of images included in said at least one preceding frame;

configured to detect said set of at least five selected key-points in a second set of images included in at least one said subsequent frame; and

based on said set of at least five key-points being selected in said in said at least one preceding frame and being detected in said at least one subsequent frame configured to substantially accurately determine a relative change between said at least one preceding frame and said at least one subsequent frame.

16. The apparatus of claim 12, wherein said means (C) further comprises:

(C1) an additional navigation device configured to estimate initial position and orientation of said image-capturing device.

17. The apparatus of claim 12, wherein said means (C) further comprises:

(C2) said additional navigation device selected from the group consisting of;

a smart phone;

a tablet, a laptop, and a personal navigation device.

18. The apparatus of claim 14, wherein said means (C) further comprises:

(C3) a processor configured to estimate said current position and orientation of said image-capturing device by replacing a set of image data related to at least five key-points detected in said preceding frame by a set of image data related to at least five same key-points detected in said subsequent frame.

19. The apparatus of claim 14, wherein said means (C) further comprises:

(C4) a Kalman processor configured to perform said estimation process by using the method of Kalman filtering.

20. An article comprising a non-transient storage medium having stored thereon instructions that, when executed, result in a computing platform having the capability to perform an estimation of position and orientation of an image-capturing device by executing an algorithm further comprising:

(A) obtaining a preceding set of frames by using said image-capturing device;

each said frame including a set of image data;

(B) estimating a previous position and orientation of said image-capturing device by using said set of image data included in at least one said preceding frame;

and

(C) estimating a current position and orientation of said image-capturing device by replacing a set of image data included in said at least one preceding frame by a set of image data included in at least one subsequent frame;

wherein said at least one subsequent frame is obtained by using said image-capturing device.