SYSTEM AND METHOD FOR IMAGE CAPTURE DEVICE POSE ESTIMATION

• US 20170249751A1
• Filed: 02/23/2017
• Published: 08/31/2017
• Est. Priority Date: 02/25/2016
• Status: Active Grant

• Current Assignee: TECHNION RESEARCH AND DEVELOPMENT FOUNDATION, LTD. (Technion-Israel Institute Of Technology)
• Original Assignee: TECHNION RESEARCH AND DEVELOPMENT FOUNDATION, LTD. (Technion-Israel Institute Of Technology)
• Inventors: INDELMAN, Vadim
• Granted Patent: US 10,546,385 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures

Abstract

A method for estimating a plurality of camera, comprising using at least one processor executing a code for: extracting a plurality of image features of a plurality of landmarks from a plurality of images captured by at least one camera from at least one pose, the plurality of landmarks calibrated with respect to a certain coordinate system; identifying among the plurality of image features at least one image feature documented in at least some of the images; producing scale values of at least one common image feature by analyzing the at least some of the images; determining a plurality of estimated poses of the at least one camera with respect to the certain coordinate system by using the scale values in calculating a minimal re-projection error between the plurality of image features and a plurality of predicted image features; and outputting the plurality of estimated poses.

Claims

1. A method for estimating a plurality of image capture device poses, comprising:

using at least one hardware processor executing a code for;

extracting a plurality of observed image features of a plurality of landmarks from a plurality of images captured by at least one image capture device from at least one pose, said plurality of landmarks calibrated with respect to a certain coordinate system;

identifying among said plurality of observed image features at least one observed image feature documented in at least some of said images;

producing image feature scale values of said at least one common observed image feature by analyzing said at least some of said images;

determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system by using said image feature scale values in calculating a minimal re-projection error between said plurality of observed image features and a plurality of predicted image features; and

outputting said plurality of estimated poses.

2. The method of claim 1, wherein said determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system comprises using a scale re-projection error term in a bundle adjustment optimization calculation;

wherein said scale re-projection error term is computed using said image feature scale values.

3. The method of claim 2, wherein said bundle adjustment optimization calculation comprises at least one cost function;

wherein said at least one cost function comprises at least one scale error term expressing a scale re-projection error using said image feature scale information; and

wherein said bundle adjustment optimization calculation comprises;

producing a plurality of estimated image capture device pose sets, each of said plurality of sets comprising a plurality of estimated image capture device poses, each of said plurality of estimated image capture device poses associated with one of said plurality of images;

producing a plurality of estimated costs by applying said at least one cost function to each of said plurality of estimated capture device pose sets;

identifying a minimum estimated cost of said plurality of estimated costs; and

selecting the estimated capture device pose set used to produce said minimum estimated cost.

4. The method of claim 3, wherein said producing a plurality of estimated image capture device pose sets comprises applying sparsity-aware iterative optimization techniques;

wherein said sparsity-aware iterative optimization techniques are members of a group comprising;

g2o, Georgia Tech smoothing and mapping (gtsam) and Ceres Solver.

5. The method of claim 4, wherein said at least one scale error term is computed;

wherein computing said at least one scale error term comprises computing a difference between a measured image feature scale and a predicted feature scale;

wherein said measured image feature scale is calculated by a feature detector; and

wherein said predicted feature scale is computed using an image capture device focal length, a distance from said common image feature'"'"'s landmark to one of said plurality of estimated camera poses and a size estimate of said common image feature'"'"'s landmark.

6. The method of claim 5,

wherein computing said predicted feature scale comprises:

dividing said size estimate by said distance to produce a ratio; and

multiplying said ratio by said focal length; and

wherein said computing a difference between a measured image feature scale and a predicted feature scale comprises subtracting said predicted feature scale from said image feature scale.

7. The method of claim 4, wherein said feature detector is Scale Invariant Feature Transform (SIFT).

8. The method of claim 4, wherein computing said difference comprises calculating a Squared Mahalanobis Distance between said measured image feature scale and said predicted feature scale, having a measurement covariance matrix of a non zero-mean Gaussian distribution of noise elements in said image feature scale values.

9. The method of claim 8, further comprising producing said measurement covariance matrix by computing a covariance matrix of a zero-mean Gaussian distribution of said noise elements in said image feature scale values using methods as known to the art.

10. The method of claim 1, wherein said identifying among said plurality of observed image features at least one observed image feature is by applying image matching algorithms.

11. The method of claim 10, wherein said image matching algorithms include Scale Invariant Feature Transform (SIFT) and Random Sample Consensus (RANSAC).

12. The method of claim 1, wherein said image feature scale values are produced while said identifying among said plurality of observed image features at least one observed image feature.

13. The method of claim 1, wherein said certain coordinate system is a world coordinate system.

14. The method of claim 1, wherein the origin of said certain coordinate system is calibrated to a pose of one of said at least one image capture devices when capturing a first image of said plurality of images.

15. A system for estimating a plurality of image capture device poses, comprising:

at least one hardware processor executing a code for;

extracting a plurality of observed image features of a plurality of landmarks from a plurality of images captured by at least one image capture device from at least one pose, said plurality of landmarks calibrated with respect to a certain coordinate system;

identifying among said plurality of observed image features at least one observed image feature documented in at least some of said images;

producing image feature scale values of said at least one common observed image feature by analyzing said at least some of said images;

determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system by using said image feature scale values in calculating a minimal re-projection error between said plurality of observed image features and a plurality of predicted image features; and

outputting said plurality of estimated poses.

16. The system of claim 15, further comprising at least one digital storage hardware, electrically connected to said at least one hardware processor.

17. The system of claim 15, further comprising at least one non-volatile data storage;

wherein said at least one hardware processor is connected to said at least one non-volatile data storage via a network.

18. The system of claim 17, wherein said non-volatile data storage is a cloud database.

19. The system of claim 15, wherein said image capture device is installed on a moving arm.

20. The system of claim 15, wherein said image capture device is installed on a vehicle.

21. The system of claim 20, wherein said vehicle is a member of a group including:

an airplane, a drone, a helicopter, a floating vehicle, a motorized vehicle, a car, a bicycle, a motorcycle, an electric vehicle, a balloon, a blimp and a cart.