METHOD FOR FAST CAMERA POSE REFINEMENT FOR WIDE AREA MOTION IMAGERY

• US 20170186164A1
• Filed: 12/29/2015
• Published: 06/29/2017
• Est. Priority Date: 12/29/2015
• Status: Active Grant

• Current Assignee: Department of the Air Force
• Original Assignee: Government of The United States As Represented By The U.S. Air Force (Government of the United States of America)
• Inventors: SEETHARAMAN, GUNASEKARAN, PALANIAPPAN, KANNAPPAN, AKBARPOUR, HADI ALI
• Granted Patent: US 9,959,625 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

The present invention provides a method for fast, robust and efficient BA pipeline (SfM) for wide area motion imagery (WAMI). The invention can, without applying direct outliers filtering (e.g. RANSAC) or re-estimation of the camera parameters (e.g. essential matrix estimation) efficiently refine noisy camera parameters in very short amounts of time. The method is highly robust owing to its adaptivity with the persistency factor of each track. The present invention highly suitable for sequential aerial imagery, particularly for WAMI, where camera parameters are available from onboard sensors.

Claims

1. A method for camera pose refinement in three dimensional reconstruction of sequential frames of imagery of an image, comprising the steps of:

acquiring camera metadata, said metadata comprising camera position metadata and camera orientation metadata;

extracting interest points from each said image frame in said sequence;

comparing descriptors of said extracted interest points for each two successive image frames;

matching said descriptors so as to generate feature tracks;

generating a persistency factor for each said feature track as a function of said feature track'"'"'s length;

computing a set of statistics for all said persistency factors when all feature tracks have been generated;

computing a triangulation based on said camera metadata and said feature tracks so as to generate estimated initial 3D interest points of said image;

weighting residuals generated from back projection error using said persistency factors;

computing an error function incorporating said weighted residuals, so as to reduce the effect of outlier noise; and

computing a bundle adjustment incorporating said weighted residuals and said camera metadata, so as to optimize said camera pose refinement and said estimated initial 3D interest points.

2. The method of claim 1, wherein said set of statistics further comprises mean and standard deviation.

3. The method of claim 2, wherein said mean is computed according to:

4. The method of claim 3, wherein said error function is computed according to

5. The method of claim 4, wherein said bundle adjustment is computed according to:

6. The method of claim 5 wherein weighting residuals is computed according to:

7. An article of manufacture comprising a non-transitory storage medium and a plurality of programming instructions stored therein, wherein said programming instructions are configured to implement upon a camera system for three dimensional image reconstruction, the following actions:

the acquisition of camera metadata, said metadata comprising camera position metadata and camera orientation metadata;

the extraction of interest points from each said image frame in said sequence;

the comparison of descriptors of said extracted interest points for each two successive image frames;

the matching of said descriptors so as to generate feature tracks;

the generation of a persistency factor for each said feature track as a function of said feature track'"'"'s length;

the computation of a set of statistics for all said persistency factors when all feature tracks have been generated;

the computation of a triangulation based on said camera metadata and said feature tracks so as to generate estimated initial 3D interest points of said image;

the weighting of residuals generated from back projection error using said persistency factors;

the computation of an error function incorporating said weighted residuals, so as to reduce the effect of outlier noise; and

the computation of a bundle adjustment incorporating said weighted residuals and said camera metadata, so as to optimize said camera pose refinement and said estimated initial 3D interest points.

8. The article of manufacture of claim 7, wherein said set of statistics further comprises mean and standard deviation.

9. The article of manufacture of claim 8, wherein said mean is computed according to:

10. The article of manufacture of claim 9, wherein said error function is computed according to

11. The article of manufacture of claim 10, wherein said bundle adjustment is computed according to:

12. The article of manufacture of claim 11 wherein weighting residuals is computed according to:

13. A system for vehicle-based three-dimensional imagery reconstruction, comprising:

an orientable camera system comprising at least one camera for producing sequential frames of imagery of an image;

at least one vehicle system having geolocation subsystems and at least one said orientable camera system part thereof; and

a computer system being specifically configured by computer programming instructions so as to

communicate with said geolocation subsystems and said camera systems;

extract geolocation data from said geolocation subsystem orientation and position data from said camera system; and

compute a refined camera pose based on said geolocation, orientation, and position data so as to produce said three-dimensional imagery reconstruction.

14. The system of claim 13, wherein said computer programming instructions further configure said computer system to

acquire camera metadata, said metadata comprising camera position metadata and camera orientation metadata;

extract interest points from each said image frame in said sequence;

compare descriptors of said extracted interest points for each two successive image frames;

match said descriptors so as to generate feature tracks;

generate a persistency factor for each said feature track as a function of said feature track'"'"'s length;

compute a set of statistics for all said persistency factors when all feature tracks have been generated;

compute a triangulation based on said camera metadata and said feature tracks so as to generate estimated initial 3D interest points of said image;

weight residuals generated from back projection error using said persistency factors;

compute an error function incorporating said weighted residuals, so as to reduce the effect of outlier noise; and

compute a bundle adjustment incorporating said weighted residuals and said camera metadata, so as to optimize said camera pose refinement and said estimated initial 3D interest points.