Photogrammetry system and method for determining relative motion between two bodies

• US 8,655,094 B2
• Filed: 05/11/2011
• Issued: 02/18/2014
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: The United States of America As Represented By The Secretary of Agriculture
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Severance, Kurt, Miller, Samuel A
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US13/105,004
• Publication Number:
  US 20120285024A1
• Time in Patent Office: 1,014 Days
• Field of Search: 382/152, 382/153, 382/154, 382/254, 348/46, 348/135, 345/419, 335/03
• US Class Current: 382/254
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A photogrammetry system and method provide for determining the relative position between two objects. The system utilizes one or more imaging devices, such as high speed cameras, that are mounted on a first body, and three or more photogrammetry targets of a known location on a second body. The system and method can be utilized with cameras having fish-eye, hyperbolic, omnidirectional, or other lenses. The system and method do not require overlapping fields-of-view if two or more cameras are utilized. The system and method derive relative orientation by equally weighting information from an arbitrary number of heterogeneous cameras, all with non-overlapping fields-of-view. Furthermore, the system can make the measurements with arbitrary wide-angle lenses on the cameras.

Claims

1. A method of determining the relative position of first and second bodies, the method comprising:

providing the first body with at least first and second imaging devices at known imaging device locations relative to the first body in a coordinate system defined by the first body;

providing the second body with at least three non-collinear targets;

providing a known first target location relative to the first body for each of the at least three targets at a first time, wherein the known first target locations are defined in the coordinate system;

utilizing the imaging devices to generate a plurality of unit vectors corresponding to a line-of-sight from the imaging devices to each of the at least three targets at a series of later times that are after the first time, wherein the imaging devices have non-overlapping fields-of-view in at least one of the later times whereby the lines-of-sight of the first imaging device are not to any targets for which the second imaging device has lines-of-sight to;

generating a plurality of equations in matrix form for each of the later times, whereby, for each of the later times, a matrix of known quantities is multiplied by a vector of unknown positions that are a function of time, and subtracting a matrix including the known imaging device locations from the product of the matrix of known quantities and the vector of unknown positions, and setting the result equal to a residual error vector, wherein the matrix of known quantities includes the known first target locations and the unit vectors, and wherein the vector of unknown positions includes three unknown Euler angles, three translations of the targets on the first body relative to the second body, and distances from the imaging devices to the targets;

determining a criteria defining an acceptable residual error vector; and

iterating the equations to converge upon a best solution whereby the residual error vector is minimized to within the criteria.

2. The method of claim 1, wherein:

the equations are iterated utilizing a mathematical algorithm that solves non-linear equations.

3. The method of claim 1, wherein:

the first and second imaging devices comprise cameras that include fish-eye lenses defining centers, the fish-eye lenses having three-dimensional projection patterns with significant curvature about the centers of the lenses.

4. The method of claim 3, wherein:

the cameras define a plurality of pixel points in an image plane, and the cameras are calibrated to determine a mathematical relationship between a vector emanating from the center of the lenses and at least a selected one of the pixel points.

5. The method of claim 4, wherein:

generating a plurality of dots in an image utilizing the cameras; and

wherein;

the locations and orientations of the cameras in a coordinate system for the first body is determined by space resection including converting coordinates of at least one of the dots into unit vectors emanating from the centers of the cameras.

6. The method of claim 5, including:

providing a plurality of sectioning targets having known locations; and

wherein;

the space resectioning includes a series of at least three iterations, and wherein;

the first iteration includes pairing several dots with several corresponding sectioning targets;

the second iteration includes utilizing a programmable digital device having an automatic pairing function that selects a plurality of dot-sectioning target pairs, followed by recomputation of the resection, and wherein the second iteration generates a residual for each of the selected dot-sectioning target pairs; and

the third iteration removes the worst of the dot-sectioning target pairs based upon predefined criteria.

7. The method of claim 6, wherein:

the predefined criteria comprises the mean value of the residuals generated by the second iteration, whereby each dot-sectioning target pair has a residual value that is greater than the mean value of the residuals generated by the second iteration.

8. The method of claim 1, wherein:

the first and second bodies are substantially rigid.

9. The method of claim 1, wherein:

the imaging device comprises a camera that generates a plurality of two-dimensional pixel points in an image plane; and

including;

calibrating the camera to determine a mathematical relationship between a selected two-dimensional pixel point in the image plane and a three-dimensional vector emanating from a center of the camera.

10. The method of claim 1, wherein:

the imaging devices comprise first and second cameras having heterogeneous lenses.

11. A method of determining the relative position of first and second rigid bodies, the method comprising:

mounting at least two cameras on a first rigid body defining a coordinate system, wherein at least one of the cameras has a wide-angle lens, the wide-angle lens defining a center and a field-of-view, and wherein the wide-angle lens has a three-dimensional projection pattern with significant curvature about the center of the wide-angle lens, and wherein the fields-of-view of the at least two cameras do not overlap;

providing a second rigid body with at least three targets that can be utilized by the cameras to form discrete images on image planes of the cameras;

calibrating the cameras;

determining the locations and orientations of the cameras in the coordinate system of the first rigid body;

generating a target database comprising first positions of the at least three targets relative to the first rigid body at a first time;

utilizing the cameras to form a plurality of images corresponding to the targets in a series of frames that are generated at a plurality of later time increments that are after the first time;

determining the locations of the targets in the images to generate known lines-of-sight from the cameras to the targets for the later time increments;

tracking the images from frame to frame;

associating images with the first positions of the targets utilizing the target database;

generating a plurality of equations that can be expressed in matrix form as Ax(t)−

b=e, wherein A comprises the known lines-of-sight and the first positions of the targets, t comprises a vector of unknown positions of the targets relative to the first positions of the targets at the later time increments, the vector including three unknown Euler angles corresponding to rotation of the second rigid body relative to the first rigid body, three unknown translations of the second rigid body relative to the first rigid body, and unknown distances of the targets relative to the centers of the lenses, x(t) comprises a function of a shorter vector t of the unknown positions, wherein x( ) returns a longer vector x, b comprises the positions of the cameras in the coordinate system of the first rigid body, and e comprises a residual error vector; and

iterating the plurality of equations over t to converge upon a best solution whereby the residual error vector e is minimized.

12. The method of claim 11, wherein:

the cameras generate two-dimensional pixel points in image planes of the cameras; and

wherein;

calibrating the cameras includes determining calibration parameters comprising a mathematical relationship between at least one of the two-dimensional pixel points and a three-dimensional vector emanating from the centers of the lenses of the cameras.

13. The method of claim 12, wherein:

calibrating the cameras includes utilizing a fourth-order polynomial to describe the three-dimensional calibration vector emanating from the centers of the lenses of the cameras.

14. The method of claim 12, wherein:

the targets define centroids; and

determining the camera locations and orientations in the coordinate system of the first rigid body includes utilizing the cameras to faun a plurality of images corresponding to the targets determining the centroids of the targets when the targets are at the first positions.

15. The method of claim 14, wherein:

the calibration parameters are utilized to convert the centroids of the targets into three-dimensional vectors emanating from the centers of the lenses of the cameras.

16. The method of claim 15, including:

determining the rotations and translations of the cameras required to transform the unit vectors such that the unit vectors point to the corresponding three-dimensional targets.

17. The method of claim 16, wherein:

determining the rotations and translations of the cameras is accomplished utilizing a non-linear matrix equation of the form P=C+DVR that is iteratively solved to determine a solution comprising three rotation angles for each camera, three camera position coordinates for each camera, and the distances from the centers of the lenses to the three-dimensional targets, wherein P comprises the positions of the targets in the coordinate system of the first body, C comprises the unknown positions of the cameras, D comprises the unknown distances from the centers of the lenses to the three-dimensional targets, V comprises the calibration vectors, and R comprises the unknown camera rotations.

18. The method of claim 17, wherein:

the at least three targets comprise substantially more than three targets;

determining the camera locations and orientations in the coordinate system of the first rigid body includes;

utilizing the cameras to generate a plurality of images corresponding to the targets when the targets are in the first positions;

manually pairing at least three images with at least three targets;

iteratively solving the equation P=C+DVR to determine a rough camera position;

utilizing an auto pair function to pair substantially more than three images with corresponding targets to form a second group of image-target pairs;

utilizing the equation P=C+DVR and the second group of image-target pairs to determine a refined camera position and second residuals corresponding to each of the image-target pairs;

deleting image-target pairs having residuals above a predefined threshold from the second group; and

solving the equation P=C+DVR utilizing the image-target pairs that remain in the second group to determine a final camera position.

19. The method of claim 18, wherein:

the predefined threshold comprises the mean value of the second residuals.

20. The method of claim 11, wherein:

the plurality of images formed by the cameras are in a digital format, and wherein the frames include a plurality of objects in addition to the images corresponding to the targets, and including;

programming a digital device to filter out and remove objects that are below a predefined size and to remove objects having holes.