Method and system for calibrating digital cameras
First Claim
1. A computer-implemented calibration process for a digital camera, comprising the following steps:
- acquiring an object having a planar surface and a pattern disposed on said planar surface;
establishing surface coordinates of at least four feature points of the pattern on the planar surface;
capturing at least two images of the planar pattern with the camera being calibrated, each of said images depicting the pattern from a different, non-parallel orientation; and
using a computer to, ascertain the image coordinates of every image point in each captured image that depicts one of said feature points of the pattern, and estimating unknown intrinsic parameters of the digital camera using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in each of the captured images.
2 Assignments
0 Petitions
Accused Products
Abstract
A digital camera is calibrated by establishing the coordinates of at least four feature points of a pattern mounted on a planar surface. At least two, and preferably three or more, images of the planar pattern are captured at different, non-parallel orientations using the digital camera. The image coordinates of the pattern'"'"'s feature points are then identified in the captured images. A closed form solution can be employed to derive all the intrinsic and extrinsic parameters needed to provide the camera calibration. Essentially, the known pattern coordinates and corresponding image coordinates are used to compute a homography for each image. Then, a process is employed that estimates the intrinsic camera parameters by analyzing the homographies associated with each image. Finally, the extrinsic parameters for each image are computed from the intrinsic parameters and the homographies. However, the images can be effected by various noise sources which could affect the accuracy of the closed form solution process. If higher accuracy is called for, a maximum likelihood inference process can be employed to either provide a more accurate first estimate, or to refine the estimates derived from the closed form solution. If radial distortion caused by the lens of the camera is also a concern, the camera parameters can be further refined by taking into account this distortion.
158 Citations
42 Claims
-
1. A computer-implemented calibration process for a digital camera, comprising the following steps:
-
acquiring an object having a planar surface and a pattern disposed on said planar surface;
establishing surface coordinates of at least four feature points of the pattern on the planar surface;
capturing at least two images of the planar pattern with the camera being calibrated, each of said images depicting the pattern from a different, non-parallel orientation; and
using a computer to, ascertain the image coordinates of every image point in each captured image that depicts one of said feature points of the pattern, and estimating unknown intrinsic parameters of the digital camera using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in each of the captured images. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
(a) computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
(b) estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
(c) estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
5. The process of claim 4, further comprising the step of refining the estimates of the intrinsic parameters and extrinsic parameters by compensating for radial distortion in the captured images caused by the lens of the camera, said refining step comprising the steps of:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing steps (a) through (c), using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameters and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing steps (a) through (c), using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating steps (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
6. The process of claim 3, wherein the steps of estimating the intrinsic parameters and estimating the extrinsic parameters comprises the steps of:
-
(a) generating a series of candidate intrinsic parameter sets for the camera;
(b) for each candidate intrinsic parameter set, computing an estimated extrinsic parameter set for each of the captured images using the intrinsic parameter set candidate, the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured images, projecting each feature point of the pattern into the captured images using the candidate intrinsic parameter set and the last computed extrinsic parameter set associated with the image into which the feature point is being projected, computing an indicator of the difference between the image coordinates of each image point in each captured image and coordinates of a corresponding projected feature point, summing the difference indicators computed for each image point in an captured image to produce a combined difference indicator, and summing the combined difference indicators of each captured image to produce a maximum likelihood estimate indicator;
(c) identifying the smallest maximum likelihood estimate indicator; and
(d) designating the candidate intrinsic parameter set, and the associated extrinsic parameter set for each image, corresponding to the smallest maximum likelihood estimate indicator, as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
7. The process of claim 6, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand, said deriving comprising the steps of:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
8. The process of claim 6, wherein the step of computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point.
-
9. The process of claim 6, further comprising the step of refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iterative procedure comprising the steps of:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration computing a new estimate of the intrinsic parameter set and extrinsic parameter sets using the estimates thereof computed in the last iteration as initializing estimates in the current iteration; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
-
10. The process of claim 6, further comprising the step of refining the estimates of the intrinsic parameter set and extrinsic parameter sets by compensating for radial distortion in the captured images caused by the lens of the camera, said refining step comprising the steps of:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing steps (a) through (d), using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing steps (a) through (d), using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating steps (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
11. The process of claim 3, wherein the steps of estimating the intrinsic parameters and estimating the extrinsic parameters comprises the steps of:
-
generating a series of candidate intrinsic parameter sets for the camera;
for each candidate intrinsic parameter set, computing an estimated extrinsic parameter set for each of the captured images using the intrinsic parameter set candidate, the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured images, computing estimates of coefficients of radial distortion attributable to the lens of the camera using the candidate intrinsic parameter set and the coordinates of the image points in each of the captured images, projecting each feature point of the pattern into the captured images using the candidate intrinsic parameter set and the last computed extrinsic parameter set associated with the image into which the feature point is being projected, re-computing the image coordinates of each projected feature point to account for radial distortion using the computed estimates of the coefficients of radial distortion and the intrinsic parameter set candidate, computing an indicator of the difference between the image coordinates of each image point in each captured image and coordinates of a corresponding radially distorted projected feature point, summing the difference indicators computed for each image point in an captured image to produce a combined difference indicator, and summing the combined difference indicators of each captured image to produce a complete maximum likelihood estimate indicator;
identifying the smallest maximum likelihood estimate indicator; and
designating the candidate intrinsic parameter set, and the associated extrinsic parameter set for each image, corresponding to the smallest maximum likelihood estimate indicator, as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
12. The process of claim 11, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand, said deriving comprising the steps of:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
13. The process of claim 11, wherein the step of computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point.
-
14. The process of claim 11, further comprising the step of refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iterative procedure comprising the steps of:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration, computing a new estimate of the intrinsic parameter set and extrinsic parameter sets using the estimates thereof computed in the last iteration as initializing estimates in the current iteration; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
-
15. A system for calibrating a digital camera, comprising:
-
a planar surface having a pattern disposed thereon, wherein surface coordinates of at least four feature points of the pattern oh the planar surface have been identified;
multiple images of the planar pattern captured using the camera to be calibrated, wherein each of said images depicts the pattern from a different, non-parallel orientation;
a general purpose computing device; and
a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, ascertain the image coordinates of every image point in each captured image depicting one of the feature points of the pattern, and estimate intrinsic parameters of the digital camera using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in each of the captured images. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28)
(a) selecting a previously unselected one of the captured images;
(b) computing a homography between the pattern on the planar surface of the object and the selected image using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the selected image;
(c) repeating (a) and (b) for each of the captured images;
(d) estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
(e) estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associate ed with the particular image for which the extrinsic parameters are being estimated.
-
-
19. The system of claim 18, further comprising a program module for refining the estimates of the intrinsic parameters and extrinsic parameters by compensating for radial distortion in the captured images caused by the lens of the camera, said refining module comprising the sub-modules for:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing (a) through (e) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameters and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing (a) through (e) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
20. The system of claim 17, wherein the program modules for estimating the intrinsic parameters and estimating the extrinsic parameters comprises sub-modules for:
-
(a) selecting a previously unselected one of the captured images;
(b) choosing a previously unselected one of the image points in the selected image;
(c) projecting the feature point of the pattern corresponding to the selected image point into the selected image using an initial estimate of an intrinsic parameter set of the camera and an extrinsic parameter set associated with the selected image;
(d) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point;
(e) repeating (b) through (d) for each image point in the selected image;
(f) summing the difference indicators computed for each image point in the selected image to produce a combined difference indicator;
(g) repeating (a) through (f) for each of the captured images;
(h) summing the combined difference indicators to produce a maximum likelihood estimate indicator;
(i) generating a series of candidate intrinsic parameter sets for the camera, exclusive of the initial estimated intrinsic parameter set;
(j) selecting a previously unselected one of he candidate intrinsic parameter sets;
(k) computing an estimated extrinsic parameter set for each of the captured images using the selected intrinsic parameter set candidate;
(l) selecting one of the captured images;
(m) choosing one of the image points in the selected image;
(n) projecting the feature point of the pattern corresponding to the chosen image point into the selected image using an the selected intrinsic parameter set candidate and the last computed extrinsic parameter set associated with the selected image;
(o) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point;
(p) repeating (n) and (o) for each remaining image point in the selected image;
(q) summing the difference indicators computed for each image point in the selected image to produce another combined difference indicator;
(r) repeating (m) through (q) for each remaining captured image;
(s) summing the combined difference indicators computed in conjunction with the selected intrinsic parameter set candidate to produce another maximum likelihood estimate indicator;
(t) repeating 0) through (s) for each of the candidate intrinsic parameter sets;
(u) identifying the smallest maximum likelihood estimate indicator; and
(v) designating the candidate intrinsic parameter set, and the extrinsic parameter set for each image, associated with the smallest maximum likelihood estimate indicator as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
21. The system of claim 20, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand via a deriving program module comprising sub-modules for:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
22. The system of claim 20, wherein the sub-module for computing the indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point.
-
23. The system of claim 20, further comprising a program module for refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iteration program module comprising sub-modules for:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration, computing a new estimate of the intrinsic parameter set and extrinsic parameter sets using the estimates thereof computed in the last iteration as initializing estimates in the current iteration; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
-
24. The system of claim 20, further comprising a program module for refining the estimates of the intrinsic parameters and extrinsic parameters by compensating for radial distortion in the captured images caused by the lens of the camera, said refining module comprising the sub-modules for:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing (a) through (v) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing (a) through (v) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
25. The system of claim 17, wherein the program modules for estimating the intrinsic parameters and estimating the extrinsic parameters comprises sub-modules for:
-
(a) computing estimates of coefficients of radial distortion attributable to the lens of the camera using initial estimates of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(b) selecting a previously unselected one of the captured images;
(c) choosing a previously unselected one of the image points in the selected image;
(d) projecting the feature point of the pattern corresponding to the selected image point into the selected image using the initial estimate of an intrinsic parameter set of the camera and an extrinsic parameter set associated with the selected image;
(e) re-computing the image coordinates of the projected feature point to account for radial distortion using the computed estimates of the coefficients of radial distortion and the initial estimate of an intrinsic parameter set;
(f) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point;
(g) repeating (c) through (f) for each image point in the selected image;
(h) summing the difference indicators computed for each image point in the selected image to produce a combined difference indicator;
(i) repeating (b) through (h) for each of the captured images;
(j) summing the combined difference indicators to produce a complete maximum likelihood estimate indicator;
(k) generating a series of candidate intrinsic parameter sets for the camera, exclusive of the initial estimated intrinsic parameter set;
(l) selecting a previously unselected one of the candidate intrinsic parameter sets;
(m) computing an estimated extrinsic parameter set for each of the captured images using the selected intrinsic parameter set candidate, the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured images;
(n) computing estimates of the coefficients of radial distortion attributable to the lens of the camera using the selected candidate intrinsic parameter set and the coordinates of the image points in each of the captured images;
(o) selecting one of the captured images;
(p) choosing one of the image points in the selected image;
(q) projecting the feature point of the pattern corresponding to the chosen image point into the selected image using an the selected intrinsic parameter set candidate and the last computed extrinsic parameter set associated with the selected image;
(r) re-computing the image coordinates of the projected feature point to account for radial distortion using the last computed estimates of the coefficients of radial distortion and selected intrinsic parameter set candidate;
(s) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point;
(t) repeating (q) and (s) for each remaining image point in the selected image;
(u) summing the difference indicators computed for each image point in the selected image to produce another combined difference indicator;
(v) repeating (p) through (u) for each remaining captured image;
(w) summing the combined difference indicators computed in conjunction with the selected intrinsic parameter set candidate to produce another complete maximum likelihood estimate indicator;
(x) repeating (l) through (w) for each of the candidate intrinsic parameter sets;
(y) identifying the smallest maximum likelihood estimate indicator; and
(z) designating the candidate intrinsic parameter set, and the extrinsic parameter set for each image, associated with the smallest maximum likelihood estimate indicator as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
26. The system of claim 25, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand via a deriving program module comprising sub-modules for:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
27. The system of claim 25, wherein the sub-module for computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point.
-
28. The system of claim 25, further comprising a program module for refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iteration program module comprising sub-modules for:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration, computing a new estimate of the intrinsic parameter set and extrinsic parameter sets using the estimates thereof computed in the last iteration as initializing estimates in the current iteration; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
-
29. A computer-readable memory for calibrating a digital camera, comprising:
-
a computer-readable storage medium; and
a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes a computer to, ascertain image coordinates of every image point in each image in a set of multiple images captured by the camera being calibrated and depicting a pattern disposed on a planar surface from a different non-parallel orientation, said pattern having at least four feature points with known surface coordinates; and
estimate intrinsic parameters of the digital camera using the surface coordinates of the feature points of the pattern and the images coordinates of the corresponding images points in each of the captured images. - View Dependent Claims (30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42)
(a) selecting a previously unselected one of the captured images;
(b) computing a homography between the pattern on the planar surface of the object and the selected image using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the selected image;
(c) repeating (a) and (b) for each of the captured images;
(d) estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
(e) estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
33. The computer-readable memory of claim 32, further comprising a program module for refining the estimates of the intrinsic parameters and extrinsic parameters by compensating for radial distortion in the captured images caused by the lens of the camera, said refining module comprising the sub-modules for:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing (a) through (e) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameters and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images by performing (a) through (e) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameters and the extrinsic parameters associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
34. The computer-readable memory of claim 31, wherein the program modules for estimating the intrinsic parameters and estimating the extrinsic parameters comprises sub-modules for:
-
(a) selecting a previously unselected one of the captured images;
(b) choosing a previously unselected one of the image points in the selected image;
(c) projecting the feature point of the pattern corresponding to the selected image point into the selected image using an initial estimate of an intrinsic parameter set of the camera and an extrinsic parameter set associated with the selected image;
(d) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point;
(e) repeating (b) through (d) for each image point in the selected image;
(f) summing the difference indicators computed for each image point in the selected image to produce a combined difference indicator;
(g) repeating (a) through (f) for each of the captured images;
(h) summing the combined difference indicators to produce a maximum likelihood estimate indicator;
(i) generating a series of candidate intrinsic parameter sets for the camera, exclusive of the initial estimated intrinsic parameter set;
(j) selecting a previously unselected one of he candidate intrinsic parameter sets;
(k) computing an estimated extrinsic parameter set for each of the captured images using the selected intrinsic parameter set candidate;
(l) selecting one of the captured images;
(m) choosing one of the image points in the selected image;
(n) projecting the feature point of the pattern corresponding to the chosen image point into the selected image using an the selected intrinsic parameter set candidate and the last computed extrinsic parameter set associated with the selected image;
(o) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point;
(p) repeating (n) and (o) for each remaining image point in the selected image;
(q) summing the difference indicators computed for each image point in the selected image to produce another combined difference indicator;
(r) repeating (m) through (q) for each remaining captured image;
(s) summing the combined difference indicators computed in conjunction with the selected intrinsic parameter set candidate to produce another maximum likelihood estimate indicator;
(t) repeating (j) through (s) for each of the candidate intrinsic parameter sets;
(u) identifying the smallest maximum likelihood estimate indicator; and
(v) designating the candidate intrinsic parameter set, and the extrinsic parameter set for each image, associated with the smallest maximum likelihood estimate indicator as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
35. The computer-readable memory of claim 34, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand via a deriving program module comprising sub-modules for:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
36. The computer-readable memory of claim 34, wherein the sub-module for computing the indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding projected feature point.
-
37. The computer-readable memory of claim 34, further comprising a program module for refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iteration program module comprising sub-modules for:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration, computing a new estimate of the intrinsic paramter set and extrinsic parameter sets using estimates thereof computed in the last iteration as initializing estimates in the current iterationl; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
-
38. The computer-readable memory of claim 34, further comprising a program module for refining the estimates of the intrinsic parameters and extrinsic parameters by compensating for radial distortion in the captured images caused by the lens of the camera, said refining module comprising the sub-modules for:
-
(aa) computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last computed estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(bb) computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last computed coefficients of radial distortion;
(cc) computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing (a) through (v) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(dd) re-computing estimates of coefficients of radial distortion attributable to the lens of the camera using the last refined estimate of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(ee) re-computing estimates of distortion-free image coordinates for each of the image points in each of the captured images using the last re-computed coefficients of radial distortion;
(ff) re-computing refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images by performing (a) through (v) using the distortion-free image coordinates in place of the previously ascertained image coordinates;
(gg) comparing the last computed estimate of the coefficients of radial distortion to the next to last estimate of the coefficients;
(hh) repeating (dd) through (gg) whenever the compared coefficients equal or exceed a prescribed difference threshold; and
(ii) designate the last re-computed refined estimates of the camera'"'"'s intrinsic parameter set and the extrinsic parameter set associated with each of the captured images as the revised final estimates whenever the compared coefficients fall below the prescribed difference threshold.
-
-
39. The computer-readable memory of claim 31, wherein the program modules for estimating the intrinsic parameters and estimating the extrinsic parameters comprises sub-modules for:
-
(a) computing estimates of coefficients of radial distortion attributable to the lens of the camera using initial estimates of the camera'"'"'s intrinsic parameter set and the coordinates of the image points in each of the captured images;
(b) selecting a previously unselected one of the captured images;
(c) choosing a previously unselected one of the image points in the selected image;
(d) projecting the feature point of the pattern corresponding to the selected image point into the selected image using the initial estimate of an intrinsic parameter set of the camera and an extrinsic parameter set associated with the selected image;
(e) re-computing the image coordinates of the projected feature point to account for radial distortion using the computed estimates of the coefficients of radial distortion and the initial estimate of an intrinsic parameter set;
(f) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point;
(g) repeating (c) through (f) for each image point in the selected image;
(h) summing the difference indicators computed for each image point in the selected image to produce a combined difference indicator;
(i) repeating (b) through (h) for each of the captured images;
(j) summing the combined difference indicators to produce a complete maximum likelihood estimate indicator;
(k) generating a series of candidate intrinsic parameter sets for the camera, exclusive of the initial estimated intrinsic parameter set;
(l) selecting a previously unselected one of the candidate intrinsic parameter sets;
(m) computing an estimated extrinsic parameter set for each of the captured images using the selected intrinsic parameter set candidate, the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured images;
(n) computing estimates of the coefficients of radial distortion attributable to the lens of the camera using the selected candidate intrinsic parameter set and the coordinates of the image points in each of the captured images;
(o) selecting one of the captured images;
(p) choosing one of the image points in the selected image;
(q) projecting the feature point of the pattern corresponding to the chosen image point into the selected image using an the selected intrinsic parameter set candidate and the last computed extrinsic parameter set associated with the selected image;
(r) re-computing the image coordinates of the projected feature point to account for radial distortion using the last computed estimates of the coefficients of radial distortion and selected intrinsic parameter set candidate;
(s) computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point;
(t) repeating (q) and (s) for each remaining image point in the selected image;
(u) summing the difference indicators computed for each image point in the selected image to produce another combined difference indicator;
(v) repeating (p) through (u) for each remaining captured image;
(w) summing the combined difference indicators computed in conjunction with the selected intrinsic parameter set candidate to produce another complete maximum likelihood estimate indicator;
(x) repeating (l) through (w) for each of the candidate intrinsic parameter sets;
(y) identifying the smallest maximum likelihood estimate indicator; and
(z) designating the candidate intrinsic parameter set, and the extrinsic parameter set for each image, associated with the smallest maximum likelihood estimate indicator as the final estimates for the intrinsic parameter set and extrinsic parameter sets, respectively.
-
-
40. The computer-readable memory of claim 39, wherein the initial estimate of the intrinsic parameter set of the camera and the extrinsic parameter sets associated with each of the captured images are derived beforehand via a deriving program module comprising sub-modules for:
-
computing a separate homography between the pattern on the planar surface of the object and each of the captured images using the surface coordinates of the feature points of the pattern and the image coordinates of the corresponding images points in the captured image for which the homography is being computed;
estimating the intrinsic parameters of the camera using the homographies computed for the captured images; and
estimating the extrinsic parameters for each of the captured images using the estimated intrinsic parameters and the homography associated with the particular image for which the extrinsic parameters are being estimated.
-
-
41. The computer-readable memory of claim 39, wherein the submodule for computing an indicator of the difference between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point comprises computing the difference in distance between the image coordinates of the selected image point and image coordinates of the corresponding distorted projected feature point.
-
42. The computer-readable memory of claim 39, further comprising a program module for refining the estimates of the intrinsic parameter set and extrinsic parameter sets via an iterative procedure, said iteration program module comprising sub-modules for:
-
assigning a number of iterations to be completed to produce the refined estimates;
for each iteration, computing a new estimate of the intrinsic parameter set and extrinsic parameter sets using the estimates thereof computed in the last iteration as initializing estimates in the current iteration; and
assigning the last computed estimates of the intrinsic parameter set and extrinsic parameter sets as the revised final estimates.
-
Specification