Method and apparatus for automatic intrinsic camera calibration using images of a planar calibration pattern

• US 10,269,140 B2
• Filed: 05/04/2018
• Issued: 04/23/2019
• Est. Priority Date: 05/04/2017
• Status: Active Grant

• Current Assignee: Genius Sports SS LLC
• Original Assignee: Second Spectrum, Inc. (Genius Sports Ltd.)
• Inventors: Dubout, Charles Xavier Quentin, Ben Shitrit, Horesh Beny
• Primary Examiner(s): Youssef, Menatoallah
• Application Number:
  US15/970,965
• Publication Number:
  US 20180322657A1
• Time in Patent Office: 354 Days
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

The present disclosure relates to a high precision method, model, and apparatus for calibrating the intrinsic camera parameter values of one or more intrinsic parameters of a camera. The intrinsic parameters include focal lengths, skew, optical center, and radial distortion coefficients. The practical implementation of the solution consists of a camera capturing several images of a planar calibration pattern from different viewpoints, and a computer implemented processing pipeline. The solution does not require additional human input and is robust to misdetection of the calibration pattern in some of the images.

Claims

1. A method for automatic camera calibration of an image capture device exhibiting substantial lens distortion, the method comprising:

obtaining, by a computing unit, a plurality of images of a planar calibration pattern captured by the image capture device from different respective angles;

detecting, by the computing unit, a plurality of distinguishable features depicted in the planar calibration pattern for each obtained image wherein at least one of the obtained images captures only a subset of the planar calibration pattern;

for each image, generating, by the computing unit, a respective lattice graph corresponding to the image based on the distinguishable features detected from the image, wherein each respective lattice graph has a plurality of nodes corresponding to distinguishable features and wherein each lattice graph is defined with reference to a global planar coordinate system having an arbitrary origin;

repeatedly calibrating, by the computing unit, a plurality of intrinsic camera parameter values corresponding to one or more intrinsic camera parameters of the image capture device based on two or more distinguishable features detected in two or more of the images and the lattice graphs corresponding to the two or more images, wherein the calibrating is performed for two or more different combinations of the images and wherein the repeated calibration comprises, for each image, a plurality of recursions each starting with a single node detected in the image, one or more n-edge vectors and one or more visited directions and continuing via a repeated selection of one or more n-closest neighbors to the one or more n-edge vectors;

selecting, by the computing unit, one or more intrinsic camera parameter values from the plurality of intrinsic camera parameter values, wherein the one or more intrinsic camera parameter values are selected to minimize a robust estimator of a reprojection error over the obtained plurality of images; and

refining, by the computing unit, calibration of the plurality of intrinsic camera parameter values based on the one or more selected intrinsic camera parameters;

wherein at least one of the intrinsic camera parameters comprises lens distortion.

2. The method of claim 1, further comprising, repeated calibrating a plurality of extrinsic camera parameter values using two and three-dimensional correspondences for the calibration images and the plurality of intrinsic camera parameter values, wherein the repeated calibrating is performed for two or more different combinations of obtained images.

3. The method of claim 1, wherein obtaining the plurality of images includes obtaining a video containing the plurality of images.

4. The method of claim 1, wherein the plurality of images is a series of standstill images.

5. The method of claim 1, wherein the plurality of images is captured by the image capture device, wherein the computing unit is contained in a form factor of the image capture device.

6. The method of claim 1, wherein the plurality of images is captured by the image capture device and received by the computing unit via a network.

7. The method of claim 1, wherein detecting the plurality of distinguishable features includes, for each image, performing a blob detection process on the image.

8. The method of claim 7, wherein the blob detection process is Laplacian of Gaussian.

9. The method of claim 7, wherein the blob detection process is Maximally Stable Extremum Regions.

10. The method of claim 1, wherein each node of the lattice graph is based on a respective center of a respective distinguishable feature of the plurality of distinguishable features.

11. The method of claim 1, wherein refining the plurality of intrinsic camera parameter values is further based on all images of the plurality of obtained images that correspond to the one or more selected intrinsic camera parameter values.

12. The method of claim 1, further comprising configuring the one or more intrinsic camera parameters of the image capture device in accordance with the refined plurality of intrinsic camera parameter values.

13. The method of claim 1, wherein the one or more intrinsic camera parameters include focal lengths, skew, optical center, and/or radial distortion coefficients.

14. A computing unit configured for automatic camera calibration of an image capture device exhibiting substantial lens distortion, the computing unit comprising:

memory storing a plurality of computer-executable instructions;

one or more processors that execute the computer-executable instructions, the computer-executable instructions causing the one or more processors to;

obtain a plurality of images of a planar calibration pattern captured by the image capture device from different respective angles;

detect a plurality of distinguishable features depicted in the planar calibration pattern for each obtained image wherein at least one of the obtained images captures only a subset of the planar calibration pattern;

generate a lattice graph having a plurality of nodes based on the distinguishable features detected from each image wherein each lattice graph is defined with reference to a global planar coordinate system having an arbitrary origin;

repeatedly calibrate a plurality of intrinsic camera parameter values corresponding to one or more intrinsic camera parameters of the image capture device based on two or more distinguishable features detected in two or more of the images and the lattice graphs corresponding to the two or more images, wherein the calibrating is performed for two or more different combinations of the images and wherein the repeated calibration comprises, for each image, a plurality of recursions each starting with a single node detected in the image, one or more n-edge vectors and one or more visited directions and continuing via a repeated selection of one or more n-closest neighbors to the one or more n-edge vectors;

select, one or more intrinsic camera parameter values from the plurality of intrinsic camera parameter values, wherein the one or more intrinsic camera parameter values are selected to minimize a robust estimator of a reprojection error over the obtained plurality of images; and

refine calibration of the plurality of intrinsic camera parameters based on the one or more selected intrinsic camera parameters;

wherein at least one of the intrinsic camera parameters comprises lens distortion.

15. The computing unit of claim 14, wherein obtaining the plurality of images includes obtaining a video containing the plurality of images.

16. The computing unit of claim 14, wherein the plurality of images is a series of standstill images.

17. The computing unit of claim 14, wherein the plurality of images is captured by the image capture device, and wherein the computing unit is integrated in a form factor of the image capture device.

18. The computing unit of claim 14, wherein the plurality of images is captured by the image capture device and received by the computing unit via a network.

19. The computing unit of claim 14, wherein detecting the plurality of distinguishable features includes, for each image, performing a blob detection process on the image.

20. The computing unit of claim 19, wherein the blob detection process is Laplacian of Gaussian.

21. The computing unit of claim 19, wherein the blob detection process is Maximally Stable Extremum Regions.

22. The computing unit of claim 21, wherein each node of the lattice graph is based on a respective center of a respective distinguishable feature of the plurality of distinguishable features.

23. The computing unit of claim 21, wherein refining the plurality of intrinsic camera parameter values is further based on all images of the plurality of obtained images that correspond to the one or more selected intrinsic camera parameter values.

24. The computing unit of claim 14, wherein the computer-executable instructions further cause the one or more processors to configure the one or more intrinsic camera parameters of the image capture device in accordance with the refined plurality of intrinsic camera parameter values.

25. The computing unit of claim 14, wherein the computer-executable instructions further cause the one or more processors to repeatedly calibrate a plurality of extrinsic camera parameter values of one or more extrinsic camera parameters of the image capture device using two and three-dimensional correspondences for the calibration images and the plurality of intrinsic camera parameter values, wherein the repeated calibrating is performed for two or more different combinations of obtained images.

26. The computing unit of claim 14, wherein the one or more intrinsic camera parameters include focal lengths, skew, optical center, and/or radial distortion coefficients.