Camera calibration system, target, and process

• US 10,547,833 B2
• Filed: 06/27/2017
• Issued: 01/28/2020
• Est. Priority Date: 06/28/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Kranski, Jeffrey Steven, Zyda, Frederick Dennis, Vesom, Grace, Tsai, Grace Shin-Yee, Grata, Jeremy A., Jia, Zhiheng, Jiang, Li
• Primary Examiner(s): Monshi, Samira
• Application Number:
  US15/635,120
• Publication Number:
  US 20170374360A1
• Time in Patent Office: 945 Days
• Field of Search: None
• CPC Class Codes: B65G 15/28 Conveyors with a load-conveying surface formed by a single flat belt, not otherwise provided for
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

An improved method, system, and apparatus is provided to perform camera calibration, where cameras are mounted onto a moving conveyance apparatus to capture images of a multi-planar calibration target. The calibration process is optimized by reducing the number of images captured while simultaneously preserving overall information density.

Claims

1. A system, comprising:

a movable platform having a mounting surface to hold a plurality of cameras;

a tessellated concave target having a plurality of planar target regions,

wherein a principle axis of the tessellated concave target is oriented towards the movable platform,

wherein the movable platform is spaced apart from the tessellated concave target; and

an input stack mechanism to load a camera onto the movable platform and an output stack mechanism to offload the camera off the movable platform.

2. The system of claim 1, wherein the movable platform is a conveyer belt.

3. The system of claim 1, wherein the movable platform is spaced apart from the tessellated concave target at a distance corresponding to a focal distance of the plurality of cameras.

4. The system of claim 1, wherein each camera from the plurality of cameras is located at a position on the movable platform, the position being spaced apart from one another camera at a set distance, wherein the each camera image capture direction is facing the tessellated concave target.

5. The system of claim 1, further comprising a data structure for storing camera calibration images, wherein a row within the data structure corresponds to a camera from the plurality of cameras located at the position on the movable platform and columns corresponds to images of the tessellated concave target captured by the camera at designated positions relative to the tessellated concave target along the movable platform.

6. A method for performing camera calibration, comprising:

loading a plurality of cameras onto a movable platform, wherein the plurality of cameras have camera image capture directions oriented towards a tessellated concave target having a plurality of planar target regions;

operating the movable platform to shift the plurality of cameras into designated positions relative to the target;

pausing movement of the movable platform to capture an image from each of the plurality of cameras when the plurality of cameras are located at the designated positions relative to the target; and

calibrating the plurality of cameras using visual patterns observed from a collection of images captured by each camera at each of the designated positions.

7. The method of claim 6, wherein some or all of the planar target regions comprises a plurality of individualized markers, and the visual patterns include images of the individualized markers and corner points of a checkerboard pattern.

8. The method of claim 6, wherein the images undergo filtering.

9. The method of claim 6, wherein calibrating of the plurality of cameras comprises identification of individualized markers from the image, wherein the individualized markers correspond to Aruco or Charuco targets.

10. The method of claim 6, wherein calibrating of the plurality of cameras comprises deriving intrinsic and extrinsic parameters for the plurality of cameras.

11. The method of claim 6, wherein each camera from the plurality of cameras is located at a position on the movable platform, the position being spaced apart from one another camera at a set distance, wherein the each camera image capture direction is facing the tessellated concave target.

12. The method of claim 6, further comprising storing camera calibration images in a data structure, wherein a row within the data structure corresponds to a camera from the plurality of cameras located at a position on the movable platform and columns corresponds to images of the tessellated concave target captured by the camera at the designated positions relative to the tessellated concave target along the movable platform.

13. A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for performing camera calibration, the method comprising:

loading a plurality of cameras onto a movable platform, wherein the plurality of cameras have camera image capture directions oriented towards a tessellated concave target having a plurality of planar target regions;

operating the movable platform to shift the plurality of cameras into designated positions relative to the target;

pausing movement of the movable platform to capture an image from each of the plurality of cameras when the plurality of cameras are located at the designated positions relative to the target; and

calibrating the plurality of cameras using visual patterns observed from a collection of images captured by each camera at each of the designated positions.

14. The computer readable medium of claim 13, wherein some or all of the planar target regions comprises a plurality of individualized markers, and the visual patterns include images of the individualized markers and corner points of a checkerboard pattern.

15. The computer readable medium of claim 13, wherein the images undergo filtering.

16. The computer readable medium of claim 13, wherein calibrating of the plurality of cameras comprises identification of individualized markers from the image, wherein the individualized markers correspond to Aruco or Charuco targets.

17. The computer readable medium of claim 13, wherein calibrating of the plurality of cameras comprises deriving intrinsic and extrinsic parameters for the plurality of cameras.

18. The computer readable medium of claim 13, wherein each camera from the plurality of cameras is located at a position on the movable platform, the position being spaced apart from one another camera at a set distance, wherein the each camera image capture direction is facing the tessellated concave target.

19. The computer readable medium of claim 13, wherein the method further comprising storing camera calibration images in a data structure, wherein a row within the data structure corresponds to a camera from the plurality of cameras located at a position on the movable platform and columns corresponds to images of the tessellated concave target captured by the camera at the designated positions relative to the tessellated concave target along the movable platform.