System and methods for calibration of an array camera

• US 9,124,864 B2
• Filed: 10/20/2014
• Issued: 09/01/2015
• Est. Priority Date: 03/10/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Pelican Imaging Corporation
• Inventors: Mullis, Robert
• Primary Examiner(s): KOSTAK, VICTOR R
• Application Number:
  US14/519,005
• Publication Number:
  US 20150035992A1
• Time in Patent Office: 316 Days
• Field of Search: 348/187, 348/188, 348/175, 348/218.1, 348/135, 348/47, 348/48, 700/258, 700/259, 702/104, 702/116, 382/103, 382/106, 382/107, 382/153, 382/154, 382/284
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 17/02 for colour television signals
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the capturing of an image of a test pattern with the array camera such that each imaging component in the array camera captures an image of the test pattern. The image of the test pattern captured by a reference imaging component is then used to derive calibration information for the reference component. A corrected image of the test pattern for the reference component is then generated from the calibration information and the image of the test pattern captured by the reference imaging component. The corrected image is then used with the images captured by each of the associate imaging components associated with the reference component to generate calibration information for the associate imaging components.

Claims

1. A method for calibrating an array camera including a plurality of imaging components, the method comprising:

capturing an image of a test pattern using the array camera wherein each of the plurality of imaging components of the array camera captures an image from a particular viewpoint;

generating scene independent geometric corrections for the image data captured by a first one of the plurality of imaging components in the array camera using data of the image of the test pattern captured by the first one of the plurality of imaging components and data describing the test pattern using a processor;

generating a corrected image of the test pattern for the first one of the plurality of imaging components based on the scene independent geometric corrections for the image data captured by the first one of the plurality of imaging components and the image of the test pattern captured by the first one of the plurality of imaging components using the processor; and

generating scene independent geometric corrections for the image data captured by a second one of the plurality of imaging components using the data of image of the test pattern captured by the second one of the plurality of imaging components and data for the corrected image of the of the test pattern of the first one of the plurality of imaging components using the processor.

2. The method of claim 1 wherein the test pattern includes a low-contrast slanted edge pattern.

3. The method of claim 2 wherein the test pattern includes a plurality of Macbeth Color Chart type patterns inset at different positions in the low-contrast slanted pattern.

4. The method of claim 1 wherein the test pattern is at a distance of at least 70 percent of the hyperfocal distance of the array camera away from the array camera during the capturing of the image of the test pattern.

5. The method of claim 1 wherein the test pattern is at a distance of at 50 percent of the hyperfocal distance of the array camera away from the array camera during the capturing of the image of the test pattern.

6. The method of claim 1 further comprising performing at least one pass/fail test of the array camera based on images of the test pattern captured by the plurality of imaging components in the array camera.

7. The method of claim 1 wherein generating scene independent geometric corrections for the image data captured by the first one of the plurality of imaging components using data of the image of the test pattern captured by the first one of the plurality of imaging components and data describing the test pattern comprises:

identifying intersection points in the image of the test pattern captured by the first one of the plurality of imaging components;

determining the uniformity characteristics of the second one of the plurality of imaging components from the identified intersection points in the image of the test pattern captured by the second one of the plurality of imaging components and the test pattern; and

deriving a set of geometric corrections for the first one of the plurality of imaging components to compensate for low frequency aberrations in the captured image of the test pattern.

8. The method of claim 1 wherein generating scene independent geometric corrections for the image data captured by the second one of the plurality of imaging components comprises:

identifying intersection points in the test pattern image captured by the second one of the plurality of imaging components;

translating the intersection points from the captured test pattern images captured by second one of the plurality of imaging components in accordance with an expected parallax shift for the second one of the plurality of imaging components relative to the first one of the plurality of imaging components; and

deriving a set of geometric corrections for the second one of the plurality of imaging components to compensate for low frequency aberrations in the captured image of the test pattern by comparing the translated intersections points in the image captured by the second one of the plurality of imaging components to corresponding intersection points in the corrected image for the first one of a plurality of imaging components.

9. The method of claim 8 wherein the expected parallax shift for the second one of the plurality of associate imaging components is based upon at least one of the physical offset of the second one of the plurality of imaging components to the first one of the plurality of imaging components, the behavior of sensor optics in the second one of the plurality of imaging components, and distance of the test pattern from the array camera.

10. The method of claim 1 further comprising storing the images captured by the plurality of imaging components to perform the calibration of the first of the plurality of imaging component and the second one of the plurality of imaging components at a later time.

11. The method of claim 1 further comprising storing the scene dependent geometric correction information for the image data captured by the first one of the plurality of imaging components in a memory.

12. The method of claim 1 further comprising storing the scene dependent geometric correction information for the image data captured by second one of the plurality of imaging components in a memory.

13. The method of claim 1, further comprising generating colorimetric corrections for the image data captured by each imaging component in the array camera using data of the image of the test pattern captured by the each of the plurality of imaging components using the processor.

14. The method of claim 1, further comprising generating photometric corrections for the image data captured by each of the plurality of imaging components in the array camera using data of the image of the test pattern captured by the first one of the plurality of imaging components using the processor.

15. A device for calibrating an array camera including a plurality of imaging components comprising:

a memory; and

a processor configured via one or more applications stored in the memory to;

receive an image of a test pattern from each of the plurality of imaging components of the array camera;

generate scene independent geometric corrections for the image data captured by a first one of the plurality of imaging components imaging component using data of the image of the test pattern captured by the first one of the plurality of imaging components and data describing the test pattern;

generate a corrected image of the test pattern for the first one of the plurality of imaging components based on the scene independent geometric corrections for the image data captured by the first one of the plurality of imaging components and the image of the test pattern captured by the first one of the plurality of imaging components; and

generate scene independent geometric corrections for the image data captured by a second one of the plurality of imaging components using the data of image of the test pattern captured by each associate imaging component and the corrected image of the first one of the plurality of imaging components using the processor.

16. A machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process for calibrating an array camera including a plurality of imaging components where the plurality of imaging components include a reference imaging component and a plurality of associate imaging components associated with the reference imaging component, the process comprising:

receiving an image of a test pattern captured by each of the plurality of imaging components of the array camera;

generating scene independent geometric corrections for the image data captured by a first one of the plurality of imaging components in the array camera using data of the image of the test pattern captured by the first one of the plurality of imaging components and data describing the test pattern using a processor;

generating a corrected image of the test pattern for the first one of the plurality of imaging components based on the scene independent geometric corrections for the image data captured by the first one of the plurality of imaging components and the image of the test pattern captured by the first one of the plurality of imaging components using the processor; and

generating scene independent geometric corrections for the image data captured by a second one of the plurality of imaging components using the data of image of the test pattern captured by the second one of the plurality of imaging components and data for the corrected image of the of the test pattern of the first one of the plurality of imaging components using the processor.