System and methods for calibration of an array camera

• US 9,741,118 B2
• Filed: 08/31/2015
• Issued: 08/22/2017
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited
• Inventors: Mullis, Robert
• Primary Examiner(s): Yenke, Brian
• Assistant Examiner(s): HAIEM, SEAN N
• Application Number:
  US14/841,694
• Publication Number:
  US 20160163051A1
• Time in Patent Office: 722 Days
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H05K 999/99 dummy group

Abstract

Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the detecting of defects in the imaging components of the array camera and determining whether the detected defects may be tolerated by image processing algorithms. The calibration process also determines translation information between imaging components in the array camera for use in merging the image data from the various imaging components during image processing. Furthermore, the calibration process may include a process to improve photometric uniformity in the imaging components.

Claims

1. A method for calibrating an array camera using a processing system wherein the array camera includes a plurality of imaging components where the plurality of imaging components includes a reference imaging component and a plurality of associate imaging components associated with the reference imaging component, the method comprising:

performing a defect detection process that includes;

identifying pixel defects in each of the plurality imaging components, and

determining whether pixel defects in a particular imaging component may be corrected in image processing using image data from at least one other one of the plurality of imaging components by;

selecting a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances,

determining a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera,

determining whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect, and

generating a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other images components intersect the parallax translation path of the selected defect; and

performing a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.

2. The method of claim 1 further comprising performing a Modulation Transfer Function (MTF) calibration process on each of the plurality of imaging components in the array camera that includes determining MTF data over the Field of View (FoV) or the Field Height (FH) for each of the imaging components.

3. The method of claim 2 wherein the MTF calibration process further comprises:

capturing image data of a target with each of the plurality imaging components;

analyzing the image data captured by the plurality of imaging components to determine MTF data over one of a FoV and FH for each of the imaging components wherein the MTF data for each of the imaging components is generated from the MTF detected in the image data; and

storing the MTF data for each of the imaging components.

4. The method of claim 1 further comprising performing a photometric calibration process of each of the plurality of imaging components in the array camera that includes determining image data adjustments for each of the imaging components that provide image data from each of the imaging components with a uniform intensity profile.

5. The method of claim 4 wherein the performing the photometric calibration comprises:

capturing image data of a flat field target under controlled lighting conditions with each of the plurality of imaging components in the array camera;

analyzing the image data of each of the imaging components to determine a per pixel adjustment needed to match cos ^4 fall-off response for an image; and

generating a grid of adjustments for each of the plurality of imaging components wherein each adjustment in the grid indicates the adjustment information for data from a set of pixels in the particular imaging component needed to match the cos ^4 fall-off response for an image.

6. The method of claim 1 wherein the identifying of defects in each of the imaging components comprises:

capturing a first set of image data of a field target under high intensity lighting with each of the plurality of imaging components;

capturing a second set of image data of a field target under low intensity lighting;

analyzing the first and second sets of image data to identify each pixel having an unexpected response value in each imaging component;

flagging the identified pixels with a category indicating the category unexpected response received for each imaging component; and

storing the identified pixels with the flagged categories.

7. The method of claim 6 wherein the detecting of defects in each imaging component further comprises:

determining the number of each category of defects identified in each imaging component;

comparing the number of each category of defects to a threshold; and

generating a disposition output for a particular imaging component in response to the number of defects in a category for the particular imaging component being greater than the threshold.

8. A device for calibrating an array camera including a plurality of imaging components where the plurality of imaging components include a reference imaging component and a plurality of associate imaging components associated with the reference imaging component comprising:

a memory; and

a processor configured via one or more applications stored in the memory to;

perform a defect detection process that includes;

identify pixel defects in each of the plurality imaging components, and

determine whether pixel defects in a particular imaging component may be corrected in image processing by;

selecting a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances,

determining a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera,

determining whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect, and

generating a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other images components intersect the parallax translation path of the selected defect; and

perform a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.

9. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process for calibrating an array camera including a plurality of imaging components where the plurality of imaging components include a reference imaging component and a plurality of associate imaging components associated with the reference imaging component, the process comprising:

performing a defect detection process that includes;

identifying pixel defects in each of the plurality imaging components, and

determining whether pixel defects in a particular imaging component may be corrected in image processing by;

selecting a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances,

determining a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera,

determining whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect, and

generating a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other images components intersect the parallax translation path of the selected defect; and

performing a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.