Systems and methods for detecting defective camera arrays and optic arrays

• US 9,807,382 B2
• Filed: 07/21/2015
• Issued: 10/31/2017
• Est. Priority Date: 06/28/2012
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited (Adeia Inc.)
• Inventors: Duparre, Jacques, McMahon, Andrew Kenneth John, Lelescu, Dan, Venkataraman, Kartik, Molina, Gabriel
• Primary Examiner(s): Natnael, Paulos M
• Assistant Examiner(s): Desir, Jean W
• Application Number:
  US14/805,412
• Publication Number:
  US 20150326852A1
• Time in Patent Office: 833 Days
• Field of Search: 348246, 3482181, 348187, 348189, 348 92, 348744, 382141
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

Systems and methods for detecting defective camera arrays, optic arrays and/or sensors are described. One embodiment includes capturing image data using a camera array; dividing the captured images into a plurality of corresponding image regions; identifying the presence of localized defects in any of the cameras by evaluating the image regions in the captured images; and detecting a defective camera array using the image processing system when the number of localized defects in a specific set of image regions exceeds a predetermined threshold, where the specific set of image regions is formed by: a common corresponding image region from at least a subset of the captured images; and any additional image region in a given image that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within said common corresponding image region within the given image.

Claims

1. A method for synthesizing a super-resolution image using a camera module that includes a plurality of localized defects, comprising:

capturing image data using a camera array comprising a plurality of cameras, where a first camera of the plurality of cameras includes a known localized defect impacting image data captured by the first camera;

disregarding image data within a region of a first image captured by the first camera using a processor configured by a super-resolution image processing application, where the disregarded image data comprises image data impacted by said known localized defect of the first camera; and

synthesizing a super-resolution image using a super-resolution process performed by the processor configured using the super-resolution image processing application, where, for a region of the super-resolution image that corresponds to the region of the first image with the disregarded image data, image data from a corresponding region of a second image captured by a second camera of the plurality of cameras is used for the super-resolution process, where the corresponding region of the second image corresponds to the region of the first image and is identified by searching for correspondence along an epipolar line up to a predetermined maximum parallax shift distance, where the epipolar line is defined parallel to the relative locations of the center of the first camera and the center of the second camera.

2. The method of claim 1, wherein the known localized defect of the first camera comprises at least one defective pixel, and disregarding the image data comprises disregarding image data captured by the at least one defective pixel of the first camera that is known to be defective.

3. An array camera, comprising:

an array camera module comprising a plurality of cameras formed by an imager array comprising a plurality of focal planes and an optic array comprising a plurality of lens stacks, where a first camera of the plurality of cameras formed by the imager array and optic array includes a known localized defect impacting image data captured by the first camera;

a processor; and

memory containing a super-resolution image processing application and defect data that identifies the first camera that includes the known localized defect and a region of the first camera that contains the known localized defect;

wherein the super-resolution processing application configures the processor to;

capture image data using the array camera module;

disregard image data within at least one region of an image captured by the first camera, where the at least one region of the image comprises image data impacted by the region of the first camera that contains the known localized defect as identified by the defect data; and

synthesize a super-resolution image, where, for a region of the super-resolution image that corresponds to the region of the first image with the disregarded image data, image data from a corresponding region of a second image captured by a second camera of the plurality of cameras is used for the synthesis, where the corresponding region corresponds to the region of the first image and is identified by searching for correspondence along an epipolar line up to a predetermined maximum parallax shift distance, where the epipolar line is defined parallel to the relative locations of the center of the first camera and the center of the second camera.

4. The array camera of claim 3, wherein the defect data further identifies at least one defective pixel within the imager array and the super-resolution processing application configures the processor to also disregard image data captured by the at least one pixel identified as defective by said defect data.