SYSTEMS AND METHODS FOR DETECTING DEFECTIVE CAMERA ARRAYS AND OPTIC ARRAYS

• US 20140002675A1
• Filed: 06/28/2013
• Published: 01/02/2014
• Est. Priority Date: 06/28/2012
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Pelican Imaging Corporation
• Inventors: Duparre, Jacques, McMahon, Andrew, Lelescu, Dan, Venkataraman, Kartik, Molina, Gabriel
• Granted Patent: US 9,100,635 B2
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

Systems and methods for detecting defective camera arrays, optic arrays and/or sensors are described. One embodiment includes capturing image data using a camera array; dividing the captured images into a plurality of corresponding image regions; identifying the presence of localized defects in any of the cameras by evaluating the image regions in the captured images; and detecting a defective camera array using the image processing system when the number of localized defects in a specific set of image regions exceeds a predetermined threshold, where the specific set of image regions is formed by: a common corresponding image region from at least a subset of the captured images; and any additional image region in a given image that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within said common corresponding image region within the given image.

Claims

1. A method for detecting a defective camera array using an image processing system, where the camera array comprises a plurality of cameras, the method comprising:

capturing image data of a known target using the plurality of cameras, where the image data forms a plurality of images;

dividing each of the plurality of images into a plurality of corresponding image regions using the image processing system;

identifying the presence of at least one localized defect in at least one of the plurality of the cameras by evaluating the image regions in the plurality of images in accordance with at least one predetermined localized defect criterion using the image processing system;

detecting a defective camera array using the image processing system when the number of localized defects in a specific set of image regions exceeds a predetermined threshold, where the specific set of image regions is formed by;

a common corresponding image region from at least a subset of the plurality of images; and

any additional image region in a given image that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within said common corresponding image region within the given image, where the epipolar line is defined by the relative location of the center of the camera that captured the given image and a predetermined viewpoint.

2. The method of claim 1, wherein identifying the presence of at least one localized defect in at least one of the plurality of the cameras by evaluating the image regions in the plurality of images in accordance with at least one predetermined localized defect criterion using the image processing system comprises identifying a plurality of defective pixels within an image region that satisfies at least one predetermined criterion.

3. The method of claim 2, wherein the predetermined criterion is that the plurality of defective pixels within the image region exceeds a predetermined number of defective pixels.

4. The method of claim 2, wherein the predetermined criterion is that the plurality of defective pixels includes a cluster of defective pixels that exceeds a predetermine size.

5. The method of claim 2, wherein defective pixels comprise hot pixels, bright pixels and dark pixels.

6. The method of claim 1, wherein identifying the presence of at least one localized defect in at least one of the plurality of the cameras by evaluating the image regions in the plurality of images in accordance with at least one predetermined localized defect criterion using the image processing system comprises:

measuring the Modulation Transfer Function (MTF) within an image region; and

determining that the MTF of the image region fails to satisfy a predetermined criterion.

7. The method of claim 6, wherein the predetermined criterion is that the on-axis MTF at a predetermined spatial frequency exceeds a first predetermined threshold, the off-axis tangential MTF at a predetermined spatial frequency exceeds a second predetermined threshold, and the off-axis sagittal MTF at a predetermined spatial frequency exceeds a third predetermined threshold.

8. The method of claim 1, wherein said plurality of corresponding images portions forms a first plurality of corresponding image regions and the method further comprises:

dividing each of the plurality of images into a second plurality of corresponding image regions using the image processing system, where the number of image regions in the first plurality of corresponding image regions differs from the number of image regions in the second plurality of corresponding image regions; and

identifying the presence of at least one localized defect in at least one of the plurality of the cameras by evaluating the image regions in the second plurality of images in accordance with at least one additional predetermined localized defect criterion using the image processing system.

9. The method of claim 1, wherein:

the plurality of images forms a reference image and a plurality of alternate view images;

the specific set of image regions is formed by;

a specific image region from the reference image;

the image regions from each of the alternate view images that correspond to the specific image region from the reference image; and

any additional image region in a given alternate view image from the plurality of alternate view images that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within the image region of the given alternate view image that corresponds to the selected image region from the reference image, where the epipolar line is defined by the relative location of the center of the camera that captured the reference image and the center of the camera that captured the given alternate view image.

10. The method of claim 1, wherein:

the plurality of images forms a plurality of images in each of a plurality of color channels; and

a specific set of image regions is formed by image regions from the plurality of images within one of the plurality of color channels.

11. The method of claim 10, wherein:

the plurality of images forms a reference image and a plurality of alternate view images and said plurality of images from one of the plurality of color channels does not include the reference image; and

the specific set of image regions is further formed by;

the image regions from each of the alternate view images within said one of the plurality of color channels that correspond to a specific image region from the reference image; and

any additional image region in a given alternate view image from said one of the plurality of color channels that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within the image region of the given alternate view image that corresponds to the selected image region from the reference image, where the epipolar line is defined by the relative location of the center of the camera that captured the reference image and the center of the camera that captured the given alternate view image.

12. The method of claim 10, further comprising detecting a defective camera array using the image processing system when the number of localized defects in a second set of image regions exceeds a second predetermined threshold, where the second set of image regions is formed by image regions from the plurality of images within a second of the plurality of color channels.

13. The method of claim 12, wherein said predetermined criterion used with respect to said specific set of image regions from said one of the plurality of color channels is different from said second predetermined criterion used with respect to said second set of image regions from said second of the plurality of color channels.

14. A method for detecting a defective optic array using an image processing system, where the optic array comprises a plurality of lens stacks, the method comprising:

dividing the image field of each of the plurality of lens stacks into a plurality of corresponding regions when using an optical test instrument;

measuring the Modulation Transfer Function (MTF) of a known target using the optical test instrument in each of the regions;

identifying the presence of at least one localized defect in at least one of the plurality of the lens stacks by evaluating the MTF measurements of the regions in the plurality of lens stacks in accordance with at least one predetermined localized defect criterion using the optical test instrument;

detecting a defective optic array using the image processing system when the number of localized defects in a specific set of regions exceeds a predetermined threshold, where the specific set of regions is formed by;

a common corresponding region from at least a subset of the plurality of lens stacks; and

any additional region in a given lens stack that forms an image within a predetermined maximum parallax shift distance along an epipolar line from said common corresponding region within the given lens stack, where the epipolar line is defined by the relative location of the center of the given lens stack and a predetermined viewpoint.

15. A method for synthesizing a super-resolution image using a camera module that includes a plurality of localized defects, comprising:

capturing image data using a camera array comprising a plurality of cameras, where at least one of the plurality of cameras includes a known localized defect impacting image data captured by the camera;

disregarding image data within a region of an image captured by the at least one of the plurality of cameras that includes a known localized defect using a processor configured by a super-resolution image processing application, where the discarded image data is from a region of the camera that is known to include said known localized defect; and

synthesizing a super-resolution image from the remaining image data captured by the cameras in the camera array using a super-resolution process performed by the processor configured using the super-resolution image processing application.

16. The method of claim 15, wherein the camera array comprises at least one camera known to include at least one defective pixel, and the method further comprises disregarding image data captured by the pixels in the at least one camera that are known to be defective.

17. An array camera, comprising:

an array camera module comprising a plurality of cameras formed by an imager array comprising a plurality of and an optic array comprising a plurality of lens stacks, where at least one of the plurality of cameras formed by the imager array and optic array includes a known localized defect impacting image data captured by the camera;

a processor; and

memory containing a super-resolution image processing application and defect data identifying said at least one of the plurality of cameras that includes a known localized defect and a region of the camera that contains the known localized defect;

wherein the super-resolution processing application configures the processor to;

capture image data using the array camera module;

with respect to each of said at least one of the plurality of cameras that includes a known localized defect, disregarding image data within at least one region identified by the defect data; and

synthesizing a super-resolution image from the remaining image data.

18. The array camera of claim 17, wherein memory further comprises defect data identifying at least one defective pixel within the imager array and the super-resolution processing application configures the processor to also disregard image data captured by the at least one pixel identified as defective by said defect data.