System and methods for calibration of an array camera

• US 9,986,224 B2
• Filed: 08/31/2015
• Issued: 05/29/2018
• Est. Priority Date: 03/10/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited (Adeia Inc.)
• Inventors: Mullis, Robert
• Primary Examiner(s): Hsia, Sherrie
• Application Number:
  US14/841,651
• Publication Number:
  US 20160165212A1
• Time in Patent Office: 1,002 Days
• Field of Search: 348187, 348188, 348175, 3482181, 348135, 348 47, 348 48, 700258, 700259, 702104, 702116, 382103, 382106, 382107, 382153, 382154, 382284
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 17/02 for colour television signals
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the capturing of an image of a test pattern with the array camera such that each imaging component in the array camera captures an image of the test pattern. The image of the test pattern captured by a reference imaging component is then used to derive calibration information for the reference component. A corrected image of the test pattern for the reference component is then generated from the calibration information and the image of the test pattern captured by the reference imaging component. The corrected image is then used with the images captured by each of the associate imaging components associated with the reference component to generate calibration information for the associate imaging components.

Claims

1. An array camera system, comprising:

an array of cameras comprising a plurality of imaging components that capture images of a scene from different viewpoints including;

a reference imaging component having a reference viewpoint; and

an associate imaging component;

a processor; and

memory in communication with the processor, said memory storing;

a software application comprising machine readable instructions;

calibration information for the reference imaging component comprising scene independent geometric corrections for image data captured by the reference imaging component to account for distortions related to the mechanical construction of the imaging component and produce a corrected reference image;

calibration information for the associate imaging component comprising scene independent geometric corrections for image data captured by the associate imaging component that correct the location of pixels in an image captured by the associate imaging component to locations of corresponding pixel locations in the corrected reference image, where corresponding pixel locations image the same point in a scene in the absence of disparity due to parallax;

wherein execution of the software application by the processor directs the processor to;

capture image data using the plurality of imaging components in the array of cameras including;

a reference image captured by the reference imaging component; and

an associate image captured by the associate imaging component;

apply corrections to the locations of pixels of the reference image using the calibration information for the reference imaging component;

apply corrections to the locations of pixels of the associate image using the calibration information for the reference imaging component;

generate a depth map by measuring disparity due to parallax between the corrected pixels in the reference image and the corrected pixels in the associate image; and

synthesize an image using the depth map and at least some of the pixels from the captured image data.

2. The array camera system of claim 1, wherein the calibration information for the reference imaging component includes characterization information and correction parameters.

3. The array camera system of claim 2, wherein the characterization information includes uniformity characteristics.

4. The array camera system of claim 1, wherein the calibration information for the reference imaging component includes compensation parameters for low frequency aberrations.

5. The array camera system of claim 4, wherein the compensation parameters include X and Y translations.

6. The array camera system of claim 4, wherein the compensation parameters include a 3rd-order radial translation.

7. The array camera system of claim 4, wherein the compensation parameters include a 5th-order radial translation.

8. The array camera system of claim 1, wherein execution of the image processing application by the processor further directs the processor to dynamically refocus an image to blur the synthesized image to make portions of a scene appear out of focus.

9. The array camera system of claim 1, wherein execution of the image processing application by the processor further directs the processor to generate a stereo pair of images using the depth map and the captured image data.

10. The array camera system of claim 9, wherein the array camera system further comprises a 3D display.

11. The array camera system of claim 1, wherein the depth map is utilized by the processor to synthesize an image from one or more virtual viewpoints.

12. The array camera system of claim 11, wherein a rendering device simulates motion parallax and dolly zoom.

13. The array camera system of claim 1, wherein the synthesized image is a super-resolution image.

14. The array camera system of claim 1, wherein the calibration information for the reference imaging component further includes colorimetric corrections for image data captured by the reference imaging component.

15. The array camera system of claim 1, wherein the calibration information for the associate imaging component further includes colorimetric corrections for image data captured by the associate imaging component.

16. The array camera system of claim 1, wherein the calibration information for the reference imaging component further includes photometric corrections for image data captured by the reference imaging component.

17. The array camera system of claim 1, wherein the calibration information for the associate imaging component further includes photometric corrections for image data captured by the associate imaging component.

18. The array camera system of claim 1, wherein the scene independent geometric corrections for image data captured by the associate imaging component are represented by a grid that provides a geometric correction prescription for pixels of the associate imaging component.