Systems and methods for photometric normalization in array cameras

• US 10,412,314 B2
• Filed: 10/09/2017
• Issued: 09/10/2019
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: McMahon, Andrew Kenneth John, Lelescu, Dan, Ciurea, Florian
• Primary Examiner(s): Hannett, James M
• Application Number:
  US15/727,932
• Publication Number:
  US 20180035057A1
• Time in Patent Office: 701 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

Systems and methods for performing photometric normalization in an array camera in accordance with embodiments of this invention are disclosed. The image data of scene from a reference imaging component and alternate imaging components is received. The image data from each of the alternate imaging components is then translated to so that pixel information in the image data of each alternate imaging component corresponds to pixel information in the image data of the reference component. The shifted image data of each alternate imaging component is compared to the image data of the reference imaging component to determine gain and offset parameters for each alternate imaging component. The gain and offset parameters of each alternate imaging component is then applied to the image data of the associate imaging to generate corrected image data for each of the alternate imaging components.

Claims

1. A method of manufacturing an array camera device, the method comprising:

assembling an array of cameras comprising a plurality of imaging components that capture images of a scene from different viewpoints, where the components include;

a set of one or more reference imaging components, each having a reference viewpoint, and

a set of one or more associate imaging components;

configuring the array of cameras to communicate with at least one processor;

configuring the processor to communicate with at least one display;

configuring the processor to communicate with at least one type of memory; and

performing a photometric normalization process for the array of cameras, where the photometric normalization process comprises;

receiving image data for a scene captured by a first one of the plurality of imaging components;

receiving image data for a scene captured by a second one of the plurality of imaging components;

determining a nominal parallax for image data of the second one of the plurality the of imaging components that translate information for a particular pixel in the image data of the second one of the plurality of imaging components to a corresponding pixel in the first one of the plurality of imaging components;

applying the nominal parallax of the second one of the plurality of imaging components to the image data of the second one of the plurality of imaging components;

applying a low pass filter to the image data from the first one of the plurality of imaging components and the shifted image data of the second one of the plurality of imaging components; and

computing gain and offset parameters for the second one of the plurality of imaging components from the low pass filtered shifted image data of the second one of the plurality of imaging components and the low pass filtered image data of the first one of the plurality of imaging components.

2. The method of claim 1 further comprising applying the gain and offset parameters of the second one of the plurality of imaging components to the image data captured by the second one of the plurality of imaging components to form photometrically normalized image data for the second one of the plurality of imaging components.

3. The method of claim 2 further comprising determining regions of high contrast in the low pass filtered shifted image data of the second one of the plurality of imaging components.

4. The method of claim 3 further comprising storing the determined regions of high contrast in the low pass filtered image data of the second one of the plurality of imaging components for further correction processing.

5. The method of claim 1 wherein the computing of the gain and offset parameters is performed on a pixel by pixel basis for the image data of the second one of the plurality of imaging components.

6. The method of claim 1 wherein the computing of the gain and offset parameters is performed on regions of pixels for the image data of the second one of the plurality of imaging components.

7. The method of claim 1 further comprising:

comparing each gain parameter and each offset parameter for the second one of the plurality of imaging components to a threshold value; and

setting each gain parameter and each offset parameter determined to at least meet the threshold value to a predetermined value.

8. A method of manufacturing an array camera device, the method comprising:

assembling an array of cameras comprising a plurality of imaging components that capture images of a scene from different viewpoints, where the components include;

a set of one or more reference imaging components, each having a reference viewpoint, and

a set of one or more associate imaging components;

configuring the array of cameras to communicate with at least one processor;

configuring the processor to communicate with at least one display;

configuring the processor to communicate with at least one type of memory; and

performing a photometric normalization process for the array of cameras, where the photometric normalization process comprises;

receiving image data for a scene captured by the reference imaging component;

receiving image data for a scene captured by each of plurality of alternate imaging components;

determining a nominal parallax for image data of each of the plurality of alternate imaging components that translate information for a particular pixel in the image data of a particular alternate imaging component to a corresponding pixel in the reference imaging component;

applying the nominal parallax of each particular alternate imaging component to the image data of the particular alternate imaging component;

applying a low pass filter to the image data from the reference imaging component and the shifted image data of each particular alternate imaging component; and

computing gain and offset parameters for each particular alternate imaging components from the low pass filtered shifted image data of the particular alternate imaging component and the low pass filtered image data of the reference imaging component.

9. The method of claim 8 further comprising applying the gain and offset parameters of each particular alternate imaging component to the image data captured by the particular alternate imaging component to form photometrically normalized image data for each particular alternate imaging component.

10. The method of claim 9 further comprising determining regions of high contrast in the low pass filtered shifted image data of each particular alternate imaging component.

11. The method of claim 10 further comprising storing determined the regions of high contrast in the low pass filtered image data of each particular alternate imaging component for further correction processing.

12. The method of claim 8, wherein the computing of the gain and offset parameters is performed on a pixel by pixel basis for the image data of each of the plurality of alternate imaging components.

13. The method of claim 8, wherein the computing of the gain and offset parameters is performed on regions of pixels for the image data of each of the plurality of alternate imaging components.

14. The method of claim 8 further comprising:

comparing each gain parameter and each offset parameter for each of the plurality of alternate imaging component to a threshold value; and

setting each gain parameter and each offset parameter determined to at least meet the threshold value to a predetermined value.