Systems and methods for photometric normalization in array cameras

• US 9,787,911 B2
• Filed: 07/30/2015
• Issued: 10/10/2017
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited (Adeia Inc.)
• Inventors: McMahon, Andrew Kenneth John, Lelescu, Dan, Ciurea, Florian
• Primary Examiner(s): Hannett, James
• Application Number:
  US14/814,291
• Publication Number:
  US 20160198096A1
• Time in Patent Office: 803 Days
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

Systems and methods for performing photometric normalization in an array camera in accordance with embodiments of this invention are disclosed. The image data of scene from a reference imaging component and alternate imaging components is received. The image data from each of the alternate imaging components is then translated to so that pixel information in the image data of each alternate imaging component corresponds to pixel information in the image data of the reference component. The shifted image data of each alternate imaging component is compared to the image data of the reference imaging component to determine gain and offset parameters for each alternate imaging component. The gain and offset parameters of each alternate imaging component is then applied to the image data of the associate imaging to generate corrected image data for each of the alternate imaging components.

Claims

1. A method performed by a processing system to provide a photometric normalization in an array camera system having a plurality of imaging components, the method comprising:

receiving image data for a scene captured by a first one of the plurality of imaging components;

receiving image data for a scene captured by a second one of the plurality of imaging components;

determining a nominal parallax for image data of the second one of the plurality the of imaging components that translate information for a particular pixel in the image data of the second one of the plurality of imaging components to a corresponding pixel in the first one of the plurality of imaging components;

applying the nominal parallax of the second one of the plurality of imaging components to the image data of the second one of the plurality of imaging components;

applying a low pass filter to the image data from the first one of the plurality of imaging components and the shifted image data of the second one of the plurality of imaging components; and

computing gain and offset parameters for the second one of the plurality of imaging components from the low pass filtered shifted image data of the second one of the plurality of imaging components and the low pass filtered image data of the first one of the plurality of imaging components.

2. The method of claim 1 further comprising applying the gain and offset parameters of the second one of the plurality of imaging components to the image data captured by the second one of the plurality of imaging components to form photometrically normalized image data for the second one of the plurality of imaging components.

3. The method of claim 2 further comprising determining regions of high contrast in the low pass filtered shifted image data of the second one of the plurality of imaging components.

4. The method of claim 3 further comprising storing the determined regions of high contrast in the low pass filtered image data of the second one of the plurality of imaging components for further correction processing.

5. The method of claim 1 wherein the computing of the gain and offset parameters is performed on a pixel by pixel basis for the image data of the second one of the plurality of imaging components.

6. The method of claim 1 wherein the computing of the gain and offset parameters is performed on regions of pixels for the image data of the second one of the plurality of imaging components.

7. The method of claim 1 further comprising:

comparing each gain parameter and each offset parameter for the second one of the plurality of imaging components to a threshold value; and

setting each gain parameter and each offset parameter determined to at least meet the threshold value to a predetermined value.

8. A system for providing a photometric normalization in an array camera system having a plurality of imaging components comprising:

an array camera system including a plurality of imaging components that each capture image data of a scene;

a memory; and

a processor that is configured by instructions stored in the memory to;

receive image data for a scene captured by a first one of the plurality of imaging components,

receive image data for a scene captured by a second one of the plurality of imaging components,

determine a nominal parallax for image data of each of the plurality of alternate imaging components that translate information for a particular pixel in the image data of a particular alternate imaging component to a corresponding pixel in the first one of the plurality of imaging components,

apply the nominal parallax of the second one of the plurality of imaging components to the image data of the second one of the plurality of imaging components,

apply a low pass filter to the image data from the first one of the plurality of imaging components and the shifted image data of the second one of the plurality of imaging components, and

compute gain and offset parameters for the second one of the plurality of imaging components from the low pass filtered shifted image data of the second one of the plurality of imaging components and the low pass filtered image data of the first one of the plurality of imaging components.

9. The system of claim 8 wherein the processor is further configured by the instructions to apply the gain and offset parameters of the second one of the plurality of imaging components to the image data captured by the second one of the plurality of imaging components to form photometrically normalized image data for the second one of the plurality of imaging components.

10. The system of claim 8 wherein the processor is further configured by the instructions to determine regions of high contrast in the low pass filtered shifted image data of the second one of the plurality of imaging components.

11. The system of claim 10 wherein the processor is further configured by the instructions to store the determined regions of high contrast in the low pass filtered image data of the second one of the plurality of imaging components for further correction processing.

12. The system of claim 8 wherein the computing of the gain and offset parameters is performed on a pixel by pixel basis for the image data of the second one of the plurality of imaging components.

13. The system of claim 8 wherein the computing of the gain and offset parameters is performed on regions of pixels for the image data of the second one of the plurality of imaging components.

14. The system of claim 8 wherein the processor is further configured by the instructions to:

compare each gain parameter and each offset parameter for the second one of the plurality of imaging components to a threshold value; and

set each gain parameter and each offset parameter determined to at least meet the threshold value to a predetermined value.

15. A non-transitory medium readable by a processor that stores instructions that when read by the processor configure the processor to perform the method comprising:

receiving image data for a scene captured by a first one of the plurality of imaging components of an array camera system;

receiving image data for a scene captured by a second one of plurality the of imaging components of the array camera system;

determining a nominal parallax for image data of the second one of the plurality of imaging components that translate information for a particular pixel in the image data of the second one of the plurality of imaging components to a corresponding pixel in the first one of the plurality of imaging components;

applying the nominal parallax of the second one of the plurality of imaging components to the image data of the second one of the plurality of imaging components;

applying a low pass filter to the image data from the first one of the plurality of imaging components and the shifted image data of the second one of the plurality of imaging components; and

computing gain and offset parameters for the second one of the plurality of imaging components from the low pass filtered shifted image data of the second one of the plurality of imaging components and the low pass filtered image data of the first one of the plurality of imaging components.

16. The non-transitory medium of claim 15 wherein the method further comprises applying the gain and offset parameters of the second one of the plurality of imaging components to the image data captured by the second one of the plurality of imaging components to form photometrically normalized image data for the second one of the plurality of imaging components.

17. The non-transitory medium of claim 16 wherein the method further comprises determining regions of high contrast in the low pass filtered shifted image data of the second one of the plurality of imaging components.

18. The non-transitory medium of claim 17 wherein the method further comprises storing the determined regions of high contrast in the low pass filtered image data of the second one of the plurality of imaging components for further correction processing.

19. The non-transitory medium of claim 15 wherein the computing of the gain and offset parameters is performed on a pixel by pixel basis for the image data of the second one of the plurality of imaging components.

20. The non-transitory medium of claim 15 wherein the computing of the gain and offset parameters is performed on regions of pixels for the image data of the second one of the plurality of imaging components.