LENS SHADING CALIBRATION FOR CAMERAS

• US 20140184813A1
• Filed: 12/31/2012
• Published: 07/03/2014
• Est. Priority Date: 12/31/2012
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Cohen, Noy, Motta, Ricardo J.
• Granted Patent: US 9,451,187 B2
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

Embodiments of the present invention are directed to methods and systems for performing automatic lens shading correction using module-specific calibrations. According to one aspect of the invention, a method is provided that is performed over three main stages. During a first stage, radial symmetric component data is determined that is common to camera modules of the type to be calibrated. During the second stage, the actual measurement of the shading profile of one or more specific camera modules is performed. In the third and final stage is the extrapolation stage, the base measurement surfaces of a camera module type determined in the first stage are extrapolated and modified with the module-specific Bezier correction surface and calibration data of the second stage. The output surfaces of this third and final stage are used to correct for the shading profile in the camera module, depending on the light-source estimation.

Claims

1. A method for performing lens shading correction, the method comprising:

generating a plurality of base surface measurements of a camera module type, the plurality of base surface measurements comprising a base surface measurement for each illuminant of a plurality of illuminants;

measuring a shading profile specific to each camera module of the camera module type to calculate a module-specific calibration surface;

creating a module-specific corrected surface measurement for each illuminant of the plurality of illuminants based on the plurality of base surface measurements of the camera module type and the module-specific calibration surface; and

wherein, each camera module of the plurality of camera modules is operable to be used in a mobile computing device.

2. The method according to claim 1, further comprising:

capturing a scene with a first camera module of the plurality of camera modules;

determining an illuminant in the scene captured by the first camera module; and

applying the module-specific corrected surface measurement corresponding to the determined illuminant to image data corresponding to the scene.

3. The method according to claim 1, wherein the generating the base surface measurement is performed offline and independently of the measuring a shading profile specific to each camera module of the camera module type.

4. The method according to claim 1, wherein the base surface measurement is stored in a memory, the memory being accessible to a camera driver executing in a processor and configured to interact with the camera module.

6. The method according to claim 1, wherein the generating the base surface measurement comprises:

obtaining a plurality of camera modules of the camera module type;

determining a calibration illumination data;

determining a target illumination data for the plurality of illuminants generating a spectral scan of each camera module with a wavelength resolution;

generating a configuration for each camera module based on the spectral scan;

generating a plurality of images under a plurality of conditions simulated based on the target illumination data and the calibration illumination data; and

averaging the plurality of images of the plurality of camera modules.

7. The method according to claim 6, wherein the generating the spectral scan of the camera module comprises scanning the plurality of camera modules with a monochromator.

8. The method according to claim 6, wherein at least one of:

the calibration illumination data; and

the target illumination data is stored as a raw image format (RAW) file.

9. The method according to claim 6, wherein the calibration illumination data comprises a spectral scan of a calibration light source used to measure the plurality of camera modules during a factory-floor calibration.

10. The method according to claim 9, wherein the target illumination data comprises a spectral scan of each illuminant of the plurality of illuminants.

11. The method according to claim 10, wherein the creating a module-specific corrected surface measurement comprises:

receiving the module-specific calibration surface;

receiving a base surface measurement data corresponding to the calibration illumination data;

receiving the base surface measurement data corresponding to a target illumination data; and

calculating the module-specific corrected surface measurement for the target illuminant based on the module-specific calibration surface, the base surface measurement data corresponding to the calibration illumination data, and the base surface measurement data corresponding to the target illumination data.

12. The method according to claim 11, wherein the module-specific calibration surface comprises a Bezier shading correction surface measurement, further wherein measuring a shading profile specific to each camera module of the camera module type comprises generating the Bezier shading correction surface for the camera module under the calibration light source.

13. The method according to claim 12, wherein the measuring a shading profile specific to each camera module is performed during a factory-floor calibration of the camera module.

14. The method according to claim 12, wherein the generating the Bezier shading correction surface comprises receiving, as input, an image of a uniform flat-field scene captured with the camera module, the image of the uniform flat-field scene comprising a plurality of pixels, each pixel of the plurality of pixels having a pixel value for each of a plurality of color channels.

15. The method according to claim 14, wherein the shading profile of a camera module comprises an intensity gradient for each of the plurality of color channels.

16. The method according to claim 14, wherein the generating the Bezier shading correction surface comprises:

receiving the shading profile of the camera module;

creating a plurality of correction surfaces based on the shading profile, each correction surface of the plurality of correction surfaces corresponding to a color channel of the plurality of color channels; and

calculating a Bezier-coefficients representation for the plurality of correction surfaces.

17. The method according to claim 16, wherein the creating a plurality of correction surfaces comprises:

finding a maximal value in each color channel of the plurality of color channels; and

for each pixel of a plurality of pixels, dividing the maximal value by the color value of the pixel.

18. The method according to claim 16, wherein the generating the Bezier shading correction surface further comprises correcting for an imbalance in green channels by multiplying a lower green channel of the plurality of color channels by a ratio of a higher green channel of the plurality of color channels and the lower green channel.

19. The method according to claim 16, wherein the calculating a Bezier-coefficients representation for the plurality of correction surfaces comprises:

dividing each correction surface into a plurality of sub-divisions;

performing a least-square optimization in each of the plurality of sub-divisions; and

determining a set of Bezier coefficients from the least-square optimizations.

20. The method according to claim 11, wherein the calculating the module-specific corrected surface measurement for the target illuminant is performed for each of the set of Bezier coefficients.

21. The method according to claim 11, wherein the calculating the module-specific corrected surface measurement for the target illuminant comprises:

generating a symmetric shading profile that matches a plurality of spatial locations corresponding to the set of Bezier coefficients;

calculating a plurality of falloff factors corresponding to the set of Bezier coefficients; and

multiplying each Bezier coefficient of the set of Bezier coefficients by a corresponding falloff factor of the plurality of falloff factors.

22. The method according to claim 11, wherein the calculating a plurality of falloff factors corresponding to the set of Bezier coefficients comprises calculating a squared radius at a spatial location of each Bezier coefficient of the set of Bezier coefficients relative to a center of the image.

23. The method according to claim 1, wherein the creating a module-specific corrected surface measurement for each illuminant of the plurality of illuminants is performed in a driver of the camera module.

24. The method according to claim 1, wherein the creating a module-specific corrected surface measurement for each illuminant of the plurality of illuminants is performed during a boot process of the mobile computing device in which the camera module is comprised.

25. The method according to claim 1, wherein the module-specific corrected surface measurements for the plurality of camera modules are stored in a memory of the mobile computing device, the memory being accessible to the driver of the camera module.

26. The method according to claim 1, wherein the module-specific calibration surface is stored in a one-time programmable memory in the mobile computing device.

5. The method according to claim 61, wherein a base surface measurement of the plurality of base surface measurements comprises at least one of:

a plurality of relative illumination characteristics of a lens comprised in the camera module;

a profile of a chief ray angle of a sensor comprised in the camera module;

a spectral response of an infrared filter comprised in the camera module.

27. An image capture device, said device comprising:

an image capture module comprising a lens, an image sensor and an image signal processor, the image capture module being configured to generate an image of a scene comprising a set of image data;

a memory, communicatively coupled to the image capture module, and configured to store the set of image data and a plurality of programmed instructions; and

a processor, communicatively coupled to the memory, and configured to execute the plurality of programmed instructions and to perform a lens shading correction method on the set of image data, the method comprising;

i) referencing a plurality of base surface measurements corresponding to a type of the image capture module and a shading profile specific to the image capture module to create a plurality of module-specific corrected surface measurements;

ii) determining an illuminant present in the scene;

iii) selecting a module-specific corrected surface measurement from the plurality of module-specific corrected surface measurements to correspond to the illuminant; and

iv) applying the selected module-specific corrected surface measurement to the image data.

28. The device according to claim 27, wherein the device comprises a mobile telephone.

29. A computer readable medium containing program instructions embodied therein for causing a camera module in an image capture device to perform lens shading correction, the program instructions comprising:

instructions to reference, from a memory in an image capture device, a plurality of base surface measurements of a camera module type corresponding to the camera module, the plurality of base surface measurements comprising a base surface measurement for each illuminant of a plurality of illuminants;

instructions to reference, from the memory, a shading profile specific to the camera module of the camera module type to calculate a module-specific calibration surface;

instructions to create a module-specific corrected surface measurement for each illuminant of the plurality of illuminants based on the plurality of base surface measurements of and the module-specific calibration surface;

instructions to capture a scene with the camera module;

instructions to determine an illuminant in the scene captured by the camera module; and

instructions to apply the module-specific corrected surface measurement corresponding to the determined illuminant to an image data corresponding to the scene.

30. The device according to claim 29, wherein the image capture device is comprised in a mobile telephone.