Systems and methods for lens shading correction

• US 9,743,057 B2
• Filed: 05/31/2012
• Issued: 08/22/2017
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cote, Guy, Petolino, Jr., Joseph Anthony, Lim, Suk Hwan, Silverstein, D. Amnon
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): PHAM, QUAN L
• Application Number:
  US13/485,235
• Publication Number:
  US 20130321678A1
• Time in Patent Office: 1,909 Days
• Field of Search: 348251
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response
• CPC Class Codes: H04N 25/673 by using reference sources
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H04N 5/142 Edging; Contouring
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84

Abstract

Systems and methods for correcting intensity drop-offs due to geometric properties of lenses are provided. In one example, a method includes receiving an input pixel of the image data, the image data acquired using an image sensor. A color component of the input pixel is determined. A gain grid is determined by pointing to the gain grid in external memory. Each of the plurality of grid points is associated with a lens shading gain selected based upon the color of the input pixel. A nearest set of grid points that enclose the input pixel is identified. Further, a lens shading gain is determined by interpolating the lens shading gains associated with each of the set of grid points and is applied to the input pixel.

Claims

1. An image signal processing circuit comprising:

an interface configured to receive image data from an image sensor of an imaging device; and

pipe processing logic configured to perform a set of processing operations on the image data, wherein one of the set of processing operations is a lens shading correction operation, and wherein the pipe processing logic comprises lens shading correction logic configured to;

receive an input pixel of the image data;

access a gain grid by pointing to the gain grid in external memory;

determine the location of the input pixel within a lens shading correction region defined within an imaging frame of the image sensor relative to the gain grid having a plurality of grid points distributed horizontally and vertically, wherein each of the plurality of grid points is associated with a lens shading gain;

determine an interpolated lens shading gain for the input pixel using bi-linear interpolation of the lens shading gains associated with a nearest set of grid points enclosing the input pixel;

determine a radial lens shading gain as a function of a global gain parameter associated with the color of the input pixel and a radial distance between the center of the lens shading correction region and the location of the input pixel;

apply the interpolated lens shading gain and the radial lens shading gain to the input pixel; and

collect and store information related to;

one or more pixel values inputted to the lens shading correction logic, one or more pixel values outputted from the lens shading correction logic, or both.

2. The image signal processing circuit of claim 1, wherein the interface comprises a Standard Mobile Imaging Architecture (SMIA) interface or a Mobile Industry Processor Interface (MIPI), or some combination thereof.

3. The image signal processing circuit of claim 1, comprising front-end processing logic configured to perform an initial set of operations on the image data before the image data is processed by the pipe processing logic, wherein the front-end processing logic is configured to receive an input pixel of the image data, determine the location of the input pixel within a lens shading correction region defined within an imaging frame of the image sensor relative to a gain grid having a plurality of grid points distributed horizontally and vertically, wherein each of the plurality of grid points is associated with a lens shading gain, determine an interpolated lens shading gain for the input pixel using bi-linear interpolation of the lens shading gains associated with a nearest set of grid points enclosing the input pixel, determine a radial lens shading gain as a function of a global gain parameter associated with the color of the input pixel and a radial distance between the center of the lens shading correction region and the location of the input pixel, apply the interpolated lens shading gain and the radial lens shading gain to the input pixel, and collect imaging statistics on the image data after the interpolated lens shading gain and the radial lens shading gain are applied.

4. The image signal processing circuit of claim 3, wherein the collected imaging statistics comprise white balance statistics, exposure statistics, focusing statistics, a number of pixels above a sensor'"'"'s saturation level before lens shading correction is applied, a number of pixels above a sensor'"'"'s saturation level after lens shading correction is applied, or some combination thereof.

5. The image signal processing circuit of claim 3, comprising control logic configured to receive the collected imaging statistics and to determine one or more control parameters for the imaging device, or one or more control parameters for the pipe processing logic, or some combination thereof.

6. An electronic device, comprising:

an imaging device comprising an image sensor; and

image processing circuitry configured to process image data acquired using the image

sensor, wherein the image processing circuitry comprises;

lens shading correction logic configured to;

determine the location of an input pixel of the image data within a lens shading correction region defined within an imaging frame of the image sensor relative to a gain grid stored in external memory and accessed by pointing to an address of the gain grid in external memory, the gain grid having a plurality of grid points distributed horizontally and vertically, wherein each of the plurality of grid points is associated with a lens shading gain;

determine an interpolated lens shading gain for the input pixel using bi-linear interpolation of the lens shading gains associated with a nearest set of grid points enclosing the input pixel;

determine a radial lens shading gain as a function of a global gain parameter associated with the color of the input pixel and a radial distance between the center of the lens shading correction region and the location of the input pixel; and

apply the interpolated lens shading gain and the radial lens shading gain to the input pixel; and

lens shading correction statistics logic configured to;

collect and store information related to;

one or more pixel values inputted to the lens shading correction logic, one or more pixel values outputted from the lens shading correction logic, or both.

7. The electronic device of claim 6, wherein the image sensor comprises a color filter array, and wherein each pixel of the image data is associated with a color component based upon the configuration of the color filter array.

8. The electronic device of claim 7, wherein the color filter array comprises a Bayer filter, a RGBW filter, or a CYGM filter.

9. The electronic device of claim 7, comprising a memory device, wherein the lens shading gains associated with the plurality of grid points of the gain grid are stored in the memory device and are shared for each of color component of the color filter array.

10. The electronic device of claim 6, comprising a display device configured to display the image data subsequent to processing by the lens shading correction logic, and wherein the processed image data exhibits increased uniformity in light intensity.

11. The electronic device of claim 6, wherein the lens shading correction statistics logic is configured to:

count a number of pixels above a pre-defined clip level of the one or more pixel values inputted to the lens shading correction logic;

count a number of pixels above the pre-defined clip level of the one or more pixel values outputted from the lens shading correction logic;

or both.