SCENE ILLUMINATION ADAPTIVE LENS SHADING CORRECTION FOR IMAGING DEVICES

• US 20100165144A1
• Filed: 12/31/2008
• Published: 07/01/2010
• Est. Priority Date: 12/31/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Lee, Ji Soo
• Granted Patent: US 8,130,292 B2
• Time in Patent Office: Days
• US Class Current: 348/229.100
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

Methods, apparatuses, and systems for scene adaptive lens shading correction. One or more gain adjustment surfaces calibrated to a reference illuminant type is stored. At least one scene adjustment surface, which varies as a function of the illuminant type, is stored for each additional illuminant type for which lens shading correction is desired. Additionally, an imaging device with a color image sensor may define a gain adjustment surface and a scene adjustment surface for each of the color channels of the image sensor. Scene adaptive lens shading correction entails determining an illuminant type for a captured image, selecting a scene adjustment surface based on the determined illuminant type, and processing the image using a gain adjustment surface modified by the scene adjustment surface.

Claims

1. A method of processing an image, the method comprising:

modifying one or more gain adjustment surfaces using at least one scene adjustment surface that is selected based on an illuminant type determined for the image; and

processing the image using the one or more modified gain adjustment surfaces.

2. The method of claim 1, wherein the illuminant type is detected by a spectrum sensor with multi-spectral imaging.

3. The method of claim 1, wherein the illuminant type is estimated based on the color temperature of the image.

4. The method of claim 1, wherein the modifying step comprises modifying a plurality of gain adjustment surfaces, one for each color channel of a pixel array in an image sensor.

5. The method of claim 4, wherein the scene adjustment surface used to modify one of the plurality of gain adjustment surfaces is further selected based on the color channel of the pixel array in the image sensor.

6. The method of claim 5, wherein each gain adjustment surface is modified by multiplying said gain adjustment surface by the selected scene adjustment surface.

7. The method of claim 5, wherein the processing step comprises adjusting a pixel signal value for each pixel within the image with a scene adapted gain adjustment value determined based on one of the modified gain adjustment surfaces.

8. A method of processing an image, the method comprising:

determining one or more gain adjustment surfaces for a reference illuminant type that is selected from a plurality of illuminant types;

determining an illuminant type for the image to be processed; and

modifying each of the gain adjustment surfaces using a scene adjustment surface determined based on the illuminant type for the image to be processed; and

processing the image using the one or more modified gain adjustment surfaces.

9. The method of claim 8, wherein the plurality of illuminant types comprises a D65 flat-field, a D50 flat-field, and a Type A flat-field.

10. The method of claim 8, wherein each gain adjustment surface is determined for the reference illuminant type for each color channel of a pixel array in an image sensor.

11. The method of claim 10, wherein the scene adjustment surface is a polynomial curve based on the deviation between two gain adjustment surfaces.

12. The method of claim 11, wherein the polynomial curve is a function of a center of the image sensor, a scaling parameter, an offset parameter, and at least one surface fitting parameter.

13. The method of claim 12, wherein the scaling parameter adjusts the strength of the polynomial curve for low-light scenes.

14. The method of claim 12, wherein the surface fitting parameters for the polynomial curve span into a plurality of sets of parameters, one set for each color channel of the pixel array in the image sensor.

15. The method of claim 14, wherein the polynomial curve is determined according to:

Scene_Adj(x, y)=Bias−

Scaler[A_N(x−

c_x)²+B_N(x−

c_)+CN(y−

c_y)²+D_N(y−

c_y)]

where x and y are pixel coordinates, A_N, B_N, C_N, and D_N are the surface fitting parameters that determine the shape of the polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c, and c_y refers to the center of the image sensor.

16. The method of claim 14, wherein the polynomial curve is determined according to:

Scene_Adj(x, y)=Bias−

Scaler[A_N(x−

c_x)²+C_N(y−

c_y)²]

where x and y are pixel coordinates, A_N and C_N are the surface fitting parameters that determine the shape of the polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c_x and c_y refers to the center of the image sensor.

17. The method of claim 14, wherein the polynomial curve is determined according to:

Scene_Adj(x, y)=Bias−

Scaler*A_N[(x−

c_x)²+(y−

c_y)²]

where x and y are pixel coordinates, A_N is the surface fitting parameter that determines the shape of the polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c_x and c_y refers to the center of the image sensor.

18. An imaging device comprising:

an imaging sensor comprising an array of imaging pixels;

a storage device for storing at least one set of gain adjustment surface parameters and at least one set of scene adjustment surface parameters; and

a signal processing circuit for adjusting signals from the array, the signal processing circuit configured to;

determine a gain adjustment value using one of the sets of stored gain adjustment surface parameters;

determine a scene adjustment value using one of the sets of stored scene adjustment surface parameters that is selected based on an illuminant type;

modify the gain adjustment value using the scene adjustment value; and

generate a corrected pixel signal using the modified gain adjustment value.

19. The imaging device of claim 18, wherein the storage device stores one set of gain adjustment surface parameters for each color channel of the array and each set of parameters is calibrated to a reference illuminant type.

20. The imaging device of claim 19, wherein the storage device stores one set of scene adjustment surface parameters for each of a plurality of illuminant types and each set of parameters is stored in association with the illuminant type.

21. The imaging device of claim 19, wherein the storage device stores one set of scene adjustment surface parameters for each color channel of the array for each of the plurality of illuminant types.

22. The imaging device of claim 18, wherein the scene adjustment value is a function of a center of the image sensor, a scaling parameter, an offset parameter, and at least one surface fitting parameter.

23. The imaging device of claim 22, wherein the signal processing circuit determines the scene adjustment value according to:

Scene_Adj(x, y)=Bias−

Scaler[A_N(x−

c_x)²+B_N(x−

c_x)+C_N(y−

c_y)²+D_N(y−

c_y)]

where x and y are pixel coordinates, A_N, B_N, C_N, and D_N are the surface fitting parameters that determine the shape of a polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c_x and c_y refers to the center of the image sensor.

24. The imaging device of claim 23, wherein the storage device stores (4*N)+2 number of parameters for each illuminant type, where N represents the number of color channels in the imaging sensor.

25. The imaging device of claim 22, wherein the signal processing circuit determines the scene adjustment value according to:

Scene_Adj(x, y)=Bias−

Scaler[A_N(x−

c_x)²+C_N(y−

c_y)²]

where x and y are pixel coordinates, A_N and C_N are the surface fitting parameters that determine the shape of a polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c_x and c_y refers to the center of the image sensor.

26. The imaging device of claim 22, wherein the processing circuit determines the scene adjustment value according to:

Scene_Adj(x, y)=Bias−

Scaler*A_N[(x−

c_x)²+(y−

c_y)²]

where x and y are pixel coordinates, A_N is the surface fitting parameter that determines the shape of a polynomial curve, Scaler is the scaling parameter, Bias is the offset parameter, and c_x and c_y refers to the center of the image sensor.

27. The imaging device of claim 26, wherein the storage device stores N+2 number of parameters for each illuminant type, where N represents the number of color channels in the imaging sensor.

28. An imaging system comprising:

an input for receiving information on a determined illuminant type of an image;

an array of pixels for capturing the image, each pixel having at least one photosensor;

a storage circuit for storing at least one scene adjustment surface, each scene adjustment surface stored in association with an illuminant type; and

a processing circuit for scene adaptive lens shading correction of pixel output signals produced by the array of pixels and corresponding to the image, the lens shading correction being performed in accordance with received information on the determined illuminant type and a scene adjustment surface corresponding to the determined illuminant type.