Fixed pattern noise subtraction in a digital image sensor

• US 7,280,141 B1
• Filed: 09/23/2003
• Issued: 10/09/2007
• Est. Priority Date: 09/23/2003
• Status: Active Grant

• Current Assignee: Pixim, Inc. (Sony Group Corp.)
• Original Assignee: Pixim, Inc. (Sony Group Corp.)
• Inventors: Frank, Michael, Motta, Ricardo J.
• Primary Examiner(s): Ometz; David
• Assistant Examiner(s): KHAN, USMAN A
• Application Number:
  US10/670,014
• Time in Patent Office: 1,477 Days
• Field of Search: 348241-251, 348252-253
• US Class Current: 348/243
• CPC Class Codes: H04N 25/63 applied to dark current
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors

Abstract

A fixed pattern noise subtraction method in a digital imaging system incorporating a digital image sensor includes: acquiring a reference image of the digital image sensor when the digital image sensor receives no illumination, storing a reference value of an operating parameter associated with the reference image where the reference image is indicative of the fixed pattern noise associated with the digital image sensor, storing a model describing the behavior of the fixed pattern noise as a function of the operating parameter. Then, the method continues with acquiring a first image, measuring a current value of the operating parameter associated with the first image, calculating a noise prediction image by extrapolation of the reference image in accordance with the model and based on the current value and the reference value of the operating parameter, and subtracting the noise prediction image from the first image to generate a final image.

Claims

1. A method for subtracting fixed pattern noise in a digital imaging system incorporating a digital image sensor, comprising:

acquiring a reference image of the digital image sensor when the digital image sensor receives no illumination;

storing a reference value of an operating parameter associated with the reference image, wherein the reference image is indicative of the fixed pattern noise associated with the digital image sensor when the operating parameter has the reference value;

storing a model describing the behavior of the fixed pattern noise as a function of the operating parameter;

acquiring a first image;

measuring a current value of the operating parameter associated with the first image;

calculating a noise prediction image by extrapolation of the reference image in accordance with the model and based on the current value and the reference value of the operating parameter; and

subtracting the noise prediction image from the first image to generate a final image.

2. The method of claim 1, wherein acquiring a reference image of the digital image sensor comprises:

acquiring a plurality of images using the digital image sensor when the digital image sensor receives no illumination; and

for each pixel in the digital image sensor, averaging pixel values of the plurality of images, the averaged pixel values for each pixel forming the reference image.

3. The method of claim 2, wherein acquiring a plurality of images using the digital image sensor when the digital image sensor receives no illumination comprises:

closing an aperture of the digital imaging system so that the digital image sensor is not exposed to illumination.

4. The method of claim 1, wherein the operating parameter comprises a temperature of the digital image sensor.

5. The method of claim 4, wherein storing a reference value of an operating parameter associated with the reference image comprises:

providing a dark pixel in the digital image sensor, the dark pixel being permanently and totally shadowed; and

measuring a leakage current associated with the dark pixel when the reference image is being acquired,

wherein the leakage current of the dark pixel is the reference value used as an indicator of the temperature of the digital image sensor when the reference image is acquired.

6. The method of claim 5, wherein measuring a current value of the operating parameter associated with the first image comprises:

measuring a leakage current associated with the dark pixel when the first image is being acquired,

wherein the leakage current of the dark pixel is the current value used as an indicator of the temperature of the digital image sensor when the first image is acquired.

7. The method of claim 1, wherein the operating parameter comprises one of a temperature, an operating voltage, an exposure time, a location of the pixels in the digital image sensor, and the reset voltage of the digital image sensor.

8. The method of claim 1, wherein subtracting the noise prediction image from the first image comprises:

determining an illumination level of the first image; and

subtracting the noise prediction image from the first image only when the illumination level is less than a pre-determined threshold.

9. The method of claim 8, wherein determining an illumination level of the first image comprises:

averaging a plurality of pixel values from the first image, the plurality of pixel values being selected from pixels distributed across the digital image sensor.

10. The method of claim 1, wherein subtracting the noise prediction image from the first image comprises:

determining an illumination level of the first image;

storing the noise prediction image in m bits when the illumination level is less than a pre-determined threshold;

storing the noise prediction image in n bits, where n is less than m, when the illumination level is greater than a pre-determined threshold; and

subtracting the noise prediction image from the first image.

11. An imaging system, comprising:

a digital image sensor for performing image capture operations, comprising;

a sensor array comprising a two-dimensional array of digital pixels, each digital pixel outputting digital signals as pixel data representing an image of a scene;

an image buffer, in communication with the sensor array, for storing the pixel data; and

a first processor, in communication with the image buffer and the sensor array, for controlling image capture and pixel data processing operations; and

a digital image processor for performing image processing operations, comprising;

a frame buffer, in communication with the digital image sensor, coupled to store the pixel data, wherein the frame buffer includes a first bit field for storing the pixel data for a first image and a second bit field for storing noise data associated with the sensor array;

a second processor, in communication with the frame buffer, for processing the pixel data stored in the frame buffer; and

a memory buffer, in communication with the second processor, for storing a reference image of the sensor array and a reference value of an operating parameter, wherein the reference image is indicative of the fixed pattern noise associated with the digital image sensor when the operating parameter has the reference value,

wherein when the digital image sensor captures a first image, a current value of the operating parameter when the first image is acquired is measured;

the second processor generates the noise data by extrapolating the reference image based on the reference value and the current value of the operating parameter and a model describing the behavior of the fixed pattern noise as a function of the operating parameter.

12. The imaging system of claim 11, wherein the reference image of the sensor array comprises an image of averaged pixel values of a plurality of images acquired using the digital image sensor when the digital image sensor receives no illumination.

13. The imaging system of claim 11, wherein the operating parameter comprises a temperature of the digital image sensor.

14. The imaging system of claim 13, wherein the sensor array comprises a dark pixel being permanently and totally shadowed, and the reference value of the operating parameter associated with the reference image comprises a leakage current value associated with the dark pixel in the sensor array, the leakage current being used as an indicator of the temperature of the digital image sensor when the reference image is acquired.

15. The imaging system of claim 14, wherein measuring a current value of the operating parameter associated with the first image comprises:

measuring a leakage current associated with the dark pixel when the first image is being acquired,

wherein the leakage current of the dark pixel is the current value used as an indicator of the temperature of the digital image sensor when the first image is acquired.

16. The imaging system of claim 11, wherein the operating parameter comprises one of a temperature, an operating voltage, an exposure time, a location of the pixels in the digital image sensor, and the reset voltage of the digital image sensor.

17. The imaging system of claim 11, wherein the second processor subtracts the noise data from the pixel data to generate a final image.

18. The imaging system of claim 17, wherein the second processor subtracts the noise data from the pixel data to generate a final image only when an illumination level of the first image is less than a pre-determined threshold.

19. The imaging system of claim 17, wherein the digital image processor further comprises a lookup table in communication with the frame buffer and the second processor and operating to perform the subtraction of the noise data from the pixel data of the first image.

20. The imaging system of claim 17, wherein the digital image processor further comprises an arithmetic unit in communication with the frame buffer and the second processor and operating to perform the subtraction of the noise data from the pixel data of the first image.