SYSTEMS AND METHOD FOR REDUCING FIXED PATTERN NOISE IN IMAGE DATA

• US 20130321671A1
• Filed: 05/31/2012
• Published: 12/05/2013
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cote, Guy, Silverstein, D. Amnon, Lim, Suk Hwan, Lin, Sheng, Guo, Haitao
• Granted Patent: US 11,089,247 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response
• CPC Class Codes: H04N 25/673 by using reference sources
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Abstract

The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Claims

1. An image signal processing system comprising:

an image processing pipeline configured to reduce fixed pattern noise in image data by;

receiving a frame of the image data comprising a plurality of pixels acquired using a digital image sensor;

determining a first plurality of correction factors configured to correct each pixel in the plurality of pixels for fixed pattern noise, wherein the first plurality of correction factors is determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data; and

applying the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

2. The image signal processing system of claim 1, wherein each correction factor in the first plurality of correction factors comprises an offset value and a gain value.

3. The image signal processing system of claim 2, wherein the offset value is retrieved from an offset look-up table comprising a plurality of offset values, wherein the offset look-up table is indexed according to a fixed pattern noise frame for a respective pixel.

4. The image signal processing system of claim 3, wherein the fixed pattern noise frame comprises a plurality of packed bits that encode two offset values and a gain value.

5. The image signal processing system of claim 2, wherein the gain value is retrieved from a gain look-up table comprising a plurality of gain values, wherein the gain look-up table is indexed according to a fixed pattern noise frame for a respective pixel.

6. The image signal processing system of claim 1, wherein the image processing pipeline is configured to:

determine a second plurality of correction factors configured to correct each row of pixels in the plurality of pixels for row fixed pattern noise; and

apply the second plurality of correction factors to the plurality of pixels.

7. The image signal processing system of claim 6, wherein the second plurality of correction factors is determined based at least in part on row fixed pattern noise statistics that correspond to the frame of the image data.

8. The image signal processing system of claim 1, wherein the image processing pipeline is configured to:

determine a second plurality of correction factors configured to correct each column of pixels in the plurality of pixels for column fixed pattern noise; and

apply the second plurality of correction factors to the plurality of pixels.

9. The image signal processing system of claim 8, wherein the second plurality of correction factors is determined based at least in part on column fixed pattern noise statistics that correspond to the frame of the image data.

10. The image signal processing system of claim 1, wherein the image processing pipeline is configured to:

determine a second plurality of correction factors configured to correct each row of pixels in the plurality of pixels for row fixed pattern noise;

determine a third plurality of correction factors configured to correct each column of pixels in the plurality of pixels for column fixed pattern noise; and

apply the second plurality of correction factors and the third plurality of correction factors to the plurality of pixels.

11. The image signal processing system of claim 1, wherein the first plurality of correction factors is determined based at least in part on a temperature value that corresponds to the digital image sensor, an integration time that corresponds to the digital image sensor, or any combination thereof.

12. A method for processing image data comprising:

receiving an image frame comprising a plurality of pixels;

determining a first plurality of correction factors configured to correct each pixel in the plurality of pixels for fixed pattern noise, wherein the first plurality of correction factors is determined based at least in part on fixed pattern noise statistics that correspond to the frame of image data; and

applying the first plurality of correction factors to the plurality of pixels.

13. The method of claim 12, wherein determining the first plurality of correction factors comprises:

retrieving a first offset value from an offset look-up table comprising a plurality of offset values, wherein the offset look-up table is indexed according to a fixed pattern noise frame for a respective pixel;

retrieving a second offset value from the offset look-up table;

applying a first weighting factor to the first offset value, thereby generating a first weighted offset value;

applying a second weighting factor to the second offset value, thereby generating a second weighted offset value; and

adding the first weighted offset value and the second offset weighted value, thereby generating an offset value for the respective pixel.

14. The method of claim 13, wherein the first offset value corresponds to least significant bits of the fixed pattern noise frame, and wherein the second offset value corresponds to a number of bits after the least significant bits in the fixed pattern noise frame.

15. The method of claim 12, wherein determining the first plurality of correction factors comprises retrieving a gain value from a gain look-up table comprising a plurality of gain values, wherein the gain look-up table is indexed according to a fixed pattern noise frame for a respective pixel.

16. The method of claim 12, wherein determining the first plurality of correction factors comprises:

retrieving a first offset value from an offset look-up table comprising a plurality of offset values, wherein the offset look-up table is indexed according to a fixed pattern noise frame for a respective pixel;

retrieving a second offset value from the offset look-up table;

applying a first weighting factor to the first offset value, thereby generating a first weighted offset value;

applying a second weighting factor to the second offset value, thereby generating a second weighted offset value;

adding the first weighted offset value and the second offset weighted value, thereby generating an offset value for the respective pixel; and

retrieving a gain value from a gain look-up table comprising a plurality of gain values, wherein the gain look-up table is indexed according to a fixed pattern noise frame for the respective pixel.

17. The method of claim 16, wherein applying the first plurality of correction factors comprises applying the offset value and the gain value to a respective pixel value of the respective pixel.

18. The method of claim 12, further comprising:

determining a second plurality of correction factors configured to correct each row of pixels in the plurality of pixels for row fixed pattern noise; and

applying the second plurality of correction factors to the plurality of pixels.

19. The method of claim 18, wherein the second plurality of correction factors corresponds to a portion of the plurality of pixels having the same color component.

20. The method of claim 12, further comprising:

determining a second plurality of correction factors configured to correct each column of pixels in the plurality of pixels for column fixed pattern noise; and

applying the second plurality of correction factors to the plurality of pixels.

21. The method of claim 20, wherein the second plurality of correction factors corresponds to a portion of the plurality of pixels having the same color component.

22. The method of claim 12, further comprising:

receiving one or more global offset values; and

applying the global offset values to the plurality of pixels before applying the first plurality of correction factors to the plurality of pixels.

23. The method of claim 22, further comprising:

receiving one or more global offset values; and

applying the global offset values to the plurality of pixels after applying the first plurality of correction factors to the plurality of pixels.

24. An image signal processing system comprising:

a statistics processing unit configured to receive a frame of image data, wherein the statistics processing unit comprises;

fixed pattern noise reduction logic configured to;

receive the frame of image data comprising a plurality of pixels;

determine a first plurality of correction factors configured to correct each pixel in the plurality of pixels for fixed pattern noise, wherein the first plurality of correction factors is determined based at least in part on fixed pattern noise statistics that correspond to the frame of image data; and

determine a second plurality of correction factors configured to correct each row of pixels in the plurality of pixels for row fixed pattern noise;

or

determine a third plurality of correction factors configured to correct each column of pixels in the plurality of pixels for column fixed pattern noise; and

apply the first plurality and the second plurality of correction factors or the third plurality of correction factors to the plurality of pixels.

25. The image signal processing system of claim 24, wherein the fixed pattern noise reduction logic is further configured to apply one or more global offset values to the plurality of pixels before applying the first plurality of correction factors and the second plurality of correction factors or the third plurality of correction factors to the plurality of pixels.

26. The image signal processing system of claim 24, wherein the fixed pattern noise reduction logic is further configured to:

determine the second plurality of correction factors and the third plurality of correction factors; and

apply the first plurality of correction factors, the second plurality of correction factors, and the third plurality of correction factors to the plurality of pixels.

27. The image signal processing system of claim 24, wherein the second plurality of correction factors is determined based at least in part on row fixed pattern noise statistics that correspond to one or more rows of pixels in the image data.

28. The image signal processing system of claim 24, wherein the third plurality of correction factors is determined based at least in part on column fixed pattern noise statistics that correspond to one or more columns of pixels in the image data.

29. An electronic device comprising:

a digital image sensor;

a sensor interface configured to communicate with the digital image sensor;

a memory device;

a display device configured to display a visual representation of an image scene corresponding to raw image data acquired by the digital image sensor; and

an image processing pipeline configured to;

receive the raw image data comprising a plurality of pixels;

determine a first plurality of correction factors configured to correct each pixel in the plurality of pixels for fixed pattern noise, wherein the first plurality of correction factors is determined based at least in part on fixed pattern noise statistics that correspond to the raw image data; and

apply the first plurality of correction factors to the plurality of pixels.

30. The electronic device of claim 29, wherein the digital image sensor comprises at least one of a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via an input/output port, or some combination thereof.

31. The electronic device of claim 29, wherein the sensor interface comprises a Standard Mobile Imaging Architecture (SMIA) interface, a serial imaging interface, a parallel imaging interface, or some combination thereof.

32. The electronic device of claim 29, comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.

33. An image signal processing system comprising:

an image processing pipeline configured to reduce fixed pattern noise in image data by;

receiving a frame of the image data comprising a plurality of pixels acquired using a digital image sensor;

determining a first plurality of correction factors configured to correct each pixel in the plurality of pixels for fixed pattern noise, wherein the first plurality of correction factors is determined based at least in part on a temperature value of the digital image sensor, an integration time of the digital image sensor, or any combination thereof; and

applying the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

34. The image signal processing system of claim 33, wherein determining the first plurality of correction factors comprises retrieving an offset value and a gain value for each pixel in the plurality of pixels from a look-up table that corresponds to the temperature value.

35. The image signal processing system of claim 34, wherein retrieving the offset value and the gain value from the look-up table comprises:

interpolating the temperature value based at least in part on one or more index values in the look-up table; and

retrieving the offset value and the gain value for each pixel from the look-up table based at least in part on the interpolated temperature value.

36. The image signal processing system of claim 33, wherein determining the first plurality of correction factors comprises retrieving an offset value and a gain value for each pixel in the plurality of pixels from a look-up table that corresponds to the integration time.

37. The image signal processing system of claim 34, wherein retrieving the offset value and the gain value from the look-up table comprises:

interpolating the integration time based at least in part on one or more index values in the look-up table; and

retrieving the offset value and the gain value for each pixel from the look-up table based at least in part on the interpolated integration time.