Systems and methods for collecting fixed pattern noise statistics of image data

• US 8,817,120 B2
• Filed: 05/31/2012
• Issued: 08/26/2014
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Silverstein, D. Amnon, Lin, Sheng, Lim, Suk Hwan, Cote, Guy, Guo, Haitao
• Primary Examiner(s): SELBY, GEVELL V
• Application Number:
  US13/485,124
• Publication Number:
  US 20130321672A1
• Time in Patent Office: 817 Days
• Field of Search: 348/223.1, 348/241, 348/243
• US Class Current: 348/220.1
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response

Abstract

The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may collect statistics associated with fixed pattern noise of image data by receiving a first frame of the image data comprising a plurality of pixels. The image processing pipeline may then determine a sum of a first plurality of pixel values that correspond to at least a first portion of the plurality of pixels such that each pixel in at least the first portion of the plurality of pixels is disposed along a first axis within the frame of the image data. After determining the sum of the first plurality of pixel values, the image processing pipeline may store the sum of the first plurality of pixel values in a memory such that the sum of the first plurality of pixel values represent the statistics.

Claims

1. An image signal processing system comprising:

an image processing pipeline configured to collect statistics associated with fixed pattern noise of image data by;

receiving a first frame of the image data comprising a plurality of pixels acquired using a digital image sensor;

determining a sum of a first plurality of pixel values that correspond to at least a first portion of the plurality of pixels, wherein each pixel in at least the first portion of the plurality of pixels is disposed along a first axis within the frame of the image data and comprise the same color component, the color component comprising red, green, blue, cyan, yellow, or magenta; and

storing the sum of the first plurality of pixel values in a memory, wherein the sum of the first plurality of pixel values represent the statistics.

2. The image signal processing system of claim 1, wherein the first axis corresponds to a horizontal axis, a vertical axis, or a diagonal axis within the frame of the image data.

3. The image signal processing system of claim 2, wherein each pixel in at least the first portion of the plurality of pixels disposed along the horizontal axis is separated by a first number of pixels along the horizontal axis.

4. The image signal processing system of claim 2, wherein each pixel in at least the first portion of the plurality of pixels disposed along the vertical axis is separated by a first number of pixels along the vertical axis.

5. The image signal processing system of claim 2, wherein each pixel in at least the first portion of the plurality of pixels disposed along the diagonal axis is separated by a first number of pixels along the horizontal axis and by a second number of pixels along the vertical axis.

6. The image signal processing system of claim 5, wherein the first number and the second number correspond to fractional values.

7. The image signal processing system of claim of claim 1, wherein the sum of the first plurality of pixel values represents row-wise, column-wise, or diagonal-wise fixed pattern noise associated with the digital image sensor.

8. The image signal processing system of claim of claim 1, wherein the image processing pipeline is configured to:

receive a second frame of the image data;

determine a sum of a second plurality of pixel values that correspond to at least a second portion of the plurality of pixels, wherein each pixel in at least the second portion of the plurality of pixels is disposed along a second axis that is different from the first axis; and

store the sum of the second plurality of pixel values in the memory.

9. The image signal processing system of claim 1, wherein the statistics are configured to estimate a signal-to-noise ratio (SNR).

10. The image signal processing system of claim 1, wherein the image processing pipeline is configured to determine one or more parameters for noise filtering configurations based at least in part on the statistics.

11. The image signal processing system of claim 10, wherein the parameters comprise one or more noise filtering strengths, one or more noise filtering coefficients, or any combination thereof.

12. A method for processing image data comprising:

receiving an image frame comprising a plurality of pixels;

receiving an orientation with respect to the plurality of pixels by;

receiving a first value that corresponds to a horizontal pixel increment; and

receiving a second value that corresponds to a vertical pixel increment, wherein each adjacent pixel in the portion of the plurality of pixels is separated by the first value along a horizontal axis of the image frame and by the second value along a vertical axis of the image frame;

accumulating a first plurality of pixel values that correspond to a portion of the plurality of pixels disposed along an axis of the image frame parallel to the orientation; and

storing the accumulated first plurality of pixel values in a memory.

13. The method of claim 12, comprising receiving a color component, wherein each pixel in the portion of the plurality of pixels corresponds to the color component.

14. The method of claim 12, comprising:

accumulating a second plurality of pixel values that corresponds to a column of pixels in the plurality of pixels; and

storing the accumulated second plurality of pixel values in the memory.

15. The method of claim 12, comprising:

accumulating a second plurality of pixel values that corresponds to a row of pixels in the plurality of pixels; and

storing the accumulated second plurality of pixel values in the memory.

16. The method of claim 12, comprising:

accumulating a second plurality of pixel values that corresponds to a column of pixels in the plurality of pixels;

accumulating a third plurality of pixel values that corresponds to a row of pixels in the plurality of pixels; and

storing the accumulated second plurality of pixels and the accumulated third plurality of pixel values in the memory.

17. An image signal processing system comprising:

a statistics processing unit configured to receive a frame of image data, wherein the statistics processing unit comprises;

fixed pattern noise logic configured to;

divide the frame of image data into a plurality of strips of image data;

accumulate a first plurality of pixel values that correspond to a first plurality of pixels disposed along a first axis of one of the plurality of strips of image data;

store the accumulated first plurality of pixel values in a memory;

accumulate a second plurality of pixel values that correspond to a second plurality of pixels disposed along a second axis of the one of the plurality of strips of image data, wherein the second axis corresponds to a row of pixels in the one of the plurality of strips; and

store the accumulated second plurality of pixel values in the memory.

18. The image signal processing system of claim 17, wherein the plurality of strips of image data correspond to a plurality of horizontal strips of image data or a plurality of vertical strips of image data.

19. The image signal processing system of claim 17, wherein the each of the plurality of strips of image data is equal in height.

20. The image signal processing system of claim 17, wherein the first axis is in a horizontal, vertical, or diagonal direction.

21. The image signal processing system of claim 17, wherein the fixed pattern noise logic is further configured to:

accumulate a second plurality of pixel values that correspond to a second plurality of pixels disposed along a second axis of the one of the plurality of strips of image data, wherein the second axis corresponds to a column of pixels in the one of the plurality of strips; and

store the accumulated second plurality of pixel values in the memory.

22. An electronic device comprising:

a digital image sensor;

a sensor interface configured to communicate with the digital image sensor;

a memory device;

a display device configured to display a visual representation of an image scene corresponding to raw image data acquired by the digital image sensor; and

an image processing pipeline configured to;

receive the raw image data comprising a plurality of pixels;

receive an orientation with respect to the plurality of pixels;

determine fixed pattern noise statistics for the plurality of pixels by;

accumulating a first plurality of pixel values that correspond to a portion of the plurality of pixels disposed along an axis parallel to the orientation, the first plurality of pixels comprising a same color component, wherein the color component comprises;

red, green, blue, cyan, yellow, or magenta;

accumulating a second plurality of pixel values that corresponds to a column of pixels in the plurality of pixels;

accumulating a third plurality of pixel values that corresponds to a row of pixels in the plurality of pixels;

or

any combination thereof; and

store the accumulated first plurality of pixel values, the accumulated second plurality of pixel values, the accumulated third plurality of pixel values, or any combinations thereof in the memory device.

23. The electronic device of claim 22, wherein the digital image sensor comprises at least one of a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via an input/output port, or some combination thereof.

24. The electronic device of claim 22, wherein the sensor interface comprises a Standard Mobile Imaging Architecture (SMIA) interface, a serial imaging interface, a parallel imaging interface, or some combination thereof.

25. The electronic device of claim 22, comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.

26. A method for processing image data comprising:

receiving an image frame comprising a plurality of pixels;

receiving an orientation with respect to the plurality of pixels;

accumulating a first plurality of pixel values that correspond to a portion of the plurality of pixels disposed along an axis of the image frame parallel to the orientation by;

determining a color component and a position in the image frame for each pixel in the portion of the plurality of pixels;

generating an array of pixel values for the portion of the plurality of pixels, wherein the array is indexed based at least in part on a respective color component and a respective position in the image frame for a respective pixel in the portion of the plurality of pixels; and

summing the pixel values in the array or one or more pixel values in the array that correspond to a color component; and

storing the accumulated first plurality of pixel values in a memory.

27. An image signal processing system comprising:

a statistics processing unit configured to receive a frame of image data, wherein the statistics processing unit comprises;

fixed pattern noise logic configured to;

divide the frame of image data into a plurality of strips of image data;

accumulate a first plurality of pixel values that correspond to a first plurality of pixels disposed along a first axis of one of the plurality of strips of image data;

accumulate a second plurality of pixel values that correspond to a second plurality of pixels disposed along a second axis of the one of the plurality of strips of image data, wherein the second axis corresponds to a column of pixels in the one of the plurality of strips;

accumulate a third plurality of pixel values that correspond to a third plurality of pixels disposed along a third axis of the one of the plurality of strips of image data, wherein the third axis corresponds to a row of pixels in the one of the plurality of strips; and

store the accumulated first plurality of pixel values, the second plurality of pixel values, and the accumulated third plurality of pixel values in the memory.