Systems and methods for statistics collection using pixel mask

• US 9,317,930 B2
• Filed: 08/09/2013
• Issued: 04/19/2016
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Kuo, David Daming, Cote, Guy
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/963,641
• Publication Number:
  US 20140133749A1
• Time in Patent Office: 984 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81

Abstract

Systems and methods are provided for collecting image statistics using a pixel mask. In one example, statistics collection logic of an image signal processor may include a pixel weighting mask and accumulation logic. The pixel weighting mask may receive a first representation of a pixel that includes a luma and chroma representation of the pixel. The pixel weighting mask may output a pixel weighting using first and second chroma components of the luma and chroma representation of the pixel. The accumulation logic may receive the first or a second representation of the pixel and the pixel weighting value. Using these, the accumulation logic may weight the second representation of the pixel or the first representation of the pixel using the pixel weighting value to obtain a weighted pixel value, adding the weighted pixel value to a statistics count.

Claims

1. An image signal processor comprising:

statistics collection logic comprising;

a pixel weighting mask configured to;

receive a first representation of a pixel, wherein the first representation of the pixel comprises a luma and chroma representation of the pixel; and

output a pixel weighting value based at least partly on first and second chroma components of the luma and chroma representation of the pixel; and

accumulation logic configured to;

receive a second representation of the pixel or the first representation of the pixel;

receive the pixel weighting value;

weight the second representation of the pixel or the first representation of the pixel using the pixel weighting value to obtain a weighted pixel value; and

add the weighted pixel value to a statistics count;

wherein the pixel weighting mask comprises a two-dimensional weighting map, wherein the two-dimensional weighting map comprises a table indexed by a first index value and a second index value, wherein the first index value is based at least partly on the first chroma component and wherein the second index value is based at least partly on the second chroma component.

2. The image signal processor of claim 1, wherein the pixel weighting mask is configured to determine the first index value by scaling, offsetting, or scaling and offsetting, the first chroma component and determine the second index value by scaling, offsetting, or scaling and offsetting, the second chroma component.

3. The image signal processor of claim 1, wherein the pixel weighting mask is configured to output a weight value greater than zero only when a luma component of the first representation of the pixel falls is within a range of acceptable values.

4. The image signal processor of claim 1, wherein:

the statistics collection logic comprises histogram determination logic configured to ascertain a two-dimensional color histogram; and

the pixel weighting mask comprises an inverse of the two-dimensional color histogram.

5. An electronic device comprising:

an image sensor configured to obtain a first frame of image data comprising a first plurality of pixels;

a storage configured to store a second frame of image data comprising a second plurality of pixels; and

an image signal processor configured to receive and process the first plurality of pixels or the second plurality of pixels, or both, wherein the image signal processor comprises;

statistics collection logic configured to;

receive a first instance of the first or second plurality of pixels;

perform a first plurality of sequential processing operations on the pixels; and

obtain at least one set of image statistics by;

weighting the pixels using a two-dimensional weighting mask that comprises a lookup table indexed based at least partly on first and second chroma components of the pixels to obtain weighted pixel values; and

accumulating the weighted pixel values; and

an image processing pipeline configured to

receive a second instance of the first or second plurality of pixels; and

perform a second plurality of sequential operations on the pixels using the at least one set of image statistics from the statistics collection logic during at least one of the operations.

6. The electronic device of claim 5, wherein the pixel weighting mask lookup table is indexed based on first and second index values determined according to the following relationship:

C1idx=(C1_scale*(C1−

C1_offset))>

>

shift; and

C2idx=(C2_scale*(C2−

C2_offset))>

>

shift,

where C1 represents the first chroma component, C2 represents the second chroma component, C1idx represents the first index value, C2idx represents the second index value, C1_scale represents a scaling value associated with the first chroma component, C2_scale represents a scaling value associated with the second chroma component, C1_offset represents an offset value associated with the first chroma component, C2_offset represents an offset value associated with the second chroma component, and shift represents a value by which to perform a bit shift.

7. The electronic device of claim 6, wherein C1_offset, C2_offset, C1, and C2 comprise a first bit depth, C1_scale and C2_scale comprise a second bit depth greater than the first bit depth by one, and shift comprises a value equal to the first bit depth.

8. The electronic device of claim 6, wherein C1_scale and C2_scale comprise 17-bit unsigned values, C1_offset and C2_offset comprise 16-bit unsigned values, and shift represents a value of 16.

9. The electronic device of claim 5, wherein the pixel weighting mask lookup table is indexed based at least partly on scaled values of the first and second chroma components, wherein the scaled values of the first and second chroma components are scaled by a value of between 0 and 1.

10. The electronic device of claim 5, comprising data processing circuitry and one or more tangible, non-transitory machine-readable media comprising instructions, when executed by the data processing circuitry, to:

cause the weighting mask of the statistics collection logic of the image signal processor to be enabled to collect the at least one set of image statistics; and

receive the at least one set of image statistics into the data processing circuitry.

11. The electronic device of claim 10, comprising instructions to cause the statistics collection logic of the image signal processor to report a sum of horizontal and vertical positions within the image frame of pixels that satisfy the weighting mask and instructions to receive the report.

12. The electronic device of claim 11, comprising instructions to compute a centroid of a window or tile, or both, to which the at least one set of image statistics pertain, for the pixels that satisfy the weighting mask.

13. The electronic device of claim 10, comprising instructions to cause the statistics collection logic of the image signal processor to selectively collect statistics in an 8-bit pixel mode or a 16-bit pixel mode.

14. The electronic device of claim 13, wherein the instructions to receive the image statistics comprise instructions to receive the image statistics as 32-bit sums when in the 8-bit mode and receive the image statistics as 40-bit sums when in the 16-bit mode.

15. A method comprising:

using image statistics collection logic of an image signal processor;

receiving a pixel of a first RGB format into the image statistics collection logic;

converting the pixel of the first RGB format into a pixel of a second RGB format and a pixel of a first luma and chroma format that includes a first chroma component and a second chroma component;

determining a pixel weight using a two-dimensional pixel weighting mask that comprises a lookup table indexed by a first index value and a second index value, wherein the first index value is based at least partly on the first chroma component and wherein the second index value is based at least partly on the second chroma component;

weighting, using the determined pixel weight, the pixel of the second RGB format or the pixel of the first luma and chroma format to obtain at least one accumulation value; and

using the at least one accumulation value to obtain a tile accumulation statistic, a window accumulation statistic, or a combination thereof.

16. The method of claim 15, wherein the pixel of the first RGB format received by the image statistics collection logic comprises a pixel of a Bayer RGB format.

17. The method of claim 16, wherein converting the pixel of the first RGB format into the pixel of the second RGB format comprises:

receiving the pixel of the first RGB format among a window size of sensor pixels comprising a plurality of 2×

2 Bayer quads;

computing an average red (R) value of the pixel;

computing an average green (G) value of the pixel;

computing an average blue (B) value of the pixel; and

using the average red (R) value, the average green (G) value, and the average blue (B) value of the pixel to compose the pixel of the second RGB format, wherein the pixel of the second RGB format comprises a pixel of a camera RGB format.

18. The method of claim 15, wherein converting the pixel of the first RGB format into the pixel of the second RGB format comprises:

converting the pixel of the first RGB format, wherein the pixel of the first RGB format comprises a pixel of the Bayer RGB format, into a pixel of a camera RGB format; and

processing the pixel of the camera RGB format using a color correction matrix to obtain the pixel of the second RGB format, wherein the pixel of the second RGB format comprises a pixel of a linear sRGB calibrated format.

19. The method of claim 15, wherein converting the pixel of the first RGB format into the pixel of the second RGB format comprises:

converting the pixel of the first RGB format, wherein the pixel of the first RGB format comprises a pixel of the Bayer RGB format, into a pixel of a camera RGB format;

processing the pixel of the camera RGB format using a first color correction matrix to obtain a pixel of a linear sRGB calibrated format; and

processing the pixel of the linear sRGB calibrated format using a non-linear lookup table to obtain the pixel of the second RGB format, wherein the pixel of the second RGB format comprises a pixel of a gamma-corrected sRGB format.

20. The method of claim 15, wherein converting the pixel of the first RGB format into the pixel of the first luma and chroma format comprises:

converting the pixel of the first RGB format, wherein the pixel of the first RGB format comprises a pixel of a Bayer RGB format, into a pixel of a camera RGB format; and

processing the pixel of the camera RGB format using linear color space conversion logic configured to implement a camera color space conversion to obtain the pixel of the first luma and chroma format, wherein the pixel of the first luma and chroma format comprises a pixel of a camera luma and chroma format.

21. The method of claim 15, wherein converting the pixel of the first RGB format into the pixel of the first luma and chroma format comprises:

converting the pixel of the first RGB format, wherein the pixel of the first RGB format comprises a pixel of the Bayer RGB format, into a pixel of a camera RGB format;

processing the pixel of the camera RGB format using a first color correction matrix to obtain a pixel of a linear sRGB calibrated format;

processing the pixel of the linear sRGB calibrated format using a non-linear lookup table to obtain a pixel of a gamma-corrected sRGB format; and

processing the pixel of the gamma-corrected sRGB format using a second color correction matrix to obtain the pixel of the first luma and chroma component.

22. A method comprising:

using statistics collection logic of an image signal processor to obtain statistics for pixels of a tile or window, or both, by;

inputting the pixels of the tile or window, or both, one-by-one into a pixel weighting mask;

determining whether the pixel weighting mask is enabled or disabled;

when the pixel weighting mask is disabled, determining a pixel weighting value of 1 for all input pixels;

when the pixel weighting mask is enabled, determining the pixel weighting value of each input pixel using the pixel weighting mask by;

determining a first chroma index by scaling, offsetting, or scaling and offsetting, a first chroma component of the input pixel;

determining a second chroma index by scaling, offsetting, or scaling and offsetting, a second chroma component of the input pixel; and

when the first chroma index value is within a range of acceptable values, the second chroma index value is within a range of acceptable values, and a luma component of the input pixel is within a range of acceptable values, indexing a lookup table of the pixel weighting mask to the first chroma index and the second chroma index to obtain the pixel weighting value;

weighting the input pixel by the pixel weighting value to obtain a weighted pixel value; and

accumulating the weighted pixel value to obtain a pixel mask sum.

23. The method of claim 22, wherein the lookup table is indexed only when a luma component of the input pixel falls within a certain range of values.

24. The method of claim 22, comprising maintaining a count equal to a total number of accumulated weighted pixels to enable later computation of an average.

25. The method of claim 22, comprising maintaining a count equal to the sum of the pixel weighting values of the accumulated weighted pixels to enable later computation of a weighted average.