Local image statistics collection

• US 9,077,943 B2
• Filed: 05/31/2012
• Issued: 07/07/2015
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lim, Suk Hwan, Silverstein, D. Amnon, Cote, Guy, Hordley, Steven David, Finlayson, Graham, Ku, Weichun, Bratt, Joseph P.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US13/484,741
• Publication Number:
  US 20130322745A1
• Time in Patent Office: 1,132 Days
• Field of Search: 382/162, 382/167, 382/168, 382/169, 382/272, 382/273, 348/223.1, 348/254, 348/259, 348/671, 348/672, 358/512, 358/518, 358/519, 358/520
• US Class Current: 1/1
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration
• CPC Class Codes: H04N 5/142 Edging; Contouring
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

Systems and methods for generating local image statistics are provided. In one example, an image signal processing system may include a statistics pipeline with image processing logic and local image statistics collection logic. The image processing logic may receive and process pixels of raw image data. The local image statistics collection logic may generate a local histogram associated with a luminance of the pixels of a first block of pixels of the raw image data or a thumbnail in which a pixel of the thumbnail represents a downscaled version of the luminance of the pixels of the first block of the pixel. The raw image data may include many other blocks of pixels of the same size as the first block of pixels.

Claims

1. A method for generating local image statistics of image data comprising:

receiving, via processing circuitry, input pixels of a block of the image data;

determining, via the processing circuitry, luminance values associated with the input pixels of the block of the image data; and

generating, via the processing circuitry, a local image statistic using the luminance values associated with the block of the image data or red, green, or blue subpixels of the block of the image data, or a combination thereof, wherein generating the local image statistic comprises;

downsampling the input pixels into a downsampled pixel with red, green, and blue components;

determining an average luminance of the red, green, and blue components of the downsampled pixel;

determining a maximal luminance of the red, green, or blue components of the downsampled pixel; and

mixing the average luminance and the maximal luminance according to a global mixing value applied to the input pixels.

2. The method of claim 1, wherein the image statistic comprises a local histogram of the block of the image data.

3. The method of claim 1, wherein the image statistic comprises a thumbnail in which the block of pixels corresponds to a single pixel of the thumbnail.

4. The method of claim 1, wherein the luminance values associated with the input pixels of the block of the image data comprise an average or weighted average luminance of all color channels of at least one of the input pixels, a maximal or weighted maximal luminance of at least one of the input pixels, a blend of the average or weighted average luminance or the maximal or weighted maximal luminance, or a logarithmic value of the average or weighted average luminance, the maximal or weighted maximal luminance, or the blend of the average or weighted average luminance or the maximal or weighted maximal luminance, or a combination thereof.

5. An image signal processing system comprising:

processing circuitry configured to perform statistics collection logic, wherein the statistics collection logic is configured to generate one or more local image statistics comprising;

a local histogram associated with a luminance of pixels of a first block of pixels of image data;

or

a thumbnail in which a pixel of the thumbnail represents a downscaled version of the luminance of the pixels of the first block of the pixels;

or

both the local histogram and the thumbnail;

wherein the image data comprises a plurality of other blocks of pixels of the same size as the first block of pixels;

wherein the statistics collection logic comprises luminance computation logic to determine the luminance, wherein the luminance computation logic comprises;

downsample logic configured to downsample pixels of the image data into a downsampled pixel with red, green, and blue components;

average luminance logic configured to determine the average luminance of the red, green, and blue components of the downsampled pixel;

maximal luminance logic configured to determine the maximal luminance of the red, green, or blue components of the downsampled pixel; and

mixing logic configured to mix the average luminance and the maximal luminance according to a global mixing value applied to all pixels.

6. The image signal processing system of claim 5, wherein the luminance computation logic comprises mixing logic configured to mix the average luminance or the maximal luminance according to a mixing value that varies based at least in part on the maximal luminance or the average luminance.

7. The image signal processing system of claim 5, wherein the luminance computation logic comprises mixing logic configured to mix the average luminance or the maximal luminance according to a mixing value, wherein the mixing value is configured to be obtained from a lookup table.

8. The image signal processing system of claim 7, wherein the processing circuitry comprises selection logic configured to index the lookup table using the average luminance or the maximal luminance.

9. The image signal processing system of claim 7, wherein the lookup table is configured to provide a mixing value that provides more of the average luminance than the maximal luminance when the average luminance or the maximal luminance, or both, are relatively low, and more of the maximal luminance when the average luminance or the maximal luminance, or both, are relatively high.

10. The image signal process of claim 5, wherein the luminance computation logic comprises logarithmic computation logic configured to convert the maximal luminance, the average luminance, or a combination thereof, into a logarithmic format.

11. The image signal processing system of claim 5, wherein the statistics collection logic comprises luminance computation logic to determine the luminance, wherein the luminance computation logic comprises:

average luminance logic configured to determine the average luminance of the red, green, and blue components of a pixel;

maximal luminance logic configured to determine the maximal luminance of the red, green, or blue components of the pixel;

or

minimal luminance logic configured to determine the minimal luminance of the red, green, or blue components of the pixel;

or

a combination thereof.

12. An electronic device comprising:

an imaging device configured to obtain Bayer image data comprising a plurality of blocks of pixels; and

processing circuitry comprising image statistics collection logic configured to;

downsample the Bayer image data into pixels of three color components; and

generate a thumbnail statistic in which each pixel of the thumbnail statistic corresponds to a luminance or a color component of the downsampled pixels of one of the blocks of pixels, wherein the image statistics collection logic is configured to generate a plurality of thumbnail statistics, wherein each of the plurality of thumbnail statistics is determined based on an average luminance, a maximal luminance, a blend of the average luminance and the maximal luminance, a logarithmic luminance, a red component, a green component, or a blue component of each of the downsampled pixels of each block of pixels.

13. The electronic device of claim 12, wherein the image statistics collection logic comprises subsampling logic configured to subsample the luminance or the color component of each block of pixels to obtain the pixels of the thumbnail statistic.

14. The electronic device of claim 12, wherein the image statistics collection logic comprises block averaging logic configured to average the luminance or the color component of each block of pixels to obtain the pixels of the thumbnail statistic.

15. The electronic device of claim 12, wherein the image statistics collection logic comprises block minimum logic configured to determine the minimal luminance or the minimal color component of each block of pixels as the pixels of the thumbnail statistic.

16. The electronic device of claim 12, wherein the image statistics collection logic comprises block maximum logic configured to determine the maximal luminance or the maximal color component of each block of pixels as the pixels of the thumbnail statistic.

17. The electronic device of claim 12, wherein the imaging device comprises a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via an input/output port, or some combination thereof.

18. The electronic device of claim 12, comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.

19. An image signal processing system comprising:

processing circuitry comprising luminance computation logic comprising;

downsample logic configured to downsample pixels of raw image data into a downsampled pixel with red, green, and blue components;

average luminance logic configured to determine an average luminance of the red, green, and blue components of the downsampled pixel;

maximal luminance logic configured to determine a maximal luminance of the red, green, or blue components of the downsampled pixel; and

mixing logic configured to blend the average and maximal luminance according to proportions that vary depending on the value of the average luminance, the maximal luminance, or a combination thereof; and

statistics collection logic configured to generate image statistics using the average luminance, the maximal luminance, the blend of the average luminance and the maximal luminance, or a value based on any of these values.

20. The image signal processing system of claim 19, wherein the mixing logic is configured to blend a more of the average luminance when the average luminance or the maximal luminance, or a combination thereof, is lower than otherwise.

21. The image signal processing system of claim 19, wherein the mixing logic is configured to blend a more of the maximal luminance when the average luminance or the maximal luminance, or a combination thereof, is higher than otherwise.

22. The image signal processing system of claim 19, comprising a lookup table configured to indicate the proportions to blend the maximal luminance and the average luminance based on the maximal luminance or the average luminance.

23. The image signal processing system of claim 19, comprising minimal luminance logic configured to determine a minimal luminance of the red, green, or blue components of the downsampled pixel, wherein:

the mixing logic is configured to blend the average luminance, the maximal luminance, or the minimal luminance, or a combination thereof, according to proportions that vary depending on the value of the average luminance, the maximal luminance, or the minimal luminance, or a combination thereof; and

the statistics collection logic is configured to generate the image statistics using the average luminance, the maximal luminance, the minimal luminance, the blend, or a value based on any of these values.