TECHNIQUES FOR COLLECTION OF AUTO-FOCUS STATISTICS

• US 20120050566A1
• Filed: 09/01/2010
• Published: 03/01/2012
• Est. Priority Date: 09/01/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Côté, Guy, Frederiksen, Jeffrey E.
• Granted Patent: US 8,922,704 B2
• Time in Patent Office: Days
• US Class Current: 348/224.1
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

Various techniques are disclosed for collecting and processing auto-focus statistics data in an image signal processor (ISP). In one embodiment, a statistics collection engine in an ISP front-end processing unit may be configured to collect coarse (based on decimated raw data) and fine auto-focus statistics. Coarse auto-focus statistics may be collected on decimated Bayer RGB data and/or on linear camera luma values. Fine auto-focus statistics may be collected on raw Bayer RGB using a combination of a horizontal filter and edge detector, or may be collected on BayerY data (by applying a 3×1 transform to the raw Bayer RGB data). Edge sums may be accumulated using the filter outputs to determine auto-focus statistics.

Claims

1. An image signal processing system comprising:

a front-end pixel processing unit configured to receive a frame of raw image data acquired using a digital image sensor, wherein the front-end pixel processing unit comprises a statistics collection engine having auto-focus (AF) statistics having fine resolution filtering logic configured to process the raw image data to provide fine resolution AF statistics and coarse resolution filtering logic configured to process the raw image data to provide coarse AF statistics.

2. The image signal processing system of claim 1, wherein the fine resolution filtering logic comprises a horizontal filter configured to apply filtering to each color component of the raw image data.

3. The image signal processing system of claim 2, wherein the horizontal filter comprises a multi-tap filter configured to apply filtering to an input pixel of the raw image data, a first set of pixels having the same color as the input pixel and located horizontally on a first side of the input pixel, and a second set of pixels having the same color as the input pixel and located horizontally on a second side of the input pixel, wherein the number of pixels in the first and second set are equal.

4. The image signal processing system of claim 3, wherein the horizontal filter comprises a seven-tap filter.

5. The image signal processing system of claim 3, comprising edge detection logic configured to receive the output of the horizontal filter, and to determine an edge detection value based at least partially upon two pixels from the first set of pixels that are filtered by the horizontal filter and two pixels from the second set of pixels that are filtered by the horizontal filter.

6. The image signal processing system of claim 5, wherein the edge detection logic is configured to be bypassed depending on the state of a control signal.

7. The image signal processing system of claim 6, wherein:

if the edge detection logic is bypassed, the horizontal filter is configured as a high pass filter and the output of the horizontal filter is used to determine edge sum values for each color component of the raw image data; and

if the edge detection logic is not bypassed, the horizontal filter is configured as a low pass filter, and the output of the edge detection logic is used to determine edge sum values for each color component of the raw image data.

9. The image signal processing system of claim 1, wherein the coarse resolution filtering logic comprises at least a first and second filter configured to receive a camera space conversion luma value derived by decimating the raw image data and applying one or more linear or non-linear transforms, and to generate a luma edge value sum based on each luma pixel in the frame.

10. A method for collecting auto-focus statistics comprising:

receiving an image frame having raw image pixels acquired using an image sensor, the raw image pixels comprising red, blue, and green color components;

applying a horizontal filter to the raw image pixels to determine edge sum values for each of the red, blue, and green color components for the image frame;

applying a transform function to convert the raw image pixels into corresponding luma values;

applying first and second filters to the converted luma values and determining luma edge sum values for each of the first and second filters for the image frame;

decimating the raw image pixels and applying one or more linear or non-linear transforms to convert the raw image pixels into a camera luma and chroma color space; and

applying third and fourth filters to the camera luma (camY) values and determining luma edge sum values for each of the third and fourth filters for the image frame.

11. The method of claim 10, comprising configuring the horizontal filter as a low pass filter if the image frame is acquired under low light conditions and applying edge detection logic to the output of the horizontal filter.

12. The method of claim 11, applying the edge detection logic comprises:

determining a first value equal to the absolute value of the sum of the product of −

2 and the horizontal filter output of a first pixel located to the immediate left of a current pixel and having the same color and the product of 2 and the horizontal filter output of a second pixel located to the immediate right of the current pixel and having the same color;

determining a second value equal to the absolute value of the difference between the horizontal filter output of a third pixel having the same color as the current pixel and located to the to the immediate right of the second pixel and the horizontal filter output of a fourth pixel having the same color as the current pixel and located to the immediate left of the first pixel; and

summing the first and second values to obtain an edge value.

13. The method of claim 12, comprising clipping the edge value between a minimum value and a maximum value.

14. The method of claim 11, wherein the third and fourth filters for filtering the camY values comprise 3×

3 filters based upon Scharr operators.

15. A method for acquiring auto-focus statistics comprising:

determining coarse auto-focus (AF) statistics by applying first and second filters to a first signal comprising decimated image data and determining a first accumulated edge sum value corresponding to a first window using the output of the first filter and a second accumulated edge sum value corresponding to a second window based on the output of the second filter;

determining fine AF statistics by applying third and fourth filters to a second signal comprising non-decimated image data and determining a third accumulated edge sum value corresponding to a third window using the output of the third filter and a fourth accumulated edge sum value corresponding to a fourth window based on the output of the fourth filter.

16. The method of claim 15, wherein the first, second, third, and fourth filters comprise programmable 3×

3 filters.

17. The method of claim 15, wherein for each of the first, second, third, and fourth accumulated edge sum values, edge sum values are determined for each pixel by:

applying a respective filter to a current pixel and a set of neighboring pixels;

clipping the output of the respective filter;

determining an edge sum value by either squaring the clipped output or obtaining an absolute value of the clipped output; and

including the edge sum value in the accumulated edge sum value.

18. An electronic device comprising:

a digital image sensor;

an interface configured to communicate with the digital image sensor;

a memory device;

a display device configured to display a visual representation of an image scene corresponding to raw image data acquired by the digital image sensor; and

an image signal processing sub-system comprising a front-end pixel processing unit having a statistics collection engine comprising auto-focus statistics logic configured to collect a combination of fine and coarse auto-focus statistics over a plurality of windows.

19. The electronic device of claim 18, wherein for each window, an accumulated value is determined and stored as auto-focus statistic.

20. The electronic device of claim 19, comprising control logic configured to adjust a focal length of a lens associated with the image sensor to bring the raw image data into focus.

21. The electronic device of claim 19, wherein the accumulated value for each window is determined based upon one of either the value of a current input pixel, the filtered value of the current input pixel, or the square of the current input pixel.

22. The electronic device of claim 21, wherein at least two of the plurality of windows overlap with one another, and wherein a pixel variance corresponding to the two overlapping windows may be determined as the difference between the input pixel squared sum and the square of the sum of the input pixel values.

23. The electronic device of claim 18, wherein digital image sensor comprises at least one of a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via the interface, or some combination thereof.

24. The electronic device of claim 18, wherein the sensor interface comprises a Standard Mobile Imaging Architecture (SMIA) interface, a serial interface, or a parallel interface.

25. The electronic device of claim 18, comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.