AUTO-FOCUS CONTROL USING IMAGE STATISTICS DATA WITH COARSE AND FINE AUTO-FOCUS SCORES

• US 20120051730A1
• Filed: 09/01/2010
• Published: 03/01/2012
• Est. Priority Date: 09/01/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Côté, Guy, Frederiksen, Jeffrey E.
• Granted Patent: US 9,398,205 B2
• Time in Patent Office: Days
• US Class Current: 396/90
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 3/10 Power-operated focusing
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

Techniques are provided for determining an optimal focal position using auto-focus statistics. In one embodiment, such techniques may include generating coarse and fine auto-focus scores for determining an optimal focal length at which to position a lens associated with the image sensor. For instance, the statistics logic may determine a coarse position that indicates an optimal focus area which, in one embodiment, may be determined by searching for the first coarse position in which a coarse auto-focus score decreases with respect to a coarse auto-focus score at a previous position. Using this position as a starting point for fine score searching, the optimal focal position may be determined by searching for a peak in fine auto-focus scores. In another embodiment, auto-focus statistics may also be determined based on each color of the Bayer RGB, such that, even in the presence of chromatic aberrations, relative auto-focus scores for each color may be used to determine the direction of focus.

Claims

1. An image signal processing system comprising:

a front-end pixel processing unit configured to receive a frame of raw image data comprising pixels acquired using an imaging device having a digital image sensor, wherein the front-end pixel processing unit comprises a statistics collection engine having auto-focus statistics logic configured to process the raw image data to collect coarse and fine auto-focus statistics; and

control logic configured to determine an optimal focal position of a lens of the imaging device using coarse and fine auto-focus scores based upon the coarse and fine auto-focus statistics and to adjust the focal position of the lens between a minimum position and a maximum position defining a total focal length to reach the optimal focal position.

2. The image signal processing system of claim 1, wherein the control logic is configured to determine the optimal focal position of the lens by:

stepping the focal position across a plurality of coarse score positions along the total focal length in a first direction beginning from the minimum position and ending at the maximum position;

determining a coarse auto-focus score for each of the plurality of coarse score positions;

identifying which of the plurality of coarse score positions has a corresponding coarse auto-focus score that decreased relative to a coarse auto-focus score corresponding to the immediately previous coarse score position;

beginning from the identified coarse score position, stepping the focal position in a second direction opposite the first direction and back towards the minimum position across a plurality of fine score positions;

determining a fine auto-focus score for each of the plurality of fine score positions; and

identifying which of the plurality of fine score positions corresponds to a peak in the fine auto-focus scores and setting the identified fine score position as the optimal focal position.

3. The image signal processing system of claim 2, wherein the step size between each of the plurality of coarse score positions is greater than the step size between each of the plurality of fine score positions.

4. The image signal processing system of claim 2, wherein the step size between each of the coarse score positions is variable based at least partially upon the magnitude of the change in coarse auto-focus scores corresponding to adjacent coarse score positions.

5. The image signal processing system of claim 4, wherein the step size between coarse score positions decreases as the magnitude of the change in coarse auto-focus scores corresponding to the coarse score positions decreases.

6. The image signal processing system of claim 2, wherein the control logic adjust the focal position of the lens using a coil, and wherein the control logic steps across coarse score positions along the total focal length to counter the effects of coil settling times.

7. The image signal processing system of claim 1, wherein the auto-focus statistics logic is configured to provide coarse auto-focus statistics by applying first and second filters to least one of camera luma values derived from the raw image data after being decimated or from decimated raw image data and to provide fine auto-focus statistics by either applying third and fourth filters to luma values obtained by applying a transform to the raw image data or by applying horizontal filtering to the raw image data.

8. The image signal processing system of claim 6, wherein the coarse auto-focus scores at each coarse position is determined based at least partially upon the sum of the outputs of the first and second filters, and wherein the fine auto-focus scores at each fine position is determined based at least partially upon the sums of the outputs of the third and fourth filters.

9. The image signal processing system of claim 6, wherein the first and second filters for filtering the camera luma values comprise 3×

3 filters based upon Scharr operators.

10. A method comprising:

receiving raw image data acquired using a digital image sensor, the raw image data representing an image scene and having red, blue, and green color components;

determining auto-focus scores corresponding to each of the red, blue, and green color components; and

selecting a focal adjustment direction based upon the relativity of the auto-focus scores corresponding to each of the red, blue, and green color components.

11. The method of claim 10, wherein an optimal focal position for the image scene corresponds to a position where the green color components has an optimal focal position.

12. The method of claim 11, comprising selecting the focal adjustment direction such that, if the red or blue color components exhibit a higher auto-focus score than the green color component at a particular focal position, focusing in the selected focal adjustment direction lowers the auto-focus score for the red or blue color components while increases the auto-focus score for the green color component.

13. The method of claim 10, wherein the raw image data comprises Bayer RGB data.

14. The method of claim 10, wherein the auto-focus scores corresponding to each of the red, blue, and green color components are obtained based upon statistics acquired by applying a multi-tap horizontal filter to the raw image data.

15. The method of claim 14, wherein the multi-tap horizontal filter comprises a seven-tap filter.

16. A method comprising:

determining coarse auto-focus scores at various steps along a focal length of a lens of an image capture device;

identifying a step at which a corresponding auto-focus score decreases relative to a previous step;

identifying a optimal focal area in the vicinity of a focal position; and

analyzing fine auto-focus scores within the optimal focal area to determine an optimal focal position for the lens.

17. The method of claim 16, wherein determining analyzing fine auto-focus scores within the optimal focal area to determine the optimal focal position comprises searching a focal position that provides a maximum fine auto-focus score within the optimal focal area.

18. The method of claim 16, wherein the coarse and fine auto-focus scores are based on white balanced luma derived from Bayer RGB data.

19. The method of claim 18, wherein the white balanced luma values for the coarse auto-focus scores are derived from decimated Bayer RGB data.

20. An electronic device comprising:

an imaging device comprising a digital image sensor and a lens;

an interface configured to communicate with the digital image sensor;

a memory device;

a display device configured to display a visual representation of an image scene corresponding to raw image data acquired by the digital image sensor; and

an image signal processing sub-system comprising a front-end pixel processing unit configured to receive a frame of raw image data comprising pixels acquired using an imaging device having a digital image sensor, wherein the front-end pixel processing unit comprises a statistics collection logic having auto-focus statistics logic configured to process the raw image data to collect coarse and fine auto-focus statistics; and

control logic configured to determine an optimal focal position of the lens of the imaging device using coarse and fine auto-focus scores based upon the coarse and fine auto-focus statistics, respectively, wherein the control logic determines the optimal focal position of the lens by determining a coarse auto-focus score for each of a plurality of coarse score positions along the total focal length in a first direction, identifying which of the plurality of coarse score positions has a corresponding coarse auto-focus score that decreased relative to a coarse auto-focus score corresponding to the immediately previous coarse score position, stepping the focal position across one or more fine score positions in a second direction opposite the first direction beginning from the identified coarse score position and searching for a peak in the fine auto-focus scores, and setting the focal position corresponding to the peak as the optimal focal position.

21. The electronic device of claim 20, wherein the step size between each of the plurality of coarse score positions is generally constant.

22. The electronic device of claim 21, wherein the step size between each of the fine score positions is generally constant, but less than the step size between each of the coarse score positions.

23. The electronic device of claim 20, wherein the sensor interface comprises a Standard Mobile Imaging Architecture (SMIA) interface.

24. The electronic device of claim 20, comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.

25. The electronic device of claim 20, wherein digital image sensor comprises at least one of a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via the interface, or some combination thereof.