Image sensor with auto-focus and pixel cross-talk compensation

• US 9,584,743 B1
• Filed: 09/09/2014
• Issued: 02/28/2017
• Est. Priority Date: 03/13/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lin, Sheng, Kleekajai, Suppawan, Li, Xiangli
• Primary Examiner(s): Berardesca, Paul
• Application Number:
  US14/481,820
• Time in Patent Office: 903 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 25/62 Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
• CPC Class Codes: H04N 25/704 Pixels specially adapted for focusing, e.g. phase difference pixel sets

Abstract

A system and method for producing a digital image. The method may compensate for cross-talk between adjacent pixels and reduce the effect of auto-focus elements integrated into a pixel array of an image sensor. A set of coefficients is obtained, where the set of coefficients represent a relative measurement between two or more pixels in the pixel array and may be obtained from a calibration operation. A predictive function is constructed based on the set of coefficients. A compensated pixel value for at least one pixel of the image sensor is calculated using the predictive function. A digital image is created based in part on the compensated value.

Claims

1. A computer-implemented method for producing a digital image using an image sensor comprised of a pixel array, the method comprising:

obtaining, using a processor, a set of coefficients representing a relative measurement between;

an affected pixel that is proximate to an auto-focus pixel having a shield element and is optically affected by the shielding element,

and an unaffected pixel that is at least one additional pixel away from the auto-focus pixel as compared to the affected pixel;

constructing, using the processor, a predictive function based on the set of coefficients wherein the predictive function is constructed by performing a non-linear regression on at least part of the set of coefficients;

computing, using the processor, a compensated pixel value for at least one pixel of the image sensor using the predictive function; and

creating a digital image, using the image sensor, based in part on the compensated value.

2. The computer-implemented method of claim 1, wherein the at least one pixel is located adjacent to the auto-focus pixel that is at least partially shielded from light by the shield element.

3. The computer-implemented method of claim 2, wherein the predictive function is an approximation of the change in the at least one pixel due in part to optical interference caused by the shield element.

4. The computer-implemented method of claim 1, wherein:

the affected pixel is one of four pixels adjacent to the auto-focus pixel having the shield element; and

the unaffected pixel is at least one pixel away from the auto-focus pixel.

5. The computer-implemented method of claim 4, wherein the affected pixel receives additional reflected light due to the shield element.

6. The computer-implemented method of claim 4, wherein the unaffected pixel does not receive additional light due to the shield element.

7. The computer-implemented method of claim 1, wherein the predictive function is a polynomial function.

8. The computer-implemented method of claim 1, wherein the predictive function is a fourth-order polynomial function.

9. The computer-implemented method of claim 1, further comprising:

dividing the set of coefficients into a matrix of grid elements, wherein each grid element represents a specified number of pixels that are adjacent to each other in the pixel array;

obtaining two or more representative coefficients that correspond to two or more grid elements of the matrix of grid elements; and

constructing the predictive function using the two or more representative coefficients.

10. The computer-implemented method of claim 9, wherein each grid element of the matrix of grid elements represents a 16 by 16 group of pixels that are adjacent to each other.

11. A computer-implemented method for producing a digital image using an image sensor comprised of a pixel array, the method comprising:

obtaining, using a processor, a set of coefficients representing a relative measurement between;

an affected pixel that is positioned proximate to an auto-focus pixel and optically affected by a shielding element, and

an unaffected pixel that is positioned further away from the auto-focus pixel as compared to the affected pixel;

constructing, using the processor, a predictive function based on the set of coefficients wherein the predictive function is constructed by performing a non-linear regression on at least part of the set of coefficients

computing, using the processor, a compensated pixel value for at least one pixel of the image sensor using the predictive function; and

creating, using the image sensor, a digital image based in part on the compensated value.

12. The computer-implemented method of claim 11, wherein the at least one pixel is located adjacent to the auto-focus pixel that is at least partially shielded from light by the shield element.

13. The computer-implemented method of claim 11, wherein:

the affected pixel is proximate to the auto-focus pixel having the shield element; and

the unaffected pixel is at least one additional pixel away from the auto-focus pixel as the affected pixel.

14. The computer-implemented method of claim 13, wherein the shield element contributes to an amount of light received by the affected pixel.

15. The computer-implemented method of claim 13, wherein the shield element does not significantly contribute to an amount of light received by the unaffected pixel.

16. A computer-implemented method for producing a digital image using an image sensor comprised of a pixel array, the method comprising:

acquiring, using the image sensor, pixel data for a pixel array of the image sensor;

obtaining, using a processor, a predictive function based on a calibration operation wherein the predictive function is constructed by performing a non-linear regression on at least part of a set of coefficients which represent a difference between;

an affected pixel that is proximate to an auto-focus pixel and optically affected by an element of the image sensor, and

an unaffected pixel that is positioned further from the auto-focus pixel as compared to the affected pixel;

computing, using the processor, a compensated pixel value for at least one pixel of the image sensor using the predictive function; and

creating a digital image, using the processor, based in part on the compensated value.

17. The computer-implemented method of claim 16, further comprising:

dividing the pixel array into a matrix of grid elements, wherein each grid element represents a specified number of pixels that are adjacent to each other in the pixel array;

obtaining a representative pixel value for at least one grid element using the predictive function; and

computing a compensated pixel value for at least one pixel of the image sensor based on the representative pixel value.

18. The computer-implemented method of claim 17, wherein each grid element of the matrix of grid elements represents a 16 by 16 group of pixels that are adjacent to each other.

19. An electronic device having a digital camera, the digital camera comprising:

an image sensor having an array of pixels;

a memory having computer-readable instructions stored thereon; and

a processor configured to produce a digital image by executing the computer-readable instructions, the instructions for;

obtaining a set of coefficients representing a relative measurement between;

an affected pixel that is positioned proximate to a shield element and optically affected by the shield element, and

an unaffected pixel that is positioned further from the shield element as compared to the affected pixel;

constructing a predictive function based on the set of coefficients wherein the predictive function is constructed by performing a non-linear regression on at least part of the set of coefficients;

computing a compensated pixel value for at least one pixel of the image sensor using at least one coefficient of the set of coefficients; and

creating the digital image based in part on the compensated value.