Dynamic range estimation with fast and slow sensor pixels

• US 10,447,951 B1
• Filed: 04/11/2018
• Issued: 10/15/2019
• Est. Priority Date: 04/11/2018
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Lee, Jisoo
• Primary Examiner(s): Giles, Nicholas G
• Application Number:
  US15/950,945
• Publication Number:
  US 20190320128A1
• Time in Patent Office: 552 Days
• CPC Class Codes: H01L 27/14601 Structural or functional details thereof
• CPC Class Codes: H01L 27/14605 Structural or functional details relating to the position of the pixel elements, e.g. smaller pixel elements in the center of the imager compared to pixel elements at the periphery
• CPC Class Codes: H01L 27/14607 Geometry of the photosensitive area
• CPC Class Codes: H01L 27/14609 Pixel-elements with integrated switching, control, storage or amplification elements scanning details of imagers (circuitry of solid-state image sensors H04N25/00); circuitry of imagers H04N25/70
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14623 Optical shielding
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14654 Blooming suppression
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/585 with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures

Abstract

A scene can be captured by integrating a first sensor pixel for a first amount of time to produce an original first photometric and integrating a second sensor pixel for the first amount of time to produce an original second photometric. The first sensor pixel can be configured to saturate with photocharge slower than the second sensor pixel. The scene can be recaptured by integrating the second sensor pixel for a second amount of time less than the first amount of time.

Claims

1. A method, comprising:

capturing a scene by integrating a first sensor pixel for a first amount of time to produce an original first photometric and integrating a second sensor pixel for the first amount of time to produce an original second photometric, the first sensor pixel being configured to saturate with photocharge slower than the second sensor pixel;

normalizing the original first photometric with respect to the second sensor pixel; and

re-capturing the scene by integrating the second sensor pixel for a second amount of time to produce a newer second photometric, the second amount of time being based on the normalized original first photometric, and the second amount of time being less than the first amount of time.

2. The method of claim 1, further comprising:

prior to re-capturing the scene, determining whether the original second photometric has been clipped based on the normalized original first photometric.

3. The method of claim 2, wherein determining whether the original second photometric has been clipped based on the normalized original first photometric comprises

determining whether the normalized original first photometric would oversaturate the second sensor pixel.

4. The method of claim 3, further comprising:

comparing the normalized original first photometric and a saturation capacity of the second sensor pixel; and

setting the second amount of time based on the comparison.

5. The method of claim 1, wherein the first sensor pixel and the second sensor pixel are configured to have equal photocharge capacity, the first sensor pixel comprises shielding, and the first sensor pixel is configured to more slowly saturate with photocharge than the second sensor pixel by virtue of the shielding.

6. The method of claim 5, wherein the shielding is applied directly over a photodiode of the first sensor pixel.

7. The method of claim 6, wherein the first sensor pixel comprises a plurality of photodiodes and the method further comprises:

performing phase detection autofocus based on the first sensor pixel.

8. The method of claim 1, further comprising:

prior to re-capturing the scene, determining whether the original second photometric is greater than or equal to a maximum photocharge capacity of the second sensor pixel;

based on an affirmative determination, determining whether the original second photometric has been clipped based on the original first photometric.

9. A method comprising:

capturing a scene with a scene dynamic range by integrating a plurality of first sensor pixels for a first amount of time to produce a plurality of original first photometrics and integrating a plurality of second sensor pixels for the first amount of time to produce a plurality of original second photometrics, each of the plurality of first sensor pixels being configured to saturate with photocharge slower than each of the plurality of second sensor pixels;

normalizing the plurality of original first photometrics with respect to the plurality of second sensor pixels;

determining whether at least some of the plurality of second sensor pixels clipped a top end of the scene dynamic range based on the normalized plurality of original first photometrics; and

based on determining that at least some of the plurality of second sensor pixels clipped the top end of the scene dynamic range, re-capturing the scene by integrating the plurality of second sensor pixels for a second amount of time to produce a plurality of newer second photometrics, the second amount of time being less than the first amount of time.

10. The method of claim 9, comprising setting the second amount of time based on the first amount of time, the plurality of original first photometrics, and the plurality of original second photometrics.

11. A processing system comprising one or more processors configured to:

capture a scene by integrating a first sensor pixel for a first amount of time to produce an original first photometric and by integrating a second sensor pixel for the first amount of time to produce an original second photometric, the first sensor pixel being configured to saturate with photocharge slower than the second sensor pixel;

normalize the original first photometric with respect to the second sensor pixel; and

re-capture the scene by integrating the second sensor pixel for a second amount of time to produce a newer second photometric, the second amount of time being based on the normalized original first photometric, and the second amount of time being less than the first amount of time.

12. The system of claim 11, wherein the one or more processors are configured to:

determine whether the original second photometric has been clipped based on the normalized original first photometric prior to recapturing the scene.

13. The system of claim 12, wherein the one or more processors are configured to:

determine whether the original second photometric has been clipped based on the original first photometric by

determining whether the normalized original first photometric would oversaturate the second sensor pixel.

14. The system of claim 13, wherein the one or more processors are configured to:

compare the normalized original first photometric and a saturation capacity of the second sensor pixel; and

set the second amount of time based on the comparison.

15. The system of claim 11, wherein the first sensor pixel and the second sensor pixel are configured to have equal photocharge capacity and the first sensor pixel is configured to more slowly saturate with photocharge than the second sensor pixel by virtue of shielding applied in the first sensor pixel.

16. The system of claim 15, wherein the shielding is applied directly over a portion of a photodiode of the first sensor pixel.

17. The system of claim 16, wherein the first sensor pixel comprises a plurality of photodiodes and the one or more processors are configured to:

perform phase detection autofocus based on the first sensor pixel.

18. The system of claim 11, wherein the one or more processors are configured to:

prior to re-capturing the scene, determine whether the original second photometric is greater than or equal to a maximum photocharge capacity of the second sensor pixel;

based on an affirmative determination, determine whether the original second photometric has been clipped based on the original first photometric.

19. The system of claim 11, wherein the one or more processors are configured to compute the second amount of time based on a difference between the original first photometric, after being scaled, and a dynamic range ceiling.

20. The system of claim 11, wherein the one or more processors are configured to compute the second amount of time based on the original first photometric after being normalized to a scale of the original second photometric, the first sensor pixel having a different structure than the second sensor pixel, the first sensor pixel and the second sensor pixel both having the same kind of color filter.