High dynamic range image sensor

• US 8,994,843 B2
• Filed: 09/01/2010
• Issued: 03/31/2015
• Est. Priority Date: 09/01/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Atanassov, Kalin M., Velarde, Ruben M., Hwang, Hau
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US12/873,981
• Publication Number:
  US 20120050557A1
• Time in Patent Office: 1,672 Days
• Field of Search: 348/259, 348/333.08, 348/335, 348/340, 348/343, 348/344, 348/369, 348/362, 348/229.1, 348/221.1, 348/208.12, 348/222.1, 348/E05.024, 348/154, 348/208.1, 348/208.4, 348/208.13
• US Class Current: 348/222.1
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures
• CPC Class Codes: H04N 25/77 Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components

Abstract

This disclosure describes techniques for producing high dynamic range images by applying a variable weighting factor to a sample prior to combining the sample with another sample. In one example, a method includes sampling a first pixel cell signal at a first time to produce a first sample, sampling a second pixel cell signal at a second time to produce a second sample, applying a variable weighting factor to the second sample, wherein the variable weighting factor is defined based on a function, and combining the first sample and the weighted second sample.

Claims

1. A method comprising:

sampling a first pixel cell signal at a first time to produce a first sample;

sampling a second pixel cell signal at a second time to produce a second sample;

applying a first variable weighting factor to the first sample, wherein the first variable weighting factor is defined based on a first function;

applying a second variable weighting factor to the second sample, wherein the second variable weighting factor is defined based on a second function; and

combining the weighted first sample and the weighted second sample according to the equation;

S_COMBINED(i, j)=S_LONG(i, j)*M(i, j)+n*S_SHORT(i, j)*(1−

M(i, j))

where S_COMBINED denotes the combination of the first sample, denoted by S_SHORT, and the second sample, denoted by S_LONG, for the pixel cell located at (i, j), n denotes a scalar value that is dependent on the difference in the time of exposure between the first time and the second time, and M(i, j) denotes the variable weighting factor.

2. The method of claim 1, wherein the value of M(i, j) is between zero and one, inclusive.

3. The method of claim 1, wherein the second time is temporally subsequent to the first time.

4. The method of claim 1, wherein combining the weighted first sample and the weighted second sample comprises combining the weighted first sample and the weighted second sample on a sensor.

5. The method of claim 1, wherein the first pixel cell signal and the second pixel cell signal are produced by one of a complementary MOS (CMOS) image sensor and a charge-coupled device (CCD) image sensor.

6. The method of claim 5, wherein sampling a first pixel cell signal at a first time to produce a first sample and sampling a second pixel cell signal at a second time to produce a second sample comprises:

opening a mechanical shutter on a CCD image sensor and exposing the image sensor to light;

transferring a first sample developed based on the light to a storage region at a first time;

reading out the first sample from the storage region;

closing the mechanical shutter at a second time, the second time being temporally subsequent to the first time;

transferring a second sample to the storage region; and

reading out the second sample from the storage region.

7. The method of claim 5, wherein sampling a first pixel cell signal at a first time to produce a first sample and sampling a second pixel cell signal at a second time to produce a second sample comprises:

aligning the first time and the second time so that the first time finishes substantially simultaneously with the second time;

exposing a plurality of even-numbered lines of the CMOS image sensor at the first time to produce the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines and exposing a plurality of odd-numbered lines of the CMOS image sensor at the second time to produce the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines;

generating the first sample for each of the plurality of spatial locations in each of the plurality of odd-numbered lines by one of downsampling and upsampling the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines; and

generating the second sample for each of the plurality of spatial locations in each of the plurality of even-numbered lines by one of downsampling and upsampling the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines.

8. The method of claim 5, wherein sampling a first pixel cell signal at a first time to produce a first sample and sampling a second pixel cell signal at a second time to produce a second sample comprises:

resetting a pixel cell and exposing the pixel cell to light;

transferring, at a first time, a first sample developed based on the light to a capacitor;

reading out, at a second time, a second sample developed based on the light; and

reading out the first sample after the second sample has been read out.

9. A system comprising:

an image sensor comprising;

a plurality of pixel cells; and

a controller that;

samples, from at least one of the plurality of pixel cells, a first pixel cell signal at a first time to produce a first sample; and

samples, from the at least one of the plurality of cells, a second pixel cell signal at a second time to produce a second sample; and

a processor that;

applies a first variable weighting factor to the first sample, wherein the first variable weighting factor is defined based on a first function;

applies a second variable weighting factor to the second sample, wherein the second variable weighting factor is defined based on a second function; and

combines the weighted first sample and the weighted second sample according to the equation;

S_COMBINED(i, j)=S_LONG(i, j)*M(i, j)+n*S_SHORT(i, j)*(1−

M(i, j))

where S_COMBINED denotes the combination of the first sample, denoted by S_SHORT, and the second sample, denoted by S_LONG, for the pixel cell located at (i, j), n denotes a scalar value that is dependent on the difference in the time of exposure between the first time and the second time, and M(i, j) denotes the variable weighting factor.

10. The system of claim 9, wherein the value of M(i, j) is between zero and one, inclusive.

11. The system of claim 9, wherein the second time is temporally subsequent to the first time.

12. The system of claim 9, wherein the processor combines the weighted first sample and the weighted second sample on a sensor.

13. The system of claim 9, wherein the first pixel cell signal and the second pixel cell signal are produced by one of a complementary MOS (CMOS) image sensor and a charge-coupled device (CCD) image sensor.

14. The system of claim 13, wherein the controller samples a first pixel cell signal at a first time to produce a first sample and samples a second pixel cell signal at a second time to produce a second sample at least by:

opening a mechanical shutter on a CCD image sensor and exposing the image sensor to light;

transferring a first sample developed based on the light to a storage region at a first time;

reading out the first sample from the storage region;

closing the mechanical shutter at a second time, the second time being temporally subsequent to the first time;

transferring a second sample to the storage region; and

reading out the second sample from the storage region.

15. The system of claim 13, wherein the controller samples a first pixel cell signal at a first time to produce a first sample and samples a second pixel cell signal at a second time to produce a second sample at least by:

aligning the first time and the second time so that the first time finishes substantially simultaneously with the second time;

exposing a plurality of even-numbered lines of the CMOS image sensor at the first time to produce the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines and exposing a plurality of odd-numbered lines of the CMOS image sensor at the second time to produce the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines;

generating the first sample for each of the plurality of spatial locations in each of the plurality of odd-numbered lines by one of downsampling and upsampling the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines; and

generating the second sample for each of the plurality of spatial locations in each of the plurality of even-numbered lines by one of downsampling and upsampling the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines.

16. The system of claim 13, wherein the controller samples a first pixel cell signal at a first time to produce a first sample and samples a second pixel cell signal at a second time to produce a second sample at least by:

resetting a pixel cell and exposing the pixel cell to light;

transferring, at a first time, a first sample developed based on the light to a capacitor;

reading out, at a second time, a second sample developed based on the light; and

reading out the first sample after the second sample has been read out.

17. A system comprising:

means for sampling a first pixel cell signal at a first time to produce a first sample;

means for sampling a second pixel cell signal at a second time to produce a second sample;

means for applying a first variable weighting factor to the first sample, wherein the first variable weighting factor is defined based on a first function;

means for applying a second variable weighting factor to the second sample, wherein the second variable weighting factor is defined based on a second function; and

means for combining the weighted first sample and the weighted second sample according to the equation;

S_COMBINED(i, j)=S_LONG(i, j)*M(i, j)+n* S_SHORT(i, j)*(1−

M(i, j))

where S_COMBINED denotes the combination of the first sample, denoted by S_SHORT, and the second sample, denoted by S_LONG, for the pixel cell located at (i, j), n denotes a scalar value that is dependent on the difference in the time of exposure between the first time and the second time, and M(i, j) denotes the variable weighting factor.

18. The system of claim 17, wherein the value of M(i, j) is between zero and one, inclusive.

19. The system of claim 17, wherein the second time is temporally subsequent to the first time.

20. The system of claim 17, wherein the means for combining the weighted first sample and the weighted second sample comprises means for combining the weighted first sample and the weighted second sample on a sensor.

21. The system of claim 17, wherein the first pixel cell signal and the second pixel cell signal are produced by one of a complementary MOS (CMOS) image sensor and a charge-coupled device (CCD) image sensor.

22. The system of claim 21, wherein the means for sampling a first pixel cell signal at a first time to produce a first sample and the means for sampling a second pixel cell signal at a second time to produce a second sample comprises:

means for opening a mechanical shutter on a CCD image sensor and exposing the image sensor to light;

means for transferring a first sample developed based on the light to a storage region at a first time;

means for reading out the first sample from the storage region;

means for closing the mechanical shutter at a second time, the second time being temporally subsequent to the first time;

means for transferring a second sample to the storage region; and

means for reading out the second sample from the storage region.

23. The system of claim 21, wherein the means for sampling a first pixel cell signal at a first time to produce a first sample and the means for sampling a second pixel cell signal at a second time to produce a second sample comprises:

means for aligning the first time and the second time so that the first time finishes substantially simultaneously with the second time;

means for exposing a plurality of even-numbered lines of the CMOS image sensor at the first time to produce the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines and exposing a plurality of odd-numbered lines of the CMOS image sensor at the second time to produce the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines;

means for generating the first sample for each of the plurality of spatial locations in each of the plurality of odd-numbered lines by one of downsampling and upsampling the first sample for each of a plurality of spatial locations in each of the plurality of even -numbered lines; and

means for generating the second sample for each of the plurality of spatial locations in each of the plurality of even-numbered lines by one of downsampling and upsampling the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines.

24. The system of claim 21, wherein the means for sampling a first pixel cell signal at a first time to produce a first sample and the means for sampling a second pixel cell signal at a second time to produce a second sample comprises:

means for resetting a pixel cell and exposing the pixel cell to light;

means for transferring, at a first time, a first sample developed based on the light to a capacitor;

means for reading out, at a second time, a second sample developed based on the light; and

means for reading out the first sample after the second sample has been read out.

25. A computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:

sample a first pixel cell signal at a first time to produce a first sample;

sample a second pixel cell signal at a second time to produce a second sample;

apply a first variable weighting factor to the first sample, wherein the first variable weighting factor is defined based on a first function;

apply a second variable weighting factor to the second sample, wherein the second variable weighting factor is defined based on a second function; and

combine the first sample and the weighted second sample according to the equation;

S_COMBINED(i, j)=S_LONG(i, j)*M(i, j)+n* S_SHORT(i, j)*(1−

M(i, j))

where S_COMBINED denotes the combination of the first sample, denoted by S_SHORT, and the second sample, denoted by S_LONG, for the pixel cell located at (i, j), n denotes a scalar value that is dependent on the difference in the time of exposure between the first time and the second time, and M(i, j) denotes the variable weighting factor.

26. The computer-readable medium of claim 25, wherein the value of M(i, j) is between zero and one, inclusive.

27. The computer-readable medium of claim 25, wherein the second time is temporally subsequent to the first time.

28. The computer-readable medium of claim 25, wherein the instructions that cause the one or more processors to combine the weighted first sample and the weighted second sample comprise instructions that cause the one or more processors to combine the weighted first sample and the weighted second sample on a sensor.

29. The computer-readable medium of claim 25, wherein the first pixel cell signal and the second pixel cell signal are produced by one of a complementary MOS (CMOS) image sensor and a charge-coupled device (CCD) image sensor.

30. The computer-readable medium of claim 29, wherein the instructions that cause the one or more processors to sample a first pixel cell signal at a first time to produce a first sample and sample a second pixel cell signal at a second time to produce a second sample comprise instructions that cause the one or more processors to:

open a mechanical shutter on a CCD image sensor and expose the image sensor to light;

transfer a first sample developed based on the light to a storage region at a first time;

read out the first sample from the storage region;

close the mechanical shutter at a second time, the second time being temporally subsequent to the first time;

transfer a second sample to the storage region; and

read out the second sample from the storage region.

31. The computer-readable medium of claim 30, wherein the instructions that cause the one or more processors to sample a first pixel cell signal at a first time to produce a first sample and sample a second pixel cell signal at a second time to produce a second sample comprise instructions that cause the one or more processors to:

align the first time and the second time so that the first time finishes substantially simultaneously with the second time;

expose a plurality of even-numbered lines of the CMOS image sensor at the first time to produce the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines and expose a plurality of odd-numbered lines of the CMOS image sensor at the second time to produce the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines;

generate the first sample for each of the plurality of spatial locations in each of the plurality of odd-numbered lines by one of downsampling and upsampling the first sample for each of a plurality of spatial locations in each of the plurality of even-numbered lines; and

generate the second sample for each of the plurality of spatial locations in each of the plurality of even-numbered lines by one of downsampling and upsampling the second sample for each of a plurality of spatial locations in each of the plurality of odd-numbered lines.

32. The computer-readable medium of claim 30, wherein the instructions that cause the one or more processors to sample a first pixel cell signal at a first time to produce a first sample and sample a second pixel cell signal at a second time to produce a second sample comprise instructions that cause the one or more processors to:

reset a pixel cell and expose the pixel cell to light;

transfer, at a first time, a first sample developed based on the light to a capacitor;

read out, at a second time, a second sample developed based on the light; and

read out the first sample after the second sample has been read out.