Optical imager using a method for adaptive real-time expanding of the dynamic range

• US 6,831,689 B2
• Filed: 04/18/2001
• Issued: 12/14/2004
• Est. Priority Date: 10/19/1998
• Status: Expired due to Term

• Current Assignee: Orly Yadid-Pecht
• Original Assignee: Orly Yadid-Pecht
• Inventors: Yadid-Pecht, Orly
• Primary Examiner(s): Christensen, Andrew
• Assistant Examiner(s): GENCO, BRIAN C
• Application Number:
  US09/837,525
• Publication Number:
  US 20020027606A1
• Time in Patent Office: 1,336 Days
• Field of Search: 348/222.1, 348/229.1, 348/230.1, 348/294, 348/297, 348/302, 348/308, 348/362
• US Class Current: 348/297
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/585 with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes

Abstract

Method and apparatus for expanding the dynamic range of an optical imager, comprising individually controlling the integration time of each pixel of a sensor array, and providing a corresponding scaling factor for the electrical output of each individual pixel during the frame time. The integration time of each pixel is controlled as a function of light intensity received by each individual pixel, by resetting the pixel after a predetermined threshold for the output signal, has been reached.

Claims

1. A method for expanding the dynamic range of an optical imager employing an array of pixels in rows, comprising:

a) determining a threshold level for the signal that is output from a pixel, whereby a signal greater than said level indicates said pixel should be reset;

b) determining the number W of comparisons to be made for each row of pixels, and time points for performing said comparisons, according to the required level of expansion;

c) for each row that is selected for comparison, individually controlling the integration time of each pixel as a function of light intensity received thereon, by performing, at each time point, W comparisons of said signal to said threshold level and resetting said pixel whenever said threshold has been reached, wherein each comparison is performed at a row that is spaced by a specific number of rows from the selected row, which corresponds to another time point, and individually controlling the integration time of each pixel in the spaced row as a function of light intensity received thereon, by comparing the signal, in said spaced row, to said threshold level and resetting that pixel whenever said threshold has been reached;

d) repeating step c) above for all rows in said array; and

e) for each pixel, accumulating data representing the number of executed reset operations and providing a corresponding scaling factor for the electrical output of each said individual pixel, by using the accumulated data.

2. A method according to claim 1, comprising:

a) providing a two-dimensional imager constructed from an active pixel array of integer M columns and integer N rows, fabricated on a semiconductor substrate, each individual pixel containing an optical sensor to receive light, a reset input and an electrical output representing the illumination received thereon;

b) providing an upper column-parallel signal chain of M capacitors for copying the outputs of a selected row, thereby to compare said outputs to a set of corresponding threshold values;

c) providing a lower column-parallel signal chain of M capacitors for copying the electrical readouts of each selected row;

d) for each pixel, determining a threshold level, above which the pixel is reset;

e) providing a digital memory, for storing the comparison results of the output of each pixel, with the set of threshold levels;

f) providing a control circuit for controlling the reading operations of each pixel, the timing of said comparisons to a set of corresponding threshold values for each pixel, and corresponding reset signals for each pixel;

g) determining the required expansion of the dynamic range by a series of W bits;

h) determining a set of W time points, at which comparisons of each pixel output with the corresponding threshold vale is carried out, the intervals between any two consecutive comparison time points being prefixed;

i) selecting a row (n) for readout;

j) determining a set of row offsets (Δ

₁, Δ

₂, . . . , Δ

_W) between the selected row n and other row n−

Δ

_i(i=1, 2, . . . , W), said set of row offsets having W prefixed values;

k) copying all pixel output values of the selected row n, into the lower capacitor chain, converting each one of said output values to a digital word and outputting said digital word;

l) resetting row n;

m) copying all pixel output values of row n−

Δ

₁ into the upper capacitor chain;

n) comparing the electrical output of each pixel in row n−

Δ

₁ with a corresponding threshold value, at the time point which is matched to this row;

o) outputting a series of M bits representing the comparison results of all pixel output values of row n−

Δ

₁;

p) storing the comparison results of step (o) above in the memory;

q) applying a reset pulse for those pixels in row n−

Δ

₁ that are expected to be saturated according to the stored comparison results;

r) consecutively repeating steps (in) to (p) above W−

1times, for all other remaining row offsets, at their matching time points, and applying a reset pulse for each pixel only if that pixel was reset in the preceding comparison time point;

s) for each pixel from the selected row, obtaining and outputting the scaling factor, stored in the memory as a W bit digital combination, representing the ratio between the full integration time and the effective integration time of each pixel, from the set of W comparison results at each time point of step h) above;

t) for each pixel from the selected row, obtaining and outputting a digital word, which corresponds to the electrical output value of the pixel, at a timing of t=0; and

u) repeating steps (l) to (t) above for all remaining rows of the pixel array.

3. A method according to claim 2, wherein the W bit digital combination is encoded and stored in the memory.

4. A method according to claim 2, wherein the intervals between any two consecutive comparison time points decrease according to a downgoing series.

5. A method according to claim 4, wherein the downgoing series is $T-\frac{T}{X_1},T-\frac{T}{X_2},\dots$



 

,T-TXW,

wherein T represents the frame time for full integration, X₁>

1 and I=1, 2, . . . , W.

6. A method according to claim 5, wherein the downgoing series is a geometric series.

7. A method according to claim 6, wherein the downgoing geometric series is $T-\frac{T}{U^1},T-\frac{T}{U^2},\dots$



 

,T-TUW

wherein U>

1.

8. A method according to claim 2, wherein the set of row offsets (Δ

₁, Δ

₂, . . . , Δ

_W) define a downgoing series.

9. A method according to claim 8, wherein the downgoing series of row offsets (Δ

₁, Δ

₂, . . . , Δ

_w) is constituted by the integer values of the downgoing series $\frac{N}{X_1},\frac{N}{X_2},\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},\frac{N}{X_W}$

where X_i>

1 and i=1, 2, . . . , W.

10. A method according to claim 8, wherein the downgoing series of row offsets (Δ

₁, Δ

₂, . . . , Δ

_w) is constituted by the integer values of the downgoing geometric series $\frac{N}{U^1},\frac{N}{U^2},\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},\frac{N}{U^W}$

wherein U>

1.

11. A method according to claim 10, wherein each row n−

Δ

_i is read at the time point $T-\frac{T}{U^i}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(i=1,2,\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},W\right).$

12. A method according to claims 7, 10 or 11, wherein U=2.

13. A method according to claim 2 wherein the electrical output values of each pixel are output directly as analog values.

14. A method according to claim 2 wherein the control circuit further comprises a conditional reset circuit, said conditional reset circuit carrying out the following operations:

a) generating a series of reset pulses of constant predetermined frequency and duty cycle by an external circuit;

b) generating a series of sampling pulses of predetermined duty cycle and of the same frequency of the reset pulses, by an external circuit, said series appearing with a constant delay with respect to the reset pulses, the time interval between consecutive sampling pulses representing the full integration time;

c) generating a series of reset enable pulses of time interval between consecutive pulses shorter than the time interval between consecutive reset pulses;

d) generating a control signal whenever the electrical output of a pixel exceeds the threshold value;

e) generating a conditional reset signal whenever there is coincidence between a reset enable pulse and a control signal;

f) starting integration at the time of generation of either a reset pulse or a conditional reset pulse; and

g) terminating the integration at the time of sampling pulse generation.

15. A method according to claim 2, wherein the electrical output value of each pixel is represented as a floating point number, where the mantissa represents the regular value obtained from the pixel'"'"'s analog to digital converter and the exponent represents the scaling factor.

16. A method according to claim 2, wherein the delay between the integration starting points of any consecutive rows is exploited for multiplexing between time and spatial domains.

17. A method according to claim 2, wherein the scaling of pixels within a frame time is carried out automatically during the frame time.

18. A method according to claim 2, wherein each pixel is an active pixel.

19. A method according to claim 2, wherein said optical sensor is a photodiode or phototransistor.

20. A method according to claim 2, wherein the number of rows and/or columns is an integer power of 2.

21. A method according to claim 2, wherein the control circuit and memory are external circuits.

22. A method according to claim 2, wherein the control circuit and memory are fabricated on the imager semiconductor substrate.

23. A method according to claim 2, wherein the pixel array is a square matrix with an identical number of rows and columns.

24. A method according to claim 2, wherein the lower column-parallel signal chain of capacitors is used for storing readouts for comparisons with threshold values, and the upper column-parallel signal chain of capacitors is used for copying the electrical readouts of each selected row.

25. A method according to claim 2, wherein the imager comprises only one column-parallel signal chain of capacitors with an associated A/D converter, which is used both for copying readouts for comparisons with threshold values and for copying the electrical readouts of each selected row.

26. A method according to claim 2, wherein comparisons are carried out by comparing to threshold levels, at at least several time points.

27. An optical imager employing an array of pixels and having expanded dynamic range, comprising:

a) means for selecting rows for comparison, with a threshold level for the signal that is output from a pixel, beyond which that pixel should be reset, said comparison occurring at predetermined time points according to a required level of expansion, the spacing between selected rows being related to said predetermined time points;

b) means for individually controlling the integration time of each pixel as a function of light intensity received thereon;

c) means for comparing said signal to said threshold level;

d) means for resetting each pixel whenever said threshold has been reached; and

e) means for accumulating data representing the number of executed reset operations and for outputting a corresponding scaling factor for the electrical output of each said individual pixel by using the accumulated data.

28. An optical imager according to claim 27, comprising:

a) an active pixel array of integer M columns and integer N rows, fabricated on a semiconductor substrate, each individual pixel containing an optical sensor to receive light, a reset input and an electrical output representing the illumination received thereon;

b) an upper column-parallel signal chain of M capacitors for copying the outputs of a selected row, so as to compare said outputs to a set of corresponding threshold values;

c) a lower column-parallel signal chain of M capacitors for copying the electrical readouts of each selected row;

d) an array of analog to digital converters representing the electrical output of each pixel in a corresponding digital word;

e) a digital memory, for storing the comparison results of the output of each pixel with the threshold levels, and the digital words corresponding to the readout of each pixel;

f) a control circuit for controlling the reading operations of each pixel, the timing of comparisons for each pixel, and corresponding reset signals for each pixel;

g) means for selecting rows for readout;

h) means for copying all pixel output values of a selected row, into the lower capacitor chain;

i) means for converting each one of the output values to a digital word;

j) means for resetting the selected row;

k) means for copying all pixel output values of rows into the upper capacitor chain;

l) means for comparing the electrical output of each pixel in a row with a corresponding threshold value, at the time point which is matched to the selected row;

m) means for outputting a series of M bits representing the comparison results of all pixel output values of each row;

n) means for storing all comparison results in the memory;

o) means for applying a reset pulse for each pixel that is expected to be saturated according to the stored comparison results;

p) means for outputting the scaling factor, stored in the memory as a W bit digital combination or as an encoded combination of said W bits; and

q) means for outputting the digital word, which corresponds to the electrical output value of the pixel, at a timing of t=0.

29. Optical imager according to claim 28, further comprising means for encoding the W bit digital combination.

30. Optical imager according to claim 28, wherein the control circuit comprises a circuit for generating conditional reset signals.

31. Optical imager according to claim 28, wherein the electrical output values of each pixel are output directly as analog values.

32. Optical imager according to claim 29, wherein the circuit for generating a conditional reset comprises:

a) means for generating a series of reset pulses of constant predetermined frequency and duty cycle by an external circuit;

b) means for generating a series of sampling pulses of predetermined duty cycle and of the same frequency of the reset pulses, by an external circuit, said series appearing with a constant delay with respect to the reset pulses, the time interval between consecutive sampling pulses representing the full integration time;

c) means for generating a series of reset enable pulses of time interval between consecutive pulses shorter than the time interval between consecutive reset pulses;

d) means for generating a control signal whenever the electrical output of a pixel exceeds the threshold value;

e) means for generating a conditional reset signal whenever there is coincidence between a reset enable pulse and a control signal;

f) circuitry for starting integration at the time of generation of either a reset pulse or a conditional reset pulse; and

g) circuitry for terminating the integration at the time of sampling pulse generation.

33. Optical imager according to claim 28, wherein the electrical output value of each pixel is represented as a floating point number, where the mantissa represents the regular value obtained from the pixel'"'"'s analog to digital converter and the exponent represents the scaling factor.

34. Optical imager according to claim 28, wherein the delay between the integration starting points of any consecutive rows is exploited for multiplexing between time and spatial domains.

35. Optical imager according to claim 28, wherein the scaling of pixels within a frame is carried out automatically during the frame time.

36. Optical imager according to claim 28, wherein each pixel is an active pixel.

37. Optical imager according to claim 28, wherein the optical sensor is a photodiode or phototransistor.

38. Optical imager according to claim 28, wherein the number of rows and/or columns is an integer power of 2.

39. Optical imager according to claim 28, wherein the accuracy of the scaling is substantially the same at any illumination level.

40. Optical imager according to claim 28, wherein the control circuit and memory are fabricated on the imager semiconductor substrate.

41. Optical imager according to claim 28, wherein the pixel array is a square matrix with an identical number of rows and columns.

42. Optical imager according to claim 28, wherein the lower column-parallel signal chain of capacitors is used for storing readouts for comparisons with threshold values, and the upper column-parallel signal chain of capacitors is used for copying the electrical readouts of each selected row.

43. Optical imager according to claim 28, wherein said imager comprises only one column-parallel signal chain of capacitors with the associated A/D converter, which is used both for copying readouts for comparisons with threshold values and for copying the electrical readouts of each selected row.

44. Optical imager according to claim 28, wherein comparisons are carried out by comparing to different threshold levels at at least several time points.