System and method for processing non-linear image data from a digital imager

• US 7,349,574 B1
• Filed: 10/14/2003
• Issued: 03/25/2008
• Est. Priority Date: 10/11/2002
• Status: Active Grant

• Current Assignee: SMAL Camera Technologies
• Original Assignee: Sensata Technologies Incorporated (Sensata Technologies Holding Plc)
• Inventors: Chang, Edward T., Sproul, Jason Y., Sodini, Charles G.
• Primary Examiner(s): Couso; Jose L.
• Application Number:
  US10/685,126
• Time in Patent Office: 1,624 Days
• Field of Search: 382168-172, 382/312, 382/321, 345589-605
• US Class Current: 382/168
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/575 with a response composed of multiple slopes
• CPC Class Codes: H04N 25/63 applied to dark current
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

A system and method process non-linear image data, still or video, from a digital imager. Noise generated by analog-to-digital converters is filtered from a pixel of digital image data. Moreover, the effects of single pixel defects in the imager are eliminated by clamping a predetermined pixel of image data within the window when the value of the predetermined pixel is greater than a maximum value of the image data of neighboring pixels or less than a minimum value of the image data of neighboring pixels. Ripples in image data are reduced by eliminating the effects of single pixel defects before filtering for crosstalk caused by electrical crosstalk between sensor elements in an imager. Dark current is removed from image data generated by an imager by subtracting a fraction of a determined dark current value from all image data generated by the imager to compensate for nonlinearities in dark current across the imager. The image data is white balanced by creating a set of scalar color adjustments from determined average color values and constraining the set of scalar adjustments to plausible lighting conditions to prevent overcompensation on images having large regions of similar hue. Lastly, utilization of a fixed set of intensity levels is optimized by remapping and restreching the image data to create new luma values for each pixel.

Claims

1. A method of optimally utilizing a fixed set of intensity levels of a video display to represent a wide dynamic range data captured by an imager, comprising:

(a) determining a target grey value;

(b) determining a power curve from the determined target grey value;

(c) limiting the determined power curve to a range of minimum and maximum power curves, the range of minimum and maximum power curves corresponding to an original compression curve utilized by the imager;

(d) remapping a luma value of each pixel using the limited power curve;

(e) determining a minimum luma value BP_min for a frame of image data;

(f) creating a histogram from the remapped luma values;

(g) determining an ideal black point value BP and an ideal white point value WP from the created histogram and determined minimum luma value; and

(h) restretching the image data, using the determined ideal black point value and an ideal white point value, to create new luma values for each pixel.

2. The method as claimed in claim 1, wherein the determination of the target grey value comprises finding an average luma for a predetermined region of an imaged scene.

3. The method as claimed in claim 2, wherein the predetermined region of the imaged scene is a center region of 256×

256 pixels.

4. The method as claimed in claim 1, wherein the determination of the target grey value comprises finding an average luma for each of a predetermined number of regions of an imaged scene.

5. The method as claimed in claim 4, wherein one of the predetermined regions of the imaged scene is a center region of 256×

256 pixels and the remaining predetermined regions of the imaged scene are regions not a power of two in size.

6. The method as claimed in claim 5, wherein the determination of the target grey value further comprises:

(a1) storing luma average for each region in an array;

(a2) computing a mean value for the array luma averages;

(a3) computing an array of absolute differences between each element of the array and the computed mean;

(a4) determining squared differences between each element and the mean;

(a5) finding a maximum squared difference;

(a6) excluding a region form determining the targeted grey value if the squared difference for that region is greater than the half the maximum squared difference and an absolute value of a difference between the luma value for that region and the computed mean value is greater than 128 so as to prevent small, extreme-valued regions from influencing an allocation of gray levels to a majority of the image; and

(a7) determining the targeted grey value from averaging luma values from all non-excluded regions.

7. The method as claimed in claim 1, wherein the determination of the power curve from the determined target grey value further comprises:

(b1) determining if the determined target grey value is greater than a predetermined value; and

(b2) multiplying all pixels by a predetermined factor when it is determined that the determined target grey value is greater than the predetermined value.

8. The method as claimed in claim 1, wherein the determination of the ideal black point value BP from the created histogram and determined minimum luma value further comprises:

(g1) determining a black threshold luma value B that is associated with a histogram bin wherein no more than 0.625% to 10% of the pixels as a scene becomes darker are in the bins below the black threshold luma value B bin;

(g2) clipping black threshold luma value B to a predetermined value;

(g3) determining the ideal black point BP to be BP=min(max(BP_min, min(16B,

     1023))).

9. The method as claimed in claim 1, wherein the restretching of the image data, using the determined ideal black point value and an ideal white point value, to create new luma values for each pixel further comprises:

(h1) determining a black stretch point R_B to be R_B=β

(WP−

BP)+BP wherein β

is a black point offset value;

(h2) determining a white stretch point R_W to be R_W=ω

(WP−

BP)+BP wherein ω

is a white point offset value;

(h3) determining a new luma value by subtracting the black stretch point R_B from the luma value for each pixel and multiplying the difference by (K)/(R_W−

R_B) wherein K is a predetermined constant; and

(h4) calculating a difference between the new luma value and the old luma value and adding this difference the color channels so as to restretch the image data without significant effects on saturation.

10. The method as claimed in claim 9, wherein K is equal to 4095.

11. The method as claimed in claim 9, wherein β

is a user controlled black point offset value and ω

is a user controlled white point offset value.

12. A method of optimally utilizing a fixed set of intensity levels of a video display to represent a wide dynamic range data captured by an imager, comprising:

(a) determining a target grey value;

(b) determining a power curve from the determined target grey value;

(c) converting luma values into logarithmic luma values, the logarithmic luma values being independent of an input data range;

(d) determining a power function based upon a ratio between a desired logarithmic luma average and an actual logarithmic luma average;

(e) determining a maximum gain limit;

(f) determining a correct luma value;

(g) determining a luma correction value from the correct luma value and the actual luma value; and

(h) applying the luma correction value to each pixel input luma value to generate a corrected pixel input luma value.

13. The method as claimed in claim 12, wherein the correct luma value a minimum value between a product of the logarithmic luma value and the determined power function and a sum of the logarithmic luma value and the determined maximum gain limit.