Method and apparatus for selectively adjusting the brightness of large regions of an image

• US 5,235,434 A
• Filed: 06/27/1991
• Issued: 08/10/1993
• Est. Priority Date: 06/27/1991
• Status: Expired due to Term

• Current Assignee: Senshin Capital LLC (Intellectual Ventures LLC)
• Original Assignee: Polaroid Corporation (PLR IP Holdings LLC)
• Inventors: Wober, Munib
• Primary Examiner(s): Coles, Sr., Edward L.
• Assistant Examiner(s): LEE, TOMMY D
• Application Number:
  US07/722,218
• Time in Patent Office: 775 Days
• Field of Search: 358/426, 358/432, 358/433, 358/448, 358/462, 358/464, 358/465, 358/447, 382/50, 382/52, 382/53, 382/54, 382/56
• US Class Current: 358/448
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: H04N 1/4072 dependent on the contents of the original
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The invention concerns a method for rendering scene information in images having a large dynamic range, i.e. sunny and shady areas. The method takes particular advantage of image segmentation and computations employed in compression of images for electronic still photography. In distinguishing large areas, the method selectively adjusts the brightness of all portions of the area without necessarily preserving contrast which avoids halo artifacts. Those portions in areas of intermediate size are subjected to a smoothing feature which avoids the production of artifacts in the form of a line at the boundary.

Claims

1. A method for adjusting the value of each one of a plurality of select pixel groups comprising one or more individual pixels which collectively define an image comprising the steps of:

comparing the value of each pixel group with at least one of a darkness threshold and a brightness threshold;

incrementally enlarging the comparison of each pixel group to an area encompassing surrounding select pixel groups until any pixel group within the comparison area exceeds the darkness threshold or is less than the brightness threshold up to a maximum area of comparison;

assigning a value of size and average to each select pixel group in accordance with the incremental amount of area thereabout entirely below the darkness threshold or above the brightness threshold up to a maximum size and average; and

comparing the size of each select pixel group with a certain maximum size and adding the assigned average value of that pixel group to the pixel group value when that pixel group is at least the maximum size or when any pixel group within a predetermined distance thereof is the maximum size, or alternatively multiplying the average value of each pixel group by a smoothing characteristic and adding the result to that pixel group value when that pixel group is less than the maximum size and no other pixel group within the predetermined distance is the maximum size.

2. The method of claim 1 wherein the maximum size and area are defined by all the pixel groups within an area spaced four pixel groups from the selected pixel group.

3. The method of claim 1 wherein the step of assigning the value of size to each pixel group comprises determining the area defined by a number of pixel groups from each select pixel group in which no pixel group has a value above the darkness threshold or below the brightness threshold.

4. The method of claim 1 wherein assigning an average to each select pixel group comprises averaging the values of all pixel groups within an assigned size surrounding each select pixel group entirely below the darkness threshold or entirely above the brightness threshold.

5. The method of claim 1 wherein the maximum size is 5 and the predetermined distance is four pixel groups.

6. The method of claim 1 wherein the smoothing characteristic is defined by √

/ (k-1)/4 where k is the size of the selected pixel group.

7. The method of claim 1 further comprising the step of selecting sequentially the pixel groups for comparison with the darkness or brightness thresholds.

8. The method of claim 7 wherein the step of selecting pixel groups comprises selecting a first pixel group for comparison with the darkness or brightness thresholds and thereafter selecting a next pixel group for comparison with the thresholds when the value of the first selected pixel group is either higher than the darkness threshold or lower than the brightness threshold or when the value of all pixel groups within an integral number of pixel groups from the first selected pixel group is either higher than the darkness threshold or lower than the brightness threshold up to the maximum size.

9. The method of claim 1 wherein the pixel groups comprise an 8×

8 array of individual pixels.

10. The method of claim 1 further comprising adjusting the brightness of each pixel group without preserving contrast at boundaries between large dark areas and large bright areas.

11. A method for selectively adjusting the brightness of a plurality of select pixel groups comprising one or more individual pixels which collectively define an image comprising the steps of sequentially selecting pixel groups from the image;

comparing the value of each pixel group selected with a threshold defining an acceptable level of brightness and expanding the comparison to other pixel groups in succeeding concentric adjacent sets if the value of the initial pixel group is not within the acceptable level of brightness and until the value of at least one of the other pixel groups in the concentric adjacent sets is within the acceptable level of brightness up to a maximum number of sets corresponding to a predefined large area;

assigning a size and average value to each pixel group in the image in accordance with the number of concentric sets of other pixel groups compared and the average value of such other pixel groups so compared which are not within the acceptable level of brightness; and

comparing the size of each pixel group with the size associated with the predefined large area and adding the average to that pixel group when the size of that pixel group corresponds to the size of the large area; and

if that pixel group does not correspond to the size of the predefined area, thereafter comparing the size of all other pixel groups within a selected number of pixel groups from that pixel group with the size associated with the predefined large area and adding the average to that pixel group when the size of the any other pixel group within the selected number of other pixel groups has a size corresponding to the predefined large area;

or alternatively multiplying the size of each pixel group times a smoothing factor when the size of that pixel group and all other pixel groups within the selected number of pixel groups is less than the size associated with the predefined large area and adding the resulting value to the size of that pixel group, so that the amount added to each pixel group operates to bring that pixel within the acceptable level of brightness.

12. The method of claim 11 wherein the threshold defines at least one of a brightness threshold and a darkness threshold and the acceptable level of brightness is between the brightness threshold and darkness threshold.

13. The method of claim 11 further comprising adjusting the values of pixel groups without preserving contrast at boundaries of predefined large areas.

14. A method for hierarchal processing of a digital image made up of an array of pixel groups of one or more pixels in which each pixel group is provided a value by selectively adjusting the brightness of pixel groups in large areas of generally uniform brightness of the image without the necessity of preserving contrast comprising the steps of:

selecting incrementally a pixel group from the array;

comparing the value of the pixel group with a threshold;

producing a first signal when the selected pixel value is at a level in comparison to the threshold indicative of a pixel group of sufficient brightness and producing a second signal when the selected pixel group value is at a level in comparison to the threshold indicative of a pixel group which may require selective brightening adjustment;

assigning a first size and average value to the selected pixel group and selecting the next succeeding pixel group in response to the first signal;

selecting each one of a first set of other pixel groups surrounding the selected pixel group in response to the second signal and comparing the value of each of the other surrounding pixel groups with the threshold, producing the first signal when the value of any of one of the surrounding pixel groups is at a sufficient brightness and producing the second signal when the value of all the pixels in the surrounding set require selective brightening adjustment;

assigning a next larger size relative to the first size and corresponding average value to the selected pixel group and selecting the next succeeding pixel group in response to the first signal;

selecting each one of a second set of other pixel groups surrounding the first set and comparing as recited above to produce one of the first and second signals accordingly and assigning a next successive larger size and corresponding average value to the selected pixel group up to a maximum number of selections; and

adjusting the value of the selected pixel group in accordance with the average and size assigned to the selected pixel group.

15. The method of claim 14 wherein the adjusting step comprises the steps of:

comparing the size of each pixel group and any other pixel group within the largest set surrounding that pixel group with the size for the maximum number of selections and algebraically adding the average to that pixel group value when the size of that pixel group is the maximum, or if any other pixel group within the largest set surrounding that pixel group equals the maximum thereby adjusting the brightness of the selected pixel group by the maximum average amount for its size;

or alternatively multiplying that pixel group value times a smoothing factor algebraically and adding the result to that pixel group value when the value of that pixel group and any other pixel groups within the largest surrounding set are not equal to the maximum so as to adjust the brightness of that pixel group to a lesser extent.

16. The method of claim 14 further comprising performing all the steps except the adjusting step on a first pass for each pixel group and performing the adjusting step on a second pass for each pixel group.

17. The method of claim 14 wherein the pixel groups are formed by compressing digital image data representing individual pixels and combining the data of a plurality of individual pixels.

18. The method of claim 17 wherein compressing digital image data comprises a discrete cosine transform (DCT) and combining the data comprises representing an average value of an n×

m array of individual pixels by a coefficient of said DCT.

19. The method of claim 18 wherein the assigning step further comprises varying the coefficient of the DCT to produce a corrected pixel group.

20. The method of claim 19 further comprising performing an inverse DCT on the corrected pixel groups for interpolating individual pixels in the image to a corrected level.

21. Apparatus for compressing the dynamic range of a digital image, made up of an array of pixel groups of one or more pixels in which each pixel group is provided a value, by selectively adjusting the brightness of pixel groups in large areas which are at an unacceptable level of brightness comprising:

a selector for selecting pixel groups;

a comparator responsive to the selector for sequentially comparing each selected pixel group with the acceptable level of brightness and producing a first output when the selected pixel group value is an acceptable level of brightness and producing a second output when the selected pixel group value is not an acceptable level of brightness indicative of a pixel group which may require selective brightening adjustment;

means for causing the selector to increment to the next succeeding pixel group in response to said first output;

means for assigning size and average values to the selected pixel group being responsive to the first output for assigning a corresponding size and corresponding average value for the selected pixel group, and being responsive to the second output for causing the selector to select other pixel groups in a set surrounding the selected pixel group for comparison by the comparator with the acceptable level of brightness, said comparator being responsive to produce said first and second outputs up to a selected maximum number of succeeding sets of surrounding pixel groups;

a size comparator for comparing the corresponding size of each selected pixel group with the maximum size and the size of other pixel groups in a select surrounding set within the maximum size and producing a first size output when the size of the selected pixel group is the same as the maximum size or the size of any other pixel group within the select surrounding set is the maximum size and providing a second size output when the size of the selected pixel group or any other pixel group within the select surrounding set is less than the maximum; and

means responsive to the first size signal for adding the value of the selected pixel group to the average value of that pixel group for the selected number of sets for that pixel group and means responsive to the second size signal for multiplying the value of the selected pixel group by a smoothing characteristic in accordance with the size of that pixel group and adding the resulting value to the value of the selected pixel group, said means producing outputs indicative of the amount of brightening adjustment for the selected pixel groups.

22. The apparatus of claim 21 wherein the acceptable level of brightness is above a darkness threshold and below a brightness threshold and the means for adding the value of the selected pixel groups to the average value of that selected pixel group operates to increase the brightness of pixel groups below the darkness threshold and to reduce the value of pixel groups above the brightness threshold.

23. Apparatus for selectively adjusting the brightness of a digital image made up of pixel data elements comprising:

comparator means for comparing each pixel data element with a threshold representative of an acceptable level of brightness and producing outputs corresponding to acceptable and unacceptable levels of brightness;

means for selectively producing a corresponding size and average value for each pixel data element in response to the comparator output representing acceptable brightness;

means responsive to the comparator means for selectively varying the size of the pixel data element to include selected adjacent pixel data elements in response to the comparator output representing an unacceptable level of brightness up to a maximum size, each size having a corresponding average value; and

means for adjusting the brightness of the pixel data element in accordance with the corresponding average value.

24. The apparatus of claim 23 wherein the means for adjusting the brightness of the pixel data element further includes means responsive to the size of the pixel data element and the size of adjacent pixel data elements;

size comparator means for comparing the size of the pixel data elements with a selected size value and means for varying the adjustment when the size of the pixel data elements are less than the selected size value.

25. The apparatus of claim 23 wherein the pixel data element represents one pixel.

26. The apparatus of claim 23 wherein the pixel data element represents an array of pixels.

27. The apparatus of claim 23 further comprising means for compressing the pixel data elements.

28. The apparatus of claim 27 wherein the means for compressing the pixel data elements comprises a discrete cosine transform.