Image sharpness management

• US 7,515,765 B1
• Filed: 08/18/2004
• Issued: 04/07/2009
• Est. Priority Date: 01/30/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bouzit, Samira, MacDonald, Lindsay William
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US10/920,585
• Time in Patent Office: 1,693 Days
• Field of Search: 382232-253, 382/205, 382/254, 382260-266, 382268-269, 382274-275, 382162-167, 37524001-24029, 358/1.9, 358 326- 327, 358517-518, 358/532, 35842601-42616
• US Class Current: 382/263
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: G06V 10/20 Image preprocessing

Abstract

Methods, media, and a system provide image sharpness management for electronic images. Pixel values for an unmodified image are processed relative to spatial frequency bands for that image. An average pixel value is acquired for each of the spatial frequency bands and used to alter the corresponding pixel value of the image. A resulting modified image exhibits enhancement of sharpness, based on adjustments for each of the spatial frequency bands. In one embodiment, the average pixel value is adjusted based on an expected viewing distance of an observer. In another embodiment, the average pixel value is adjusted based on the ability of a display device to reproduce spatial frequencies in the image.

Claims

1. A method comprising:

using at least one processor to perform steps, comprising,

transforming an image into a plurality of spatial frequency bands;

calculating average values for each of the spatial frequency bands;

weighting each of the spatial frequency bands by a Modulation Transfer Function (MTF) ratio value; and

after said weighting, adjusting a relative amplitude of different ones of the spatial frequency bands to generate a modified image using an adjustment value associated with the average values.

2. The method of claim 1 further comprising the step of defining the spatial frequency bands in octaves by the ratios f_N/2^M+1:

f_N/2^M for M=1 . . . 6 where f_N/2 is a Nyquist frequency of the sampling method.

3. The method of claim 1 further comprising the step of defining each spatial frequency band sufficiently for the plurality of spatial frequency bands to approximate a continuous function of spatial frequency.

4. The method of claim 1 further comprising the step of weighting each of the spatial frequency bands by a Contrast Sensitivity Function (CSF) value before performing the adjusting.

5. The method of claim 1 further comprising the step of separating the image into luminance and chromatic components, and then processing the method with the luminance component, where the chromatic component remains unchanged.

6. The method of claim 1 further comprising the step of normalizing the modified image to preserve an overall power of the image.

7. The method of claim 1 wherein the step of adjusting further includes normalizing a total average of the average values to preserve an original average lightness associated with the image.

8. The method of claim 1 further comprising the step of scaling the adjustment value by a configurable and selectable sharpness factor.

9. A method comprising:

using at least one processor to perform steps comprising,

transforming an image into a plurality of spatial frequency bands;

calculating an average pixel value at each of the spatial frequency bands;

calculating a mean Contrast Sensitivity Function (CSF) value at each of the spatial frequency bands;

calculating a mean Modulation Transfer Function (MTF) ratio value at each of the spatial frequency bands;

combining each average pixel value, each mean CSF value, and each MTF ratio value into an adjustment value; and

adding the adjustment value to each pixel value of the image to generate a modified image.

10. The method of claim 9 wherein the step of transforming further includes generating the spatial frequency bands by a set of convolution kernels of diameter 2^M−

1 pixels, wherein M is an integer that begins at 1 and continues to 6 or greater.

11. The method of claim 9 wherein the step of adding further includes adding a non-linear function of a weighted sum of the average values for each of the spatial frequency bands to each of the pixel values.

12. The method of claim 9 wherein the step of calculating of the mean CSF value at each of the spatial frequencies further includes using an expected viewing distance of an average observer who views the modified image.

13. The method of claim 9 wherein the step of calculating of the mean MTF value at each of the spatial frequencies further includes using an ideal MTF value for a display that presents the modified image.

14. The method of claim 9 wherein the step of combining further includes obtaining a total average for each of the average pixel values, each of the mean CSF values, and each of the MTF values, and wherein the total average is the adjustment value.

15. The method of claim 9 further comprising the step of separating the image into luminance and chromatic components, then processing the method with the luminance component, where the chromatic component remains unchanged.

16. The method of claim 9 wherein the step of adjusting further includes multiplying by a customized sharpness factor in the range of 0 to 1.

17. A system, comprising:

a display to present an image;

an image sharpness manager that communicates with the display, wherein the image sharpness manager is configured to;

process each pixel of the image relative to a plurality of spatial frequency bands, and adjust the value of each processed pixel based on a function of weighted values of the spatial frequency bands, wherein the function comprises at least one factor for a Modulation Transfer Function ratio value.

18. The system of claim 17 further comprising a device that implements the image sharpness manager, wherein the device is at least one of a digital camera, a phone, an appliance, intelligent apparel, a computing device, a computing peripheral device, and a television.

19. The system of claim 17, wherein each spatial frequency band is defined by a region of diameter 2^M−

1 pixels, where M is an integer number that begins with 1 and continues to at least 6.

20. The system of claim 17, wherein the image sharpness manager is configured to adjust the value for each processed pixel to enhance the image for at least one of contrast, spatial losses, and sharpness.

21. The system of claim 17, wherein the function further comprises at least one factor for a mean Contrast Sensitivity Function (CSF) value in each of the spatial frequency bands.

22. The system of claim 17, wherein the image sharpness manager is configured to adjust the value for each processed pixel to alter the image based on an expected viewing distance of an observer.

23. The system of claim 17, wherein the image sharpness manager is configured to separate the image into luminance and chromatic components, and to then adjust the luminance component of each pixel value, without changing the chromatic component.

24. A computer readable medium having instructions thereon, which, when the instructions are executed by a computer perform a method, the method comprising:

transforming an image into a plurality of spatial frequency bands;

calculating average values for each of the spatial frequency bands;

weighting each of the spatial frequency bands by a value based at least in part on a Modulation Transfer Function (MTF) ratio value; and

adjusting a relative amplitude of different ones of the spatial frequency bands using the average and weighted values to generate a modified image.

25. The medium of claim 24 further comprising the instructions of:

separating the image into luminance and chromatic components, and subsequently executing the further instructions on the medium only with respect to the luminance component, where the chromatic component remains unchanged.

26. The medium of claim 24 wherein the instructions further comprise weighting each of the spatial frequency bands in part by a Contrast Sensitivity Function (CSF) value.

27. The medium of claim 26 wherein the weighting instructions further include defining the CSF value based on an expected viewing distance of an average observer who views the modified images and defining the MTF ratio value based on an ideal MTF ratio value for a display that presents the modified image.

28. A method comprising:

using at least one computing device to perform steps comprising,

transforming an image into a plurality of spatial frequency bands;

calculating an average pixel value at each of the spatial frequency bands;

calculating a mean Contrast Sensitivity Function (CSF) value at each of the spatial frequency bands;

calculating a mean Modulation Transfer Function (MTF) ratio value at each of the spatial frequency bands;

combining each average pixel value, each mean CSF value, and each MTF ratio value into an adjustment value; and

adding the adjustment value to each pixel value of the image to generate a modified image.

29. The method of claim 28 wherein the step of transforming step further includes generating the spatial frequency bands by a set of convolution kernels of diameter 2^M−

1 pixels, wherein M is an integer that begins at 1 and continues to 6 or greater.

30. The method of claim 28 wherein the step of adding step further includes adding a non-linear function of a weighted sum of the average values for each of the spatial frequency bands to each of the pixel values.

31. The method of claim 28 wherein the step of calculating of the mean CSF value at each of the spatial frequencies further includes using an expected viewing distance of an average observer who views the modified image.

32. The method of claim 28 wherein the step of calculating of the mean MTF value at each of the spatial frequencies further includes using an ideal MTF value for a display that presents the modified image.

33. The method of claim 28 wherein the step of combining further includes obtaining a total average for each of the average pixel values, each of the mean CSF values, and each of the MTF values, and wherein the total average is the adjustment value.

34. The method of claim 28 further comprising the step of separating the image into luminance and chromatic components, then processing the method with the luminance component, where the chromatic component remains unchanged.

35. The method of claim 28 wherein the step of adjusting further includes multiplying by a customized sharpness factor in the range of 0 to 1.