Systems and methods for luma sharpening

• US 9,031,319 B2
• Filed: 05/31/2012
• Issued: 05/12/2015
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cote, Guy, Lim, Suk Hwan, Mori, Munehiro, Lin, Sheng
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/485,341
• Publication Number:
  US 20130321700A1
• Time in Patent Office: 1,076 Days
• US Class Current: 382/167
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81

Abstract

Systems, methods, and devices for sharpening image data are provided. One example of an image signal processing system includes a YCC processing pipeline that includes luma sharpening logic. The luma sharpening logic may sharpen the luma component while avoiding sharpening some noise. Specifically, a multi-scale unsharp mask filter may obtain unsharp signals by filtering an input luma component, and sharp component determination logic may determine sharp signals representing differences between the unsharp signals and the luma component. Sharp lookup tables may “core” the sharp signals, which may prevent some noise from being sharpened. Output logic may determine a sharpened output luma signal by combining the sharp signals with, for example, luma component or one of the unsharp signals.

Claims

1. An image signal processing system comprising:

a YCC processing pipeline configured to process image data in a YCC format, wherein the image data comprises a luma component and two chroma components, wherein the YCC processing pipeline comprises luma sharpening logic configured to sharpen the luma component, wherein the luma sharpening logic comprises;

a multi-scale unsharp mask filter configured to obtain a plurality of unsharp signals by filtering the luma component;

sharp component determination logic configured to determine a plurality of sharp signals representing differences between the one or more unsharp signals and the luma component;

a plurality of sharp lookup tables configured to vary the sharp signals so as to suppress the sharp signals when the sharp signals are beneath a coring threshold to reduce sharpening of noise; and

output logic configured to determine an output luma signal by combining one or more of the sharp signals with the luma component or one of the unsharp signals, or a combination thereof.

2. The image signal processing system of claim 1, wherein the multi-scale unsharp mask filter comprises a plurality of Gaussian filters.

3. The image signal processing system of claim 1, wherein the luma sharpening logic comprises coring thresholding logic configured to core the sharp signals before the sharp signals reach the sharp lookup tables by varying the sharp signals based at least partly on pixel luminance intensity.

4. The image signal processing system of claim 1, wherein the luma sharpening logic comprises coring thresholding logic configured to core the sharp signals before the sharp signals reach the sharp lookup tables by varying the sharp signals so as to suppress the sharp signals more when one of the unsharp signals is lower than when the one of the unsharp signals is higher.

5. The image signal processing system of claim 1, wherein the luma sharpening logic comprises coring thresholding logic configured to core the sharp signals by subtracting a coring threshold signal from the sharp signals before the sharp signals reach the sharp lookup tables, wherein the coring threshold signal is obtained by a coring threshold lookup table indexed to the luma component, one of the unsharp signals, or a combination thereof, and wherein the coring threshold lookup table is configured to provide a higher coring threshold when the indexing signal is lower than when the indexing signal is higher.

6. The image signal processing system of claim 5, comprising selection logic configured to select the luma component or one of the unsharp signals to index the coring threshold lookup table.

7. The image signal processing system of claim 1, comprising radial gain modulation logic configured to modulate the sharp signals by a radial gain that varies depending on a distance from an optical center of the image data.

8. The image signal processing system of claim 1, wherein the luma sharpening logic comprises:

edge detection logic configured to generate an edge signal indicative of an edge boundary in the image data based at least in part on the luma component of the image data;

an edge lookup table configured to core the edge signal by suppressing the edge signal more when the edge signal is lower than when the edge signal is higher to reduce sharpening of noise;

wherein the output logic is configured to determine the output luma signal by combining the luma component or one of the unsharp signals, or both, with the one or more of the sharp signals or the edge signal, or the sharp signals and the edge signal.

9. The image signal processing system of claim 8, wherein the edge detection logic comprises a sobel filter.

10. The image signal processing system of claim 1, wherein the luma sharpening logic comprises output modulation logic configured to modulate the one or more of the sharp signals before the one or more of the sharp signals are combined with the luma component or the one of the unsharp signals, or the combination thereof, wherein the output modulation logic is configured to suppress the one or more sharp signals more when at least one of the sharp signals is higher than when the at least one of the sharp signals is lower.

11. The image signal processing system of claim 10, wherein the output modulation logic comprises a modulation lookup table indexed to the at least one of the sharp signals, wherein the table is configured to output a modulation value that is lower when the at least one of the sharp signals is higher and a modulation value that is higher when the at least one of the sharp signals is lower, and wherein the output modulation logic is configured to multiply the one or more sharp signals by the modulation value.

12. The image signal processing system of claim 11, wherein the output modulation logic comprises selection logic configured to select the at least one of the sharp signals.

13. The image signal processing system of claim 1, wherein the luma sharpening logic comprises output modulation logic configured to modulate the one or more of the sharp signals before the one or more of the sharp signals are combined with the luma component or the one of the unsharp signals, or the combination thereof, wherein the output modulation logic is configured to increase the one or more sharp signals more when at least one of the sharp signals is higher than when the at least one of the sharp signals is lower.

14. An electronic device comprising:

an imaging device configured to obtain image data; and

an image signal processor configured to process a pixel of the image data, wherein the image signal processor comprises;

luma sharpening logic configured to sharpen a luma component of the pixel by;

determining a plurality of sharp signals at least partly by subtracting a plurality of filtered values of the luma component from the luma component;

summing the sharp signals to obtain a total sharp signal;

reducing the total sharp signal by an amount that is greater when at least one of the sharp signals is lower and therefore more likely to be noise, and an amount that is lesser when the at least one of the sharp signals is higher and therefore less likely to be noise; and

adding the total sharp signal to the luma component or the filtered values of the luma component, or a combination thereof;

or

chroma suppression logic configured to suppress a chroma component of the pixel by;

determining a first attenuation factor based at least in part on one of the sharp signals;

determining a second attenuation factor based at least in part on the luma component; and

suppressing the chroma component based at least in part on the second attenuation factor, or a combination of the first attenuation factor and the second attenuation factor;

or

both the luma suppression logic and the chroma suppression logic.

15. The electronic device of claim 14, wherein the first attenuation factor is configured to attenuate the chroma component more when the one of the sharp signals is higher and less when the one of the sharp signals is lower.

16. The electronic device of claim 14, wherein the first attenuation factor is determined by indexing the one of the sharp signals to a first chroma attenuation lookup table that substantially does not attenuate the chroma component when the one of the sharp signals is substantially 0 but attenuates the chroma component monotonically as the one of the sharp signals increases.

17. The electronic device of claim 14, wherein the first attenuation factor is determined by indexing the one of the sharp signals to a first chroma attenuation lookup table that gradually increasingly attenuates the chroma component as the one of the sharp signals increases until the one of the sharp signals exceeds a threshold, after which the first chroma attenuation lookup table more rapidly attenuates the chroma component until completely attenuating the chroma component as the one of the sharp signals increases.

18. The electronic device of claim 14, wherein the second attenuation factor is configured substantially not to attenuate the chroma component unless the luma component is beneath a first threshold or above a second threshold.

19. The electronic device of claim 14, wherein the second attenuation factor is determined by indexing the one of the sharp signals to a second chroma attenuation lookup table that attenuates the chroma component completely when the luma component is outside of a first and second threshold but does not attenuate the chroma component when the luma component is between the first and second threshold.

20. A method for sharpening image data comprising:

receiving a luma component of a pixel into luma sharpening logic of an image signal processor;

filtering the luma component using a multi-scale unsharp mask to obtain a plurality of unsharp signals;

determining a plurality of sharp signals based at least in part on a difference between the luma component and the unsharp signals;

coring the sharp signals by subtracting a coring threshold from each of the sharp signals, wherein the coring threshold is determined using a lookup table indexed to the luma component or one of the unsharp signals; and

determining a luma-sharpened value of the pixel by adding at least the sharp signals to the luma component or to one of the unsharp signals.

21. The method of claim 20, comprising applying a radial gain to each of the sharp signals before determining the luma-sharpened value of the pixel, wherein the radial gain comprises a spatially varying gain that depends on a spatial relationship of the pixel to an optical center of the image data.

22. The method of claim 21, wherein the radial gain is obtained using a radial gain lookup table indexed to a radius of the pixel from the optical center.