Image enhancement unit and method for image quality improvement of a video stream

• US 20050099545A1
• Filed: 05/04/2004
• Published: 05/12/2005
• Est. Priority Date: 11/10/2003
• Status: Active Grant

• Current Assignee: Zhangjiagang Kangde Xin Optronics Material Co., Ltd. (Kangde Xin Composite Material Group Co., Ltd.)
• Original Assignee: Huaya Microelectronics, Inc.
• Inventors: Zhu, Ge
• Granted Patent: US 7,561,209 B2
• Time in Patent Office: Days
• US Class Current: 348/630
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: H04N 23/86 for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

An image enhancement unit and method for enhancing the image quality of a video stream is presented. The image enhancement unit combines a contour enhancement unit, a luminance contrast stretching unit, a color transient improvement unit, and a color saturation control unit to enhance both the luminance and chrominance values in the video stream. Specifically, the contour enhancement unit processes the luminance values to improve the contours of the images in the video steam. The luminance contrast stretching unit further improves the luminance by enhancing the contrast in the images. The color transient improvement unit processes the chrominance values to remove blurring around along the edges of color transitions and the color saturation control unit increases the color saturation to improve the appearance of the images.

Claims

1. An image enhancement unit for improving the quality of an image having a plurality of pixels, the image enhancement unit comprising:

a contour enhancement unit configured to process luminance information from the image; and

a color transient improvement unit configured to process chrominance information from the image.

2. The image enhancement unit of claim 1, wherein the contour enhancement unit enhances a plurality of local contours of the image based on a contour enhancement factor.

3. The image enhancement unit of claim 2, wherein the contour enhancement factor is user programmable.

4. The image enhancement unit of claim 1, wherein the contour enhancement unit comprises:

a contour detection unit configured to determine the dominant contour direction of a current pixel; and

a contour enhanced luminance calculation unit coupled to the contour detection unit and configured to generate an enhanced luminance value for the current pixel, wherein the enhanced luminance value enhances the contour of the current pixel in the dominant contour direction.

5. The image enhancement unit of claim 4, wherein the contour detection unit is configured calculate a plurality of slopes for a plurality of slope directions and to select a dominant slope direction as the dominant contour direction, wherein the dominant slope direction has a largest magnitude from among the slope directions.

6. The image enhancement unit of claim 4, wherein an enhanced slope of the dominant contour direction calculated using the enhanced luminance value is equal to a contour enhancement factor multiplied by a slope of the dominant contour direction.

7. The image enhancement unit of claim 4, wherein the contour enhancement unit further comprises a contour threshold comparison unit coupled to the contour detection unit and the contour enhanced luminance calculation unit.

8. The image enhancement unit of claim 7, wherein the contour threshold unit controls the contour enhanced luminance calculation unit based on the magnitude of the dominant contour direction and the magnitude of the secondary contour direction.

9. The image enhancement unit of claim 1, wherein, the color transient improvement unit further comprises:

a transient characterization unit configured to calculate one or more transient characterization values for a color transient detection window containing a current pixel;

a transient threshold unit coupled to the transient characterization unit and configured to determine whether a color transient exists in the color transient detection window by comparing the transient characterization values to a chrominance color improvement threshold; and

a transient correction unit coupled to the transient threshold unit and configured to generate an improved U chrominance value and an improved V chrominance value for the current pixel.

10. The image enhancement unit of claim 9, wherein the color transient improvement unit further comprises a line averaging unit coupled to the transient characterization unit and configured to calculate an average U chrominance value and an average V chrominance value for each column of the color transient detection window to form a plurality of average U chrominance values and a plurality of average V chrominance values.

11. The image enhancement unit of claim 10, wherein the transient characterization unit is configured to calculate a first U transient characterization value corresponding to a degree that the plurality of average U chrominance values is monotonically increasing and to calculate a first V transient characterization value corresponding to a degree that the plurality of average V chrominance values is monotonically increasing.

12. The image enhancement unit of claim 11, wherein the transient characterization unit is configured to calculate a second U transient characterization value corresponding to a degree that the plurality of average U chrominance values is monotonically decreasing and to calculate a second V transient characterization value corresponding to a degree that the plurality of average V chrominance values is monotonically decreasing.

13. The image enhancement unit of claim 9, wherein the color transient improvement unit further comprises an averaging unit configured to calculating a left side average U chrominance value, a right side average U chrominance value, a left side average V chrominance value, and a right side average V chrominance value.

14. The image enhancement unit of claim 1, further comprising a color saturation control unit coupled to the color transient improvement unit.

15. The image enhancement unit of claim 14, wherein the image has a maximum chrominance value and a minimum chrominance value and wherein the color saturation control unit further comprises:

a scaled saturation enhancement factor calculation unit configured to generate a scaled saturation enhancement factor using a base saturation enhancement factor;

a multiplier coupled to receive the scaled saturation enhancement factor and configured to generate a saturation enhanced U chrominance value by multiplying a current U chrominance value of a current pixel by the scaled saturation enhancement factor and to generate a saturation enhanced V chrominance value by multiplying a current V chrominance value of a current pixel by the scaled saturation enhancement factor; and

wherein the saturation enhanced U chrominance value divided by the saturation enhanced V chrominance value is equal to the current U chrominance value divided by the current V chrominance value of the current pixel.

16. The image enhancement unit of claim 15, wherein the scaled saturation enhancement factor calculation unit comprises:

an index calculation and control circuit coupled to receive the base saturation enhancement factor, the current U chrominance value, and the current V chrominance value and configured to generate an index;

a lookup table coupled to the index calculation and control circuit and configured to provide a tabled saturation enhancement factor.

17. The image enhancement unit of claim 16, wherein the index calculation and control circuit is also configured to generate a preset saturation enhancement factor, wherein the preset saturation enhancement factor is equal to one when the current U chrominance is equal to the maximum chrominance value, when the current U chrominance value is equal to the minimum chrominance value, when the current V chrominance value is equal to the maximum chrominance value, and when the current V chrominance value is equal to the minimum chrominance value.

18. The image enhancement unit of claim 17, wherein the preset saturation enhancement factor is equal to the base saturation enhancement factor when the current U chrominance is not equal to the maximum chrominance value, the current U chrominance value is not equal to the minimum chrominance value, the current V chrominance value is not equal to the maximum chrominance value, and the current V chrominance value is not equal to the minimum chrominance value.

19. The image enhancement unit of claim 16, further comprising a multiplexing circuit having a first input port coupled to receive the base saturation enhancement factor, a second input port coupled to receive the tabled saturation enhancement factor, a third input port coupled to receive a value of one, and an output port configured to drive the scaled saturation enhancement factor.

20. The image enhancement unit of claim 16, wherein the index calculation and control circuit sets the index equal to a maximum value from among the absolute value of the current U chrominance value and the absolute value of the current V chrominance value.

21. The image enhancement unit of claim 16, wherein the lookup table comprises a plurality of table entries each corresponding to a table index value.

22. The image enhancement unit of claim 21, wherein a table entry is equal to 2 to a negative power of a bit precision value multiplied by an integer value of the maximum chrominance value multiplied by 2 carried to the power of a bit precision value divided by the corresponding table index.

23. The image enhancement unit of claim 1, further comprising a luminance contrast stretching unit coupled to the contour enhancement unit.

24. A method of enhancing an image having a plurality of pixels, the method comprising:

performing contour enhancement on luminance data in the image; and

performing color transient improvement on chrominance data in the image.

25. The method of claim 24, wherein the performing contour enhancement on luminance data in the image further comprises:

determining a dominant contour direction for a current pixel; and

enhancing a contour of the current pixel by a contour enhancement factor along the dominant contour direction.

26. The method of claim 24, wherein the determining a dominant contour direction for a current pixel comprises:

calculating a plurality of slopes for a plurality of slope directions for the current pixel;

selecting a dominant slope direction from the plurality of slope directions as the dominant direction of the contour of the current pixel, wherein the dominant slope direction has a largest absolute slope value from among the slope directions.

27. The method of claim 26, wherein the calculating a plurality of slopes for the current pixel comprises:

calculating a vertical slope for the current pixel;

calculating a horizontal slope for the current pixel; and

calculating a diagonal slope for the current pixel.

28. The method of claim 27, wherein the diagonal slope for the current pixel is equal to the luminance of the current pixel minus an average luminance of a first pixel above the current pixel, a second pixel below the current pixel, a third pixel to the left of the current pixel, and a fourth pixel to the right of the current pixel.

29. The method of claim 27, wherein the vertical slope is equal to a first average luminance of the current pixel, a first pixel above of the current pixel and a second pixel below the current pixel minus a second average luminance of a third pixel above and to the left of the current pixel, a fourth pixel to the left of the current pixel, a fifth pixel below and to the left of the current pixel, a sixth pixel above and to the right of the current pixel, a seventh pixel to the right of the current pixel, and an eighth pixel below and to the right of the current pixel.

30. The method of claim 27, wherein the horizontal slope is equal to a first average luminance of the current pixel, a first pixel to the left of the current pixel and a second pixel to the right of the current pixel minus a second average luminance of a third pixel above and to the left of the current pixel, a fourth pixel above the current pixel, a fifth pixel above and to the right of the current pixel, a sixth pixel below and to the left of the current pixel, a seventh pixel below the current pixel, and an eighth pixel below and to the right of the current pixel.

31. The method of claim 25, further comprising calculating a secondary contour direction for the current pixel.

32. The method of claim 31, wherein the enhancing a contour of the current pixel by a contour enhancement factor along the dominant contour direction is performed only when the magnitude of the dominant contour direction minus the magnitude of the secondary contour direction is greater than a contour enhancement threshold.

33. The method of claim 25, wherein the enhancing a contour of the current pixel by a contour enhancement factor along the dominant contour direction further comprises replacing the luminance value of the current pixel with an enhanced luminance value such that an enhanced slope of the dominant contour direction calculated using the enhanced luminance value is equal to a contour enhancement factor multiplied by a slope of the dominant contour direction.

34. The method of claim 33, wherein the enhanced slope of the dominant contour direction is calculated using the enhanced luminance value of the current pixel and the slope of the dominant contour direction is calculated using the luminance value of the current pixel.

35. The method of claim 24, wherein the performing color transient improvement on chrominance data in the image further comprises:

defining a color transient detection window around a current pixel;

calculating one or more transient characterization values for the color transient detection window, wherein the transient characterization values correspond to a likelihood of a color transition occurring in the color transient detection window; and

generating an improved U chrominance value and an improved V chrominance value for the current pixel.

36. The method of claim 35, wherein the color transient detection window includes 3 lines of 7 pixels centered around the current pixel.

37. The method of claim 35, wherein calculating one or more transient characterization values for the color transient detection window further comprises:

calculating an average U chrominance value for each column of the color transient detection window to form a plurality of average U chrominance values; and

calculating an average V chrominance value for each column of the color transient detection window to form a plurality of average V chrominance values.

38. The method of claim 37, wherein calculating one or more transient characterization values for the color transient detection window further comprises:

calculating a first U transient characterization value corresponding to a degree that the plurality of average U chrominance values is monotonically increasing; and

calculating a first V transient characterization value corresponding to a degree that the plurality of average V chrominance values is monotonically increasing.

39. The method of claim 38, wherein calculating one or more transient characterization values for the color transient detection window further comprises:

calculating a second U transient characterization value corresponding to a degree that the plurality of average U chrominance values is monotonically decreasing; and

calculating a second V transient characterization value corresponding to a degree that the plurality of average V chrominance values is monotonically decreasing.

40. The method claim 35, wherein generating an improved U chrominance value and an improved V chrominance value for the current pixel comprises:

defining a U chrominance proxy for the U chrominance value of the current pixel;

defining a V chrominance proxy for the V chrominance value of the current pixel;

calculating an average left side U chrominance value;

calculating an average right side U chrominance value;

calculating an average left side V chrominance value;

calculating an average right side V chrominance value;

setting the improved U chrominance value and the improved V chrominance value equal to the average left side U chrominance value and the average left side V chrominance value, respectively, when a color transient exists in the color transient detection window, the U chrominance proxy is closer to the average left side U chrominance value than to the average right side U chrominance value, and the V chrominance proxy is closer to the average left side V chrominance value than to the average right side V chrominance value; and

setting the improved U chrominance value and the improved V chrominance value equal to the average right side U chrominance value and the average right side V chrominance value, respectively, when a color transient exists in the color transient detection window, the U chrominance proxy is closer to the average right side U chrominance value than to the average left side U chrominance value, and the V chrominance proxy is closer to the average right side V chrominance value than to the average left side V chrominance value.

41. The method of claim 40, wherein

the average left side U chrominance value is equal to an average of U chrominance values from pixels in the color transient detection window to the left of the current pixel;

and

the average right side U chrominance value is equal to an average of U chrominance values from pixels in the color transient detection window to the right of the current pixel.

42. The method of claim 40, wherein

the average left side V chrominance value is equal to an average of V chrominance values from pixels in the color transient detection window to the left of the current pixel;

and

the average right side V chrominance value is equal to an average of V chrominance values from pixels in the color transient detection window to the right of the current pixel.

43. The method of claim 24, further comprising enhancing a color saturation of the image.

44. The method of claim 43, wherein the image has a maximum chrominance value and a minimum chrominance value and wherein the enhancing a color saturation of the image further comprises:

generating a scaled saturation enhancement factor using a base saturation enhancement factor;

generating a saturation enhanced U chrominance value by multiplying a current U chrominance value of a current pixel by the scaled saturation enhancement factor, wherein the saturation enhanced U chrominance value is within a range bounded by maximum chrominance value and the minimum chrominance value inclusive;

generating a saturation enhanced V chrominance value by multiplying a current V chrominance value of the current pixel by the scaled saturation enhancement factor, wherein the saturation enhanced V chrominance value within a range bounded by maximum chrominance value and the minimum chrominance value inclusive; and

wherein the saturation enhanced U chrominance value divided by the saturation enhanced V chrominance value is equal to the current U chrominance value divided by the current V chrominance value of the current pixel.

45. The method of claim 44, wherein generating a scaled saturation enhancement factor based on a base saturation enhancement factor comprises:

calculating a maximum U enhancement factor;

calculating a maximum V enhancement factor;

setting the scaled saturation enhancement factor to a smallest value from among the maximum U enhancement factor, the maximum V enhancement factor, and the base saturation enhancement factor.

46. The method of claim 45 wherein the maximum U enhancement factor is equal to a largest value from among an absolute value of dividing the maximum chrominance value by the current U chrominance value and the absolute value of dividing the minimum chrominance value by the current U chrominance value.

47. The method of claim 46, wherein the maximum U enhancement factor is set to a predefined large number when the current U chrominance value is equal to zero.

48. The method of claim 45 wherein the maximum V enhancement factor is equal to a largest value from among an absolute value of dividing the maximum chrominance value by the current V chrominance value and the absolute value of dividing the minimum chrominance value by the current V Chrominance value.

49. The method of claim 44, wherein generating a scaled saturation enhancement factor based on a base saturation enhancement factor further comprises:

generating an index for a look-up table that provides a tabled saturation enhancement factor;

selecting the scaled saturation enhancement factor from among a tabled saturation enhancement factor, the base saturation enhancement factor and one based on the index.

50. The method of claim 49, wherein selecting the scaled saturation enhancement factor from among a tabled saturation enhancement factor, the base saturation enhancement factor and one based on the index comprises:

setting the scaled saturation enhancement factor equal to one when the index is greater than or equal to the maximum chrominance value;

setting the scaled saturation enhancement factor equal to the base saturation enhancement factor when the index multiplied by the base saturation enhancement factor is less than or equal to the maximum chrominance value; and

setting the scaled saturation enhancement factor equal to the tabled saturation enhancement factor when the index is less than the maximum chrominance value and the index multiplied by the base saturation enhancement factor is greater than the maximum chrominance value.

51. The method of claim 49, wherein the generating an index for a lookup table that provides a tabled saturation enhancement factor comprises setting the index equal to the greater from among the absolute value of the current U chrominance value and the absolute value of the current V chrominance value.

52. The method of claim 51, wherein the generating an index for a table that provides a tabled saturation enhancement factor further comprises setting the index to the maximum chrominance value when the index is greater than the maximum chrominance value.

53. The method of claim 51, wherein the lookup table comprises a plurality of table entries each corresponding to a table index value.

54. The method of claim 53, wherein a table entry is equal to 2 to a negative power of a bit precision value multiplied by an integer value of the maximum chrominance value multiplied by 2 carried to the power of a bit precision value divided by the corresponding table index.