Video image quality

• US 7,430,333 B2
• Filed: 11/17/2004
• Issued: 09/30/2008
• Est. Priority Date: 10/21/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jiang, Yu, Keman, Li, Shipeng
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US10/990,874
• Publication Number:
  US 20060088210A1
• Time in Patent Office: 1,413 Days
• Field of Search: 382/167, 382/274, 382/114, 382/254, 348/E9.042, 348/62, 348/97, 348/615, 348/E13.033, 345/8
• US Class Current: 382/254
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Video image quality may be improved by correcting exposure levels and/or enhancing contrast amounts on each frame. One or more of the following phases may be implemented: skin-color model building, face detecting, exposure level correcting, and contrast enhancing. In a described implementation, each pixel of an image frame is re-exposed using a ratio of contrast amount control variables (CACVs). More specifically, a pixel may be converted to a corresponding light intensity using a first CACV, and the corresponding light intensity may be reconverted to a pixel using a second CACV to enhance the contrast and possibly reduce fuzziness of the image frame. In another described implementation, a Gaussian skin-color model is built for each image frame during runtime. The Gaussian skin-color model is built with training pixels that are selected responsive to a defined skin color range, which is created offline from manually-selected skin pixels of multiple test sequences.

Claims

1. A computer-readable medium having instructions stored thereon for execution by a processor to perform a method comprising:

converting a first pixel value from an image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconverting the corresponding light intensity or a derivative thereof to a second pixel value for the image frame using a second contrast amount control variable value.

2. The computer-readable medium of claim 1, wherein the converting and the reconverting each comprise applying an exposure-density function having an exponential variable comprising the contrast amount control variable.

3. The computer-readable medium of claim 1, further comprising:

repeating the converting and the reconverting for each pixel of the image frame.

4. The computer-readable medium of claim 1, wherein the first contrast amount control variable value is smaller than the second contrast amount control variable value.

5. The computer-readable medium of claim 1, wherein the derivative of the corresponding light intensity comprises a sum of (i) the corresponding light intensity and (ii) a difference between an ideal exposure level and an average luminance value of a detected skin portion.

6. The computer-readable medium of claim 5, wherein the reconverting the corresponding light intensity comprises

reconverting the derivative of the corresponding light intensity to the second pixel value for the image frame using the second contrast amount control variable value to thereby correct an exposure level of at least the detected skin portion of the image frame along with enhancing a contrast amount of the image frame.

7. The computer-readable medium of claim 1, wherein the converting and the reconverting comprise accessing at least one lookup table.

8. The computer-readable medium of claim 1, further comprising:

building a skin-color model for the image frame by selecting training pixels based on a defined skin color range that is created offline.

9. The computer-readable medium of claim 8, further comprising:

detecting a skin portion of the image frame using the skin-color model and a predetermined skin pixel probability threshold.

10. The computer-readable medium of claim 9, further comprising:

correcting an exposure level of the image frame responsive to an average luminance value of the detected skin portion.

11. The computer-readable medium of claim 1, wherein the computer- readable comprise at least one of (i) one or more storage media or (ii) one or more transmission media.

12. A device comprising:

a contrast enhancer that is adapted to enhance a contrast of an image frame by re-exposing each pixel of the image frame using an exposure-density function and a ratio of contrast amount control variables.

13. The device as recited in claim 12, wherein the ratio of contrast amount control variables comprises a first contrast amount control variable value and a second contrast amount control variable value;

and wherein the contrast enhancer is further adapted to convert respective pixels of the image frame to respective corresponding light intensities using the first contrast amount control variable value and to reconvert the respective corresponding light intensities back to respective pixel values using the second contrast amount control variable value.

14. The device as recited in claim 13, further comprising:

at least one lookup table;

wherein the contrast enhancer is further adapted to effectuate the conversion and the reconversion using the at least one lookup table.

15. The device as recited in claim 12, wherein the exposure- density function comprises:

$I=f<br><br>\left(S\right)=\frac{G}{1+{\mathrm{e<br><br>}}^{-A\times <br><br>S}},$

wherein the constant G corresponds to a number of gray levels, the constant A corresponds to the contrast amount control variable, the variable S corresponds to light intensity, and the variable I corresponds to luminance pixel values.

16. The device as recited in claim 12, wherein the contrast enhancer comprises an exposure level corrector and contrast enhancer that is adapted to correct an exposure level of at least an area of interest of the image frame while enhancing the contrast of the image frame.

17. The device as recited in claim 12, further comprising:

a skin-color model builder that is capable of accepting a defined skin color range and the image frame as input, the skin-color model builder adapted to select training pixels from the image frame based on the defined skin color range and to build a Gaussian skin-color model at runtime for the image frame using the selected training pixels;

a face detector that is a capable of accepting the Gaussian skin-color model as input, the face detector adapted to scan pixels of the image frame and to add a particular pixel to a set of face pixels if a probability that the particular pixel belongs to a skin color class exceeds a predetermined skin pixel probability threshold; and

an exposure level corrector that is capable of accepting the set of face pixels as input, the exposure level corrector adapted (i) to calculate a difference between an ideal exposure level and an average luminance value for the set of face pixels and (ii) to re-expose each pixel from the image frame responsive to the calculated difference;

wherein the exposure level corrector and the contrast enhancer are capable of operating substantially simultaneously.

18. An arrangement for video image processing in conjunction with contrast enhancement, the arrangement comprising:

conversion means for converting a first pixel value of an image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconversion means for reconverting the corresponding light intensity or a derivative thereof to a second pixel value for the image frame using a second contrast amount control variable value.

19. The arrangement as recited in claim 18, further comprising:

re-exposure means for re-exposing multiple pixels of the image frame responsive to a luminance value of a detected skin portion of the image frame.

20. The arrangement as recited in claim 19, wherein at least the reconversion means and the re-exposure means operate substantially simultaneously.

21. The arrangement as recited in claim 19, wherein the luminance value comprises an average luminance value;

and wherein the arrangement further comprises;

calculation means for calculating a difference between the average luminance value of the detected skin portion and an ideal exposure level; and

wherein the re-exposure means operates by re-exposing the multiple pixels of the image frame responsive to the calculated difference.

22. The arrangement as recited in claim 19, further comprising:

selection means for selecting training pixels from the image frame based on a predefined skin color range; and

build means for building a skin-color model using the selected training pixels.

23. The arrangement as recited in claim 22, further comprising:

calculation means for calculating a probability that a pixel of the image frame belongs to a skin color class as established by the skin-color model;

determination means for determining if the calculated probability of the pixel exceeds a predetermined skin pixel probability threshold; and

addition means for adding the pixel to a set of skin pixels, which establish the detected skin portion, in accordance with the determination of the determination means.

24. The arrangement as recited in claim 18, wherein the arrangement comprises at least one of (i) one or more processor-accessible media or (ii) at least one device.

25. A computer-readable medium having instructions stored thereon for execution by a processor to perform a method comprising:

correcting an exposure level of an image frame; and

enhancing a contrast amount of the image frame using a ratio of contrast amount control variable values and an exposure-density function.

26. The computer-readable medium of claim 25, wherein the correcting and the enhancing are performed partially or fully simultaneously.

27. The computer-readable medium of claim 25, wherein the correcting and the enhancing are effectuated using at least one lookup table.

28. The computer-readable medium of claim 27, wherein the correcting and the enhancing are effectuated using at least three lookup tables;

and wherein a first and a second lookup table of the at least three lookup tables are created offline, and a third lookup table of the at least three lookup tables is established at runtime.

29. The computer-readable medium of claim 25, wherein the correcting comnprises:

calculating a difference between an ideal exposure level and an average luminance value for a region of the image frame; and

re-exposing each pixel of the image frame responsive to the calculated difference.

30. The computer-readable medium of claim 29, further comprising:

skipping the re-exposing and the enhancing when the average luminance value for the region of the image frame is relatively close to the ideal exposure level.

31. The computer-readable medium of claim 25, wherein the enhancing a contrast amount of the image frame comprises:

converting a first pixel value from the image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconverting the corresponding light intensity or a derivative thereof to a second pixel value for the image frame using a second contrast amount control variable value;

wherein the first contrast amount control variable value and the second contrast amount control variable value comprise the ratio of contrast amount control variable values.

32. The computer-readable medium of claim 31, wherein the correcting an exposure level of an image frame comprises:

calculating a difference between an ideal exposure level and an average luminance value for a region of the image frame; and

the reconverting the derivative of the corresponding light intensity to the second pixel value for the image frame comprises;

reconverting the derivative of the corresponding light intensity to the second pixel value for the image frame using the second contrast amount control variable value;

wherein the derivative of the corresponding light intensity is produced using the calculated difference.

33. The computer-readable medium of claim 32, wherein the correcting and the enhancing are performed using a lookup table that is produced at runtime and that incorporates the calculated difference, and wherein the lookup table maps the first pixel value to the second pixel value.

34. The computer-readable medium of claim 25, further comprising:

detecting a skin portion of the image frame using a skin-color model;

wherein the correcting an exposure level of an image frame comprises;

calculating a difference between an ideal exposure level and an average luminance value for the detected skin portion of the image frame; and

re-exposing each pixel of the image frame responsive to the calculated difference.

35. The computer-readable medium of claim 34, further comprising:

building the skin-color model during runtime.

36. The computer-readable medium of claim 35, wherein the building the skin-color model during runtime comprises:

selecting training pixels from the image frame based on a skin color range that is defined offline; and

building the skin-color model using the selected training pixels.

37. A method comprising:

building a skin-color model using, selected training pixels;

detecting a facial portion of an image frame using the skin-color model and a skin pixel probability threshold;

correcting an exposure level of the image frame with regard to the detected facial portion; and

enhancing a contrast amount of the image frame by re-exposing pixels of the image frame with a first contrast amount control variable value and a second contrast amount control variable value.

38. The method as recited in claim 37, further comprising:

selecting the selected training pixels from the image frame based on a predefined skin color range, the predefined skin color range created offline.

39. The method as recited in claim 37, wherein the enhancing comprises:

converting first pixel values from the image frame to corresponding light intensities using the first contrast amount control variable value; and

reconverting the corresponding light intensities or derivatives thereof to second pixel values for the image frame using the second contrast amount control variable value;

wherein the converting and the reconverting are effectuated, at least partially, using an exposure-density function.