Video image quality

• US 20060088210A1
• Filed: 11/17/2004
• Published: 04/27/2006
• Est. Priority Date: 10/21/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jiang, Yu, Keman, Li, Shipeng
• Granted Patent: US 7,430,333 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Video image quality may be improved by correcting exposure levels and/or enhancing contrast amounts on each frame. One or more of the following phases may be implemented: skin-color model building, face detecting, exposure level correcting, and contrast enhancing. In a described implementation, each pixel of an image frame is re-exposed using a ratio of contrast amount control variables (CACVs). More specifically, a pixel may be converted to a corresponding light intensity using a first CACV, and the corresponding light intensity may be reconverted to a pixel using a second CACV to enhance the contrast and possibly reduce fuzziness of the image frame. In another described implementation, a Gaussian skin-color model is built for each image frame during runtime. The Gaussian skin-color model is built with training pixels that are selected responsive to a defined skin color range, which is created offline from manually-selected skin pixels of multiple test sequences.

Claims

1. One or more processor-accessible media comprising processor-executable instructions that, when executed, direct a device to perform actions comprising:

converting a first pixel value from an image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconverting the corresponding light intensity or a derivative thereof to a second pixel value for the image frame using a second contrast amount control variable value.

2. The one or more processor-accessible media as recited in claim 1, wherein the actions of converting and reconverting each comprise an action of:

applying an exposure-density function having an exponential variable comprising the contrast amount control variable.

3. The one or more processor-accessible media as recited in claim 1, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

repeating the actions of converting and reconverting for each pixel of the image frame.

4. The one or more processor-accessible media as recited in claim 1, wherein the first contrast amount control variable value is smaller than the second contrast amount control variable value.

5. The one or more processor-accessible media as recited in claim 1, wherein the derivative of the corresponding light intensity comprises a sum of (i) the corresponding light intensity and (ii) a difference between an ideal exposure level and an average luminance value of a detected skin portion.

6. The one or more processor-accessible media as recited in claim 5, wherein the action of reconverting comprises an action of:

reconverting the derivative of the corresponding light intensity to the second pixel value for the image frame using the second contrast amount control variable value to thereby correct an exposure level of at least the detected skin portion of the image frame along with enhancing a contrast amount of the image frame.

7. The one or more processor-accessible media as recited in claim 1, wherein the actions of converting and reconverting comprise at least one action of:

accessing at least one lookup table.

8. The one or more processor-accessible media as recited in claim 1, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

building a skin-color model for the image frame by selecting training pixels therefor based on a defined skin color range that is created offline.

9. The one or more processor-accessible media as recited in claim 8, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

detecting a skin portion of the image frame using the skin-color model and a predetermined skin pixel probability threshold.

10. The one or more processor-accessible media as recited in claim 9, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

correcting an exposure level of the image frame responsive to an average luminance value of the detected skin portion.

11. The one or more processor-accessible media as recited in claim 1, wherein the one or more processor-accessible media comprise at least one of (i) one or more storage media or (ii) one or more transmission media.

12. A device comprising:

a contrast enhancer that is adapted to enhance a contrast of an image frame by re-exposing each pixel of the image frame using an exposure-density function and a ratio of contrast amount control variables.

13. The device as recited in claim 12, wherein the ratio of contrast amount control variables comprises a first contrast amount control variable value and a second contrast amount control variable value;

and wherein the contrast enhancer is further adapted to convert respective pixels of the image frame to respective corresponding light intensities using the first contrast amount control variable value and to reconvert the respective corresponding light intensities back to respective pixel values using the second contrast amount control variable value.

14. The device as recited in claim 13, further comprising:

at least one lookup table;

wherein the contrast enhancer is further adapted to effectuate the conversion and the reconversion using the at least one lookup table.

15. The device as recited in claim 12, wherein the exposure-density function comprises:

$I=f<br><br>\left(S\right)=\frac{G}{1+{\mathrm{e<br><br>}}^{-A\times <br><br>S}},$

wherein the constant G corresponds to a number of gray levels, the constant A corresponds to the control amount control variable, the variable, the variable S corresponds to light intensity, and the variable I corresponds to luminance pixel values.

16. The device as recited in claim 12, wherein the contrast enhancer comprises an exposure level corrector and contrast enhancer that is adapted to correct an exposure level of at least an area of interest of the image frame while enhancing the contrast of the image frame.

17. The device as recited in claim 12, further comprising:

a skin-color model builder that is capable of accepting a defined skin color range and the image frame as input, the skin-color model builder adapted to select training pixels from the image frame based on the defined skin color range and to build a Gaussian skin-color model at runtime for the image frame using the selected training pixels;

a face detector that is a capable of accepting the Gaussian skin-color model as input, the face detector adapted to scan pixels of the image frame and to add a particular pixel to a set of face pixels if a probability that the particular pixel belongs to a skin color class exceeds a predetermined skin pixel probability threshold; and

an exposure level corrector that is capable of accepting the set of face pixels as input, the exposure level corrector adapted (i) to calculate a difference between an ideal exposure level and an average luminance value for the set of face pixels and (ii) to re-expose each pixel from the image frame responsive to the calculated difference;

wherein the exposure level corrector and the contrast enhancer are capable of operating substantially simultaneously.

18. An arrangement for video image processing in conjunction with contrast enhancement, the arrangement comprising:

conversion means for converting a first pixel value of an image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconversion means for converting the corresponding light intensity or a derivative therof to a second pixel value for the image frame using a second contrast amount control variable value.

19. The arrangement as recited in claim 18, further comprisingL

re-exposure means for re-exposing multiple pixels of the image frame responsive to a luminance value of a detected skin portion of the image frame.

20. The arrangement as recited in claim 19, wherein at least the reconversion means and the re-exposure means operate substantially simultaneously.

21. The arrangement as recited in claim 19, wherein the luminance value comprises an average luminance value;

and wherein the arrangement further comprises;

calculation means for calculating a difference between the average luminance value of the detected skin portion and an ideal exposure level; and

wherein the re-exposure means operates by re-exposing the multiple pixels of the image frame responsive to the calculated difference.

22. The arrangement as recited in claim 19, further comprising:

selection means for selecting training pixels from the image frame based on a predefined skin color range; and

build means for building a skin-color model using the selected training pixels.

23. The arrangement as recited in claim 22, further comprising:

calculation means for calculating a probability that a pixel of the image frame belongs to a skin color class as established by the skin-color model;

determination means for determining if the calculated probability of the pixel exceeds a predetermined skin pixel probability threshold; and

addition means for adding the pixel to a set of skin pixels, which establish the detected skin portion, in accordance with the determination of the determination means.

24. The arrangement as recited in claim 18, wherein the arrangement comprises at least one of (i) one or more processor-accessible media or (ii) at least one device.

25. One or more processor-accessible media comprising processor-executable instructions that, when executed, direct a device to perform actions comprising:

correcting an exposure level of an image frame; and

enhancing a contrast amount of the image frame using a ratio of contrast amount control variable values and an exposure-density function.

26. The one or more processor-accessible media as recited in claim 25, wherein the actions of correcting and enhancing are performed partially or fully simultaneously.

27. The one or more processor-accessible media as recited in claim 25, wherein the actions of correcting and enhancing are effectuated using at least one lookup table.

28. The one or more processor-accessible media as recited in claim 27, wherein the actions of correcting and enhancing are effectuated using at least three lookup tables;

and wherein a first and a second lookup table of the at least three lookup tables are created offline, and a third lookup table of the at least three lookup tables is established at runtime.

29. The one or more processor-accessible media as recited in claim 25, wherein the action of correcting comprises actions of:

calculating a difference between an ideal exposure level and an average luminance value for a region of the image frame; and

re-exposing each pixel of the image frame responsive to the calculated difference.

30. The one or more processor-accessible media as recited in claim 29, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

skipping the actions of re-exposing and enhancing when the average luminance value for the region of the image frame is relatively close to the ideal exposure level.

31. The one or more processor-accessible media as recited in claim 25, wherein the action of enhancing comprises actions of:

converting a first pixel value from the image frame to a corresponding light intensity using a first contrast amount control variable value; and

reconverting the corresponding light intensity or a derivative thereof to a second pixel value for the image frame using a second contrast amount control variable value;

wherein the first contrast amount control variable value and the second contrast amount control variable value comprise the ratio of contrast amount control variable values.

32. The one or more processor-accessible media as recited in claim 31, wherein:

the action of correcting comprises an action of;

calculating a difference between an ideal exposure level and an average luminance value for a region of the image frame; and

the action of reconverting comprises an action of;

reconverting the derivative of the corresponding light intensity to the second pixel value for the image frame using the second contrast amount control variable value;

wherein the derivative of the corresponding light intensity is produced using the calculated difference.

33. The one or more processor-accessible media as recited in claim 32, wherein the actions of correcting and enhancing are performed using a lookup table that is produced at runtime and that incorporates the calculated difference, and wherein the lookup table maps the first pixel value to the second pixel value.

34. The one or more processor-accessible media as recited in claim 25, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

detecting a skin portion of the image frame using a skin-color model;

wherein the action of correcting comprises actions of;

calculating a difference between an ideal exposure level and an average luminance value for the detected skin portion of the image frame; and

re-exposing each pixel of the image frame responsive to the calculated difference.

35. The one or more processor-accessible media as recited in claim 34, wherein the processor-executable instructions, when executed, direct the device to perform a further action comprising:

building the skin-color model during runtime.

36. The one or more processor-accessible media as recited in claim 35, wherein the action of building comprises actions of:

selecting training pixels from the image frame based on a skin color range that is defined offline; and

building the skin-color model using the selected training pixels.

37. A method comprising:

building a skin-color model using selected training pixels;

detecting a facial portion of an image frame using the skin-color model and a skin pixel probability threshold;

correcting an exposure level of the image frame with regard to the detected facial portion; and

enhancing a contrast amount of the image frame by re-exposing pixels of the image frame with a first contrast amount control variable value and a second contrast amount control variable value.

38. The method as recited in claim 37, further comprising:

selecting the selected training pixels from the image frame based on a predefined skin color range, the predefined skin color range created offline.

39. The method as recited in claim 37, wherein the enhancing comprises:

converting first pixel values from the image frame to corresponding light intensities using the first amount control variable value; and

converting the corresponding light intensities or derivatives thereof to second pixel values for the image frame using the second contrast amount control variable value;

wherein the converting and the reconverting are effectuated, at least partially, using an exposure-density function.