Video image quality

• US 20060088209A1
• Filed: 10/21/2004
• Published: 04/27/2006
• Est. Priority Date: 10/21/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jiang, Yu, Keman, Li, Shipeng
• Granted Patent: US 7,542,600 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Video image quality may be improved by correcting exposure levels and/or enhancing contrast amounts on each frame. One or more of the following phases may be implemented: skin-color model building, face detecting, exposure level correcting, and contrast enhancing. In a described implementation, a Gaussian skin-color model is built for each image frame during runtime. The Gaussian skin-color model is built with training pixels that are selected responsive to a defined skin color range, which is created offline from manually-selected skin pixels of multiple test sequences. In another described implementation, each pixel of an image frame is re-exposed using a ratio of contrast amount control variables (CACVs). More specifically, a pixel may be converted to a corresponding light intensity using a first CACV, and the corresponding light intensity may be reconverted to a pixel using a second CACV to enhance the contrast and possibly reduce fuzziness of the image frame.

Claims

1. One or more processor-accessible media comprising processor-executable instructions that, when executed, direct a device to perform actions comprising:

selecting training pixels from an image frame based on a defined skin color range;

building a skin-color model using the selected training pixels;

detecting a skin portion of the image frame using the skin-color model; and

adjusting the image frame with regard to the detected skin portion.

2. The one or more processor-accessible media as recited in claim 1, wherein the action of adjusting comprises an action of:

correcting an exposure level of the image frame responsive to an average luminance value of the detected skin portion.

3. The one or more processor-accessible media as recited in claim 1, wherein the action of adjusting comprises an action of:

enhancing a contrast amount of the image frame using an exposure-density function.

4. The one or more processor-accessible media as recited in claim 1, wherein the action of building comprises an action of:

building a two-dimensional Gaussian skin-color model.

5. The one or more processor-accessible media as recited in claim 1, wherein the action of selecting comprises an action of:

selecting the training pixels based on the defined skin color range;

(i) the defined skin color range comprising at least one range of Gaussian distribution centers, and (ii) the defined skin color range created offline using a collection of test sequences, respective sets of skin pixels manually selected from respective ones of the collected test sequences, and respective Gaussian distribution centers calculated from respective Gaussian skin-color models built responsive to the respective sets of manually-selected skin pixels of the respective ones of the collected test sequences.

6. The one or more processor-accessible media as recited in claim 1, wherein the action of selecting comprises an action of:

selecting the training pixels based on the defined skin color range in a YCbCr color space, the defined skin color range including a range corresponding to Cb and a range corresponding to Cr.

7. The one or more processor-accessible media as recited in claim 1, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising:

determining a center of the skin-color model; and

ascertaining if the determined center comports with at least one reliability range;

wherein if the determined center is ascertained to comport with the at least one reliability range, the action of detecting comprises using a current skin-color model built for a current image frame to detect the skin portion of the current image frame.

8. The one or more processor-accessible media as recited in claim 1, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising:

determining a center of the skin-color model; and

ascertaining if the determined center comports with at least one reliability range;

wherein if the determined center is not ascertained to comport with the at least one reliability range, the action of detecting comprises using a previous skin-color model built for a previous image frame to detect the skin portion of a current image frame.

9. The one or more processor-accessible media as recited in claim 1, wherein the processor-executable instructions, when executed, direct the device to perform further actions comprising:

determining a center of the skin-color model; and

ascertaining if the determined center comports with at least one reliability range;

wherein the at least one reliability range comprises a summed reliability range and a difference reliability range that are produced offline from tracked centers of multiple respective Gaussian skin-color models of multiple respective test sequences.

10. The one or more processor-accessible media as recited in claim 1, wherein the one or more processor-accessible media comprise at least one of (i) one or more storage media or (ii) one or more transmission media.

11. A device comprising:

a skin-color model builder that is capable of accepting a defined skin color range and an image frame as input, the skin-color model builder adapted to select training pixels from the image frame based on the defined skin color range and to build a skin-color model at runtime for the image frame using the selected training pixels.

12. The device as recited in claim 11, wherein the device comprises (i) a mobile telephone, (ii) video conferencing equipment, or (iii) at least part of a personal computer.

13. The device as recited in claim 11, wherein the defined skin color range is expressed in terms of chrominance-B and chrominance-R in a YCbCr color space.

14. The device as recited in claim 11, wherein the skin-color model builder is further adapted to scan the image frame for pixels having chrominance values fitting within the defined skin color range and to select those pixels as the training pixels.

15. The device as recited in claim 11, wherein the skin-color model builder is further capable of accepting at least one reliability range as input, the skin-color model builder further adapted to perform a reliability examination on the skin-color model built for the image frame using the at least one reliability range.

16. The device as recited in claim 15, wherein the skin-color model built by the skin-color model builder comprises a Gaussian skin-color model;

and wherein the at least one reliability range includes a summed reliability range and a difference reliability range that are determined from multiple tracked centers of multiple Gaussian skin-color models from multiple test sequences, and wherein the reliability examination comprises ascertaining if a center of the built Gaussian skin-color model comports with the summed reliability range and the difference reliability range.

17. The device as recited in claim 16, further comprising:

a face detector that is adapted to detect at least part of a face of the image frame utilizing a given Gaussian skin-color model;

wherein the Gaussian skin-color model built by the skin-color model builder for the image frame is utilized by the face detector to detect the at least part of the face when the built Gaussian skin-color model is ascertained to comport with the summed reliability range and the difference reliability range, and another Gaussian skin-color model built for a previous image frame is utilized by the face detector to detect the at least part of the face when the built Gaussian skin-color model is ascertained to not comport with the summed reliability range and the difference reliability range.

18. The device as recited in claim 11, further comprising:

a face detector that is a capable of accepting the skin-color model from the skin-color model builder, the face detector adapted to scan pixels of the image frame and to use the skin-color model to add a particular pixel to a set of face pixels if a probability that the particular pixel belongs to a skin color class exceeds a predetermined skin pixel probability threshold.

19. The device as recited in claim 11, further comprising:

an exposure level corrector that is adapted (i) to calculate a difference between an ideal exposure level and an average luminance value for a set of face pixels that are detected in the image frame using the skin-color model and (ii) to re-expose each pixel from the image frame responsive to the calculated difference.

20. The device as recited in claim 11, further comprising:

a contrast enhancer that is adapted to convert each pixel from the image frame to a corresponding light intensity using a first contrast amount control variable value and to reconvert the corresponding light intensity back to a pixel value using a second contrast amount control variable value.

21. The device as recited in claim 11, further comprising:

an exposure level corrector and contrast enhancer that is adapted to re-expose each pixel from the image frame utilizing an exposure-density function (i) responsive to a difference between an ideal exposure level and an average luminance value of a set of skin pixels of the image frame and (ii) using a ratio of contrast amount control variables.

22. An arrangement for video image processing in conjunction with skin color modeling, the arrangement comprising:

selection means for selecting training pixels from an image frame using a predefined skin color range; and

build means for building a skin-color model using the selected training pixels.

23. The arrangement as recited in claim 22, further comprising:

means for conducting a reliability examination to ascertain if at least one attribute of the skin-color model comports with a reliability range; and

utilization means for utilizing the skin-color model built for the image frame when detecting a skin portion in the image frame if the at least one attribute is ascertained to comport with the reliability range and for utilizing another skin-color model built for a previous image frame when detecting a skin portion in the image frame if the at least one attribute is not ascertained to comport with the reliability range.

24. The arrangement as recited in claim 22, further comprising:

detection means for detecting a skin portion of the image frame using the skin-color model and a skin pixel probability threshold.

25. The arrangement as recited in claim 22, further comprising:

correction means for correcting an exposure level of the image frame responsive to an ideal exposure level and an average luminance value of a set of skin pixels detected using the skin-color model.

26. The arrangement as recited in claim 22, further comprising:

enhancement means for enhancing a contrast amount of the image frame by re-exposing pixels of the image frame using a ratio of contrast amount control variables.

27. The arrangement as recited in claim 22, further comprising:

combination means for correcting an exposure level and enhancing a contrast amount of the image frame using an exposure-density function along with (i) a difference between an ideal exposure level and an average luminance value of a skin portion detected using the skin-color model and (ii) a ratio of contrast amount control variables.

28. The arrangement as recited in claim 22, wherein the arrangement comprises at least one of (i) one or more processor-accessible media or (ii) at least one device.

29. A device that is configured to perform actions comprising:

building a skin-color model for an image frame using a defined skin color range;

detecting a facial portion in the image frame using the skin-color model; and

correcting an exposure level of the image frame based on a luminance value of the detected facial portion.

30. The device as recited in claim 29, wherein the action of building comprises an action of:

building the skin-color model at runtime for multiple frames.

31. The device as recited in claim 30, wherein the action of building comprises a further action of:

building the skin-color model for one image frame out of every predetermined number of image frames.

32. The device as recited in claim 30, wherein the skin-color model comprises a Gaussian skin-color model;

and wherein the defined skin color range is created offline using a collection of test sequences;

respective sets of skin pixels manually selected from respective ones of the collected test sequences, and respective Gaussian distribution centers calculated from respective Gaussian skin-color models built responsive to the respective sets of manually-selected skin pixels of the respective ones of the collected test sequences;

the defined skin color range created based on the calculated respective Gaussian distribution centers.

33. The device as recited in claim 29, wherein the device is configured to perform a further action comprising:

enhancing a contrast amount of the image frame.

34. The device as recited in claim 29, wherein the luminance value comprises an average luminance value;

and

wherein the device is configured to perform a further action comprising;

skipping the action of correcting when the average luminance value of the detected facial portion is relatively close to an ideal exposure level.

35. The device as recited in claim 29, wherein the action of detecting comprises an action of:

determining if respective individual probabilities associated with respective individual pixels of multiple pixels of the image frame exceed a predetermined threshold, each respective individual probability indicating if an associated respective individual pixel is likely to belong to a skin color class in accordance with the skin-color model.

36. The device as recited in claim 29, wherein the action of building comprises action of:

selecting training pixels from the image frame based on the defined skin color range; and

building the skin-color model using the selected training pixels.

37. The device as recited in claim 29, wherein the skin-color model comprises a Gaussian skin-color model;

wherein the action of building comprises actions of;

selecting training pixels from the image frame based on the defined skin color range; and

building the Gaussian skin-color model using the selected training pixels; and

wherein the device is configured to perform further actions comprising;

determining a center of the Gaussian skin-color model;

ascertaining if the determined center comports with at least one reliability range;

if the determined center is ascertained to comport with the at least one reliability range, utilizing the Gaussian skin-color model built for the image frame in the action of detecting; and

if the determined center is not ascertained to comport with the at least one reliability range, utilizing a previous Gaussian skin-color model built for a previous image frame in the action of detecting.

38. The device as recited in claim 37, wherein the at least one reliability range comprises a summed reliability range and a difference reliability range, the summed and difference reliability ranges produced from multiple tracked Gaussian centers of multiple Gaussian skin-color models built offline from multiple test sequences.

39. A runtime method comprising:

selecting training pixels from an image frame based on a predefined skin color range, the predefined skin color range created offline;

building a Gaussian skin-color model using the selected training pixels;

detecting a facial portion of the image frame using the Gaussian skin-color model and a skin pixel probability threshold; and

correcting an exposure level of the image frame with regard to the detected facial portion.