Cartoon face generation

• US 8,437,514 B2
• Filed: 10/02/2007
• Issued: 05/07/2013
• Est. Priority Date: 10/02/2007
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Wen, Fang, Liang, Lin, Xu, Ying-Qing, Zou, Xin
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US11/865,833
• Publication Number:
  US 20090087035A1
• Time in Patent Office: 2,044 Days
• Field of Search: 382/118, 382/225, 382/165, 345/473
• US Class Current: 382/118
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

A face cartooning system is described. In one implementation, the system generates an attractive cartoon face or graphic of a user'"'"'s facial image. The system extracts facial features separately and applies pixel-based techniques customized to each facial feature. The style of cartoon face achieved resembles the likeness of the user more than cartoons generated by conventional vector-based cartooning techniques. The cartoon faces thus achieved provide an attractive facial appearance and thus have wide applicability in art, gaming, and messaging applications in which a pleasing degree of realism is desirable without exaggerated comedy or caricature.

Claims

1. A method, comprising:

extracting visual features of a face portrayed in an image, the visual features including a facial shadow that represents three-dimensional information of the face;

assigning pixels of the facial shadow into clusters according to an attribute;

for each cluster in the facial shadow, selecting a mean color of a corresponding cluster as a representative color of the corresponding cluster;

propagating each representative color across all the pixels of the corresponding cluster in the facial shadow; and

recombining the visual features to form a cartoon of the face.

2. The method as recited in claim 1, further comprising aligning the face for mapping the visual features with points, wherein extracting the visual features includes extracting the visual features and associated alpha-matte masks based on the points.

3. The method as recited in claim 1, wherein the visual features further include at least one of facial skin, eyes, brows, nose, cheeks, lips, and inner mouth.

4. The method as recited in claim 2, wherein the mapping the visual features includes applying a Bayesian Tangent Shape Model (BSTM).

5. The method as recited in claim 1, wherein assigning pixels of the facial shadow into clusters further includes grouping the pixels into the clusters according to a lightness attribute.

6. The method as recited in claim 1, further comprising:

clustering pixels in an additional visual feature that is a facial skin into additional clusters according to the attribute;

selecting a color of a second lightest cluster of the additional clusters as a true color of the facial skin.

7. The method as recited in claim 6, further comprising shifting the lightness and chroma of the pixels for the additional visual feature to achieve a cartoon style.

8. The method as recited in claim 1, further comprising learning an affine transformation from a cheek shape of a reference face to a cheek shape of the face in the image to produce a forehead and ear geometry for the face in the image based on a forehead and ear geometry of the reference face.

9. The method as recited in claim 1, further comprising adopting a different cartooning technique for each different visual feature of the face.

10. The method as recited in claim 1, further comprising adopting a same cartooning technique for each different visual feature of the face, and using different parameters for the cartooning technique for each visual feature.

11. The method as recited in claim 1, further comprising:

applying a pixel-based cartooning technique to an extracted skin region of the face;

applying a pixel-based cartooning technique to an extracted head shape feature of the face; and

combining the skin region of the face and the head shape feature of the face to create a base for recombining the remaining visual features of the face.

12. The method as recited in claim 11, further comprising combining the base with a shadow visual feature of the face to form a second base for recombining the subsequent remaining visual features of the face.

13. The method as recited in claim 1, further comprising combining accessories with the cartoon of the face, wherein the accessories include one of a neck, hair, hat, eyeglasses, facial hair, and clothing.

20. The method as recited in claim 2, further comprising receiving user input of an adjustment to at least one point of the points, wherein extracting the visual features includes extracting the visual features and associated alpha-matte masks based on the points that includes the at least one point that is adjusted.

14. A system, comprising:

a computing device having a cartooning engine, the cartooning engine comprising;

a decomposition engine that extracts visual features from a face portrayed in an image;

a cartoonizer that assigns pixels of the visual feature into clusters according to an attribute, and for each cluster, selects a mean color of a corresponding cluster as a representative color of the corresponding cluster, propagates each representative color across all the pixels of the corresponding cluster, and shifts pixel color of the visual feature;

a geometry engine that learns an affine transformation from a cheek shape of a reference face to a cheek shape of the face in the image to obtain a forehead and ear geometry for a base-head background according to a forehead and ear geometry of the reference face; and

a compositor that recombines the visual features onto the base-head using an alpha-matte technique to form a cartoon of the face.

15. The system as recited in claim 14, wherein the cartooning engine further comprises a face processor, including a face detector, pre-processing components to enhance contrast and normalize color, and an alignment engine to align the face in the image for mapping facial features.

16. The system as recited in claim 15, wherein the cartooning engine further comprises a user interface for a user to interactively assist mapping the facial features.

17. The system as recited in claim 14, wherein the cartoonizer renders each of the visual features separately into a cartoon style, and wherein the visual features include a skin feature, a shadow feature, a facial geometry feature, an eyebrows features, an eyes feature, a lips feature, and an inner-mouth feature.

18. The system as recited in claim 14, wherein the cartooning engine further comprises an accessories engine that adds user-selected pre-drawn visual features to the cartoon of the face.

19. A system, comprising:

a computing device having a cartooning engine, the cartooning engine comprising;

a decomposition engine that extracts visual features from a face portrayed in an image;

a cartoonizer including a cartooner component that assigns pixels of a visual feature into at least five clusters according to a lightness attribute, selects a color from a second lightest cluster of the at least five clusters as a representative color for all the pixels in the visual feature, propagates the representative color across all the pixels of the visual feature, and shifts pixel color of the visual feature; and

a compositor that recombines the visual features using an alpha-matte technique to form a cartoon of the face.