Cartoon personalization

• US 8,831,379 B2
• Filed: 08/28/2008
• Issued: 09/09/2014
• Est. Priority Date: 04/04/2008
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wen, Fang, Tang, Xiao-Ou, Xu, Ying-Qing
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/200,361
• Publication Number:
  US 20090252435A1
• Time in Patent Office: 2,203 Days
• Field of Search: 382/284, 382/118, 382/181, 345/629, 345/648, 345/655
• US Class Current: 382/284
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

Embodiments that provide cartoon personalization are disclosed. In accordance with one embodiment, cartoon personalization includes selecting a face image having a pose orientation that substantially matches an original pose orientation of a character in a cartoon image. The method also includes replacing a face of the character in the cartoon image with the face image. The method further includes blending the face image with a remainder of the character in the cartoon image.

Claims

1. A method comprising:

detecting one or more face images from at least one photographic image;

selecting a face image having a pose orientation that substantially matches an original pose orientation of a character in a cartoon image;

replacing a face of the character in the cartoon image with the face image; and

blending the face image with a remainder of the character in the cartoon image by operations including;

converting, by one or more processors, the face image into a color space that includes an illumination channel and a plurality of color channels;

performing a shading on the illumination channel of the face image based at least in part on illumination information of the cartoon image, the shading including one or more uniform color regions and an illumination value for each region;

performing an illumination blending after the face image replaces the face of the character in the cartoon image by keeping illumination smooth across the regions; and

performing color adjustment on the plurality of color channels in the color space.

2. The method of claim 1, further comprising:

determining a suitability of the face image for integration into the cartoon image, the suitability being based at least in part on whether the face image is within at least one of an acceptable illumination range or greater than a minimal resolution threshold,

wherein the replacing the face of the character in the cartoon image with the face image includes replacing the face of the character if the face image is suitable for integration into the cartoon image.

3. The method of claim 1, further comprising:

obtaining one or more landmarks of each detected face image for determining orientation of each face image by using a Bayesian Tangent Shape Model (BTSM) on each face image.

4. The method of claim 1, further comprising grouping the face image with another similar face image using at least one of offline pre-processing or online clustering.

5. The method of claim 1, wherein the replacing includes:

determining a portion of the character to be replaced-with the face image using an affine transform, the portion including the face of the character;

transforming the face image to replace the portion of the character via warping using the affine transform; and

substituting the portion of the character with the transformed face image.

6. The method of claim 1, further comprising:

determining the pose orientation of the face image by estimating a symmetric axis in a two-dimensional face plane of the face image; and

calculating the pose orientation based at least in part on a linear distance between facial feature points in a left part of the face image and the symmetric axis and a linear distance between facial feature points in a right part of the image and the symmetric axis.

7. The method of claim 1, further comprising:

selecting a plurality of face images based at least in part on a size of individual ones of the face images or a brightness of individual ones of the face images; and

selecting the face image from the plurality of face images.

8. A memory storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

detecting one or more face images from at least one photographic image;

selecting a face image having a pose orientation that substantially matches an original pose orientation of a character in a cartoon image;

replacing a face of the character in the cartoon image with the face image; and

blending the face image with a remainder of the character in the cartoon image by operations including;

converting the face image into a color space that includes an illumination channel;

performing a shading on the illumination channel of the face image based at least in part on illumination information of the cartoon image, the shading including one or more uniform color regions and an illumination value for each region; and

performing an illumination blending after the face image replaces the face of the character in the cartoon image by keeping illumination smooth across the regions.

9. The memory of claim 8, wherein the color space further includes a plurality of color channels.

10. The memory of claim 9, wherein the operations further include performing color adjustment on the plurality of color channels in the color space.

11. The memory of claim 8, wherein the acts further comprise:

determining a suitability of the face image for integration into the cartoon image, the suitability being based at least in part on whether the face image is within at least one of an acceptable illumination range or greater than a minimal resolution threshold,

wherein the replacing the face of the character in the cartoon image with the face image includes replacing the face of the character if the face image is suitable for integration into the cartoon image.

12. The memory of claim 8, wherein the replacing includes:

determining a portion of the character to be replaced-with the face image using an affine transform, the portion including the face of the character;

transforming the face image to replace the portion of the character via warping using the affine transform; and

substituting the portion of the character with the transformed face image.

13. The memory of claim 8, wherein the acts further comprise:

determining the pose orientation of the face image by estimating a symmetric axis in a two-dimensional face plane of the face image; and

calculating the pose orientation based at least in part on a linear distance between facial feature points in a left part of the face image and the symmetric axis and a linear distance between facial feature points in a right part of the image and the symmetric axis.

14. The memory of claim 8, wherein the acts further comprise:

selecting a plurality of face images based at least in part on a size of individuals ones of the face images or brightness of individual ones of the face images; and

selecting the face image from the plurality of face images.

15. A system comprising:

a face detection engine that detects a plurality of face images from at least one photographic image;

a face pose engine that selects a face image from the plurality of face images, the face image having a pose orientation that matches an original pose orientation of a character in a cartoon image;

a geometry engine that replaces a face of the character in the cartoon image with the face image; and

a blending engine that blends the face image with a remainder of the character in the cartoon image by operations including;

converting the face image into a color space that includes an illumination channel;

performing a shading on the illumination channel of the face image based at least in part on illumination information of the cartoon image, the shading including one or more uniform color regions and an illumination value for each region; and

performing an illumination blending after the face image replaces the face of the character in the cartoon image by keeping illumination smooth across the regions.

16. The system of claim 15, wherein the color space further includes a plurality of color channels.

17. The system of claim 16, wherein the operations further include performing color adjustment on the plurality of color channels in the color space.

18. The system of claim 15, wherein the geometry engine replaces the face of the character with the face image by:

determining a portion of the character to be replaced-with the face image using an affine transform, the portion including the face of the character;

transforming the face image to replace the portion of the character via warping using the affine transform; and

substituting the portion of the character with the transformed face image.

19. The system of claim 15, wherein the face pose engine further:

determines the pose orientation of the face image by estimating a symmetric axis in a two-dimensional face plane of the face image; and

calculates the pose orientation based at least in part on a linear distance between facial feature points in a left part of the face image and the symmetric axis and a linear distance between facial feature points in a right part of the image and the symmetric axis.

20. The system of claim 15, further comprising a user interface module that provides a user interface for a user to select the cartoon image.