GENERATING CARTOON IMAGES FROM PHOTOS

• US 20180268595A1
• Filed: 03/14/2018
• Published: 09/20/2018
• Est. Priority Date: 03/20/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sarna, Aaron, Krishnan, Dilip, Cole, Forrester, Mosseri, Inbar
• Granted Patent: US 10,529,115 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites

Abstract

A system and method for generating cartoon images from photos are described. The method includes receiving an image of a user, determining a template for a cartoon avatar, determining an attribute needed for the template, processing the image with a classifier trained for classifying the attribute included in the image, determining a label generated by the classifier for the attribute, determining a cartoon asset for the attribute based on the label, and rendering the cartoon avatar personifying the user using the cartoon asset.

Claims

1. A computer-implemented method comprising:

receiving, using one or more computing devices, an image of a user;

determining, using the one or more computing devices, a template for a cartoon avatar;

determining, using the one or more computing devices, an attribute needed for the template;

processing, using the one or more computing devices, the image with a classifier trained for classifying the attribute included in the image;

determining, using the one or more computing devices, a label generated by the classifier for the attribute;

determining, using the one or more computing devices, a cartoon asset for the attribute based on the label; and

rendering, using the one or more computing devices, the cartoon avatar personifying the user using the cartoon asset.

2. The computer-implemented method of claim 1, wherein determining the cartoon asset for the attribute based on the label comprises:

determining a set of cartoon assets associated with the attribute, wherein each cartoon asset has an associated label;

mapping the label to the set of cartoon assets;

determining whether the cartoon asset is available within the set of cartoon assets based on the mapping; and

responsive to determining that the cartoon asset is available within the set of cartoon assets, selecting, from the set of cartoon assets, the cartoon asset for the rendering.

3. The computer-implemented method of claim 2, wherein determining the set of cartoon assets associated with the attribute comprises:

determining a cartoon style of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon style.

4. The computer-implemented method of claim 2, wherein determining the set of cartoon assets associated with the attribute comprises:

determining a cartoon expression of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon expression.

5. The computer-implemented method of claim 2, further comprising:

responsive to determining that the cartoon asset is unavailable within the set of cartoon assets, selecting a default cartoon asset for the rendering.

6. The computer-implemented method of claim 1, wherein rendering the cartoon avatar personifying the user comprises:

organizing the cartoon asset into a position relative to other cartoon assets corresponding to other facial attributes in the template; and

rendering the cartoon avatar based on the organized cartoon asset.

7. The computer-implemented method of claim 1, further comprising:

presenting the cartoon avatar on a display of a user device;

receiving user input on the display;

editing the cartoon asset for the attribute included in the cartoon avatar based on the user input; and

re-rendering the cartoon avatar based on the edited cartoon asset.

8. A computer program product comprising a non-transitory computer readable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform the operations comprising:

receiving an image of a user;

determining a template for a cartoon avatar;

determining an attribute needed for the template;

processing the image with a classifier trained for classifying the attribute included in the image;

determining a label generated by the classifier for the attribute;

determining a cartoon asset for the attribute based on the label; and

rendering the cartoon avatar personifying the user using the cartoon asset.

9. The computer program product of claim 8, wherein determining the cartoon asset for the attribute based on the label comprises:

determining a set of cartoon assets associated with the attribute, wherein each cartoon asset has an associated label;

mapping the label to the set of cartoon assets;

determining whether the cartoon asset is available within the set of cartoon assets based on the mapping; and

responsive to determining that the cartoon asset is available within the set of cartoon assets, selecting, from the set of cartoon assets, the cartoon asset for the rendering.

10. The computer program product of claim 9, wherein determining the set of cartoon assets associated with the attribute comprises:

determining a cartoon style of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon style.

11. The computer program product of claim 9, wherein determining the set of cartoon assets associated with the attribute comprises:

determining a cartoon expression of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon expression.

12. The computer program product of claim 9, further comprising:

responsive to determining that the cartoon asset is unavailable within the set of cartoon assets, selecting a default cartoon asset for the rendering.

13. The computer program product of claim 8, wherein rendering the cartoon avatar personifying the user comprises:

organizing the cartoon asset into a position relative to other cartoon assets corresponding to other facial attributes in the template; and

rendering the cartoon avatar based on the organized cartoon asset.

14. The computer program product of claim 8, further comprising:

presenting the cartoon avatar on a display of a user device;

receiving user input on the display;

editing the cartoon asset for the attribute included in the cartoon avatar based on the user input; and

re-rendering the cartoon avatar based on the edited cartoon asset.

15. A system comprising:

a processor; and

a memory storing instructions that, when executed, cause the system to perform operations comprising;

receiving an image of a user;

determining a template for a cartoon avatar;

determining an attribute needed for the template;

processing the image with a classifier trained for classifying the attribute included in the image;

determining a label generated by the classifier for the attribute;

determining a cartoon asset for the attribute based on the label; and

rendering the cartoon avatar personifying the user using the cartoon asset.

16. The system of claim 15, wherein to determine the cartoon asset for the attribute based on the label, the operations further comprise:

determining a set of cartoon assets associated with the attribute, wherein each cartoon asset has an associated label;

mapping the label to the set of cartoon assets;

determining whether the cartoon asset is available within the set of cartoon assets based on the mapping; and

responsive to determining that the cartoon asset is available within the set of cartoon assets, selecting, from the set of cartoon assets, the cartoon asset for the rendering.

17. The system of claim 16, wherein to determine the set of cartoon assets associated with the attribute, the operations further comprise:

determining a cartoon style of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon style.

18. The system of claim 16, wherein to determine the set of cartoon assets associated with the attribute, the operations further comprise:

determining a cartoon expression of the cartoon avatar; and

determining the set of cartoon assets associated with the attribute based on the cartoon expression.

19. The system of claim 16, further comprising:

responsive to determining that the cartoon asset is unavailable within the set of cartoon assets, selecting a default cartoon asset for the rendering.

20. The system of claim 15, wherein to render the cartoon avatar personifying the user, the operations further comprise:

organizing the cartoon asset into a position relative to other cartoon assets corresponding to other facial attributes in the template; and

rendering the cartoon avatar based on the organized cartoon asset.