Avatar based ideogram generation

• US 10,360,708 B2
• Filed: 06/30/2016
• Issued: 07/23/2019
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Bondich, Artem, Maltsev, Volodymyr
• Primary Examiner(s): Lhymn, Sarah
• Application Number:
  US15/199,472
• Publication Number:
  US 20180005420A1
• Time in Patent Office: 1,118 Days
• Field of Search: 345619, 345629, 345630, 345632, 345633, 345634, 345636, 345637, 345638
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

Systems, devices, media, and methods are presented for generating ideograms from a set of images received in an image stream. The systems and methods detect at least a portion of a face within the image and identify a set of facial landmarks within the portion of the face. The systems and methods determine one or more characteristics representing the portion of the face, in response to detecting the portion of the face. Based on the one or more characteristics and the set of facial landmarks, the systems and methods generate a representation of a face. The systems and methods position one or more graphical elements proximate to the graphical model of the face and generate an ideogram from the graphical model and the one or more graphical elements.

Claims

1. A method comprising:

detecting, by one or more processors of a first mobile computing device, a portion of a face within one or more images;

identifying a set of facial landmarks within the portion of the face depicted within the one or more images;

in response to detecting the portion of the face and identifying the set of facial landmarks, determining one or more characteristics representing the portion of the face depicted in the one or more images;

based on the one or more characteristics and the set of facial landmarks, generating a graphical model of a face including the portion of the face depicted within the one or more images, the graphical model generated by rendering a base face and applying one or more generated features corresponding to the one or more characteristics and the set of facial landmarks;

positioning one or more graphical elements proximate to the graphical model of the face;

generating a digital sticker from the graphical model and the one or more graphical elements, the digital sticker combining the graphical model and the one or more graphical elements to represent at least a portion of a message and configured for presentation within a messaging application, wherein a first graphical element of the one or more graphical elements has a first size and a second graphical element of the one or more graphical elements has a second size;

determining configuration information of the messaging application comprising minimum dimension information or maximum dimension information of digital stickers in the messaging application;

determining that the second size of the second graphical element is closer to the maximum dimension information of digital stickers in the messaging application than the first size of the first graphical element;

based on determining that the second size of the second graphical element is closer to the maximum dimension information than the first size of the first graphical element, determining the second graphical element is a prioritized element;

based on determining that the second graphical element is the prioritized element, scaling the first size of the first graphical element to generate a modified size of the first graphical element based on the second size of the second graphical element;

scaling the graphical model to generate a scaled graphical model based on the second size of the second graphical element and the modified size of the first graphical element; and

transmitting the digital sticker to a second mobile computing device via the messaging application.

2. The method of claim 1, wherein positioning the one or more graphical elements further comprises:

determining that a position type of a graphical element is a background type, the graphical element included in the one or more graphical elements; and

positioning the graphical element behind at least a portion of the graphical model such that the portion of the graphical model obstructs at least a portion of the graphical element.

3. The method of claim 1, wherein positioning the one or more graphical elements further comprises:

determining that a position type of a graphical element is a foreground type, the graphical element included in the one or more graphical elements; and

positioning at least a portion of the graphical model behind the graphical element such that the graphical element obstructs the portion of the graphical model.

4. The method of claim 1, wherein the graphical model is a three-dimensional graphical model and the one or more graphical elements are two-dimensional graphical elements, and generating the digital sticker further comprises:

in response to positioning the one or more graphical elements, rendering the three-dimensional graphical model as a two-dimensional graphical model; and

rendering the digital sticker as a two-dimensional digital sticker combining the two-dimensional graphical model and the one or more graphical elements.

5. The method of claim 1 further comprising:

determining that second graphical element is of a background type and the first graphical element is of a foreground type; and

based on determining that second graphical element is of the background type and the first graphical element is of the foreground type, determining the second graphical element is the prioritized element.

6. The method of claim 1, further comprising:

receiving a user selection of a theme associated with the digital sticker; and

selecting, in response to receiving the user selection of the theme, the first and second graphical elements from a plurality of graphical elements as the one or more graphical elements to generate the digital sticker, the first and second graphical elements being associated with the theme and being of different types.

7. The method of claim 1, further comprising:

generating at least a portion of a body model connected to the graphical model of the face to generate a composite model, the body model having a skeletal representation movable to position at least a portion of the composite model;

determining a pose corresponding to the one or more graphical elements;

positioning one or more portions of the skeletal representation of the composite model to represent the pose; and

generating the digital sticker with the one or more graphical element and the composite model positioned in the pose.

8. A system comprising:

one or more processors of a first mobile computing device; and

a non-transitory processor-readable storage medium storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

detecting, by the one or more processors, a portion of a face within one or more Images;

identifying a set of facial landmarks within the portion of the face depicted within the one or more Images;

in response to detecting the portion of the face and identifying the set of facial landmarks, determining one or more characteristics representing the portion of the face depicted in the one or more images;

based on the one or more characteristics and the set of facial landmarks, generating a graphical model of a face including the portion of the face depicted within the one or more images, the graphical model generated by rendering a base face and applying one or more generated features corresponding to the one or more characteristics and the set of facial landmarks;

positioning one or more graphical elements proximate to the graphical model of the face;

generating a digital sticker from the graphical model and the one or more graphical elements, the digital sticker combining the graphical model and the one or more graphical elements to represent at least a portion of a message and configured for presentation within a messaging application;

determining configuration information of the messaging application comprising minimum dimension information or maximum dimension information of digital stickers in the messaging application;

determining that the second size of the second graphical element is closer to the maximum dimension information of digital stickers in the messaging application than the first size of the first graphical element;

based on determining that the second size of the second graphical element is closer to the maximum dimension information than the first size of the first graphical element, determining the second graphical element is a prioritized element;

based on determining that the second graphical element is the prioritized element, scaling the first size of the first graphical element to generate a modified size of the first graphical element based on the second size of the second graphical element;

scaling the graphical model to generate a scaled graphical model based on the second size of the second graphical element and the modified size of the first graphical element; and

transmitting the digital sticker to a second mobile computing device via the messaging application.

9. The system of claim 8, wherein the graphical model is a three-dimensional graphical model and the one or more graphical elements are two-dimensional graphical elements, and generating the digital sticker further comprises:

in response to positioning the one or more graphical elements, rendering the three-dimensional graphical model as a two-dimensional graphical model; and

rendering the digital sticker as a two-dimensional digital sticker combining the two-dimensional graphical model and the one or more graphical elements.

10. The system of claim 8, wherein the operations further comprise:

determining that second graphical element is of a background type and the first graphical element is of a foreground type; and

based on determining that second graphical element is of the background type and the first graphical element is of the foreground type, determining the second graphical element is the prioritized element.

11. The system of claim 8, wherein the operations further comprise:

generating at least a portion of a body model connected to the graphical model of the face to generate a composite model, the body model having a skeletal representation movable to position at least a portion of the composite model;

determining a pose corresponding to the one or more graphical elements;

positioning one or more portions of the skeletal representation of the composite model to represent the pose; and

generating the digital sticker with the one or more graphical element and the composite model positioned in the pose.

12. A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by one or more processors of a first mobile computing device, cause the first mobile computing device to perform operations comprising:

detecting the a portion of a face within one or more images;

identifying a set of facial landmarks within the portion of the face depicted within the one or more Images;

in response to detecting the portion of the face and identifying the set of facial landmarks, determining one or more characteristics representing the portion of the face depicted in the one or more images;

based on the one or more characteristics and the set of facial landmarks, generating a graphical model of a face including the portion of the face depicted within the one or more images, the graphical model generated by rendering a base face and applying one or more generated features corresponding to the one or more characteristics and the set of facial landmarks;

positioning one or more graphical elements proximate to the graphical model of the face;

generating a digital sticker from the graphical model and the one or more graphical elements, the digital sticker combining the graphical model and the one or more graphical elements to represent at least a portion of a message and configured for presentation within a messaging application;

determining configuration information of the messaging application comprising minimum dimension information or maximum dimension information of digital stickers in the messaging application;

determining that the second size of the second graphical element is closer to the maximum dimension information of digital stickers in the messaging application than the first size of the first graphical element;

based on determining that the second size of the second graphical element is closer to the maximum dimension information than the first size of the first graphical element, determining the second graphical element is a prioritized element;

based on determining that the second graphical element is the prioritized element, scaling the first size of the first graphical element to generate a modified size of the first graphical element based on the second size of the second graphical element;

scaling the graphical model to generate a scaled graphical model based on the second size of the second graphical element and the modified size of the first graphical element; and

transmitting the digital sticker to a second mobile computing device via the messaging application.

13. The non-transitory processor-readable storage medium of claim 12, wherein the graphical model is a three-dimensional graphical model and the one or more graphical elements are two-dimensional graphical elements, and generating the digital sticker further comprises:

in response to positioning the one or more graphical elements, rendering the three-dimensional graphical model as a two-dimensional graphical model; and

rendering the digital sticker as a two-dimensional digital sticker combining the two-dimensional graphical model and the one or more graphical elements.

14. The non-transitory processor-readable storage medium of claim 12, wherein the operations further comprise:

generating at least a portion of a body model connected to the graphical model of the face to generate a composite model, the body model having a skeletal representation movable to position at least a portion of the composite model;

determining a pose corresponding to the one or more graphical elements;

positioning one or more portions of the skeletal representation of the composite model to represent the pose; and

generating the digital sticker with the one or more graphical element and the composite model positioned in the pose.