DYNAMIC MASK APPLICATION

• US 20180182141A1
• Filed: 12/22/2016
• Published: 06/28/2018
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Caballero, Maria Luz, Fowler, Molly Jane, Lu, Congxi, Hodgkins, Charles Joseph, Marrero, Daniel Moreno
• Granted Patent: US 10,636,175 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

In one embodiment, a method includes identifying an emotion associated with an identified first object in one or more input images, selecting, based on the emotion, a mask from a set of masks, where the mask specifies one or more mask effects, and for each of the input images, applying the mask to the input image. Applying the mask includes generating graphical features based on the identified first object or a second object in the input images according to instructions specified by the mask effects, and incorporating the graphical features into an output image. The emotion may be identified based on graphical features of the identified first object. The graphical features of the identified object may include facial features. The selected mask may be selected from a lookup table that maps the identified emotion to the selected mask.

Claims

1. A method comprising:

by a computer system, identifying an emotion associated with an identified first object in one or more input images;

by the computer system, selecting, based on the emotion, a mask from a plurality of masks, wherein the mask specifies one or more mask effects and displays a stylized emotion reflective of the emotion associated with the identified first object; and

by the computer system, for each of the input images, applying the mask to the input image, wherein applying the mask comprises;

generating one or more graphical features based at least in part on the stylized emotion reflective of the emotion associated with the identified first object according to instructions specified by the mask effects; and

incorporating the one or more graphical features into the identified first object or a second object in the input images to generate an output image.

2. The method of claim 1, wherein the emotion is identified based on one or more graphical features of the identified first object.

3. The method of claim 2, wherein the graphical features of the identified object comprise one or more facial features.

4. The method of claim 1, wherein the selected mask is selected from a lookup table that maps the identified emotion to the selected mask to display the stylized emotion reflective of the identified emotion.

5. The method of claim 1, wherein the mask is selected from the plurality of masks based further on one or more user-related factors associated with a user of a social-networking system to display the stylized emotion reflective of the emotion associated with the identified first object.

6. The method of claim 5, wherein the user-related factors comprise user-profile data, social data, one or more user interests, location data, or a combination thereof.

7. The method of claim 5, wherein the mask is selected from the plurality of masks when a selection condition associated with the mask is satisfied by the user-related factors associated with the user.

8. The method of claim 7, wherein the selection condition is satisfied when, for an attribute associated with at least one of the user-related factors, a value of the attribute associated with the mask matches a value of the attribute associated with the user.

9. The method of claim 1, wherein one or more emotions are associated with the mask, and the mask is selected when the identified emotion corresponds to at least one of the emotions associated with the mask in order to display the stylized emotion reflective of the emotion associated with the identified first object.

10. The method of claim 9, wherein the identifying an emotion generates a confidence value for the identified emotion,

wherein each of the one or more emotions associated with the mask is further associated with a threshold confidence condition, and the mask is selected when the confidence value satisfies the threshold confidence condition.

11. The method of claim 1, wherein the plurality of masks comprises a first mask that generates one or more first features associated with a first emotion of a person or character and a second mask that generates one or more second features associated with a second emotion of the person or character,

wherein the first mask is selected when the identified emotion is the first emotion, and the second mask is selected when the identified emotion is the second emotion.

12. The method of claim 1, wherein the mask effects include a mask image, and the graphical features incorporated into the output image include one or more features from the mask image.

13. The method of claim 12, wherein the features from the mask image are scaled based on the size of the first or second object.

14. The method of claim 12, wherein the features from the mask image are positioned in the output image at locations that are determined based on locations of corresponding features of the first or second object in the input images according to mask geometry information associated with the mask effects.

15. The method of claim 1, wherein the graphical features generated by the mask effects to display the stylized emotion reflective of the emotion associated with the identified first object comprise one or more of:

exaggerated facial features, renderings of inanimate physical objects, or stylized facial features.

16. The method of claim 1, wherein the identified first object in the input image comprises an image of a human, human face, animal, animal face, or inanimate object.

17. The method of claim 1, wherein the second object is identified by input received from a user of the social-networking system, and the graphical features are generated based on the second object.

18. The method of claim 1, wherein the input images are captured using a camera.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

identify an emotion associated with an identified first object in one or more input images;

select, based on the emotion, a mask from a plurality of masks, wherein the mask specifies one or more mask effects and displays a stylized emotion reflective of the emotion associated with the identified first object; and

for each of the input images, apply the mask to the input image, wherein to apply the mask, the software is further operable when executed to;

generate one or more graphical features based at least in part on the stylized emotion reflective of the emotion associated with the identified first object according to instructions specified by the mask effects; and

incorporate the one or more graphical features into the identified first object or a second object in the input images to generate an output image.

20. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

identify an emotion associated with an identified first object in one or more input images;

select, based on the emotion, a mask from a plurality of masks, wherein the mask specifies one or more mask effects and displays a stylized emotion reflective of the emotion associated with the identified first object; and

for each of the input images, apply the mask to the input image, wherein to apply the mask, the processor is further operable when executing the instructions to;

generate one or more graphical features based at least in part on the stylized emotion reflective of the emotion associated with the identified first object according to instructions specified by the mask effects; and

incorporate the one or more graphical features into the identified first object or a second object in the input images to generate an output image.