Animated expressive icon

• US 10,726,603 B1
• Filed: 02/28/2018
• Issued: 07/28/2020
• Est. Priority Date: 02/28/2018
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Al Majid, Newar Husam, Boyd, Nathan Kenneth, Chang, Sheldon, Samaranayake, Chamal, Voss, Jeremy
• Primary Examiner(s): Merouan, Abderrahim
• Application Number:
  US15/908,422
• Time in Patent Office: 881 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/18 Commands or executable codes
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04M 2201/42 Graphical user interfaces

Abstract

Embodiments described herein include an expressive icon system to present an animated graphical icon, wherein the animated graphical icon is generated by capture facial tracking data at a client device. In some embodiments, the system may track and capture facial tracking data of a user via a camera associated with a client device (e.g., a front facing camera, or a paired camera), and process the facial tracking data to animate a graphical icon.

Claims

1. A method comprising:

identifying a set of facial landmarks;

generating a set of points representative of the set of facial landmarks, each point among the set of points corresponding to a facial landmark from among the set of facial landmarks;

capturing, by one or more processors of a first client device, facial tracking data based on positions of the set of points that correspond with the set of facial landmarks;

detecting one or more contextual conditions at the first client device;

accessing a user profile associated with a second client device, the user profile comprising user profile data that assigns a graphical icon to the one or more contextual conditions;

selecting the graphical icon from among a plurality of graphical icons at the first client device based on the user profile data that assigns the graphical icon to the one or more contextual conditions;

transposing the facial tracking data upon the graphical icon;

causing display of a presentation of the graphical icon at the second client device; and

animating the presentation of graphical icon based on the positions of the set of points from the facial tracking data captured at the first client device.

2. The method of claim 1, wherein the capturing the facial tracking data further comprises:

recording audio data at the first client device; and

wherein the graphical icon comprises at least the facial tracking data and the audio data; and

presenting the audio data in conjunction with the graphical icon.

3. The method of claim 1, wherein the graphical icon includes a bitmoji, and wherein the method further comprises:

receiving a selection of the bitmoji from among a set of bitmoji; and

wherein the presenting the bitmoji based on the facial tracking data includes animating the bitmoji based on the facial tracking data.

4. The method of claim 1, wherein the method further comprises:

generating an animation based on the graphical icon and the facial tracking data at the first client device;

flattening the animation at the first client device; and

wherein the presenting the graphical icon based on the facial tracking data at the second client device includes presenting the flattened animation.

5. The method of claim 1, wherein the first client device is associated with a user profile, and the method further comprises:

selecting the graphical icon from among a set of graphical icons based on the user profile associated with the first client device.

6. The method of claim 1, wherein the method further comprises:

generating a message at the first client device, the message comprising the graphical icon and the facial tracking data assigned to the graphical icon; and

wherein the receiving the graphical icon at the second client device includes receiving the message from the first client device.

7. The method of claim 1, wherein the presenting the graphical icon at the second client device includes:

presenting an ephemeral message that includes the graphical icon at the second client device.

8. A system comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising;

identifying a set of facial landmarks;

generating a set of points representative of the set of facial landmarks, each point among the set of points corresponding to a facial landmark from among the set of facial landmarks;

capturing, by one or more processors of a first client device, facial tracking data based on positions of the set of points that correspond with the set of facial landmarks;

detecting one or more contextual conditions at the first client device;

accessing a user profile associated with a second client device, the user profile comprising user profile data that assigns a graphical icon to the one or more contextual conditions;

selecting the graphical icon from among a plurality of graphical icons at the first client device based on the user profile data that assigns the graphical icon to the one or more contextual conditions;

transposing the facial tracking data upon the graphical icon;

causing display of a presentation of the graphical icon at the second client device; and

animating the presentation of graphical icon based on the positions of the set of points from the facial tracking data captured at the first client device.

9. The system of claim 8, wherein the capturing the facial tracking data further comprises:

recording audio data at the first client device; and

wherein the graphical icon comprises at least the facial tracking data and the audio data; and

presenting the audio data in conjunction with the graphical icon.

10. The system of claim 8, wherein the graphical icon includes a bitmoji, and wherein the instructions cause the system to perform operations further comprising:

receiving a selection of the bitmoji from among a set of bitmoji; and

wherein the presenting the bitmoji based on the facial tracking data includes animating the bitmoji based on the facial tracking data.

11. The system of claim 8, wherein the instructions cause the system to perform operations further comprising:

generating an animation based on the graphical icon and the facial tracking data at the first client device;

flattening the animation at the first client device; and

wherein the presenting the graphical icon based on the facial tracking data at the second client device includes presenting the flattened animation.

12. The system of claim 8, wherein the first client device is associated with a user profile, and the instructions cause the system to perform operations further comprising:

selecting the graphical icon from among a set of graphical icons based on the user profile associated with the first client device.

13. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

identifying a set of facial landmarks;

generating a set of points representative of the set of facial landmarks, each point among the set of points corresponding to a facial landmark from among the set of facial landmarks;

capturing, by one or more processors of a first client device, facial tracking data based on positions of the set of points that correspond with the set of facial landmarks;

detecting one or more contextual conditions at the first client device;

accessing a user profile associated with a second client device, the user profile comprising user profile data that assigns a graphical icon to the one or more contextual conditions;

selecting the graphical icon from among a plurality of graphical icons at the first client device based on the user profile data that assigns the graphical icon to the one or more contextual conditions;

transposing the facial tracking data upon the graphical icon;

causing display of a presentation of the graphical icon at the second client device; and

animating the presentation of graphical icon based on the positions of the set of points from the facial tracking data captured at the first client device.

14. The non-transitory machine-readable storage medium of claim 13, wherein the capturing the facial tracking data further comprises:

recording audio data at the first client device; and

wherein the graphical icon comprises at least the facial tracking data and the audio data; and

presenting the audio data in conjunction with the graphical icon.

15. The non-transitory machine-readable storage medium of claim 13, wherein the graphical icon includes a bitmoji, and wherein the instructions cause the machine to perform operations further comprising:

receiving a selection of the bitmoji from among a set of bitmoji; and

wherein the presenting the bitmoji based on the facial tracking data includes animating the bitmoji based on the facial tracking data.

16. The non-transitory machine-readable storage medium of claim 15, wherein the first client device is associated with a user profile, and the instructions cause the machine to perform operations further comprising:

selecting the graphical icon from among a set of graphical icons based on the user profile associated with the first client device.

17. The non-transitory machine-readable storage medium of claim 13, wherein the instructions cause the machine to perform operations further comprising:

generating an animation based on the graphical icon and the facial tracking data at the first client device;

flattening the animation at the first client device; and

wherein the presenting the graphical icon based on the facial tracking data at the second client device includes presenting the flattened animation.