Emoji recording and sending

• US 10,997,768 B2
• Filed: 06/09/2020
• Issued: 05/04/2021
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Barlier, Guillaume Pierre Andre, Bauer, Sebastian, Guzman, Aurelio, Scapel, Nicolas, Yerkes, Giancarlo
• Primary Examiner(s): Poon, King Y
• Assistant Examiner(s): Peren, Vincent
• Application Number:
  US16/896,457
• Publication Number:
  US 20200302669A1
• Time in Patent Office: 329 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/08 Annexed information, e.g. attachments
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04W 4/12 : Messaging; Mailboxes; Announcements

Abstract

The present disclosure generally relates to generating and modifying virtual avatars. An electronic device having a camera and a display apparatus displays a virtual avatar that changes appearance in response to changes in a face in a field of view of the camera. In response to detecting changes in one or more physical features of the face in the field of view of the camera, the electronic device modifies one or more features of the virtual avatar.

Claims

1. An electronic device, comprising:

one or more cameras;

a display apparatus;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, via the display apparatus, a virtual avatar, wherein the virtual avatar includes;

a first avatar feature reactive to changes in a first physical feature of a face within a field of view of the one or more cameras;

a second avatar feature reactive to changes in the first physical feature; and

a third avatar feature not primarily reactive to changes in the first physical feature;

while displaying the virtual avatar, detecting changes in the first physical feature; and

in response to detecting the changes in the first physical feature;

modifying the first avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include a change of a first magnitude, modifying the first avatar feature by a first modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include a change of a second magnitude different from the first magnitude, modifying the first avatar feature by a second modification magnitude based on the second magnitude, wherein the second modification magnitude is different from the first modification magnitude;

modifying the second avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include the change of the first magnitude, modifying the second avatar feature by a third modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include the change of the second magnitude different from the first magnitude, modifying the second avatar feature by a fourth modification magnitude based on the second magnitude, wherein the fourth modification magnitude is different from the third modification magnitude; and

forgoing modification of the third avatar feature based on the detected changes in the first physical feature.

2. The electronic device of claim 1, wherein the first avatar feature anatomically corresponds to the first physical feature, and wherein the second avatar feature does not anatomically correspond to the first physical feature.

3. The electronic device of claim 1, wherein the third avatar feature is reactive to changes in a second physical feature, wherein the third avatar feature anatomically corresponds to the second physical feature, the one or more programs further including instructions for:

while displaying the virtual avatar, detecting changes in the second physical feature; and

in response to detecting the changes in the second physical feature;

modifying the third avatar feature based on the detected changes in the second physical feature.

4. The electronic device of claim 3, wherein the second avatar feature is reactive to changes in the second physical feature.

5. The electronic device of claim 3, wherein the first avatar feature is reactive to changes in the second physical feature.

6. The electronic device of claim 1, wherein:

the changes in the first physical feature include a change of a first type;

modifying the first avatar feature based on the detected changes in the first physical feature includes modifying the first avatar feature based on a second type of change; and

modifying the second avatar feature based on the detected changes in the first physical feature includes modifying the second avatar feature based on a third type of change, the third type of change being different than the second type of change.

7. The electronic device of claim 1, wherein the virtual avatar further includes a fourth avatar feature reactive to changes in the first physical feature, the one or more programs further including instructions for:

in response to detecting the changes in the first physical feature;

modifying the fourth avatar feature based on the detected changes in the first physical feature.

8. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with one or more cameras and a display apparatus, the one or more programs including instructions for:

displaying, via the display apparatus, a virtual avatar, wherein the virtual avatar includes;

a first avatar feature reactive to changes in a first physical feature of a face within a field of view of the one or more cameras;

a second avatar feature reactive to changes in the first physical feature; and

a third avatar feature not primarily reactive to changes in the first physical feature;

while displaying the virtual avatar, detecting changes in the first physical feature; and

in response to detecting the changes in the first physical feature;

modifying the first avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include a change of a first magnitude, modifying the first avatar feature by a first modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include a change of a second magnitude different from the first magnitude, modifying the first avatar feature by a second modification magnitude based on the second magnitude, wherein the second modification magnitude is different from the first modification magnitude;

modifying the second avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include the change of the first magnitude, modifying the second avatar feature by a third modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include the change of the second magnitude different from the first magnitude, modifying the second avatar feature by a fourth modification magnitude based on the second magnitude, wherein the fourth modification magnitude is different from the third modification magnitude; and

forgoing modification of the third avatar feature based on the detected changes in the first physical feature.

9. The non-transitory computer-readable storage medium of claim 8, wherein the first avatar feature anatomically corresponds to the first physical feature, and wherein the second avatar feature does not anatomically correspond to the first physical feature.

10. The non-transitory computer-readable storage medium of claim 8, wherein the third avatar feature is reactive to changes in a second physical feature, wherein the third avatar feature anatomically corresponds to the second physical feature, the one or more programs further including instructions for:

while displaying the virtual avatar, detecting changes in the second physical feature; and

in response to detecting the changes in the second physical feature;

modifying the third avatar feature based on the detected changes in the second physical feature.

11. The non-transitory computer-readable storage medium of claim 10, wherein the second avatar feature is reactive to changes in the second physical feature.

12. The non-transitory computer-readable storage medium of claim 10, wherein the first avatar feature is reactive to changes in the second physical feature.

13. The non-transitory computer-readable storage medium of claim 8, wherein:

the changes in the first physical feature include a change of a first type;

modifying the first avatar feature based on the detected changes in the first physical feature includes modifying the first avatar feature based on a second type of change; and

modifying the second avatar feature based on the detected changes in the first physical feature includes modifying the second avatar feature based on a third type of change, the third type of change being different than the second type of change.

14. The non-transitory computer-readable storage medium of claim 8, wherein the virtual avatar further includes a fourth avatar feature reactive to changes in the first physical feature, the one or more programs further including instructions for:

in response to detecting the changes in the first physical feature;

modifying the fourth avatar feature based on the detected changes in the first physical feature.

15. A method, comprising:

at an electronic device having one or more cameras and a display apparatus;

displaying, via the display apparatus, a virtual avatar, wherein the virtual avatar includes;

a first avatar feature reactive to changes in a first physical feature of a face within a field of view of the one or more cameras;

a second avatar feature reactive to changes in the first physical feature; and

a third avatar feature not primarily reactive to changes in the first physical feature;

while displaying the virtual avatar, detecting changes in the first physical feature; and

in response to detecting the changes in the first physical feature;

modifying the first avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include a change of a first magnitude, modifying the first avatar feature by a first modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include a change of a second magnitude different from the first magnitude, modifying the first avatar feature by a second modification magnitude based on the second magnitude, wherein the second modification magnitude is different from the first modification magnitude;

modifying the second avatar feature based on the detected changes in the first physical feature, including;

in accordance with a determination that the changes in the first physical feature include the change of the first magnitude, modifying the second avatar feature by a third modification magnitude based on the first magnitude; and

in accordance with a determination that the changes in the first physical feature include the change of the second magnitude different from the first magnitude, modifying the second avatar feature by a fourth modification magnitude based on the second magnitude, wherein the fourth modification magnitude is different from the third modification magnitude; and

forgoing modification of the third avatar feature based on the detected changes in the first physical feature.

16. The method of claim 15, wherein the first avatar feature anatomically corresponds to the first physical feature, and wherein the second avatar feature does not anatomically correspond to the first physical feature.

17. The method of claim 15, wherein the third avatar feature is reactive to changes in a second physical feature, wherein the third avatar feature anatomically corresponds to the second physical feature, the method further comprising:

while displaying the virtual avatar, detecting changes in the second physical feature; and

in response to detecting the changes in the second physical feature;

modifying the third avatar feature based on the detected changes in the second physical feature.

18. The method of claim 17, wherein the second avatar feature is reactive to changes in the second physical feature.

19. The method of claim 17, wherein the first avatar feature is reactive to changes in the second physical feature.

20. The method of claim 15, wherein:

the changes in the first physical feature include a change of a first type;

modifying the first avatar feature based on the detected changes in the first physical feature includes modifying the first avatar feature based on a second type of change; and

modifying the second avatar feature based on the detected changes in the first physical feature includes modifying the second avatar feature based on a third type of change, the third type of change being different than the second type of change.

21. The method of claim 15, wherein the virtual avatar further includes a fourth avatar feature reactive to changes in the first physical feature, the method further comprising:

in response to detecting the changes in the first physical feature;

modifying the fourth avatar feature based on the detected changes in the first physical feature.