EMOJI RECORDING AND SENDING

• US 20180335929A1
• Filed: 03/29/2018
• Published: 11/22/2018
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: SCAPEL, Nicolas, GUZMAN, Aurelio, BAUER, Sebastian, YERKES, Giancarlo, BARLIER, Guillaume Pierre Andre
• Granted Patent: US 10,845,968 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/08 Annexed information, e.g. attachments
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04W 4/12 Messaging; Mailboxes; Announcements

Abstract

The present disclosure generally relates to generating and modifying virtual avatars. An electronic device having a camera and a display apparatus displays a virtual avatar that changes appearance in response to changes in a face in a field of view of the camera. In response to detecting changes in one or more physical features of the face in the field of view of the camera, the electronic device modifies one or more features of the virtual avatar.

Claims

1. An electronic device, comprising:

one or more cameras;

a display apparatus;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, via the display apparatus, a graphical representation that changes appearance in response to changes in a face in a field of view of the one or more cameras, wherein the graphical representation includes;

a first portion, and

a second portion that is different from the first portion;

while displaying, via the display apparatus, the graphical representation, detecting a change in pose of the face within the field of view of the one or more cameras;

in response to detecting the change in pose of the face, changing an appearance of the graphical representation, including;

in accordance with a determination that the change in pose of the face includes a first type of change in pose of the face, changing the appearance of the graphical representation includes moving the first portion of the graphical representation relative to the second portion of the graphical representation, including;

in accordance with a determination that the first type of change in pose of the face has a first magnitude, moving the first portion of the graphical representation by a first amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face; and

in accordance with a determination that the first type of change in pose of the face has a second magnitude that is greater than the first magnitude, moving the first portion of the graphical representation by a second amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face, wherein the second amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation is greater than the first amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation; and

in accordance with a determination that the change in pose of the face includes a second type of change in pose of the face, changing the appearance of the graphical representation includes moving both the first portion of the graphical representation and the second portion of the graphical representation based on a magnitude of the second type of change in pose of the face.

2. The electronic device of claim 1, wherein:

the first portion is reactive to the first type of change in pose of the face and the second type of change in pose of the face; and

the second portion has reduced reactivity to the first type of change in pose of the face and is reactive to the second type of change in pose of the face.

3. The electronic device of claim 1, wherein the first portion is an upper portion of the graphical representation and the second portion is a lower portion of the graphical representation.

4. The electronic device of claim 1, wherein changing the appearance of the graphical representation further includes:

in accordance with a determination that the change in pose of the face includes both the first type of change in pose of the face and the second type of change in pose of the face, changing the appearance of the graphical representation includes;

moving the first portion of the graphical representation relative to the second portion of the graphical representation; and

moving both the first portion of the graphical representation and the second portion of the graphical representation based on the magnitude of the second type of change in pose of the face.

5. The electronic device of claim 1, wherein the change in pose of the face includes only the first type of change, and wherein moving the first portion of the graphical includes moving the first portion of the graphical representation without moving the second portion of the graphical representation.

6. The electronic device of claim 1, wherein the change in pose of the face includes only the second type of change, and wherein moving both the first portion of the graphical representation and the second portion of the graphical representation based on a magnitude of the second type of change in pose of the face includes maintaining a relative position of the first portion of the graphical representation with respect to a position of the second portion of the graphical representation.

7. The electronic device of claim 1, wherein the graphical representation further includes a third portion that is different from the first portion and the second portion, further in accordance with a determination that the change in pose of the face includes the first type of change in pose of the face, moving the third portion of the graphical representation relative to the second portion of the graphical representation, wherein the movement of the third portion relative to the second portion is less than the movement of the first portion relative to the second portion.

8. The electronic device of claim 7, wherein the third portion of the graphical representation is positioned between the first portion of the graphical representation and the second portion of the graphical representation.

9. The electronic device of claim 8, wherein moving the third portion of the graphical representation relative to the second portion of the graphical representation includes moving the third portion of the graphical representation about an axis extending between the first portion of the graphical representation and the second portion of the graphical representation.

10. The electronic device of claim 1, wherein, prior to detecting the change in pose of the face within the field of view of the one or more cameras, the first portion of the graphical representation is not visually delineated from the second portion of the graphical representation.

11. The electronic device of claim 1, wherein the first portion of the graphical representation is at least a portion of a head feature of the graphical representation and the second portion of the graphical representation is at least a portion of a neck feature of the graphical representation.

12. The electronic device of claim 1, wherein:

prior to detecting the change in pose of the face within the field of view of the one or more cameras, the face is oriented in a first orientation with respect to the field of view of the one or more cameras; and

prior to detecting the change in pose of the face within the field of view of the one or more cameras, the graphical representation is displayed in a second orientation different than the first orientation.

13. The electronic device of claim 1, wherein movement of the first portion of the graphical representation or movement of the second portion of the graphical representation occurs in accordance to one or more physics models.

14. The electronic device of claim 1, the one or more programs further including instructions for:

while displaying the graphical representation, detecting a second change in pose of the face within the field of view of the one or more cameras; and

in response to detecting the second change in pose of the face, changing the appearance of the graphical representation, including;

in accordance with a determination that the second change in pose of the face includes a third type of change in pose of the face, changing the appearance of the graphical representation includes moving the first portion of the graphical representation relative to the second portion of the graphical representation in accordance with a magnitude of the third type of change in pose of the face; and

in accordance with a determination that the second change in pose of the face includes a fourth type of change in pose of the face, changing the appearance of the graphical representation includes moving both the first portion of the graphical representation and the second portion of the graphical representation based on a magnitude of the fourth type of change in pose of the face.

15. The electronic device of claim 14, wherein the third type of change in pose of the face is the first type of change in pose of the face, and the fourth type of change in pose of the face is the second type of change in pose of the face.

16. The electronic device of claim 1, wherein the graphical representation is a virtual avatar.

17. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with one or more cameras and a display apparatus, the one or more programs including instructions for:

displaying, via the display apparatus, a graphical representation that changes appearance in response to changes in a face in a field of view of the one or more cameras, wherein the graphical representation includes;

a first portion, and

a second portion that is different from the first portion;

while displaying, via the display apparatus, the graphical representation, detecting a change in pose of the face within the field of view of the one or more cameras;

in response to detecting the change in pose of the face, changing an appearance of the graphical representation, including;

in accordance with a determination that the change in pose of the face includes a first type of change in pose of the face, changing the appearance of the graphical representation includes moving the first portion of the graphical representation relative to the second portion of the graphical representation, including;

in accordance with a determination that the first type of change in pose of the face has a first magnitude, moving the first portion of the graphical representation by a first amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face; and

in accordance with a determination that the first type of change in pose of the face has a second magnitude that is greater than the first magnitude, moving the first portion of the graphical representation by a second amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face, wherein the second amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation is greater than the first amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation; and

in accordance with a determination that the change in pose of the face includes a second type of change in pose of the face, changing the appearance of the graphical representation includes moving both the first portion of the graphical representation and the second portion of the graphical representation based on a magnitude of the second type of change in pose of the face.

18. A method, comprising:

at an electronic device having one or more cameras and a display apparatus;

displaying, via the display apparatus, a graphical representation that changes appearance in response to changes in a face in a field of view of the one or more cameras, wherein the graphical representation includes;

a first portion, and

a second portion that is different from the first portion;

while displaying, via the display apparatus, the graphical representation, detecting a change in pose of the face within the field of view of the one or more cameras;

in response to detecting the change in pose of the face, changing an appearance of the graphical representation, including;

in accordance with a determination that the change in pose of the face includes a first type of change in pose of the face, changing the appearance of the graphical representation includes moving the first portion of the graphical representation relative to the second portion of the graphical representation, including;

in accordance with a determination that the first type of change in pose of the face has a first magnitude, moving the first portion of the graphical representation by a first amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face; and

in accordance with a determination that the first type of change in pose of the face has a second magnitude that is greater than the first magnitude, moving the first portion of the graphical representation by a second amount relative to the second portion of the graphical representation in response to the first type of change in pose of the face, wherein the second amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation is greater than the first amount of movement of the first portion of the graphical representation relative to the second portion of the graphical representation; and

in accordance with a determination that the change in pose of the face includes a second type of change in pose of the face, changing the appearance of the graphical representation includes moving both the first portion of the graphical representation and the second portion of the graphical representation based on a magnitude of the second type of change in pose of the face.