EMOJI RECORDING AND SENDING

• US 20180335927A1
• Filed: 01/12/2018
• Published: 11/22/2018
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: ANZURES, Freddy Allen, BAUER, Sebastian, DYE, Alan C., GUZMAN, Aurelio, KING, Nicholas V., PHAM, Hoan, SCAPEL, Nicolas, WILSON, Christopher, YERKES, Giancarlo
• Granted Patent: US 10,521,091 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/08 Annexed information, e.g. attachments
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04W 4/12 Messaging; Mailboxes; Announcements

Abstract

The present disclosure generally relates to generating and modifying virtual avatars. An electronic device having a camera and a display apparatus displays a virtual avatar that changes appearance in response to changes in a face in a field of view of the camera. In response to detecting changes in one or more physical features of the face in the field of view of the camera, the electronic device modifies one or more features of the virtual avatar.

Claims

1. An electronic device, comprising:

a display;

a camera;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a virtual avatar generation interface;

displaying a preview of a virtual avatar in the virtual avatar generation interface,

wherein the preview of the virtual avatar reacts to changes in an appearance of a face that is in a field of view of the camera;

while displaying the preview of the virtual avatar, detecting an input in the virtual avatar generation interface;

in response to detecting the input in the virtual avatar generation interface;

in accordance with a determination that the input starts on the preview of the virtual avatar, generating a static virtual avatar that represents an expression of the face in the field of view of the camera at a respective time, wherein the respective time is determined based on a timing of the input and after generating the static virtual avatar, the static virtual avatar is displayed on the display prior to sending the static virtual avatar to a recipient; and

in accordance with a determination that the input includes activation of a record affordance in the virtual avatar generation interface, generating an animated virtual avatar that represents a sequence of changes in the expression of the face in the field of view of the camera over a period of time, wherein the period of time is determined based on a timing of the input.

2. The electronic device of claim 1, the one or more programs further including instructions for:

displaying a messaging interface including a message area, wherein the message area includes messages from two or more participants of a communication, and the virtual avatar generation interface is displayed concurrently with the messaging interface.

3. The electronic device of claim 2, wherein displaying the virtual avatar generation interface includes replacing display of a virtual keyboard of the messaging interface with display of the virtual avatar generation interface.

4. The electronic device of claim 2, wherein the messaging interface includes a message composition area and the input is a tap on the preview of the virtual avatar, and wherein the display of the static virtual avatar on the display prior to sending the static virtual avatar to the recipient is a display in the message composition area.

5. The electronic device of claim 1, wherein the avatar generation interface includes a static virtual avatar area that includes a collection of one or more previously generated virtual avatars, the one or more programs further including instructions for:

in response to a user input, adding the generated virtual avatar to the collection of one or more previously generated virtual avatars.

6. The electronic device of claim 5, the one or more programs further including instructions for:

after adding the virtual avatar to the collection of virtual avatars, receiving, from a user, a request to share the collection of virtual avatars with a second user, and in response, sending the collection of virtual avatars to the second user.

7. The electronic device of claim 2, wherein the input starts on the preview of the virtual avatar and ends at a location within the message area, the one or more programs further including instructions for:

after displaying the static virtual avatar, sending the static virtual avatar to the recipient, wherein the recipient is a participant associated with the communication.

8. The electronic device of claim 7, wherein the static virtual avatar has an appearance that is determined based on an expression of the face in the field of view of the camera at the time that the input was detected on the preview of the virtual avatar.

9. The electronic device of claim 7, the one or more programs further including instructions for:

in response to detecting the start of the input on the preview of the virtual avatar, displaying the preview of the virtual avatar ceasing to react to changes in an appearance of a face that is in a field of view of the camera.

10. The electronic device of claim 1, the one or more programs further including instructions for:

in response to expiration of the period of time for generating the animated virtual avatar, displaying a send affordance in place of the record affordance; and

in response to receiving input selecting the send affordance, sending the generated animated virtual avatar to a remote user.

11. The electronic device of claim 2, the one or more programs further including instructions for:

in response to expiration of the period of time for generating the animated virtual avatar, displaying a confirm affordance in place of the record affordance; and

in response to receiving input selecting the confirm affordance, displaying a representation of the animated virtual avatar in a message composition area of the messaging interface.

12. The electronic device of claim 1, wherein the period of time is based on a predetermined amount of time, the one or more programs further including instructions for:

after generating the animated virtual avatar, ceasing to display the preview of the virtual avatar and displaying a looping version of the animated virtual avatar, wherein displaying the looping version of the animated virtual avatar includes displaying the animation sequence two or more times.

13. The electronic device of claim 12, wherein the looping version of the animated virtual avatar is displayed using a first virtual avatar template, the one or more programs further including instructions for:

displaying representations of a plurality of other virtual avatar templates including a representation of a second virtual avatar template, wherein the second virtual avatar template is different from the first virtual avatar template; and

after starting to display the looping version of the animated virtual avatar and in response to receiving user input selecting the second virtual avatar template representation, updating the display of the looping version of the animated virtual avatar to reflect the second virtual avatar template.

14. The electronic device of claim 1, wherein the preview of the virtual avatar is automatically displayed in response to launching the virtual avatar generation interface.

15. The electronic device of claim 1, the one or more programs further including instructions for:

in response to detecting a characteristic of a first physical feature of the face in the field of view of the camera, updating a first physical feature of the displayed preview of the virtual avatar based on the detected characteristic, wherein a type of the first physical feature of the face is the same as a type of the first physical feature of the displayed preview.

16. The electronic device of claim 1, the one or more programs further including instructions for:

in response to detecting a characteristic of a second physical feature of the face in the field of view of the camera, updating a second physical feature of the displayed preview of the virtual avatar based on the detected characteristic, wherein a type of the second physical feature of the face is different than a type of the second physical feature of the displayed preview.

17. The electronic device of claim 1, the one or more programs further including instructions for:

in response to detecting movement of the face in the field of view of the camera, updating a third physical feature of the displayed preview of the virtual avatar based on a physics model for the virtual avatar and the detected movement.

18. The electronic device of claim 1, wherein the preview of the virtual avatar is based on a predefined virtual avatar template, the one or more programs further including instructions for:

updating the preview of the virtual avatar based on one or more predefined behaviors associated with the predefined virtual avatar template.

19. The electronic device of claim 1, the one or more programs further including instructions for:

in response to a determination that the face is no longer detected in the field of view of the camera, fading the display of the preview of the virtual avatar.

20. The electronic device of claim 1, the one or more programs further including instructions for:

in response to a determination that the face is no longer detected in the field of view of the camera, displaying a message indicating that the face is no longer being detected by the camera.

21. The electronic device of claim 1, the one or more programs further including instructions for:

in response to a determination that the face is no longer detected in the field of view of the camera, updating the display of the preview of the virtual avatar based on a change in appearance of the face that was occurring during a time period before the face was no longer detected in the field of view.

22. The electronic device of claim 21, wherein updating the display of the preview of the virtual avatar based on a change in appearance of the face that was occurring during a time period before the face was no longer detected in the field of view of the camera includes slowing the updating of the preview of the virtual avatar over time so that the updating of the virtual avatar stops.

25. The electronic device of claim 1, wherein the generated static virtual avatar is displayed concurrently with the preview of the virtual avatar prior to sending the static virtual avatar to the recipient.

26. The electronic device of claim 1, wherein:

the input is a drag gesture that starts on the preview of the virtual avatar; and

the static virtual avatar is displayed while continuing to detect the drag gesture prior to sending the static virtual avatar to the recipient.

23. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display and a camera, the one or more programs including instructions for:

displaying a virtual avatar generation interface;

displaying a preview of a virtual avatar in the virtual avatar generation interface, wherein the preview of the virtual avatar reacts to changes in an appearance of a face that is in a field of view of the camera;

while displaying the preview of the virtual avatar, detecting an input in the virtual avatar generation interface;

in response to detecting the input in the virtual avatar generation interface;

in accordance with a determination that the input starts on the preview of the virtual avatar, generating a static virtual avatar that represents an expression of the face in the field of view of the camera at a respective time, wherein the respective time is determined based on a timing of the input and after generating the static virtual avatar, the static virtual avatar is displayed on the display prior to sending the static virtual avatar to a recipient; and

in accordance with a determination that the input includes activation of a record affordance in the virtual avatar generation interface, generating an animated virtual avatar that represents a sequence of changes in the expression of the face in the field of view of the camera over a period of time, wherein the period of time is determined based on a timing of the input.

24. A method comprising:

at an electronic device having a camera and a display;

displaying a virtual avatar generation interface;

displaying a preview of a virtual avatar in the virtual avatar generation interface, wherein the preview of the virtual avatar reacts to changes in an appearance of a face that is in a field of view of the camera;

while displaying the preview of the virtual avatar, detecting an input in the virtual avatar generation interface;

in response to detecting the input in the virtual avatar generation interface;

in accordance with a determination that the input starts on the preview of the virtual avatar, generating a static virtual avatar that represents an expression of the face in the field of view of the camera at a respective time, wherein the respective time is determined based on a timing of the input and after generating the static virtual avatar, the static virtual avatar is displayed on the display prior to sending the static virtual avatar to a recipient; and

in accordance with a determination that the input includes activation of a record affordance in the virtual avatar generation interface, generating an animated virtual avatar that represents a sequence of changes in the expression of the face in the field of view of the camera over a period of time, wherein the period of time is determined based on a timing of the input.