Animating digital graphics overlaid on visual media items based on dynamic attributes

• US 10,460,499 B2
• Filed: 09/27/2017
• Issued: 10/29/2019
• Est. Priority Date: 09/27/2017
• Status: Active Grant

• Current Assignee: WhatsApp LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: WhatsApp LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Stukalov, Dmitri
• Primary Examiner(s): Wu, Chong
• Application Number:
  US15/717,795
• Publication Number:
  US 20190096113A1
• Time in Patent Office: 762 Days
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

This disclosure covers methods, computer-readable media, and systems that animate a digital graphic associated with a video or other visual media item based on a detected dynamic attribute. In particular, the disclosed methods, computer-readable media, and systems detect sensor data from a client device or a motion of an object within a video or other visual media item. Based on the detected sensor data or motion of an object within a visual media item, the methods, computer-readable media, and systems overlay and animate an emoji or other digital graphic selected by a user on a video or other visual media item.

Claims

1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to:

present a visual media item within a graphical user interface of the computing device;

receive user input to overlay a digital graphic on the visual media item within the graphical user interface;

detect a motion of an object within the visual media item; and

present the digital graphic as an overlay on the visual media item with an animation effect based on the motion of the object within the visual media item.

2. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to:

receive the user input to overlay the digital graphic on the visual media item by receiving user input to overlay the digital graphic on a video within the graphical user interface; and

present the digital graphic as the overlay on the visual media item with the animation effect by presenting the digital graphic as the overlay on the video with the animation effect based on the motion of the object within the visual media item.

3. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to detect the motion of the object within the visual media item by detecting the motion of the object relative to an additional object within the visual media item.

4. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to detect the motion of the object within the visual media item by comparing a reference background image for the visual media item with a foreground pixel map for the visual media item.

5. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to detect the motion of the object within the visual media item by detecting one or more of:

a speed of the object as the object moves within the visual media item;

a jumping motion of the object within the visual media item;

a rotational motion of the object within the visual media item;

a running motion of a person within the visual media item;

or

a change in orientation of the object within the visual media item.

6. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to:

motion of the object within the visual media item by determining a magnitude of the motion of the object within the visual media item; and

present the digital graphic as the overlay on the visual media item with the animation effect by presenting the digital graphic as the overlay on the visual media item with the animation effect having a frequency based on the magnitude of the motion of the object within the visual media item.

7. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to transmit, from the computing device to another computing device, an indication of the digital graphic and an indication of the animation effect, the indications causing the digital graphic to appear as the overlay on the visual media item with the animation effect.

8. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to present the digital graphic as the overlay on the visual media item with the animation effect by mapping the motion of the object to the animation effect, the animation effect being one of a plurality of animation effects associated with the digital graphic.

9. A system comprising:

at least one processor; and

at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause a computing device to;

present a visual media item within a graphical user interface of the computing device;

receive user input to overlay a digital graphic on the visual media item within a graphical user interface;

detect a motion of an object within the visual media item; and

present the digital graphic as an overlay on the visual media item with an animation effect based on the motion of the object within the visual media item.

10. The system of claim 9, further comprising instructions that, when executed by the at least one processor, cause the system to detect the motion of the object within the visual media item by detecting the motion of the object relative to an additional object within the visual media item.

11. The system of claim 9, further comprising instructions that, when executed by the at least one processor, cause the system to detect motion of the object within the visual media item by comparing a reference background image for the visual media item with a foreground pixel map for the visual media item.

12. The system of claim 9, further comprising instructions that, when executed by the at least one processor, cause the system to:

present the digital graphic as the overlay on the visual media item initially without the animation effect based on receiving the user input; and

present the digital graphic with the animation effect based on detecting the motion of the object within the visual media item.

13. The system of claim 9, further comprising instructions that, when executed by the at least one processor, cause the system to detect the motion of the object within the visual media item by detecting a pattern of the object'"'"'s motion within the visual media item.

14. The system of claim 9, further comprising instructions that, when executed by the at least one processor, cause the system to present the digital graphic as the overlay on the visual media item with the animation effect by:

identifying metadata associated with the digital graphic comprising a plurality of animation-effect options;

mapping the motion of the object to an animation-effect option of the plurality of animation-effect options, the animation-effect option corresponding to the animation effect; and

in response to mapping the motion of the object to the animation-effect option, present the digital graphic as the overlay on the visual media item with the animation effect.

15. A method comprising:

presenting a visual media item within a graphical user interface of a computing device;

receiving user input to overlay a digital graphic on the visual media item within the graphical user interface;

detecting a motion of an object within the visual media item; and

presenting the digital graphic as an overlay on the visual media item with an animation effect based on the motion of an object within the visual media item.

16. The method of claim 15, wherein:

receiving the user input to overlay the digital graphic on the visual media item comprises receiving user input to overlay the digital graphic on a video within the graphical user interface; and

presenting the digital graphic as the overlay on the visual media item with the animation effect comprises presenting the digital graphic as the overlay on the video with the animation effect based on the motion of the object within the visual media item.

17. The method of claim 15, wherein detecting the motion of the object within the visual media item comprises detecting the motion of the object relative to an additional object within the visual media item.

18. The method of claim 15, further comprising:

presenting the digital graphic as the overlay on the visual media item initially without the animation effect based on receiving the user input; and

presenting the digital graphic with the animation effect based on detecting the motion of the object within the visual media item.

19. The method of claim 15, wherein detecting the motion of the object within the visual media item comprises detecting one or more of:

a speed of the object as the object moves within the visual media item;

a jumping motion of the object within the visual media item;

a rotational motion of the object within the visual media item;

a running motion of a person within the visual media item;

or

a change in orientation of the object within the visual media item.

20. The method of claim 15, wherein:

detecting the motion of the object within the visual media item comprises determining a magnitude of the motion of the object within the visual media item; and

presenting the digital graphic as the overlay on the visual media item with the animation effect comprises presenting the digital graphic as the overlay on the visual media item with the animation effect having a frequency based on the magnitude of the motion of the object within the visual media item.