Digital Video Effects

• US 20070216675A1
• Filed: 08/28/2006
• Published: 09/20/2007
• Est. Priority Date: 03/16/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Sun, Jian, Shum, Heung-Yeung, Tang, Xiaoou, Zhang, Weiwei
• Granted Patent: US 8,026,931 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Digital video effects are described. In one aspect, a foreground object in a video stream is identified. The video stream comprises multiple image frames. The foreground object is modified by rendering a 3-dimensional (3-D) visual feature over the foreground object for presentation to a user in a modified video stream. Pose of the foreground object is tracked in 3-D space across respective ones of the image frames to identify when the foreground object changes position in respective ones of the image frames. Based on this pose tracking, aspect ratio of the 3-D visual feature is adaptively modified and rendered over the foreground object in corresponding image frames for presentation to the user in the modified video stream.

Claims

1. A method at least partially implemented by a computing device, the method comprising:

identifying a foreground object in a video stream comprising multiple image frames;

rendering a three-dimensional (3-D) visual feature over the foreground object to add a digital video effect to the video stream and generate a modified foreground object; and

tracking pose of the foreground object in 3-D space across respective ones of the image frames to maintain rendered aspect ratio of the 3-D visual feature on the foreground object as the foreground object changes position in respective ones of the image frames.

2. The method of claim 1, wherein the video stream is associated with a communication session between at least two users.

3. The method of claim 1, wherein the method is a networked service provided to one or more users of remote computing device(s).

4. The method of claim 1, wherein tracking the pose further comprises tracking facial features to identify rotational and translational vectors across respective ones of the image frames;

and

wherein the method further comprises rendering the 3-D visual feature on the foreground object based on the rotational and translational vectors.

5. The method of claim 1, further comprising presenting a user interface to a user for the user to select at least the 3-D visual feature.

6. The method of claim 1, further comprising presenting a modified video stream comprising the modified foreground to one or more users.

7. The method of claim 1, further comprising:

identifying a background from one or more of the image frames;

modifying the background to generate a modified background, the modifying being independent of any modification to the foreground object; and

wherein the modifying adaptively modifies the background responsive to one or more of background changes and detected camera shake across respective ones of the image frames.

8. The method of claim 8, further comprising presenting a user interface to a user for the user to select a modification to apply to the background.

9. The method of claim 8, wherein modifying the background comprises selectively blurring the background, replacing the background with an image, or replacing the background with an animation.

10. The method of claim 8, further comprising presenting a modified video stream to one or more users, the modified video stream comprising the modified background and the modified foreground.

11. A computing device comprising:

a processor; and

a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for;

generating a video stream comprising a 3-D image of a first person involved in a video communication session with a second person using a remote computing device;

separating a foreground object representing the 3-D image from a background of the video stream;

adaptively rendering a 3-D feature over particular ones of multiple video frames that comprise the foreground object in multiple translational and rotational poses to generate a modified video stream, the 3-D feature being rendered over facial features of the first person such that aspect ratios of the 3-D feature are maintained in view of the translational and rotational poses; and

communicating the modified video stream to the remote computing device for presentation to the second person.

12. The computing device of claim 11, further comprising determining the 3-D feature from a set of pre-configured user preferences.

13. The computing device of claim 11, further comprising:

presenting a user interface to the first person, the user interface comprising a video effects area that provides one or more selectable costume options to the first person;

responsive to the first person selecting a particular one option of the one or more selectable costume options, setting the 3-D feature to the particular one option.

14. The computing device of claim 13, wherein the user interface further comprises a first display area for presenting an image of the first person and a second display area for presenting an image of the second person, the user interface being used by the first and second person for real-time communications between the first and the second person, the image of the first person being a modified image overlain with the 3-D feature responsive to user input, the modified image representing what is viewed by the second person during the real-time communications.

15. The computing device of claim 11, further comprising:

segmenting the foreground object from a background represented by at least a subset of frames of the video stream;

dynamically altering the background to adapt to changes to the background across respective ones of the frames and translational and rotational movement of the foreground object, the dynamic altering generating modified background frames; and

wherein the modified video stream comprises the modified background frames.

16. The computing device of claim 15, further comprising:

presenting a user interface to the first person, the user interface comprising a video effects area that provides one or more selectable background modification options to the first person;

responsive to the first person selecting a particular one option of the background modification options, applying the particular one option to adaptively modify the background across respective ones of the frames that represent the background, the modified video stream comprising such adaptive background modifications.

17. The computing device of claim 16, wherein the user interface further comprises a first display area for presenting, responsive to user input, the modified video stream, the modified video stream representing what is viewed by the second person during the real-time communications.

18. A tangible computer-readable storage medium comprising computer-program instructions executable by a processor, the computer-program instructions, when executed by the processor, for performing operations comprising:

providing a user with one or more video stream background modification options;

presenting the user with one or more costume overlay options,

responsive to selection by the user of a particular background modification option of the video stream background modification options, adaptively modifying background of a video stream using the particular background modification option,

responsive to selection by the user of a particular costume overlay option of the costume overlay options, adaptively rendering a 3-D image associated with the costume overlay option over portions of frames that comprise the video stream, the portions representing facial features; and

communicating the video stream to a remote computing device for presentation to a different user.

19. The tangible computer-readable storage medium of claim 18, wherein the computer-program instructions further comprise instructions for presenting a different video stream to the user, the different video stream showing an image of the different user that has been altered with one or more 3-D-visual features that follow translational and rotational movement of the different user across respective frames of the different video stream,

20. The tangible computer-readable storage medium of claim 19, wherein the different video stream further comprises an adaptively altered background that is blurred or replaced as per input from the different user, the altered background being adapted to allow for translational and rotational movement of the different user across respective frames of the different video stream.