Real time video processing for changing proportions of an object in the video
First Claim
1. A computer implemented method for real time video processing for changing proportions of an object in a video, the method comprising:
- receiving a video stream depicting a face that at least partially and at least occasionally is presented in frames of the video stream;
receiving a user request for changing a proportion of the face depicted in the video stream;
after receiving the user request for changing the proportion of the face, detecting the face in the video stream by detecting feature reference points of the face;
tracking the detected face in the video stream, wherein the tracking comprises creating a first mesh based on the detected feature reference points of the face and aligning the first mesh to the face in each frame;
while tracking the detected face with the first mesh, transforming a set of pixels within the frames of the video stream representing a portion of the feature reference points to generate transformed frames of the video stream, wherein the transforming comprises deforming a first portion of the face depicted in the video stream by a first deformation amount, in accordance with the user request for changing the proportion, while deforming a second portion of the face depicted in the video stream by a second deformation amount;
maintaining the first mesh while the face is present in the frames of the video stream; and
providing the transformed frames within the video.
4 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems are presented for transforming frames of a video. The method provides an object in the video that at least partially and at least occasionally is presented in frames of a video. An object is detected in the video. Detection of the object comprises detecting feature reference points of the object. The object is tracked in the video. The tracking comprises creating a mesh that is based on the detected feature reference points of the object and aligning the mesh to the object in each frame. A first set of node points is generated on the mesh based on a request for changing proportions. A second set of nodes points is generated based on the first set of node points. The frames of the video are transformed in such way that the object'"'"'s proportions are transformed in accordance with the second set of the node points using the mesh.
131 Citations
18 Claims
-
1. A computer implemented method for real time video processing for changing proportions of an object in a video, the method comprising:
-
receiving a video stream depicting a face that at least partially and at least occasionally is presented in frames of the video stream; receiving a user request for changing a proportion of the face depicted in the video stream; after receiving the user request for changing the proportion of the face, detecting the face in the video stream by detecting feature reference points of the face; tracking the detected face in the video stream, wherein the tracking comprises creating a first mesh based on the detected feature reference points of the face and aligning the first mesh to the face in each frame; while tracking the detected face with the first mesh, transforming a set of pixels within the frames of the video stream representing a portion of the feature reference points to generate transformed frames of the video stream, wherein the transforming comprises deforming a first portion of the face depicted in the video stream by a first deformation amount, in accordance with the user request for changing the proportion, while deforming a second portion of the face depicted in the video stream by a second deformation amount; maintaining the first mesh while the face is present in the frames of the video stream; and providing the transformed frames within the video. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A mobile computerized system comprising a central processing unit and a memory, the memory storing instructions for:
-
receiving a video stream depicting a face that at least partially and at least occasionally is presented in frames of the video stream; receiving a user request for changing a proportion of the face depicted in the video stream; after receiving the user request for changing the proportion of the face, detecting the face in the video stream by detecting feature reference points of the face; tracking the detected face in the video stream, wherein the tracking comprises creating a first mesh based on the detected feature reference points of the face and aligning the first mesh to the face in each frame; while tracking the detected face with the first mesh, transforming a set of pixels within the frames of the video stream representing a portion of the feature reference points to generate transformed frames of the video stream, wherein the transforming comprises deforming a first portion of the face depicted in the video stream by a first deformation amount, in accordance with the user request for changing the proportion, while deforming a second portion of the face depicted in the video stream by a second deformation amount; maintaining the first mesh while the face is present in the frames of the video stream; and providing the transformed frames. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
Specification