Method for real-time video processing involving retouching of an object in the video
First Claim
1. A computer-implemented method for real time, video processing for retouching an object in a video, the method being performed in connection with a computerized system comprising a processing unit and a memory, the method comprising:
- receiving a video stream depicting an object, the object being at least partially and at least occasionally presented in frames of the video stream;
automatically generating a request for retouching the object depicted in the received video stream;
generating a list of at least one element of the object, the at least one element selected based on the request for retouching and a degree of retouching;
detecting the at least one element of the object in the video and parameters of the at east one element;
calculating new parameters of the at least one element according to the degree of retouching;
detecting characteristic points for each of the at least one element of the object;
generating a mesh based on the characteristic points for each of the at least one element of the object;
tracking the at least one element of the object in the video, wherein the tracking comprises aligning the mesh for each of the at least one element with a position of the corresponding each of the at least one element;
generating a square grid associated with a background of the object in the video stream;
transforming the frames of the video stream to generate modified frames within the video stream such that the background of the object is blurred using the square grid and the at least one element of the object is modified by changing the parameters of the at least one element to the new parameters for the at least one element of the object at one or more of the characteristic points using the mesh; and
providing the modified frames including the at least one modified element within the video stream such that a modified version of the at least one element of the object is included in frames of the video occurring after the changing of the parameters of the at least one element to the new parameters.
4 Assignments
0 Petitions
Accused Products
Abstract
A method for real time video processing for retouching an object in a video is presented The method includes providing an object in the video steam, where the object is at least partially and at least occasionally presented in frames of the video. The method sets a degree of retouching and generates a list of at least one element of the object selected based on a request of retouching and the degree of retouching. The method detects the at least one element of the object in the video and parameters of the at least one element and calculates new parameters of the at least one element according to the degree of retouching. Characteristic points are detected for each of the at least one element of the object and a mesh is generated based on the characteristic points for each of the at least one element of the object. The at least one element of the object in the video is tracked by aligning the mesh for each of the at least one element with a position of the corresponding each of the at least one element. The frames of the video are transformed such that the at least one element of the object is modified by the application of the new parameters to the at least one element of the object using the mesh.
128 Citations
22 Claims
-
1. A computer-implemented method for real time, video processing for retouching an object in a video, the method being performed in connection with a computerized system comprising a processing unit and a memory, the method comprising:
-
receiving a video stream depicting an object, the object being at least partially and at least occasionally presented in frames of the video stream; automatically generating a request for retouching the object depicted in the received video stream; generating a list of at least one element of the object, the at least one element selected based on the request for retouching and a degree of retouching; detecting the at least one element of the object in the video and parameters of the at east one element; calculating new parameters of the at least one element according to the degree of retouching; detecting characteristic points for each of the at least one element of the object; generating a mesh based on the characteristic points for each of the at least one element of the object; tracking the at least one element of the object in the video, wherein the tracking comprises aligning the mesh for each of the at least one element with a position of the corresponding each of the at least one element; generating a square grid associated with a background of the object in the video stream; transforming the frames of the video stream to generate modified frames within the video stream such that the background of the object is blurred using the square grid and the at least one element of the object is modified by changing the parameters of the at least one element to the new parameters for the at least one element of the object at one or more of the characteristic points using the mesh; and providing the modified frames including the at least one modified element within the video stream such that a modified version of the at least one element of the object is included in frames of the video occurring after the changing of the parameters of the at least one element to the new parameters. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computer-implemented method of real time video processing for retouching an object in a video, the method being performed in connection with a computerized system comprising a processing unit and a memory, the method comprising:
-
receiving a video stream depicting an object being at least partially and at least occasionally presented in frames of the video stream; automatically generating a request for retouching the object depicted in the video stream; setting a degree of retouching; generating an average image of the object based on a plurality of frames of the video stream and detecting feature reference points of the object; generating a list of at least one element of the object, the at least one element selected based on the request for retouching and the degree of retouching; detecting the at least one element of the object in the video stream and parameters of the at least one element; calculating new parameters of the at least one element according to the degree of retouching; calculating characteristic points for each of the at least one element of the object; generating a first mesh based on the characteristic points for each of the at least one element of the object; generating a covering layer for the video frames of the video stream based on the average image of the object and the calculated new parameters of the at least one element, wherein the covering layer comprises the average image with the at least one element of the object modified by the application of the calculated new parameters using the first mesh; tracking the feature reference points of the object in the video, wherein the tracking comprises creating a second mesh that is based on the positions of the feature reference points of the object, and aligning the second mesh to the object in each frame; transforming the covering layer based on the second mesh in each frame; aligning the covering layer with the object using the second mesh; transforming the frames of the video stream such that covering layer covers up the object in each frame within the video stream; and providing the modified frames including the at least one modified element within the video stream such that a modified version of the at least one element of the object is included in frames of the video occurring after the changing of the parameters of the at least one element to the new parameters.
-
-
14. A mobile computerized system comprising a central processing unit and a memory, the memory storing instructions for:
-
receiving a video stream depicting an object, the object being at east partially and at least occasionally presented in frames of the video stream; automatically generating a request for retouching the object depicted in the video stream; generating a list of at least one element of the object, the at least one element selected based on the request for retouching and a degree of retouching; detecting the at least one element of the object in the video stream and parameters of the at least one element; calculating new parameters of the at least one element according to the degree of retouching; detecting characteristic points for each of the at least one element of the object; generating a mesh based on the characteristic points for each of the at least one element of the object; tracking the at least one element of the object in the video stream, wherein the tracking comprises aligning the mesh for each of the at least one element with a position of the corresponding each of the at least one element; generating a square grid associated with a background of the object in the video stream; transforming the frames of the video stream to generate modified frames within the video stream such that the background of the object is blurred using the square grid and the at least one element of the object is modified by changing the parameters of the at least one element to the new parameters for the at least one element of the object at one or more of the characteristic points using the mesh; and providing the modified frames including the at least one modified element within the video stream such that a modified version of the at least one element of the object is included in frames of the video stream occurring after the changing of the parameters of the at least one element to the new parameters. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22)
-
Specification