Methods and apparatus for face fitting and editing applications
First Claim
1. A method, comprising:
- causing one or more processors to performidentifying sets of two-dimensional local feature points on a face in each image of a set of images, whereinthe set of images comprises a sequence of frames a video stream; and
generating a three-dimensional face model for the face in the each image as a combination of a set of predefined three-dimensional face models, wherein said generating further comprisesreducing an error between a projection of vertices of the set of predefined three-dimensional face models and the two-dimensional local feature points of the each image,constraining facial expression of the three-dimensional face model to change smoothly from image to image in the sequence of video frames, andrepresenting each of the set of predefined three-dimensional face models using a three-dimensional tensor decomposition defined bya vector of coefficients representing an expression dimension that varies for each frame, anda vector of identity coefficients that is fixed across frames.
2 Assignments
0 Petitions
Accused Products
Abstract
Various embodiments of methods and apparatus for face fitting are disclosed. In one embodiment, sets of two-dimensional local feature points on a face in each image of a set of images are identified. The set of images includes a sequence of frames a video stream. A three-dimensional face model for the face in the each image is generated as a combination of a set of predefined three-dimensional face models. In some embodiments, the generating includes reducing an error between a projection of vertices of the set of predefined three-dimensional face models and the two-dimensional local feature points of the each image, and constraining facial expression of the three-dimensional face model to change smoothly from image to image in the sequence of video frames.
-
Citations
20 Claims
-
1. A method, comprising:
-
causing one or more processors to perform identifying sets of two-dimensional local feature points on a face in each image of a set of images, wherein the set of images comprises a sequence of frames a video stream; and generating a three-dimensional face model for the face in the each image as a combination of a set of predefined three-dimensional face models, wherein said generating further comprises reducing an error between a projection of vertices of the set of predefined three-dimensional face models and the two-dimensional local feature points of the each image, constraining facial expression of the three-dimensional face model to change smoothly from image to image in the sequence of video frames, and representing each of the set of predefined three-dimensional face models using a three-dimensional tensor decomposition defined by a vector of coefficients representing an expression dimension that varies for each frame, and a vector of identity coefficients that is fixed across frames. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system, comprising:
-
at least one processor; and a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to; identify sets of two-dimensional local feature points on a face in each image of a set of images, wherein the set of images comprises a sequence of frames a video stream; and generate a three-dimensional face model for the face in the each image as a combination of a set of predefined three-dimensional face models, wherein the program instructions executable by the at least one processor to generate the three-dimensional face model for the face in the each image further comprise program instructions executable by the at least one processor to represent each of the set of predefined three-dimensional face models using a three-dimensional tensor decomposition defined by a vector of coefficients representing an expression dimension that varies for each frame, and a vector of identity coefficients that is fixed across frames, and constrain facial expression of the three-dimensional face model. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:
-
identifying sets of two-dimensional local feature points on a face in each image of a set of images, wherein the set of images comprises a sequence of frames a video stream; and generating a three-dimensional face model for the face in the each image as a combination of a set of predefined three-dimensional face models, wherein the program instructions computer-executable to implement said generating further comprise program instructions computer-executable to implement reducing an error between a projection of vertices of the set of predefined three-dimensional face models and the two-dimensional local feature points of the each image, representing each of the set of predefined three-dimensional face models using a three-dimensional tensor decomposition defined by a vector of coefficients representing an expression dimension that varies for each frame, and a vector of identity coefficients that is fixed across frames. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification