METHOD OF AUGMENTED MAKEOVER WITH 3D FACE MODELING AND LANDMARK ALIGNMENT
1 Assignment
0 Petitions
Accused Products
Abstract
Generation of a personalized 3D morphable model of a user'"'"'s face may be performed first by capturing a 2D image of a scene by a camera. Next, the user'"'"'s face may be detected in the 2D image and 2D landmark points of the user'"'"'s face may be detected in the 2D image. Each of the detected 2D landmark points may be registered to a generic 3D face model. Personalized facial components may be generated in real time to represent the user'"'"'s face mapped to the generic 3D face model to form the personalized 3D morphable model. The personalized 3D morphable model may be displayed to the user. This process may be repeated in real time for a live video sequence of 2D images from the camera.
413 Citations
44 Claims
-
1-23. -23. (canceled)
-
24. A method of generating a personalized 3D morphable model of a user'"'"'s face comprising:
-
capturing at least one 2D image of a scene by a camera; detecting the user'"'"'s face in the at least one 2D image; detecting 2D landmark points of the user'"'"'s face in the at least one 2D image; registering each of the 2D landmark points to a generic 3D face model; and generating in real time personalized facial components representing the user'"'"'s face mapped to the generic 3D face model to form the personalized 3D morphable model, based at least in part on the 2D landmark points registered to the generic 3D face model. - View Dependent Claims (25, 26, 27)
-
-
28. A system to generate a personalized 3D morphable model representing a user'"'"'s face comprising:
-
a 2D landmark points detection component to accept at least one 2D image from a camera, the at least one 2D image including a representation of the user'"'"'s face, and to detect 2D landmark points of the user'"'"'s face in the at least one 2D image; a 3D facial part characterization component to accept a generic 3D face model and to facilitate the user to interact with segmented 3D face regions; a 3D landmark points registration component, coupled to the 2D landmark points detection component and the 3D facial part characterization component, to accept the generic 3D face model and the 2D landmark points, to register each of the 2D landmark points to the generic 3D face model, and to estimate a re-projection error in registering each of the 2D landmark points to the generic 3D face model; and a personalized avatar generation component, coupled to the 2D landmark points detection component and the 3D landmark points registration component, to accept the at least one 2D image from the camera, the one or more 2D landmark points as registered to the generic 3D face model, and the re-projection error, and to generate in real time personalized facial components representing the user'"'"'s face mapped to the 3D personalized morphable model. - View Dependent Claims (29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39)
-
-
40. A method of generating a personalized 3D morphable model representing a user'"'"'s face, comprising:
-
accepting at least one 2D image from a camera, the at least one 2D image including a representation of the user'"'"'s face; detecting the user'"'"'s face in the at least one 2D image; detecting 2D landmark points of the detected user'"'"'s face in the at least one 2D image; accepting a generic 3D face model and the 2D landmark points, registering each of the 2D landmark points to the generic 3D face model, and estimating a re-projection error in registering each of the 2D landmark points to the generic 3D face model; performing stereo matching for a pair of 2D images to recover a camera pose of the user; performing dense matching and bundle optimization operations to rectify a pair of 2D images such that an epipolar line corresponds to a scan tine, based at least in part on calibrated camera parameters; performing denoising/orientation propagation operations to represent the personalized 3D morphable model with an adequate number of point clouds while depicting an geometry shape having a similar appearance; performing texture mapping/image blending operations to produce avatar parameters representing the user'"'"'s face to enhance the visual effect of the avatar parameters to be photo-realistic under various lighting conditions and viewing angles; mapping the avatar parameters to the generic 3D face model to generate the personalized facial components; and generating in real time the personalized 3D morphable model east in part from the personalized facial components. - View Dependent Claims (41, 42, 43, 44)
-
Specification