System and method providing improved head motion estimations for animation
First Claim
1. A method to estimate head motion between two images of a face, the method comprising operations of:
- identifying locations of a plurality of distinct facial features in the two images, the locations corresponding to a number of unknowns determined upon estimation of head motion, the number of unknowns being determined by a number of equations;
converting the identified locations into a set of physical face parameters based on the symmetry of the distinct facial features, the set of physical face parameters reducing the number of unknowns as compared to the number of equations used to determine the unknowns; and
estimating head motion from identified points in the two images, the identified points being based on the set of physical face parameters; and
wherein the operation of estimating head motion comprise operations of;
calculating an estimation objective function to determine an initial estimate of head rotation, the estimation objective function comprising a set of terms to estimate;
(a) each of the set of physical face parameters. (b) a first head pose transform corresponding to the first image, and (c) a second head pose transform corresponding to the second image;
placing an inequality constraint on a particular face parameter of the set of physical face parameters, such that the particular face parameter is constrained within a predetermined minimum and maximum value;
converting the inequality constraint to an equality constraint using a penalty function; and
adding the equality constraint to the initial estimate.
2 Assignments
0 Petitions
Accused Products
Abstract
The system provides improved procedures to estimate head motion between two images of a face. Locations of a number of distinct facial features are identified in two images. The identified locations can correspond to the eye comers, mouth corners and nose tip. The locations are converted into as a set of physical face parameters based on the symmetry of the identified distinct facial features. The set of physical parameters reduces the number of unknowns as compared to the number of equations used to determine the unknowns. An initial head motion estimate is determined by: (a) estimating each of the set of physical parameters, (b) estimating a first head pose transform corresponding to the first image, and (c) estimating a second head pose transform corresponding to the second image. The head motion estimate can be incorporated into a feature matching algorithm to refine the head motion estimation and the physical facial parameters. In one implementation, an inequality constraint is placed on a particular physical parameter—such as a nose tip, such that the parameter is constrained within a predetermined minimum and maximum value. The inequality constraint is converted to an equality constraint by using a penalty function. Then, the inequality constraint is used during the initial head motion estimation to add additional robustness to the motion estimation.
-
Citations
29 Claims
-
1. A method to estimate head motion between two images of a face, the method comprising operations of:
-
identifying locations of a plurality of distinct facial features in the two images, the locations corresponding to a number of unknowns determined upon estimation of head motion, the number of unknowns being determined by a number of equations;
converting the identified locations into a set of physical face parameters based on the symmetry of the distinct facial features, the set of physical face parameters reducing the number of unknowns as compared to the number of equations used to determine the unknowns; and
estimating head motion from identified points in the two images, the identified points being based on the set of physical face parameters; and
wherein the operation of estimating head motion comprise operations of;
calculating an estimation objective function to determine an initial estimate of head rotation, the estimation objective function comprising a set of terms to estimate;
(a) each of the set of physical face parameters. (b) a first head pose transform corresponding to the first image, and (c) a second head pose transform corresponding to the second image;
placing an inequality constraint on a particular face parameter of the set of physical face parameters, such that the particular face parameter is constrained within a predetermined minimum and maximum value;
converting the inequality constraint to an equality constraint using a penalty function; and
adding the equality constraint to the initial estimate.
-
-
2. A method to estimate head motion between two images of a face, the method comprising operations of:
-
identifying locations of a plurality of distinct facial features in the two images, the locations corresponding to a number of unknowns determined upon estimation of head motion, the number of unknowns being determined by a number of equations;
converting the identified locations into a set of physical face parameters based on the symmetry of the distinct facial features, the set of physical face parameters reducing the number of unknowns as compared to the number of equations used to determine the unknowns;
adding an inequality constraint on a particular face parameter of the physical face parameters, such that the particular face parameter is constrained within a predetermined minimum and maximum value;
converting the inequality constraint to an equality constraint using a penalty function; and
estimating head motion from identified points in the two images, the identified points being based on the set of physical face parameters. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9)
-
-
10. One or more computer-readable media containing computer-program instructions executable by a processor to estimate motion between two images, the computer-program instructions comprising instructions for:
-
determining locations of a plurality of distinct features in the two images;
converting the identified locations into a set of parameters based on the symmetry of the distinct features; and
estimating motion between the two images based on the set of physical face parameters by;
(1) calculating an estimation objective function comprising a set of terms to estimate;
(a) each of the parameters, (b) a first transform corresponding to the first image, and (c) a second transform corresponding to the second image; and
(2) multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of data corresponding to the estimation objective function. - View Dependent Claims (11, 12, 13, 14, 15, 16)
-
-
17. A method to estimate head motion between two images of a face, the method comprising:
-
identifying locations of a plurality of distinct facial features in the two images, the locations corresponding to a number of unknowns determined upon estimation of head motion, the number of unknowns being determined by a number of equations;
converting the locations into set of physical face parameters based on the symmetry of the identified distinct facial features, the set of physical face parameters reducing the number of unknowns as compared to the number of equations used to determine the unknowns;
determining an initial estimation of head motion between the two images using the set of physical face parameters as follows;
(1) calculating an estimation objective function comprising a number of terms to estimate each of;
(a) the set of coordinates, (b) a first head pose transform corresponding to the first image, and (c) a second head pose transform corresponding to the second image; and
(2) multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of data corresponding to the estimation objective function; and
refining the initial estimation of head motion by incorporating the initial estimation into a feature matching algorithm. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A computing device to estimate motion between two images, the computing device comprising:
-
a processor; and
a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for;
identifying locations of a plurality of distinct features in the two images, the locations corresponding to symmetrical features shared between the two images, the locations corresponding to a number of unknowns determined upon estimation of motion between the two images;
converting the locations into a set of parameters based on the symmetry of the identified distinct features, the parameters reducing the number of unknowns as compared to a number of equations used to determine the unknowns;
estimating motion between the two images using the parameters by;
(1) calculating an estimation objective function comprising a number of terms to estimate each of;
(a) the parameters, (b) a first image pose transform corresponding to the first image, and (c) a second image pose transform corresponding to the second image;
(2) multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of that corresponding data;
(3) providing an initial estimate of image rotation by estimation of each of the parameters, a first image pose transform corresponding to the first image, and a second image pose transform corresponding to the second image;
(4) adding an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within a predetermined minimum and maximum value;
(5) converting the inequality constraint to an equality constraint using a penalty fiction; and
(6) adding the equality constraint to the initial estimate.
-
-
28. A method to estimate motion between two images, the method comprising:
-
identifying locations of a plurality of distinct features in the two images, the locations corresponding to symmetrical features shared between the two images, the locations corresponding to a number of unknowns determined upon estimation of motion between the two images;
converting the locations into a set of parameters based on the symmetry of the identified distinct features, the parameters reducing the number of unknowns as compared to a number of equations used to determine the unknowns; and
estimating motion between the two images using the parameters by;
providing an initial estimate of image rotation by estimation of each of the parameters, a first image pose transform corresponding to the first image, and a second image pose transform corresponding to the second image;
multiplying each term of a set of terms used to determine the initial estimate by a weighted contribution factor based on the confidence of data corresponding to the initial estimate;
adding an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within a predetermined minimum and maximum value;
converting the inequality constraint to an equality constraint using a penalty function; and
adding the equality constraint to the initial estimate. - View Dependent Claims (29)
-
Specification