System and method providing improved head motion estimations for animation

• US 7,853,053 B2
• Filed: 03/31/2010
• Issued: 12/14/2010
• Est. Priority Date: 12/06/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Liu, Zicheng
• Primary Examiner(s): Seth; Manav
• Application Number:
  US12/751,705
• Publication Number:
  US 20100189310A1
• Time in Patent Office: 258 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/115, 382117-118, 382/154, 382/181, 348/154, 348/155
• US Class Current: 382/118
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

The computer-readable media provides improved procedures to estimate head motion between two images of a face. Locations of a number of distinct facial features are determined in two images. The locations are converted into as a set of physical face parameters based on the symmetry of the identified distinct facial features. An estimation objective function is determined by: (a) estimating each of the set of physical parameters, (b) estimating a first head pose transform corresponding to the first image, and (c) estimating a second head pose transform corresponding to the second image. The motion is estimated between the two images based on the set of physical face parameters by multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of data corresponding to the estimation objective function.

Claims

1. One or more computer-readable storage media containing a program that is executable by a computer to estimate motion between two images, the program configured to perform the following actions:

determining locations of a plurality of distinct facial features in the two images;

converting the determined locations into a set of physical face parameters based on a symmetry of the distinct facial features;

calculating an estimation objective function comprising a set of terms to estimate;

(a) each of the parameters, (b) a first transform corresponding to a first image of the two images, and (c) a second transform corresponding to a second image of the two images; and

estimating motion between the two images based on the set of physical face parameters by multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of data corresponding to the estimation objective function.

2. One or more computer-readable storage media as recite in claim 1, further comprising adding an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within predetermined minimum and maximum values.

3. One or more computer-readable storage media as recited in claim 2, wherein the predetermined minimum value and the predetermined maximum value are based on absolute values of other of the locations.

4. One or more computer-readable storage media as recited in claim 2, further comprising converting the inequality constraint to an equality constraint using a penalty function.

5. One or more computer-readable storage media as recited in claim 4, wherein the estimating further comprises adding the equality constraint to the initial estimate.

6. One or more computer-readable storage media as recited in claim 4, wherein the calculating further comprises multiplying each term of the estimation objective function and the equality constraint by a weighted contribution factor based on a confidence of data corresponding to the estimation objective function.

7. One or more computer-readable storage media as recited in claim 1, wherein the determining comprises accepting input from a human user.

8. One or more computer-readable storage media as recited in claim 1, wherein the determined locations correspond to the eye corners, mouth corners and nose tip.

9. One or more computer-readable storage media containing a program that is executable by a computer to estimate motion between two images, the program comprising the following actions:

determining locations of a plurality of distinct facial features in the two images;

converting the determined locations into a set of physical face parameters based on a symmetry of the distinct facial features; and

estimating motion between the two images based on the set of physical face parameters, the estimating further comprising;

calculating an estimation objective function to determine an initial estimate of image rotation, the estimation objective function comprising a set of terms to estimate;

(a) each of the parameters, (b) a first pose transform corresponding to a first image of the two images, and (c) a second pose transform corresponding to a second image of the two images;

placing an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within a predetermined minimum and maximum value;

converting the inequality constraint to an equality constraint using a penalty function; and

adding the equality constraint to the initial estimate.

10. One or more computer-readable storage media as recited in claim 9, wherein the estimating further comprises multiplying each term of the estimation objective function by a weighted contribution factor based on a confidence of data corresponding to the estimation objective function.

11. One or more computer-readable storage media as recited in claim 9, wherein the predetermined minimum value and the predetermined maximum value are based on absolute values of other of the locations.

12. One or more computer-readable storage media as recited in claim 9, wherein the determining comprises accepting input from a human user.

13. One or more computer-readable storage media as recited in claim 9, wherein the determined locations correspond to the eye corners, mouth corners and nose tip.

14. One or more computer-readable storage media containing a program that is executable by a computer to estimate motion between two images, the program configured to perform the following actions:

determining locations of a plurality of distinct facial features in the two images;

converting the determined locations into a set of physical face parameters based on the symmetry of the distinct facial features;

adding an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within predetermined minimum and maximum values, the predetermined minimum value and the predetermined maximum value are based on absolute values of other of the locations; and

estimating motion between the two images based on the set of physical face parameters.

15. One or more computer-readable storage media as recited in claim 14, wherein the estimating motion further comprises calculating an estimation objective function comprising a set of terms to estimate:

(a) each of the parameters, (b) a first transform corresponding to a first image of the two images, and (c) a second transform corresponding to a second image of the two images.

16. One or more computer-readable storage media as recited in claim 14, wherein the estimating further comprises multiplying each term of the estimation objective function by a weighted contribution factor based on a confidence of data corresponding to the estimation objective function.

17. One or more computer-readable storage media as recited in claim 14, further comprising converting the inequality constraint to an equality constraint using a penalty function.

18. One or more computer-readable storage media as recited in claim 14, wherein the estimating further comprises:

calculating an estimation objective function to determine an initial estimate of image rotation, the estimation objective function comprising a set of terms to estimate;

(a) each of the parameters, (b) a first pose transform corresponding to a first image of the two images, and (c) a second pose transform corresponding to a second image of the two images;

placing an inequality constraint on a particular parameter of the parameters, such that the particular parameter is constrained within predetermined minimum and maximum values

converting the inequality constraint to an equality constraint using a penalty function; and

adding the equality constraint to the initial estimate.

19. One or more computer-readable storage media as recited in claim 14, wherein the determining comprises accepting input from a human user.

20. One or more computer-readable storage media as recited in claim 14, wherein the determined locations correspond to the eye corners, mouth corners and nose tip.