Facial image processing

• US 20060015308A1
• Filed: 09/01/2005
• Published: 01/19/2006
• Est. Priority Date: 08/30/2000
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Kang, Sing Bing, Olynyk, Kirk, Guenter, Brian K., Raghupathy, Sashi, Marschner, Stephen
• Granted Patent: US 7,433,807 B2
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject'"'"'s face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

Claims

1. A computer-implemented facial image-processing method comprising:

illuminating a face with multiple different light sources;

measuring range map data from said illuminating;

measuring image data from said illuminating;

deriving a 3-dimensional surface from the range map data;

computing surface normals to the 3-dimensional surface; and

processing the surface normals and the image data to derive an albedo map.

2. The method of claim 1, wherein at least one of the light sources is polarized.

3. The method of claim 1, wherein all of the light sources are polarized.

4. The method of claim 1 further comprising prior to deriving the 3-dimensional surface, filtering the range map data.

5. A computer-implemented facial image-processing method comprising:

illuminating a face with illumination; and

contemporaneously capturing structure data describing the face'"'"'s structure and reflectance data describing reflectance properties of the face from the illumination.

6. The method of claim 5, wherein said illuminating comprises using multiple light sources.

7. The method of claim 6, wherein one of the light sources projects a pattern on the face from which the structure data can be ascertained.

8. The method of claim 6, wherein one of the light sources comprises an infrared light source.

9. The method of claim 6, wherein all of the light sources comprise infrared light sources.

10. The method of claim 5, wherein said illuminating comprises using multiple polarized light sources.

11. The method of claim 5, wherein said illuminating comprises illuminating the face with light sources at different frequencies.

12. The method of claim 5, wherein said capturing comprises using a camera having a polarizer that suppresses specularly-reflected light so that diffuse component reflection data is captured.

13. The method of claim 12, wherein one of the light sources projects a pattern on the face from which the structure data can be ascertained.

14. The method of claim 13, wherein the one light source comprises an infrared light source.

15. The method of claim 5, wherein said illuminating comprises illuminating the face with multiple narrow-band light sources.

16. A computer-implemented facial image-processing method comprising:

illuminating a face with a first polarized light source that is selected so that specularly-suppressed reflective properties of the face can be ascertained;

illuminating the face with a second structured light source that projects a pattern onto the face, while simultaneously illuminating the face with the first polarized light source;

capturing both specularly-suppressed reflection data and structure data from the simultaneous illumination.

17. The method of claim 16, wherein the light sources provide light at different frequencies.

18. The method of claim 16, wherein the light sources provide infrared light.

19. The method of claim 16 further comprising processing the captured data to provide both (a) data that describes dimensional aspects of the face and (b) data that describes diffuse reflective properties of the face.

20. The method of claim 19, wherein the data that describes the diffuse reflective properties of the face comprises an albedo map.