Facial image processing methods and systems

• US 7,363,201 B2
• Filed: 11/02/2004
• Issued: 04/22/2008
• Est. Priority Date: 08/30/2000
• Status: Expired due to Term

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Kang, Sing Bing, Olynyk, Kirk, Guenter, Brian K., Raghupathy, Sashi, Marschner, Stephen
• Primary Examiner(s): JONES, HUGH M
• Application Number:
  US10/980,073
• Publication Number:
  US 20050089199A1
• Time in Patent Office: 1,267 Days
• Field of Search: 703/2, 382/275, 382/294
• US Class Current: 703/2
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject'"'"'s face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

Claims

1. A facial image-processing system comprising:

means for illuminating a face with multiple different light sources, wherein at least one of the light sources is polarized;

means for measuring range map data from said illuminating;

means for measuring image data from said illuminating;

means for deriving a 3-dimensional surface from the range map data;

means for computing surface normals to the 3-dimensional surface; and

means for processing the surface normals and the image data to derive an albedo map.

2. The system of claim 1, wherein at least one of the light sources is infrared.

3. The system of claim 1, wherein the light sources are at different frequencies.

4. A facial image-processing system comprising:

means for illuminating a face with multiple different light sources, wherein all of the light sources are polarized;

means for measuring range map data from said illuminating;

means for measuring image data from said illuminating;

means for deriving a 3-dimensional surface from the range map data;

means for computing surface normals to the 3-dimensional surface; and

means for processing the surface normals and the image data to derive an albedo map.

5. A facial image-processing system comprising:

one or more processors;

one or more computer-readable media;

computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising;

illuminating a face with multiple different light sources, wherein at least one of the light sources is polarized;

measuring range map data from said illuminating;

measuring image data from said illuminating;

deriving a 3-dimensional surface from the range map data;

computing surface normals to the 3-dimensional surface; and

processing the surface normals and the image data to derive an albedo map.

6. The system of claim 5, wherein at least one of the light sources is infrared.

7. The system of claim 5, wherein the light sources are at different frequencies.

8. A facial image-processing system comprising:

one or more processors;

one or more computer-readable media;

computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising;

illuminating a face with multiple different light sources, wherein all of the light sources are polarized;

measuring range map data from said illuminating;

measuring image data from said illuminating;

deriving a 3-dimensional surface from the range map data;

computing surface normals to the 3-dimensional surface; and

processing the surface normals and the image data to derive an albedo map.