ENHANCED REAL-TIME FACE MODELS FROM STEREO IMAGING

• US 20110102553A1
• Filed: 06/27/2010
• Published: 05/05/2011
• Est. Priority Date: 02/28/2007
• Status: Abandoned Application

• Current Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Ionita, Mircea C., Bigioi, Petronel, Corcoran, Peter
• Application Number:
  US12/824,204
• Publication Number:
  US 20110102553A1
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A stereoscopic image of a face is generated. A depth map is created based on the stereoscopic image. A 3D face model of the face region is generated from the stereoscopic image and the depth map. The 3D face model is applied to process an image.

Claims

1. An image processing method using a 3D face model, comprising:

generating a stereoscopic image of a face, including using a dual-lens camera, or using a method including moving a camera relative to the face to capture facial images from more than one perspective, or applying a depth from defocus process including capturing at least two differently focused images of an approximately same scene, or combinations thereof;

creating a depth map based on the stereoscopic image;

generating a 3D face model of the face region from the stereoscopic image and the depth map; and

applying the 3D face model to process an image.

2. The method of claim 1, further comprising applying a foreground/background separation operation, wherein the modeled face comprises a foreground region.

3. The method of claim 1, further comprising applying progressive blurring to the face region based on distances of different portions of the face model from the camera as determined from either the depth map, or the 3D model, or both.

4. The method of claim 3, further comprising applying selective blurring to the face based on a combination of distance from the camera and the type of face region.

5. The method of claim 4, wherein the type of face region comprises a hair region, one or both eyes, a nose or nose region, a mouth or mouth region, a cheek portion, a chin or chin region, or combinations thereof.

6. The method of claim 3, further comprising applying selective blurring to the face based on a combination of distance from the camera and the type of face region.

7. The method of claim 1, wherein said 3D face model comprises a first set of one or more illumination components corresponding to a frontally illuminated face and a second set of one or more illumination components corresponding to a directionally illuminated face.

8. The method of claim 7, further comprising applying a foreground/background separation operation, wherein the modeled face comprises a foreground region.

9. The method of claim 7, further comprising applying progressive directional illumination to the face based on distances of different portions of the face from the camera as determined from the depth map or the 3D model, or both.

10. The method of claim 7, further comprising applying selective directional illumination to the face based on a combination of distance from the camera and type of face region.

11. The method of claim 10, wherein the type of face region comprises a hair region, one or both eyes, a nose or nose region, a mouth or mouth region, a cheek portion, a chin or chin region, or combinations thereof.

24. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a method as recited at any of claims 1-23.

25. One or more computer readable storage media having code embedded therein for programming a processor to perform a method as recited at any of claims 1-23.

12. A method of determining a characteristic of a face within a scene captured in a digital image, comprising:

acquiring digital images from at least two perspectives including a face within a scene, and generating a stereoscopic image based thereon;

generating and applying a 3D face model based on the stereoscopic image, the 3D face model comprising a class of objects including a set of model components;

obtaining a fit of said model to said face including adjusting one or more individual values of one or more of the model components of said 3D face model;

based on the obtained fit of the model to said face in the scene, determining at least one characteristic of the face; and

electronically storing, transmitting, applying face recognition to, editing, or displaying a modified version of at least one of the digital images or a 3D image based on the acquired digital images including the determined characteristic or a modified value thereof, or combinations thereof.

13. The method of claim 12, wherein the model components comprise eigenvectors, and the individual values comprise eigenvalues of the eigenvectors.

14. The method of claim 12, wherein the at least one determined characteristic comprises a feature that is independent of directional lighting.

15. The method of claim 12, further comprising determining an exposure value for the face, including obtaining a fit to the face to a second 3D model that comprises a class of objects including a set of model components that exhibit a dependency on exposure value variations.

16. The method of claim 15, further comprising reducing an effect of a background region or density contrast caused by shadow, or both.

17. The method of claim 12, further comprising controlling a flash to accurately reflect a lighting condition, including obtaining a flash control condition by referring to a reference table and controlling a flash light emission according to the flash control condition.

18. The method of claim 17, further comprising reducing an effect of contrasting density caused by shadow or black compression or white compression or combinations thereof.

19. The method of claim 12, wherein the set of model components comprises a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components which are independent of directional lighting variations.

20. The method of claim 19, further comprising applying a foreground/background separation operation, wherein the modeled face comprises a foreground region.

21. The method of claim 19, further comprising applying progressive directional illumination to the face based on distances of different portions of the face from the camera as determined from the depth map or the 3D model, or both.

22. The method of claim 19, further comprising applying selective directional illumination to the face based on a combination of distance from the camera and type of face region.

23. The method of claim 22, wherein the type of face region comprises a hair region, one or both eyes, a nose or nose region, a mouth or mouth region, a cheek portion, a chin or chin region, or combinations thereof.