Separating directional lighting variability in statistical face modelling based on texture space decomposition

• US 8,542,913 B2
• Filed: 03/07/2013
• Issued: 09/24/2013
• Est. Priority Date: 02/28/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.), National University of Ireland Galway
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.), National University of Ireland Galway
• Inventors: Ionita, Mircea, Bacivarov, Ioana, Corcoran, Peter
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): PARK, SOO JIN
• Application Number:
  US13/789,616
• Publication Number:
  US 20130182920A1
• Time in Patent Office: 201 Days
• Field of Search: 382155-160
• US Class Current: 382/159
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A technique for determining a characteristic of a face or certain other object within a scene captured in a digital image including acquiring an image and applying a linear texture model that is constructed based on a training data set and that includes a class of objects including a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components which are independent of directional lighting variations. A fit of the model to the face or certain other object is obtained including adjusting one or more individual values of one or more of the model components of the linear texture model. Based on the obtained fit of the model to the face or certain other object in the scene, a characteristic of the face or certain other object is determined.

Claims

1. A method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, comprising:

providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;

applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);

determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;

constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and

combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.

2. The method of claim 1, further comprising:

(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,

(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;

(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and

(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.

3. The method of claim 2, further comprising:

(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;

(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and

(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.

4. The method of claim 1, wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.

5. One or more non-transitory processor-readable media having code embodied therein for programming a processor to perform a method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, the method comprising:

(a) providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;

(b) applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);

(c) determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;

(d) constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and

(e) combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.

6. The one or more non-transitory processor-readable media of claim 5, the method further comprising:

(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,

(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;

(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and

(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.

7. The one or more non-transitory processor-readable media of claim 6, the method further comprising:

(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;

(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and

(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.

8. The one or more non-transitory processor-readable media of claim 5, wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.

9. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, wherein the method comprises:

(a) providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;

(b) applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);

(c) determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;

(d) constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and

(e) combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.

10. The device of claim 9, wherein the method further comprises:

(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,

(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;

(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and

(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.

11. The device of claim 10, wherein the method further comprises:

(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;

(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and

(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.

12. The device of claim 9, wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.