Facial recognition and the open mouth problem

• US 7,421,098 B2
• Filed: 02/16/2005
• Issued: 09/02/2008
• Est. Priority Date: 10/07/2002
• Status: Expired due to Term

• Current Assignee: Intel Benelux B. V.
• Original Assignee: TECHNION RESEARCH AND DEVELOPMENT FOUNDATION, LTD. (Technion-Israel Institute Of Technology)
• Inventors: Bronstein, Michael, Bronstein, Alexander, Kimmel, Ron
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/058,171
• Publication Number:
  US 20050180613A1
• Time in Patent Office: 1,294 Days
• Field of Search: 345/419, 345/473, 345/621, 345/622, 382/107, 382/115, 382/118, 382/154, 382/282, 382/283, 382/190, 382/203
• US Class Current: 382/118
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method, for electronic face recognition using geodesic representations, of processing such a geodesic representation of a face having a mouth and being able to adopt open-mouthed or closed mouth states, so that the electronic recognition works regardless of whether the mouth in the representation is open or closed. The method comprises: identifying a mouth region within a face; applying a topological constraint to the mouth region, and representing the face with the constraint. The topological constraint is chosen to render the geodesic representation invariant to the mouth state being open or closed.

Claims

1. A method of automatic face recognition, comprising providing an electronic representation of a predetermined face having a mouth and being able to adopt open-mouthed or closed mouth states, for electronic face processing to compare with a candidate face, said providing an electronic representation comprising:

identifying a mouth region within said predetermined face;

applying a topological constraint to said identified mouth region; and

representing said face using said topological constraint to mask said identified mouth region with a mask being substantially invariant to a change in extent of openness of said mouth, thereby to render said representation invariant to a change in an open or closed mouth state.

2. The method of claim 1, wherein said topological constraint is selected such that a geodesic contour travels about said mouth region over a substantially similar path irrespective of whether said face is in an open mouth state or a closed mouth state.

3. The method of claim 2, wherein said topological constraint comprises setting said mouth region into a permanently open state.

4. The method of claim 3, wherein setting said mouth region into a permanently open state comprises cropping about said mouth region.

5. The method of claim 2, wherein said topological constraint comprises setting said mouth region into a permanently closed state.

6. The method of claim 1, wherein said identifying a mouth region comprises identifying lips.

7. The method of claim 6, wherein said identifying lips comprises texture processing.

8. The method of claim 1, comprising cropping said face by:

selecting a first geodesic contour about an invariant reference point on said face, setting a region within said first geodesic contour as a first mask, selecting a second geodesic about a boundary of said identified mouth region, setting a region within said second geodesic as a second mask, and forming a final mask from a union of said first mask and said second mask.

9. Apparatus for electronic face recognition by providing an electronic representation of a predetermined face having a mouth and being able to adopt a mouth open state comprising at least an open-mouthed state and a closed mouth state, for comparison with a candidate face for electronic face processing to recognize said candidate face, the apparatus comprising:

a mouth region identifier for identifying a mouth region within said predetermined face; and

a constraint application unit for applying a topological constraint to said identified mouth region, the apparatus being configured to represent said face and to use said topological constraint to mask said identified mouth region with a mask being substantially invariant to a change in extent of openness of said mouth, thereby to render said representation invariant to a change in a mouth open or closed state.

10. The apparatus of claim 9, wherein said topological constraint is selected such that a geodesic contour travels about said mouth region over a substantially similar path irrespective of said mouth open state.

11. The apparatus of claim 10, wherein said topological constraint comprises setting said mouth region into a permanently open state.

12. The apparatus of claim 11, wherein setting said mouth region into a permanently open state comprises cropping about said mouth region.

13. The apparatus of claim 10, wherein said topological constraint comprises setting said mouth region into a permanently closed state.

14. The apparatus of claim 9, wherein said identifying a mouth region comprises identifying lips.

15. The apparatus of claim 14, wherein said identifying lips comprises texture processing.

16. The apparatus of claim 9, further comprising a cropping unit for cropping said face, said cropping unit comprising:

a first mask unit for selecting a first geodesic contour about an invariant reference point on said face, and setting a region within said first geodesic contour as a first mask, a second mask unit for selecting a second geodesic about a boundary of said identified mouth region, and setting a region within said second geodesic as a second mask, and a third mask unit for forming a final mask from a union of said first mask and said second mask.

17. Apparatus according to claim 9, comprising:

a geodesic converter, for receiving 3-dimensional topographical data of said geometric body as a triangulated manifold, and for converting said triangulated manifold into a series of geodesic distances between pairs of points of said manifold, and a multi-dimensional scaler, connected subsequently to said geodesic converter, for forming a low dimensional Euclidean representation of said series of geodesic distances, said low dimensional Euclidean representation providing a bending invariant representation of said geometric body suitable for matching with other geometric shapes, said geodesic converter and said multidimensional scaler both being located prior to said mouth region identifier.

18. Apparatus according to claim 17, further comprising a subsampler located prior to said geodesic converter, configured to subsample said triangulated manifold, and to provide to said geodesic converter a subsampled triangulated manifold.

19. Apparatus according to claim 18, wherein said manifold comprises a plurality of vertices and wherein said subsampler is operable to select a first vertex and to iteratively select a next vertex having a largest geodesic distance from vertices already selected, until a predetermined number of vertices has been selected.

20. Apparatus according to claim 19, wherein said subsampler is operable to use the fast marching method for triangulated domains to calculate geodesic distances between vertices for said iterative selection.

21. Apparatus according to claim 18, wherein said geometric body is a face, having soft geometric regions, being regions of said face susceptible to short term geometric changes, and hard geometric regions, being regions substantially insusceptible to said short term geometric changes, said apparatus comprising a preprocessor, located prior to said subsampler, for removing said soft geometric regions from said face.

22. Apparatus according to claim 21, wherein said preprocessor is operable to identify said soft regions by identification of an orientation point on said face.

23. Apparatus according to claim 22, wherein said orientation point is at least one of a nose tip, a pair of eyeball centers and a mouth center.

24. Apparatus according to claim 21, wherein said preprocessor is further operable to center said face.

25. Apparatus according to claim 21, wherein said preprocessor is further operable to crop said face.

26. Apparatus according to claim 21, wherein said preprocessor is operable to carry out removal of said soft regions by application of a geodesic mask.

27. Apparatus according to claim 17, wherein said geodesic converter is operable to use the fast marching method for triangulated domains to calculate said geodesic distances.

28. Apparatus according to claim 17, further comprising a triangulator for forming said triangulated manifold from scan data of a geometric body.

29. Apparatus according to claim 28, further operable to embed said triangulated manifold into a space of higher than two dimensions, thereby to include additional information with said topographical information.

30. Apparatus according to claim 29, wherein said additional information is any one of a group comprising texture information, albedo information, grayscale information, and color information.

31. Apparatus according to claim 17, wherein said subsampler comprises an optimizer for allowing a user to select an optimum subsampling level by defining a trade-off between calculation complexity and representation accuracy.

32. Apparatus according to claim 17, wherein said multi-dimensional scalar is configured such that said Euclidean representation comprises a predetermined number of eigenvalues extractable to be used as co-ordinates in a feature space.

33. Apparatus according to claim 32, wherein said predetermined number of cigenvalues is at least three, and said feature space has a number of dimensions corresponding to said predetermined number.