Face recognition system

• US RE36,041 E
• Filed: 11/16/1994
• Issued: 01/12/1999
• Est. Priority Date: 11/01/1990
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Pentland, Alex P., Turk, Matthew
• Primary Examiner(s): Mancuso, Joseph
• Application Number:
  US08/340,615
• Time in Patent Office: 1,518 Days
• Field of Search: 382/115, 382/118, 382/200, 382/201, 382/195, 382/278, 382/279, 382/724, 382/204, 348/143, 348/1, 340/825.36, 340/825.49, 455/2
• US Class Current: 382/118
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: G06K 9/00228 Detection; Localisation; Normalisation
• CPC Class Codes: G06K 9/00241 using holistic features extraction or representation of holistic features per se G06K9/00275
• CPC Class Codes: G06K 9/00275 Holistic features and representations, i.e. based on the facial image taken as a whole statistical feature extraction in general G06K9/6232; matching eigenfaces G06K9/00288
• CPC Class Codes: G06K 9/6232 Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
• CPC Class Codes: G06K 9/6247 based on an approximation criterion, e.g. principal component analysis
• CPC Class Codes: G07C 9/00158 by means of a personal physical data
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/94 Vector quantisation
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015

Abstract

A recognition system for identifying members of an audience, the system including an imaging system which generates an image of the audience; a selector module for selecting a portion of the generated image; a detection means which analyzes the selected image portion to determine whether an image of a person is present; and a recognition module responsive to the detection means for determining whether a detected image of a person identified by the detection means resembles one of a reference set of images of individuals.

Claims

1. A recognition system for identifying members of an audience, the system comprising:

an imaging system which generates an image of the audience;

a selector module for selecting a portion of said generated image;

means for representing a reference set of images of individuals as a set of eigenvectors in a multi-dimensional image space;

a detection means which determines whether the selected image portion contains an image that can be classified as an image of a person, said detection means including means for representing said selected image portion as an input vector in said multi-dimensional image space and means for computing the distance between a point identified by said input vector and a multi-dimensional subspace defined by said set of eigenvectors, wherein said detection means uses the computed distance to determine whether the selected image portion contains an image that can be classified as an image of a person; and

a recognition module responsive to said detection means for determining whether a detected image of a person identified by said detection means resembles one of the reference set of images of individuals.

2. The recognition system of claim 1 wherein said detection means further comprises a thresholding means for determining whether an image of a person is present by comparing said computed distance to a preselected threshold.

3. The recognition system of claim 1 wherein said . .selection means.!. .Iadd.selector module .Iaddend.comprises a motion detector for identifying the selected portion of said image by detector motion.

4. The recognition system of claim 3 wherein said . .selection means.!. .Iadd.selector module .Iaddend.further comprises a locator module for locating the portion of said image corresponding to a face of the person based on motion detected by said motion detector.

5. The recognition system of claim 1 wherein said image of a person is an image of a person'"'"'s face and wherein said reference set comprises images of faces of said individuals.

6. The recognition system of claim 1 wherein said recognition module comprises means for representing each member of said reference set as a corresponding point in said subspace.

7. The recognition system of claim 6 wherein the location of each point in subspace associated with a corresponding member of said reference set is determined by projecting a vector associated with that member onto said subspace.

8. The recognition system of claim 7 wherein said recognition module further comprises means for projecting said input vector onto said subspace.

9. The recognition system of claim 8 wherein said recognition module further comprises means for selecting a particular member of said reference set and means for computing a distance within said subspace between a point identified by the projection of said input vector onto said subspace and the point in said subspace associated with said selected member.

10. The recognition system of claim 8 wherein said recognition module further comprises means for determining for each member of said reference set a distance in subspace between the location associated with that member in subspace and the point identified by the projection of said input vector onto said subspace.

11. The recognition system of claim 10 wherein said image of a person is an image of a person'"'"'s face and wherein said reference set comprises images of faces of said individuals.

12. A method for identifying members of an audience, the method comprising:

generating an image of the audience;

selecting a portion of said generated image;

representing a reference set of images of individuals as a set of eigenevectors in a multi-dimensional image space;

representing said selected image portion as an input vector in said multi-dimensional image space;

computing the distance between a point identified by said input vector and a multi-dimensional subspace defined by said set of eigenvectors;

using the computed distance to determine whether the selected image portion contains an image that can be classified as an image of a person; and

if it is determined that the selected image contains an image that can be classified as an image of a person determining whether said image of a person resembles one of a reference set of images of individuals.

13. The method of claim 12 further comprising the step of determining which one, if any, of the members of said reference set said image of a person resembles.

14. The method of claim 12 wherein the image of the audience is a sequence of image frames and wherein the method further comprises detecting motion within the sequence of image frames and wherein the selected image portion is determined on the basis of the detected motion.

15. The method of claim 12 wherein the step of determining whether the selected image portion contains an image that can be classified as an image of a person further comprises comparing said computed distance to a preselected threshold.

16. The method of claim 15 wherein the step of determining whether said image of a person resembles a member of said reference set comprises representing each member of said reference set as a corresponding point in said subspace.

17. The method of claim 16 wherein the step of determining whether said image of a person resembles a member of said reference set further comprises determining the location of each point in subspace associated with a corresponding member of said reference set by projecting a vector associated with that member onto said subspace.

18. The method of claim 17 wherein the step of determining whether said image of a person resembles a member of said reference set further comprises projecting said input vector onto said subspace.

19. The method of claim 18 wherein the step of determining whether said image of a person resembles a member of said reference set further comprises selecting a member of said reference set and computing a distance within said subspace between a point identified by the projection of said input vector onto said subspace and the point in said subspace associated with said selected member.

20. The method of claim 18 wherein the step of determining whether said image of a person resembles a member of said reference set further comprises determining for each member of said reference set a distance in subspace between the location for that member in subspace and the point identified by the projection of said input vector onto said subspace.

21. The method of claim 20 wherein said image of a person is an image of a person'"'"'s face and wherein said reference set comprises images of faces of said individuals. .Iadd.

22. A recognition system comprising:

an imaging system which generates an image;

a selector module for selecting a portion of said generated image;

means for representing a reference set of images of individuals as a set of eigenvectors in a multi-dimensional image space;

a detection means which determines whether the selected image portion contains an image that can be classified as an image of a person, said detection means including means for representing said selected image portion as an input vector in said multi-dimensional image space and means for computing the distance between a point identified by said input vector and a multi-dimensional subspace defined by said set of eigenvectors, wherein said detection means uses the computed distance to determine whether the selected image portion contains an image that can be classified as an image of a person; and

a recognition module responsive to said detection means for determining whether a detected image of a person identified by said detection means resembles one of the reference set of images of individuals. .Iaddend..Iadd.23. The recognition system of claim 22 wherein said detection means further comprises a thresholding means for determining whether an image of a person is present by comparing said computed distance to a preselected threshold. .Iaddend..Iadd.24. The recognition system of claim 22 wherein said image of a person is an image of a person'"'"'s face and wherein said reference set comprises images of faces of said individuals. .Iaddend..Iadd.25. The recognition system of claim 22 wherein said recognition module comprises means for representing each member of said reference set as a corresponding point in said subspace.

24. individuals. .Iaddend..Iadd.32. The method of claim 31 further comprising the step of determining which one, if any, of the members of said reference set said image of a person'"'"'s face resembles. .Iaddend..Iadd.33. The method of claim 31 wherein the step of determining whether the selected image portion contains an image that can be classified as an image of a person'"'"'s face further comprises comparing said computed distance to a preselected threshold. .Iaddend..Iadd.34. The method of claim 33 wherein the step of determining whether said image of a person'"'"'s face resembles a member of said reference set comprises representing each member of said reference set as a corresponding point in said subspace. .Iaddend..Iadd.35. The method of claim 34 wherein the step of determining whether said image of a person'"'"'s face resembles a member of said reference set further comprises determining the location of each point in subspace associated with a corresponding member of said reference set by projecting a vector associated with that member onto said subspace.

23. Iaddend..Iadd.26. The recognition system of claim 25 wherein the location of each point in subspace associated with a corresponding member of said reference set is determined by projecting a vector associated with that member onto said subspace. .Iaddend..Iadd.27. The recognition system of claim 26 wherein said recognition module further comprises means for projecting said input vector onto said subspace. .Iaddend..Iadd.28. The recognition system of claim 27 wherein said recognition module further comprises means for selecting a particular member of said reference set and means for computing a distance within said subspace between a point identified by the projection of said input vector onto said subspace and the point in said subspace associated with said selected member. .Iaddend..Iadd.29. The recognition system of claim 27 wherein said recognition module further comprises means for determining for each member of said reference set a distance in subspace between the location associated with that member in subspace and the point identified by the projection of said input vector onto said subspace. .Iaddend..Iadd.30. The recognition system of claim 24 wherein said means for representing said reference set includes means for adding a member to said reference set by protecting into said subspace an input vector having a computed distance indicative of an image of a face. .Iaddend..Iadd.31. A method comprising:

generating an image;

selecting a portion of said generated image;

representing a reference set of images of faces of individuals as a set of eigenvectors in a multi-dimensional image space;

representing said selected image portion as an input vector in said multi-dimensional image space;

computing the distance between a point identified by said input vector and a multi-dimensional subspace defined by said set of eigenvectors;

using the computed distance to determine whether the selected image portion contains an image that can be classified as an image of a person'"'"'s face; and

if it is determined that the selected image contains an image that can be classified as an image of a person'"'"'s face, determining whether said image of a person'"'"'s face resembles one of a reference set of images of faces of

25. Iaddend..Iadd. The method of claim 35 wherein the step of determining whether said image of a person'"'"'s face resembles a member of said reference set further comprises projecting said input vector onto said subspace. .Iaddend..Iadd.37. The method of claim 36 wherein the step of determining whether said image of a person'"'"'s face resembles a member of said reference set further comprises determining for each member of said reference set a distance in subspace between the location for that member in subspace and the point identified by the projection of said input vector onto said subspace. .Iaddend.