Continuous video monitoring using face recognition for access control

• US 6,111,517 A
• Filed: 12/30/1996
• Issued: 08/29/2000
• Est. Priority Date: 12/30/1996
• Status: Expired due to Term

• Current Assignee: Rockefeller University
• Original Assignee: Visionics Corp. (Advent International Corporation)
• Inventors: Griffin, Paul A., Atick, Joseph J., Redlich, A. Norman
• Primary Examiner(s): Holloway, III, Edwin C.
• Application Number:
  US08/774,556
• Time in Patent Office: 1,338 Days
• Field of Search: 340/825.34, 340/825.31, 382/115, 382/118, 382/203, 382/276, 348/156, 348/14, 902/3, 704/273, 704/1
• US Class Current: 340/5.83
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

A continuous monitoring system for regulating access to a computer system or other restricted environment is disclosed. The system employs real-time face recognition to initially detect the presence of an authorized individual and to grant the individual access to the computer system. In some embodiments, the system also employs real-time face recognition to continuously or periodically track the continued presence of the authorized individual. Access to the computer system is revoked when the individual'"'"'s presence is no longer detected. In other embodiments, the system employs a screen saver program to deny access to the computer system when a predetermined period of user inactivity is detected. Other aspects of the invention include a stranger detector which warns the authorized individual of the approach of an unauthorized individual, a multimedia messaging center which permits unauthorized individuals to leave messages for authorized individuals, and an adaptive enrollment program which permits the system to update the stored video images of authorized individuals to reflect the individuals'"'"' current appearance.

Claims

1. A method of regulating continued access to a restricted environment, comprising:

storing a facial representation of an individual authorized to have access to the restricted environment;

periodically acquiring a facial representation of an individual desiring continued access to the restricted environment;

determining repeatedly whether the individual seeking continued access is the authorized individual by comparing the most recently acquired representation to the stored representation to determine the degree to which the most recently acquired representation corresponds to the stored representation;

storing additional identifying features representative of an authorized individual;

acquiring additional identifying features of the individual desiring continued access to the restricted environment;

comparing the acquired additional features and the stored additional features when the degree of similarity between the most recently acquired facial representation and the stored facial representation does not confirm the identity of the individual seeking continued access;

revoking access to the restricted environment if the step of determining and the step of comparing indicate that the individual seeking continued access is not the authorized individual.

2. The method of claim 1 wherein the additional identifying features comprise the head location, head shape, and head size of the authorized individual.

3. The method of claim 1 wherein the stored representation is modified by at least a portion of the acquired representations.

4. The method of claim 1 further comprising:

determining if no individual is seeking access to the restricted environment; and

revoking access to the restricted environment if the determination indicates that no individual is seeking access to the restricted environment.

5. The method of claim 1 wherein the restricted environment is a physical environment.

6. The method of claim 1 wherein a video camera is employed to acquire a facial image of the individual seeking access to the restricted environment.

7. The method of claim 1 wherein the comparison is performed in real-time.

8. The method of claim 1 wherein the step of acquiring a facial representation comprises the steps of:

detecting the presence of a face in a field of view;

aligning the detected face;

normalizing the detected face; and

representing the detected face as a template.

9. The method of claim 1, further comprising:

monitoring the environment for periods of inactivity greater than a predetermined length; and

revoking access to the environment when such period of inactivity is detected.

10. The method of claim 1 further comprising a method of regulating initial access to the restricted environment, comprising:

storing facial representations of individuals authorized to have access to the restricted environment;

acquiring a representation of the face of an individual who approaches the restricted environment during a period when access to the restricted environment is impeded;

comparing the initial acquired representation to the stored representations; and

denying initial access to the restricted environment if the comparison indicates that the individual seeking initial access is not an authorized individual.

11. The method of claim 10 further comprising:

discerning the presence of a face in the field of view of a video camera.

12. The method of claim 1 wherein the restricted environment is a virtual environment.

13. The method of claim 12 wherein the virtual environment is a computer network environment.

14. The method of claim 12 wherein the virtual environment is a PC environment.

15. A method of regulating initial and continued access to a restricted environment, comprising:

storing facial representations of individuals authorized to have access to the restricted environment;

acquiring a first representation of the face of an individual who approaches the restricted environment during a period when access to the restricted environment is impeded;

comparing the first acquired representation to the stored representations;

denying initial access to the restricted environment if the comparison indicates that the individual seeking initial access is not an authorized individual and granting initial access to the restricted environment if the comparison indicates that the individual seeking initial access is an authorized individual;

acquiring a second facial representation of an individual desiring continued access to the restricted environment;

determining whether the individual seeking continued access is the authorized individual by comparing the second acquired representation to at least one of the stored representations;

revoking access to the restricted environment if the determination indicates that the individual seeking continued access is not the authorized individual;

if initial access is denied, displaying a greeting to the unauthorized individual; and

storing a message for an authorized individual from the unauthorized individual.

16. The method of claim 15 wherein the message comprises a text portion.

17. The method of claim 15 wherein the message comprises an audio portion.

18. The method of claim 15 wherein the message comprises a visual portion.

19. A method of regulating continued access to a restricted environment, comprising:

storing a facial representation of an individual authorized to have access to the restricted environment;

acquiring a facial representation of an individual desiring continued access to the restricted environment;

determining whether the individual seeking continued access is the authorized individual by comparing the acquired representation to the stored representation;

revoking access to the restricted environment if the determination indicates that the individual seeking continued access is not the authorized individual;

detecting the approach of an unauthorized second individual to the restricted environment; and

revoking access of the authorized individual to the restricted environment when the approach of the unauthorized individual is detected.

20. A method of regulating initial access to a restricted environment, comprising:

storing a plurality of facial representations of individuals authorized to have access to the restricted environment;

associating one or more words with each authorized individual;

acquiring a representation of the face of an individual who approaches the restricted environment during a period when access to the restricted environment is impeded;

recognizing the linguistic content of the one or more words when spoken by an individual seeking access to the restricted environment;

comparing the acquired representation only against the facial representations of individuals with whom the recognized one or more words are associated;

denying initial access to the restricted environment if the comparison indicates that the individual seeking initial access is not an authorized individual.

21. A continuous monitoring system for regulating access to a restricted environment, comprising:

a CPU;

a memory connected to the CPU;

a tracking path stored in the memory comprising data regarding an individual authorized to have access to the restricted environment;

the tracking path comprising at least a facial representation of the authorized individual;

a video input device connected to the CPU and having a field of view;

an image translator resident in the CPU for repeatedly receiving images in the field of view and generating therefrom data regarding an individual located in the field of view;

a comparator resident in the CPU for repeatedly comparing the generated data with the tracking path, the results of each comparison constituting a comparison result;

an access control device resident in the CPU operative in response to the comparison results;

wherein the generated data comprises a facial representation of the individual in the field of view when it is possible to acquire a facial representation of the individual;

wherein the tracking path and the generated data further comprise additional identifying data regarding the authorized individual located in the field of view; and

wherein the comparator compares the additional generated data and the additional tracking path data when it is impossible to acquire a facial representation of the individual in the field of view.

22. The system of claim 21 wherein the generated data comprises a facial representation of the individual in the field of view.

23. The system of claim 21 wherein the tracking path comprises the most current data available regarding the authorized individual.

24. The system of claim 21 wherein the access control device denies access to the restricted environment when the comparison result indicates the absence of the authorized individual.

25. The system of claim 21 wherein the image translator, comparator, and access control device operate in real-time.

26. The system of claim 21 wherein the additional data comprise the head location, head shape, and head size of the authorized individual.

27. The system of claim 26 wherein the current data is at least partially derived from the additional generated data.

28. A monitoring system for regulating continued access to a restricted environment, comprising:

means for storing a facial representation of an individual authorized to have access to the restricted environment;

means for acquiring a facial representation of an individual desiring continued access to the restricted environment;

means for repeatedly determining whether the individual seeking continued access is the authorized individual by comparing the acquired representation and the stored representation;

means for revoking access to the restricted environment if the determination indicates that the individual seeking continued access is not the authorized individual;

second means for storing additional identifying features representative of an authorized individual;

second means for acquiring additional identifying features of an individual desiring continued access to the restricted environment;

wherein the means for determining comprises a means for comparing the acquired additional features and the stored additional features when the degree of similarity between the most recently acquired facial representation and the stored facial representation does not confirm the identity of the individual seeking continued access.

29. The system of claim 28 further comprising:

means for periodically acquiring subsequent facial representations of at least one individual desiring continued access to the restricted environment;

means for repeatedly determining the degree to which the most recently acquired representation corresponds to the stored representation of the authorized individual.

30. The system of claim 28 wherein the additional identifying features comprise the head location, head shape, and head size of the authorized individual.

31. The system of claim 28 wherein the stored representation is modified by at least a portion of the acquired representations used to identify the individual desiring continued access to the restricted environment.

32. The system of claim 28 further comprising means for determining if no individual is seeking access to the restricted environment;

and

means for revoking access to the restricted environment if the determination indicates that no individual is seeking access to the restricted environment.

33. The continuous video monitoring system of claim 28 wherein the restricted environment is a physical environment.

34. The system of claim 28 wherein the means for acquiring comprises a video camera.

35. The system of claim 28 wherein the means for comparing operates in real-time.

36. The system of claim 28 wherein the means for acquiring a facial representation comprises:

means for detecting the presence of a face in a field of view;

means for aligning the detected face;

means for normalizing the detected face; and

means for representing the detected face as a template.

37. The system of claim 28, further comprising:

means for monitoring the environment for periods of inactivity greater than a predetermined length; and

means for revoking access to the environment when such period of inactivity is detected.

38. The system of claim 28 further comprising means for regulating initial access to the restricted environment, comprising:

means for storing facial representations of individuals authorized to have access to the restricted environment;

means for acquiring a representation of the face of an individual who approaches the restricted environment during a period when access to the restricted environment is impeded;

face recognition means for comparing the initial acquired representation to the stored representations; and

means for denying initial access to the restricted environment if the comparison indicates that the individual seeking initial access is not an authorized individual.

39. The system of claim 38 wherein the means for acquiring comprises:

a video input device having a field of view, and

means for discerning the presence of a face in the field of view.

40. The system of claim 28 wherein the restricted environment is a virtual environment.

41. The system of claim 40 wherein the virtual environment is a computer network environment.

42. The system of claim 41 wherein the virtual environment is a PC environment.

43. A monitoring system for regulating initial and continued access to a restricted environment, comprising:

means for storing facial representations of individuals authorized to have access to the restricted environment;

means for acquiring a first representation of the face of an individual who approaches the restricted environment during a period when access to the restricted environment is impeded;

face recognition means for comparing the first acquired representation to the stored representations;

means for denying initial access to the restricted environment if the comparison indicates that the individual seeking initial access is not an authorized individual and for granting initial access to the restricted environment if the comparison indicates that the individual seeking initial access is an authorized individual;

means for acquiring subsequent facial representations of an individual desiring continued access to the restricted environment;

means for repeatedly determining whether the individual seeking continued access is the authorized individual by comparing the subsequent acquired representations and at least one of the stored representations;

means for revoking access to the restricted environment if the means for determining determines that the individual seeking continued access is not the authorized individual;

means, operative upon detection of an unauthorized individual, for displaying a greeting to the unauthorized individual and message means for enabling the unauthorized individual to leave a message for an authorized individual.

44. The system of claim 43 wherein the message means comprises means for leaving a typewritten message.

45. The system of claim 43 wherein the message means comprises means for leaving a spoken message.

46. The system of claim 43 wherein the message means comprises means for leaving a message comprising both audio and visual components.

47. A monitoring system for regulating continued access to a restricted environment, comprising:

means for storing a facial representation of an individual authorized to have access to the restricted environment;

means for acquiring a facial representation of an individual desiring continued access to the restricted environment;

means for repeatedly determining whether the individual seeking continued access is the authorized individual by comparing the acquired representation and the stored representation;

means for revoking access to the restricted environment if the determination indicates that the individual seeking continued access is not the authorized individual;

means for detecting the approach of an unauthorized second individual to the restricted environment; and

means, triggered by detection of the unauthorized second individual, for revoking access of the authorized individual to the restricted environment.

48. A system for regulating initial access to a restricted environment, comprising:

means for storing a plurality of facial representations of individuals authorized to have access to the restricted environment;

means for associating one or more words with each authorized individual;

means for acquiring a representation of the face of an individual who approaches the restricted envioronment during a period when access to the restricted environment is impeded;

means for recognizing the linguistic content of the one or more words when spoken by an individual seeking access to the restricted environment;

means for comparing the acquired representation only against the facial representations of individuals with whom the one or more words are associated; and

means for denying intial access to the restricted environment if the comparsion indicates that the individual seeking initial access is not an authorized individual.