Apparatus and methods for user recognition employing behavioral passwords

• US 6,421,453 B1
• Filed: 05/15/1998
• Issued: 07/16/2002
• Est. Priority Date: 05/15/1998
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kanevsky, Dimitri, Maes, Stephane H.
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US09/079,754
• Time in Patent Office: 1,523 Days
• Field of Search: 382/115, 382/116, 382/117, 382/118, 382/119, 382/120-124, 235/380, 235/492, 704/208, 704/246, 704/273, 704/250, 902/3, 902/4, 902/25, 348/161, 340/5.2, 340/5.52, 340/5.53, 340/5.54, 340/5.8, 340/5.81, 340/5.82, 340/5.83
• US Class Current: 382/115
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

A method for controlling access of an individual to one of a computer and a service and a facility comprises the steps of: pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session; extracting the predefined sequence of intentional gestures from the individual during a recognition session; and comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual.

Claims

1. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the method further comprises the steps of;

determining whether the individual has performed any accidental movements more than a predetermined number of times during a plurality of recognition sessions; and

adding the accidental movements to the predefined sequence of intentional gestures, when the individual has performed the accidental movements more than the predetermined number of times.

2. The method of claim 1, further comprising the step of identifying the accidental movements as optional in recognizing the individual during said comparing step.

3. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the method further comprises the step of, during the recognition session, identifying the predefined sequence of intentional gestures within a larger sequence of gestures performed by the individual by using pre-specified visually observable gestures to respectively indicate a beginning and an end of the predefined sequence of intentional gestures.

4. The method of claim 3, wherein the predefined sequence of intentional gestures used to recognize the individual comprise a sequence of actions that include the individual tapping an object in a pre-specified manner.

5. The method of claim 4, wherein at least one fingerprint of the individual is extracted from the individual while the individual is tapping, to recognize the individual based on the at least one fingerprint.

6. The method of claim 3, wherein said comparing step uses semantic gesture units corresponding to the predefined sequence of intentional gestures to recognize the individual.

15. The method of claim 3, wherein a gesture in the sequence of intentional gestures is one of touching an object, touching a body part, and waving.

16. The method of claim 3, further comprising the steps of:

digitizing video input data corresponding to the extracted sequence of intentional gestures to obtain a string of digitized image frames;

segmenting the string of frames into sub-strings of frames that represent single gestures; and

concatenating the sub-strings of frames associated with an encompassing single gesture.

17. The method of claim 16, further comprising the step of clustering vectors that represent the individual in a similar position into a respective cluster.

18. The method of claim 17, wherein the clustering step further comprises the step of replacing each vector in a respective cluster by a single cluster vector having a group value associated therewith.

19. The method of claim 18, wherein the group value of the single cluster vector is an average of the values of the vectors in the respective cluster.

20. The method of claim 17, wherein the clustering step further comprises the step of replacing the vectors in a respective cluster by a single cluster vector having a group value associated therewith.

21. The method of claim 20, wherein the group value of the single cluster vector is an average of the values of the vectors in the respective cluster.

24. The method of claim 3, wherein the individual is considered recognized if the predefined sequence of intentional gestures substantially matches the extracted sequence of intentional gestures.

25. The method of claim 3, wherein the predefined sequence of intentional gestures comprise visually observable gestures, and said method further comprises the steps of:

pre-storing at least one characteristic associated with the performance of the visually observable gestures performed during the enrollment session;

extracting the at least one characteristic from the individual during the recognition session; and

comparing the pre-stored at least one characteristic to the extracted at least one characteristic to recognize the individual.

26. The method of claim 25, wherein the at least one characteristic is one of a speed of performing at least one gesture of the gesture sequence, a speed of transitioning from a first gesture to a second gesture, and a speed of performing the entire gesture sequence.

27. The method of claim 3, further comprising the steps of:

pre-storing at least one unintentional gesture associated with, and performed during, the performance of the intentional gesture sequence of the enrollment session;

extracting the at least one unintentional gesture from the individual during the recognition session; and

comparing the pre-stored at least one unintentional gesture to the extracted at least one unintentional gesture to recognize the individual.

28. The method of claim 27, wherein the at least one unintentional gesture is a result of one of reflex and habit with respect to the performance of the intentional gesture sequence.

29. The method of claim 27, further comprising the steps of:

pre-storing at least one characteristic associated with the performance of the one unintentional gesture performed during the enrollment session;

extracting the at least one characteristic from the individual during the recognition session; and

comparing the pre-stored at least one characteristic to the extracted at least one characteristic to recognize the individual.

30. The method of claim 29, wherein the at least one characteristic is one of a speed of performing the at least one unintentional gesture.

31. The method of claim 3, further comprising the steps of:

pre-storing at least one biometric feature associated with the individual during the enrollment session;

extracting the at least one biometric feature from the individual during the recognition session; and

comparing the pre-stored at least one biometric feature to the extracted at least one biometric feature to recognize the individual.

32. The method of claim 31, wherein the at least one biometric feature is one of voice print, face recognition, signature recognition, face temperature infrared pattern, lip reading, writing instrument velocity, writing instrument pressure, fingerprint, retinal print, body geometry, and body part geometry.

33. The method of claim 3, further comprising the steps of:

pre-storing at least one non-biometric feature associated with the individual during the enrollment session;

extracting the at least one non-biometric feature from the individual during the recognition session; and

comparing the pre-stored at least one non-biometric feature to the extracted at least one non-biometric feature to recognize the individual.

34. The method of claim 33, wherein the at least one non-biometric feature is one of a password, a personal identification number (PIN), and personal information.

49. The method of claim 3, wherein the predefined sequence of intentional gestures is unrelated to handwriting, and said method further comprises the step of comparing frames corresponding to the predefined sequence of intentional gestures with a library of basic positions to identify gestures within the predefined sequence of intentional gestures.

50. The method of claim 3, wherein the sequence of intentional gestures is unrelated to handwriting, and said comparing step comprises the step of comparing a representation of the predefined sequence of intentional gestures to Hidden Markov Models (HMMs) that model a user performing a particular gesture that is also unrelated to the handwriting.

51. The method of claim 3, further comprising the steps of:

representing each frame corresponding to the predefined sequence of intentional gestures as a vector;

clustering the frames into clusters;

segmenting the clusters into strokes, wherein a stroke is a series of images that represent a user performing a particular gesture; and

concatenating strokes to form gesture symbols based on Hidden Markov Models (HMMs), each of the HMMs having a plurality of states and a plurality of output arcs, wherein the plurality of states correspond to a set of strokes, and the plurality of output arcs correspond to labels for the set of strokes.

52. The method of claim 5, further comprising the step of interpreting strings of gesture symbols as a semantic unit.

53. The method of claim 3, further comprising the step of identifying pre-specified visually detectable gestures that mark a beginning and an end of the predefined sequence of gestures.

7. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising the steps of:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual,

wherein said apparatus further comprises;

means for digitizing video input data corresponding to the extracted sequence of intentional gestures to obtain a string of digitized image frames;

means for segmenting the string of frames into sub-strings of frames that represent single gestures; and

means for concatenating the sub-strings of frames associated with an encompassing single gesture; and

wherein the segmenting means further comprises;

means for comparing each digitized image frame in the string of frames with a library of gestures to associated the digitized image frame with a single gesture;

means for utilizing predefined criteria to determine the digitized image frame in the string of frames that represents an end of a single gesture; and

means for defining a sub-string of frames other than a first sub-string of frames as the frame representing the end of the single gesture, a beginning frame preceding the frame representing the end of a next single gesture, and the frames therebetween.

8. The apparatus of claim 7, wherein the first sub-string of frames is defined as a first digitized image frame in the string of frames, the frame representing the end of the single gesture, and the frames therebetween.

9. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the apparatus further comprises;

means for pre-storing at least one non-biometric feature associated with the individual during the enrollment session;

means for extracting the at least one non-biometric feature from the individual during the recognition session; and

means for comparing the pre-stored at least one non-biometric feature to the extracted at least one non-biometric feature to recognize the individual; and

wherein the at least one non-biometric feature comprises at least one synthesized sound outside of a range of human hearing.

10. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the predefined sequence of intentional gestures comprises a sequence of hand signals performed in an enclosure that restricts visibility to the sequence of hand signals.

11. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the apparatus further comprises;

means for determining whether the individual has performed any accidental movements more than a predetermined number of times during a plurality of recognition sessions; and

means for adding the accidental movements to the predefined sequence of intentional gestures, when the individual has performed the accidental movements more than the predetermined number of times.

12. The apparatus of claim 11, further comprising means for identifying the accidental movements as optional in recognizing the individual to said means for comparing.

35. The apparatus of claim 11, wherein a gesture in the sequence of intentional gestures is one of touching an object, and touching a body part.

36. The apparatus of claim 11, further comprising:

(a) a digitizer for digitizing video input data corresponding to the extracted sequence of intentional gestures to obtain a string of digitized image frames;

(b) means for segmenting the string of frames into sub-strings of frames that represent single gestures; and

(c) means for concatenating the sub-strings of frames associated with an encompassing single gesture.

37. The method of claim 36, further comprising the step of clustering the digitized image frames that represent the individual in a similar position into a respective cluster.

38. The apparatus of claim 11, wherein the predefined sequence of intentional gestures comprise visually observable gestures, and said apparatus further comprises:

means for pre-storing at least one characteristic associated with the performance of the visually observable gestures performed during the enrollment session;

means for extracting the at least one characteristic from the individual during the recognition session; and

means for comparing the pre-stored at least one characteristic to the extracted at least one characteristic to recognize the individual.

39. The apparatus of claim 38, wherein the at least one characteristic is one of a speed of performing at least one gesture of the gesture sequence, a speed of transitioning from a first gesture to a second gesture, and a speed of performing the entire gesture sequence.

40. The apparatus of claim 11, further comprising:

means for pre-storing at least one unintentional gesture associated with, and performed during, the performance of the intentional gesture sequence of the enrollment session;

means for extracting the at least one unintentional gesture from the individual during the recognition session; and

means for comparing the pre-stored at least one unintentional gesture to the extracted at least one unintentional gesture to recognize the individual.

41. The apparatus of claim 40, wherein the at least one unintentional gesture is a result of one of reflex and habit with respect to the performance of the intentional gesture sequence.

42. The apparatus of claim 40, further comprising:

means for pre-storing at least one characteristic associated with the performance of the one unintentional gesture performed during the enrollment session;

means for extracting the at least one characteristic from the individual during the recognition session; and

means for comparing the pre-stored at least one characteristic to the extracted at least one characteristic to recognize the individual.

43. The apparatus of claim 42, wherein the at least one characteristic is one of a speed of performing the at least one unintentional gesture.

44. The apparatus of claim 11, further comprising:

means for pre-storing at least one biometric feature associated with the individual during the enrollment session;

means for extracting the at least one biometric feature from the individual during the recognition session; and

means for comparing the pre-stored at least one biometric feature to the extracted at least one biometric feature to recognize the individual.

45. The apparatus of claim 11, further comprising:

means for pre-storing at least one non-biometric feature associated with the individual during the enrollment session;

means for extracting the at least one non-biometric feature from the individual during the recognition session; and

means for comparing the pre-stored at least one non-biometric feature to the extracted at least one non-biometric feature to recognize the individual.

13. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the apparatus further comprises, during the recognition session, means for identifying the predefined sequence of intentional gestures within a larger sequence of gestures performed by the individual by using pre-specified visually observable gestures to respectively indicate a beginning and an end of the predefined sequence of intentional gestures.

14. An apparatus for controlling access of an individual to one of a computer and a service and a facility, the apparatus comprising the steps of:

means for pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

means for extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

means for comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the apparatus further comprises;

means for representing each frame corresponding to the predefined sequence of intentional gestures as a vector; and

means for clustering the frames, wherein said clustering step comprises the step of replacing a similar sequence of vectors corresponding to a single gesture with a same number of identical vectors to reduce a number of different vectors per the single gesture.

22. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual,

wherein said method further comprises the steps of;

digitizing video input data corresponding to the extracted sequence of intentional gestures to obtain a string of digitized image frames;

segmenting the string of frames into sub-strings of frames that represent single gestures; and

concatenating the sub-strings of frames associated with an encompassing single gesture; and

wherein the segmenting step further comprises the steps of;

comparing each digitized image frame in the string of frames with a library of gestures to associated the digitized image frame with a single gesture;

utilizing predefined criteria to determine the digitized image frame in the string of frames that represents an end of a single gesture; and

defining a sub-string of frames other than a first sub-string of frames as the frame representing the end of the single gesture, a beginning frame preceding the frame representing the end of a next single gesture, and the frames therebetween.

23. The method of claim 22, wherein the first sub-string of frames is defined as a first digitized image frame in the string of frames, the frame representing the end of the single gesture, and the frames therebetween.

46. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the method further comprises the steps of;

pre-storing at least one non-biometric feature associated with the individual during the enrollment session;

extracting the at least one non-biometric feature from the individual during the recognition session; and

comparing the pre-stored at least one non-biometric feature to the extracted at least one non-biometric feature to recognize the individual; and

wherein the at least one non-biometric feature comprises at least one synthesized sound outside of a range of human hearing.

47. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the predefined sequence of intentional gestures comprises a sequence of hand signals performed in an enclosure that restricts visibility to the sequence of hand signals.

48. A method for controlling access of an individual to one of a computer and a service and a facility, the method comprising the steps of:

pre-storing a predefined sequence of intentional gestures performed by the individual during an enrollment session;

extracting the predefined sequence of intentional gestures from the individual during a recognition session; and

comparing the pre-stored sequence of intentional gestures to the extracted sequence of intentional gestures to recognize the individual; and

wherein the method further comprises the steps of;

representing each frame corresponding to the predefined sequence of intentional gestures as a vector; and

clustering the frames, wherein said clustering step comprises the step of replacing a similar sequence of vectors corresponding to a single gesture with a same number of identical vectors to reduce a number of different vectors per the single gesture.