Authenticating a user using hand gesture

• US 9,274,607 B2
• Filed: 03/15/2013
• Issued: 03/01/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Bruno Delean
• Original Assignee: Bruno Delean
• Inventors: DeLean, Bruno
• Primary Examiner(s): Hicks, Charles
• Assistant Examiner(s): Truong, Nguyen H
• Application Number:
  US13/837,689
• Publication Number:
  US 20140267009A1
• Time in Patent Office: 1,082 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The present document describes a method for authenticating a user into a system using gestures. The user may draw the gesture on a touch sensitive device (e.g. touchpad), or make the gesture in the air in front of a camera. In the touchpad embodiment, the trajectory defined by the gesture is received ready from the touchpad. In the camera embodiment, the trajectory is built by analyzing the images of an image stream to find a hand (or another subject i.e. meta-subject). The trajectory may then be built by monitoring the change of position of the hand in the succession of images. The trajectory is analyzed to determine the key-code defined by the gesture, and to determine whether or not it is the authenticated user who is performing the gesture (as opposed to an intruder) based on the speed and the distance between the trajectory and a straight line.

Claims

1. A computer implemented method for authenticating a user into a system using a gesture drawn by the user, the method comprising:

dividing a trajectory defined by said gesture into a plurality of portions;

defining, for each portion, a virtual segment based on an average direction of the portion;

measuring a difference (d) between the virtual segment and the trajectory associated therewith at a number n of different locations within the trajectory, thus obtaining a set of d<sub>i=1−

n</sub>;

classifying each set of d<sub>i=1−

n</sub> as a sample point in a multidimensional space of at least n dimensions, said multidimensional space including sample points for the user (Yes samples) and sample points for other people (No samples);

determining for each set of d<sub>i=1−

n</sub> a probability that the portion associated with that set is drawn by the user, said determining comprising;

counting a number of Yes samples and a number of No samples within a predefined volume around the sample point associated with the set of d<sub>1−

n</sub>;

calculating a first ratio of Yes samples versus No samples within the predefined volume;

dividing the first ratio by a second ratio representing the number of Yes samples versus No samples in the entire multidimensional space, to produce said probability;

aggregating the probabilities associated with each portion to produce an aggregated probability for the gesture;

outputting said aggregated probability to the system for comparison with a predetermined threshold to take an authentication decision or action based on said comparison.

2. The method of claim 1, further comprising:

associating a value representing a direction to each virtual segment; and

classifying the set of d<sub>i=1−

n</sub> associated with the virtual segment in an n+1 dimensional space comprising one dimension for each di and one dimension for the value.

3. The method of claim 2, further comprising determining the value by comparing the virtual segment to a cardinal direction vector, wherein the value is an integer representing a number associated with one of the direction vectors.

4. The method of claim 3, further comprising concatenating the values associated with the virtual segments into a key-code, and outputting the key-code to the system for verification with a predefined key-code.

5. The method of claim 4, further comprising, if the key-code matches the predefined key-code, and the aggregated probability is lower than the predetermined threshold preventing the user from accessing the system.

6. The method of claim 3, wherein the cardinal direction vector comprises eight direction vectors.

7. The method of claim 2, wherein d=4, and the multidimensional space comprises 5 dimensions.

8. The method of claim 1, further comprising setting the size of the volume around the sample point as a function of a density of the multidimensional space such that the volume contains a predefined percentage of an entire count of samples in the multidimensional space.

9. The method of claim 1, further comprising setting the size of the volume around the sample point as a function of a smallest volume that may be found in the multidimensional space which includes a specific number of Yes samples.

10. The method of claim 1, further comprising performing a learning phase, said learning phase comprising:

prompting the user to perform said gesture for a number of times;

repeating the steps of detecting to classifying for each gesture;

classifying the sample points associated with each gesture in the multidimensional space as YES samples.

11. The method of claim 1, the method further comprising receiving the trajectory from a touch sensitive device.

12. The method of claim 1, further comprising building said trajectory from an image stream showing the gesture made using a meta-subject.

13. The method of claim 12, further comprising detecting a meta-subject and a position of said meta-subject in the image stream, and building said trajectory based on a sequential change of positions of the meta-subject in the image stream.

14. The method of claim 13, wherein detecting comprises scanning each image of the image stream in search for the meta-subject using a pre-loaded image of the meta-subject.

15. The method of claim 14 further comprising convoluting the images of the image stream and the pre-loaded image of the meta-subject and performing the comparison on binary versions of the images.

16. The method of claim 15, wherein scanning comprises:

dividing a given binary image of the image stream into a plurality of areas;

providing different versions of the binary image of the meta subject, each version having a different resolution, and dividing each version into one or more tiles, thus producing a number m of tiles from all the different versions;

performing a sum of square difference (SSD) between each area of the given image and each tile, to produce a set of SSD values including m SSD values;

classifying said set of SSD values as a sample point in an m-dimensional space including sample points representing images that include only the meta-subject (Yes samples) and images that do not include the meta-subject (No samples);

counting a number of Yes samples and a number of No samples within a predefined volume around the sample point associated with the area;

calculating a third ratio of Yes samples versus No samples within the predefined volume; and

dividing the third ratio by a fourth ratio representing the number of Yes samples versus No samples in the entire m-dimensional space, thus producing an image-probability indicative of the presence of the meta-subject in the given image.

17. The method of claim 16, wherein if the image-probability is greater than a predetermined threshold, outputting a position of the meta-subject in the given image.

18. The method of claim 17, further comprising outputting a size of the meta-subject in the given image.

19. The method of claim 18, further comprising setting dimensions of the areas based on the size of the meta-subject found in the given image.

20. The method of claim 16 further comprising:

morphing the binary image in a plurality of dimensions to produce morphed versions of the image of the meta-subject, and

repeating the steps of providing and performing for each morphed version to produce a plurality of sets of SSD values for each area;

outputting the SSD set having the lowest values for classification in the m-dimensional space.

21. The method of claim 16, wherein adjacent areas of the given image have overlapping pixels.

22. The method of claim 21, wherein each area is shifted by four pixels with respect to the next/previous area.

23. The method of claim 16, wherein the image of the meta-subject is an image of a body part.

24. The method of claim 23, wherein the body part is one of:

eye, hand, head, fist, neck, ear.

25. The method of claim 21, wherein the image of the meta-subject is pre-loaded.

26. The method of claim 21, wherein the image of the meta-subject is received by the user and represents a body part of the user.

27. The method of claim 21, wherein the image of the meta-subject is pre-loaded and represents a body part of someone other than the user.

28. The method of claim 16, wherein the meta-subject is a physical object excluding body parts.

29. The method of claim 16, further comprising loading the sample points from a local database, a remote server, or both thereof.

30. The method of claim 29, wherein each sample point in the m-dimensional space represents a set of m-values associated with a different image.

31. The method of claim 16, further comprising scanning the image to detect an additional meta-subject, and upon detection of said additional meta-subject in the given image, increasing the image-probability.

32. The method of claim 10, further comprising:

detecting an average duration for performing the gesture; and

upon authentication, analyzing portions of the trajectory that are received within said average duration, thereby introducing errors that disqualify gestures that are slower or faster than the user gesture.

33. A computer implemented method for authenticating a user into a system using a gesture drawn by the user, the method comprising:

dividing a trajectory defined by said gesture into a plurality of portions;

defining, for each portion, a virtual segment based on an average direction of the portion;

measuring a difference (d) between the virtual segment and the trajectory associated therewith at a number n of different locations within the trajectory, thus obtaining a set of d<sub>i=1−

n</sub>;

classifying each set of d<sub>i=1−

n</sub> as a sample point in a multidimensional space of at least n dimensions, said multidimensional space including sample points for the user (Yes samples) and sample points for other people (No samples);

determining for at least one set of d<sub>i=1−

n</sub> a probability that the portion associated with that set is drawn by the user, said determining comprising;

counting a number of Yes samples and a number of No samples within a predefined volume around the sample point associated with the set of d<sub>1−

n</sub>;

calculating a first ratio of Yes samples versus No samples within the predefined volume;

dividing the first ratio by a second ratio representing the number of Yes samples versus No samples in the entire multidimensional space, to produce said probability;

outputting the probability associated with each portion to the system for comparison with a predetermined threshold to take an authentication decision or action based on said comparison.

34. An apparatus for authenticating a user into a system using a gesture drawn by the user, the system comprising:

a trajectory analyzer adapted to;

divide a trajectory defined by said gesture into a plurality of portions,

define, for each portion, a virtual segment based on an average direction of the portion;

measure a difference (d) between the virtual segment and the trajectory associated therewith at a number n of different locations within the trajectory, to obtaining a set of d<sub>i=1−

n</sub>;

an authentication module adapted to;

classify each set of d<sub>i=1−

n</sub> as a sample point in a multidimensional space of at least n dimensions, said multidimensional space including sample points for the user (Yes samples) and sample points for other people (No samples); and

determine for each set of d<sub>i=1−

n</sub> a probability that the portion associated with that set is drawn by the user, by counting a number of Yes samples and a number of No samples within a predefined volume around the sample point associated with the set of d<sub>i=1−

n</sub>;

calculating a first ratio of Yes samples versus No samples within the predefined volume; and

dividing the first ratio by a second ratio representing the number of Yes samples versus No samples in the entire multidimensional space, to produce said probability;

a probability aggregator for aggregating the probabilities associated with each portion to produce an aggregated probability for the gesture, wherein the aggregated probability is sent to the system for comparison with a predetermined threshold to take an authentication decision or a decision based on said comparison.

35. The apparatus of claim 34, wherein the apparatus associates a value representing a direction to each virtual segment;

and classifies the set of d<sub>i=1−

n</sub> associated with the virtual segment in an n+1 dimensional space comprising one dimension for each di and one dimension for the value.

36. The apparatus of claim 35, wherein the apparatus determines the value by comparing the virtual segment to a cardinal direction vector, wherein the value is an integer representing a number associated with one of the direction vectors.

37. The apparatus of claim 36, wherein the apparatus concatenates the values associated with the virtual segments into a key-code, and outputs the key-code to the system for verification with a predefined key-code.

38. The apparatus of claim 37, wherein the apparatus prevents the user from accessing the system if the key-code matches the predefined key-code, and the aggregated probability is lower than the predetermined threshold.

39. The apparatus of claim 36, wherein the cardinal direction vector comprises eight direction vectors.

40. The apparatus of claim 35, wherein d=4, and the multidimensional space comprises 5 dimensions.

41. The apparatus of claim 34, wherein the apparatus sets the size of the volume around the sample point as a function of a density of the multidimensional space such that the volume contains a predefined percentage of an entire count of samples in the multidimensional space.

42. The apparatus of claim 34, wherein the apparatus sets the size of the volume around the sample point as a function of a smallest volume that may be found in the multidimensional space which includes a specific number of Yes samples.

43. The apparatus of claim 34, wherein the apparatus performs a learning phase, said learning phase comprising:

prompting the user to perform said gesture for a number of times;

repeating the steps of detecting to classifying for each gesture;

classifying the sample points associated with each gesture in the multidimensional space as YES samples.

44. The apparatus of claim 34, wherein the apparatus receives the trajectory from a touch sensitive device.

45. The apparatus of claim 34, wherein the apparatus builds the trajectory from an image stream showing the gesture made using a meta-subject.

46. The apparatus of claim 45, wherein the apparatus detects a meta-subject and a position of said meta-subject in the image stream, and builds said trajectory based on a sequential change of positions of the meta-subject in the image stream.

47. The apparatus of claim 46, wherein the apparatus scans each image of the image stream in search for the meta-subject using a pre-loaded image of the meta-subject.

48. The apparatus of claim 47, wherein the apparatus convolutes the images of the image stream and the pre-loaded image of the meta-subject and performs the comparison on binary versions of the images.

49. The apparatus of claim 48, wherein the apparatus is adapted to:

divide a given binary image of the image stream into a plurality of areas;

provide different versions of the binary image of the meta subject, each version having a different resolution, and divide each version into one or more tiles, thus producing a number m of tiles from all the different versions;

perform a sum of square difference (SSD) between each area of the given image and each tile, to produce a set of SSD values including m SSD values;

classify said set of SSD values as a sample point in an m-dimensional space including sample points representing images that include only the meta-subject (Yes samples) and images that do not include the meta-subject (No samples);

count a number of Yes samples and a number of No samples within a predefined volume around the sample point associated with the area;

calculate a third ratio of Yes samples versus No samples within the predefined volume; and

divide the third ratio by a fourth ratio representing the number of Yes samples versus No samples in the entire m-dimensional space, thus producing an image-probability indicative of the presence of the meta-subject in the given image.

50. The apparatus of claim 49, wherein the apparatus outputs a position of the meta-subject in the given image if the image-probability is greater than a predetermined threshold.

51. The apparatus of claim 50, wherein the apparatus outputs a size of the meta-subject in the given image.

52. The apparatus of claim 51, wherein the apparatus sets dimensions of the areas based on the size of the meta-subject found in the given image.

53. The apparatus of claim 49, wherein the apparatus is adapted to:

morph the binary image in a plurality of dimensions to produce morphed versions of the image of the meta-subject, and

repeat the steps of provide and perform for each morphed version to produce a plurality of sets of SSD values for each area; and

output the SSD set having the lowest values for classification in the m-dimensional space.

54. The apparatus of claim 49, wherein adjacent areas of the given image have overlapping pixels.

55. The apparatus of claim 54, wherein each area is shifted by four pixels with respect to the next/previous area.

56. The apparatus of claim 49, wherein the image of the meta-subject is a body part.

57. The apparatus of claim 56, wherein the body part is one of:

eye, hand, head, fist, neck, ear.

58. The apparatus of claim 49, wherein the image of the meta-subject is pre-loaded.

59. The apparatus of claim 49, wherein the image of the meta-subject is received by the user and represents a body part of the user.

60. The apparatus of claim 49, wherein the image of the meta-subject is pre-loaded and represents a body part of someone other than the user.

61. The apparatus of claim 49, wherein the meta subject is a physical object excluding body parts.

62. The apparatus of claim 49, wherein the apparatus loads the sample points from a local database, a remote server, or both thereof.

63. The apparatus of claim 62, wherein each sample point in the m-dimensional space represents a set of m-values associated with a different image.

64. The apparatus of claim 49, wherein the system scans the image to detect an additional meta-subject, and upon detection of said additional meta-subject in the given image, increasing the image-probability.

65. The apparatus of claim 43, wherein the apparatus detects an average duration for performing the gesture;

and analyzes only portions of the trajectory that are received within said average duration, thereby introducing errors that disqualify gestures that are slower or faster than the user gesture.

66. The apparatus of claim 34, wherein the apparatus is embedded within the system.

67. The apparatus of claim 34, wherein the apparatus sends the aggregated probability to the system over a telecommunications network.