Controlling resource access based on user gesturing in a 3D captured image stream of the user

• US 7,971,156 B2
• Filed: 01/12/2007
• Issued: 06/28/2011
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Goldman, Steven D., Sessa, Anthony J., Arnold, Kenneth C., Paolini, Michael A., Albertson, Jacob C.
• Primary Examiner(s): Pitaro; Ryan F
• Application Number:
  US11/622,687
• Publication Number:
  US 20080170776A1
• Time in Patent Office: 1,628 Days
• Field of Search: None
• US Class Current: 715/863
• CPC Class Codes: G06F 21/35 communicating wirelessly
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04W 12/68 Gesture-dependent or behaviour-dependent

Abstract

A computer-implemented method, system, and program product comprises a gesture processing system for capturing a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one stereoscopic image device aimed at the user to identify and track at least one particular three-dimensional movement of the user. The gesture processing system predicts at least one defined movement within the captured three-dimensional movement matching at least one gesture definition of at least one gesture type from among a plurality of gesture definitions. A resource control system receives a request to access a resource based on the predicted at least one gesture type. The resource control system only allows user access to the resource by matching the predicted at least one gesture type with at least one required gesture type.

Claims

1. A computer-implemented method for controlling access to a resource, comprising:

sending, by a resource control system for controlling electronic access by a user to a resource, to the user a communication comprising a performance instruction specifying a plurality of instruction sequences each specified by separate a name of a separate instruction sequence from among the plurality of instruction sequences, at least one required gesture to be made by the user to access the resource, and at least required one background item to be worn by the user when making the sequence of gestures, prior to the user requesting to access the resource;

responsive to the user requesting to access the resource, accessing the performance instructions previously submitted to the user and selecting one instruction sequence of the plurality of instruction sequences based on a time of day;

prompting the user with the name of the selected instruction sequence;

capturing a three-dimensional movement of a user and an image of the three-dimensional movement, wherein the three-dimensional movement and image is determined using at least one image capture device aimed at the user;

predicting at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement;

predicting at least one background item definition of at least one background item type from among a plurality of background item definitions matching at least one item worn by the user within the image;

and

only allowing user access to the resource if the predicted at least one gesture type matches with the at least one required gesture for the selected instruction sequence and if the predicted at least one background item matches the required background item for the selected instruction sequence.

2. The method according to claim 1, wherein capturing a three-dimensional movement of the user further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

3. The method according to claim 1, wherein predicting at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement further comprises calculating a percentage probability that the at least one defined movement represents the at least one gesture type defined in the at least one gesture definition.

4. The method according to claim 1, wherein predicting at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement further comprises calculating a percentage probability that the at least one defined movement maps to the at least one gesture type defined in the at least one gesture definition, wherein the at least one gesture definition specifies a particular gesture definition of the at least one gesture type by the user.

5. The method according to claim 1, wherein predicting at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement further comprises identifying at least one of body movement of the user comprising at least one of a head movement, a facial expression, an eye movement, a muscle contraction, and a change in skin surface characteristics of the user within the at least one defined movement matching at least one of the plurality of gesture definitions.

6. The method according to claim 1, further comprising:

capturing a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the user;

predicting a sequence of at least one gesture type matching a sequence of a plurality of defined movements within the captured three-dimensional movement;

requesting access to the resource based on the sequence of predicted matching gesture types each specified according to a separate time range when captured; and

only allowing user access to the resource by matching the predicted matching gesture types at the separate time ranges with a required series of least one required gesture type with the separate time ranges matching a time-based pattern.

7. The method according to claim 1, further comprising:

generating an operation control signal for at least one computer system based the operation control signal matching the at least one gesture type.

8. The method of claim 1, further comprising:

selecting a time based pattern;

prompting the user with the name of the selected instruction sequence and an instruction to enter the gesture following the time based pattern;

outputting the time based pattern via a tactile detectable device detectable by the user;

comparing the captured three-dimensional movement with the time based pattern to detect whether the user gestured to mimic the time based pattern; and

only allowing user access to the resource if the predicted at least one gesture type matches the at least one required gesture for the selected instruction sequence and the user enters the at least one gesture type to mimic the time based pattern.

9. A system for controlling access to a resource, comprising:

a resource control system for controlling electronic access by a user to a resource;

the resource control system for sending a communication to the user comprising a performance instruction specifying a plurality of instruction sequences each specified by separate a name of a separate instruction sequence from among the plurality of instruction sequences, at least one required gesture to be made by the user to access the resource, and at least required one background item to be worn by the user when making the sequence of gestures, prior to the user requesting to access the resource;

the resource control system, responsive to the user requesting to access the resource, for accessing the performance instructions previously submitted to the user and selecting one instruction sequence of the plurality of instruction sequences based on a time of day;

the resource control system for prompting the user with the name of the selected instruction sequence;

a gesture processing system communicatively connected to the resource control system via a network;

the gesture processing system for capturing a three-dimensional movement of a user and an image of the three-dimensional movement, wherein the three-dimensional movement and image is determined using at least one image capture device aimed at the user;

the gesture processing system for predicting at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement;

the gesture processing system for predicting at least one background item definition of at least one background item type from among a plurality of background item definitions matching at least one item worn by the user within the image;

the resource control system for only allowing user access to the resource if the predicted at least one gesture type matches with the at least one required gesture for the selected instruction sequence and if the predicted at least one background item matches the required background item for the selected instruction sequence.

10. The system according to claim 9, further comprising the gesture processing system for capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

11. The system according to claim 9, further comprising the gesture processing system for calculating a percentage probability that the at least one defined movement represents the at least one gesture type defined in the at least one gesture definition.

12. The system according to claim 9, further comprising the gesture processing system for calculating a percentage probability that the at least one defined movement maps to the at least one gesture type defined in the at least one gesture definition, wherein the at least one gesture definition specifies a particular gesture definition of the at least one gesture type by the user.

13. The system according to claim 9, further comprising the gesture processing system for identifying at least one of body movement of the user comprising at least one of a head movement, a facial expression, an eye movement, a muscle contraction, and a change in skin surface characteristics of the user within the at least one defined movement matching at least one of the plurality of gesture definitions.

14. The system according to claim 9, further comprising:

the gesture processing system for capturing a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the user;

the gesture processing system for predicting a sequence of at least one gesture type matching a sequence of a plurality of defined movements within the captured three-dimensional movement;

the resource control system for requesting access to the resource based on the sequence of predicted matching gesture types each specified according to a separate time range when captured; and

the resource control system for only allowing user access to the resource by matching the predicted matching gesture types at the separate time ranges with a required series of least one required gesture type with the separate time ranges matching a time-based pattern.

15. The system of claim 9, further comprising:

the resource control system for selecting a time based pattern;

the resource control system for prompting the user with the name of the selected instruction sequence and an instruction to enter the gesture following the time based pattern;

the resource control system for outputting the time based pattern via a tactile detectable device detectable by the user;

the gesture processing system for comparing the captured three-dimensional movement with the time based pattern to detect whether the user gestured to mimic the time based pattern; and

the resource control system for only allowing user access to the resource if the predicted at least one gesture type matches the at least one required gesture for the selected instruction sequence and the user enters the at least one gesture type to mimic the time based pattern.

16. A program product comprising a storage-type computer-usable medium including a computer-readable program for controlling access to a resource, wherein the computer-readable program when executed on a computer cause the computer to:

send, by a resource control system for controlling electronic access by a user to a resource, a communication to the user comprising a performance instruction specifying a plurality of instruction sequences each specified by separate a name of a separate instruction sequence from among the plurality of instruction sequences, at least one required gesture to be made by the user to access the resource, and at least required one background item to be worn by the user when making the sequence of gestures, prior to the user requesting to access the resource;

responsive to the user requesting to access the resource, access the performance instructions previously submitted to the user and selecting one instruction sequence of the plurality of instruction sequences based on a time of day;

prompt the user with the name of the selected instruction sequence;

capture a three-dimensional movement of a user and an image of the three-dimensional movement, wherein the three-dimensional movement and image is determined using at least one image capture device aimed at the user;

predict at least one gesture definition of at least one gesture type from among a plurality of gesture definitions matching at least one defined movement within the captured three-dimensional movement;

predict at least one background item definition of at least one background item type from among a plurality of back round item definitions matching at least one item worn by the user within the image;

and

only allow user access to the resource if the predicted at least one gesture type matches with the at least one required gesture for the selected instruction sequence and if the predicted at least one background item matches the required background item for the selected instruction sequence.

17. The program product of claim 16, wherein said computer-readable program is transmitted over a network.

18. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to capture the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

19. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to calculate a percentage probability that the at least one defined movement maps to the at least one gesture type defined in the at least one gesture definition, wherein the at least one gesture definition specifies a particular gesture definition of the at least one gesture type by the user.

20. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

select a time based pattern;

prompt the user with the name of the selected instruction sequence and an instruction to enter the gesture following the time based pattern;

output the time based pattern via a tactile detectable device detectable by the user;

compare the captured three-dimensional movement with the time based pattern to detect whether the user gestured to mimic the time based pattern; and

only allow user access to the resource if the predicted at least one gesture type matches the at least one required gesture for the selected instruction sequence and the user enters the at least one gesture type to mimic the time based pattern.