Gesture identification

• US 9,785,241 B2
• Filed: 09/27/2013
• Issued: 10/10/2017
• Est. Priority Date: 08/26/2013
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Mason, Robert A.
• Primary Examiner(s): Okebato, Sahlu
• Application Number:
  US14/039,677
• Publication Number:
  US 20150054748A1
• Time in Patent Office: 1,474 Days
• Field of Search: 345173
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Various methods and systems are provided to allow a user to perform finger gesture motions, such as making typing, swiping, tapping, or other types of finger motions to provide device input, such as typing, clicking, or selecting data into a webpage, application, operating system, or toolbar, that was normally performed by the user having to use a keyboard, mouse, stylus, microphone, touchscreen, or another input device.

Claims

1. A system comprising:

a non-transitory memory; and

one or more hardware processors coupled to the non-transitory memory and configured to execute instructions to cause the system to perform operations comprising;

determining an indication by a user to conduct a transaction using a merchant device, wherein the transaction includes a plurality of user inputs needed to complete the transaction;

responsive to the determining the indication, identifying, during a first user input of the plurality of user inputs, a first finger gesture motion made by the user and captured by a sensor associated with the merchant device, wherein the first finger gesture motion does not touch the merchant device and the first finger gesture motion corresponds to a first non-alphanumeric symbol of a plurality of non-alphanumeric symbols recognizable by the merchant device;

determining, during the first user input, that the first finger gesture motion corresponds with a gesture mapping of the first non-alphanumeric symbol that is associated with a device input and stored in a database, wherein the device input is configured to transition the transaction from one user input to another user input of the plurality of user inputs;

transitioning the transaction from the first user input to a second user input based on the first non-alphanumeric symbol and the first user input;

storing an association of the gesture mapping of the first non-alphanumeric symbol and the device input in a table lookup; and

prioritizing the association in the table lookup based on a frequency the association is accessed.

2. The system of claim 1, wherein the first non-alphanumeric symbol corresponds to a user interface element displayed to the user on the merchant device.

3. The system of claim 2, wherein the user interface element corresponds to an action performed during a payment process.

4. The system of claim 2, wherein the user interface element corresponds to an action performed during a search process.

5. The system of claim 1, wherein the determining the first finger gesture motion corresponds with the gesture mapping of the first non-alphanumeric symbol that is associated with the device input and stored in a database is executed within a tolerance or an error threshold.

6. The system of claim 1, wherein the operations further comprise requesting the user, during the first user input, to repeat the first finger gesture motion or enter a second finger gesture motion in response to determining that the first finger gesture motion does not correspond with any gesture mapping.

7. The system of claim 1, wherein the operations further comprise:

responsive to the determining the first finger gesture motion corresponds with the gesture mapping that is associated with the device input, presenting a notification of a successful match.

8. The system of claim 1, wherein the determining the first finger gesture motion corresponds with the gesture mapping of the first non-alphanumeric symbol that is associated with the device input is based on a set of discrete points that represent a path taken by the first finger gesture motion.

9. The system of claim 1, wherein the determining the first finger gesture motion corresponds with the gesture mapping of the first non-alphanumeric symbol that is associated with the device input is based on an order of two or more finger gesture motions including the first finger gesture motion.

17. The system of claim 1, wherein the operations further comprise:

performing a calibration user input, wherein the calibration user input includes

generating the gesture mapping of the first non-alphanumeric symbol based on the first finger gesture motion of the user;

associating the gesture mapping with the device input that corresponds with the on the first finger gesture motion received; and

storing the association of the gesture mapping and the device input in the database associated with a user profile of the user.

10. A method comprising:

determining an indication by a user to conduct a transaction using a merchant device, wherein the transaction includes a plurality of user inputs needed to complete the transaction;

responsive to the determining the indication, identifying, during a first user input of the plurality of user inputs, a first finger gesture motion made by the user and captured by a sensor associated with the merchant device, wherein the first finger gesture motion does not touch the merchant device and the first finger gesture motion corresponds to a first non-alphanumeric symbol of a plurality of non-alphanumeric symbols recognizable by the merchant device;

determining, during the first user input, that the first finger gesture motion corresponds with a gesture mapping of the first non-alphanumeric symbol that is associated with a device input and stored in a database, wherein the device input is configured to transition the transaction from one user input to another user input of the plurality of user inputs;

transitioning the transaction from the first user input to a second user input based on the first non-alphanumeric symbol and the first user input;

storing an association of the gesture mapping of the first non-alphanumeric symbol and the device input in a table lookup; and

prioritizing the association in the table lookup based on a frequency the association is accessed.

11. The method of claim 10, wherein the first non-alphanumeric symbol to a user interface element displayed to the user on the merchant device.

12. The method of claim 10, further comprising requesting the user, during the first user input, to repeat the first finger gesture motion or enter a second finger gesture motion in response to determining that the first finger gesture motion does not correspond with any gesture mapping.

13. The method of claim 10, wherein the determining the first finger gesture motion corresponds with the gesture mapping of the first non-alphanumeric symbol that is associated with the device input is based on a set of discrete points that represent a path taken by the first finger gesture motion.

14. The method of claim 10, wherein the determining the first finger gesture motion corresponds with the gesture mapping of the first non-alphanumeric symbol that is associated with the device input is executed within a tolerance or an error threshold.

18. The method of claim 10 comprising:

performing a calibration user input, wherein the calibration user input includes

generating the gesture mapping of the first non-alphanumeric symbol based on the first finger gesture motion of the user;

associating the gesture mapping with the device input that corresponds with the on the first finger gesture motion received; and

storing the association of the gesture mapping and the device input in the database associated with a user profile of the user.

15. A non-transitory machine readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

determining an indication by a user to conduct a transaction using a merchant device, wherein the transaction includes a plurality of user inputs needed to complete the transaction;

responsive to the determining the indication, identifying, during a first user input of the plurality of user inputs, a first finger gesture motion made by the user and captured by a sensor associated with the merchant device, wherein the first finger gesture motion does not touch the merchant device and the first finger gesture motion corresponds to a first non-alphanumeric symbols of a plurality of non-alphanumeric symbols recognizable by the merchant device;

determining, during the first user input, that the first finger gesture motion corresponds with a gesture mapping of the first non-alphanumeric symbol that is associated with a device input and stored in a database, wherein the device input is configured to transition the transaction from one user input to another user input of the plurality of user inputs;

transitioning the transaction from the first user input to a second user input based on the first non-alphanumeric symbol and the first user input;

storing an association of the gesture mapping of the first non-alphanumeric symbol and the device input in a table lookup; and

prioritizing the association in the table lookup based on a frequency the association is accessed.

16. The non-transitory machine readable medium of claim 15, wherein the first non-alphanumeric symbol corresponds to a user interface element displayed to the user on the merchant device.

19. The non-transitory machine readable medium of claim 15, wherein the operations comprise:

performing a calibration user input, wherein the calibration user input includes

generating the gesture mapping of the first non-alphanumeric symbol based on the first finger gesture motion of the user;

associating the gesture mapping with the device input that corresponds with the on the first finger gesture motion received; and

storing the association of the gesture mapping and the device input in the database associated with a user profile of the user.

20. The non-transitory machine readable medium of claim 15, wherein the operations comprise:

requesting the user, during the first user input, to repeat the first finger gesture motion or enter a second finger gesture motion in response to determining that the first finger gesture motion does not correspond with any gesture mapping.