Authentication via item recognition

• US 10,397,208 B2
• Filed: 12/11/2015
• Issued: 08/27/2019
• Est. Priority Date: 12/11/2015
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Eramian, David Edward
• Primary Examiner(s): Pham, Luu T
• Assistant Examiner(s): Baum, Ronald
• Application Number:
  US14/967,112
• Publication Number:
  US 20170171177A1
• Time in Patent Office: 1,355 Days
• Field of Search: 726 7
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/45 Structures or tools for the administration of authentication
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/105 Multiple levels of security
• CPC Class Codes: H04L 63/107 wherein the security policies are location-dependent, e.g. entities privileges depend on current location or allowing specific operations only from locally connected terminals
• CPC Class Codes: H04W 12/06 Authentication
• CPC Class Codes: H04W 12/65 Environment-dependent, e.g. using captured environmental data
• CPC Class Codes: H04W 88/02 Terminal devices

Abstract

Systems and methods for authenticating a user are provided. A user specifies one or more items for authentication and provides images of the one or more items. When the user opens up a mobile application, a camera on the mobile device is activated and takes a picture of an item in its field of view. If the picture matches one of the images of the items, the user is authenticated.

Claims

1. A system for authenticating a user, comprising:

a non-transitory memory; and

one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising;

receiving, from a mobile device of a user, a request for accessing a user account;

in response to receiving the request, obtaining an image captured by a camera of the mobile device;

applying one or more image recognition algorithms to the captured image to extract a first set of features related to a first item in the captured image;

comparing the first set of features to stored features associated with a plurality of reference items designated for authenticating the user of the user account to determine that the first item in the captured image matches a first reference item in the plurality of reference items;

applying the one or more image recognition algorithms to the captured image to extract a second set of features related to a second item in the captured image;

comparing the second set of features to the stored features associated with the plurality of reference items designated for authenticating the user of the user account to determine that the second item in the captured image matches a second reference item in the plurality of reference items, wherein the first and second reference items are associated with a reference location;

determining a color of a third item in the captured image;

determining that a location of the mobile device corresponds to the reference location associated with the first and second reference items based at least in part on the color of the third item in the captured image;

determining, based on analyzing the captured image, that the first item and the second item are within a geographical boundary associated with the location of the mobile device;

in response to determining that the location of the mobile device corresponds to the reference location and that the first item and the second item are within the geographical boundary, granting the mobile device access to the user account according to a first access level;

retrieving additional descriptions of the first reference item, wherein the additional descriptions represent one or more characters or symbols appearing on the first reference item;

applying at least one of an optical character recognition algorithm or a pattern recognition algorithm to a portion of the captured image representing the first item to extract a third set of features related specifically to the first item, wherein the third set of features comprises at least one of a character or a symbol that appears on the first item;

comparing the third set of features against the additional descriptions associated with the first reference item to determine that the third set of features matches the additional descriptions; and

in response to determining that the third set of features matches the additional descriptions, granting the mobile device access to the user account according to a second access level that is less restrictive than the first access level.

2. The system of claim 1, wherein the plurality of reference items comprises at least one of a piece of jewelry, a key, a piece of clothing, an accessory, a keychain, a picture, a tool, or a book.

3. The system of claim 1, wherein each reference item in the plurality of reference items is assigned one or more access levels for accessing the user account, and wherein the first access level and the second access level are determined based on the one or more access levels assigned to at least one of the first reference item or the second reference item.

4. The system of claim 1, wherein determining that the location of the mobile device corresponds to the reference location is further based on metadata extracted from the captured image.

5. The system of claim 1, wherein determining that the location of the mobile device corresponds to the reference location is further based on global positioning system (GPS) coordinates obtained from a geo-location component of the mobile device.

6. The system of claim 1, wherein the operations further comprise generating a score representing an accuracy of determining that the first item in the captured image matches the first reference item.

7. The system of claim 6, wherein the first access level is determined based at least in part on the score.

8. The system of claim 1, wherein the first access level indicates an access to view transactions related to the user account, wherein the second access level indicates an access to perform a transaction with the user account.

9. A method of authenticating a user, comprising:

receiving, from a mobile device of a user by one or more hardware processors, a request for accessing a user account;

in response to receiving the request, automatically obtaining an image captured by a camera of the mobile device;

extracting, from the captured image by the one or more hardware processors, a first set of features associated with at least a first item and a second item in the captured image by applying one or more image recognition algorithms to the captured image;

comparing, by the one or more hardware processors, the first set of features and the second set of features to stored features associated with a plurality of reference items designated for authenticating the user of the user account to determine that the first item in the captured image matches a first reference item in the plurality of reference items and that the second item in the captured image matches a second reference item in the plurality of reference items, wherein the first and second reference items are associated with a reference location;

determining, by the one or more hardware processors, a color of a third item in the captured image;

determining, by the one or more hardware processors, that a location of the mobile device corresponds to the reference location associated with the first and second reference items based at least in part on the color of the third item in the captured image;

determining, by the one or more hardware processors based on analyzing the captured image, that the first item and the second item are within a geographical boundary associated with the location of the mobile device;

in response to determining that the location of the mobile device corresponds to the reference location and that the first item and the second item are within the geographical boundary, granting, by the one or more hardware processors, the mobile device access to the user account according to a first access level;

retrieving, by the one or more hardware processors, additional descriptions of the first reference item, wherein the additional descriptions represent one or more characters or symbols appearing on the first reference item;

extracting, from the captured image by the one or more hardware processors, a second set of features by applying at least one of an optical character recognition algorithm or a pattern recognition algorithm to a portion of the captured image representing the first item, wherein the second set of features comprises at least one of a character or a symbol that appears on the first item;

comparing, by the one or more hardware processors, the second set of features against the additional descriptions associated with the first reference item to determine that the second set of features matches the additional descriptions; and

in response to determining that the second set of features matches the additional descriptions, granting, by the one or more hardware processors, the mobile device access to the user account according to a second access level that is less restrictive than the first access level.

10. The method of claim 9, wherein each reference item in the plurality of reference items is assigned one or more access levels for accessing the user account, and wherein the first access level and the second access level are determined based on the one or more access levels assigned to at least one of the first reference item or the second reference item.

11. The method of claim 9, wherein determining that the location of the mobile device corresponds to the reference location is further based on global positioning system (GPS) coordinates obtained from a geo-location component of the mobile device.

12. The method of claim 9, further comprising generating a score representing an accuracy of determining that the first item in the captured image matches the first reference item.

13. The method of claim 12, wherein the first access level is determined based at least in part on the score.

14. The method of claim 9, wherein the first access level designates a first spending limit with the user account and the second access level designates a second spending limit that is higher than the first spending limit.

15. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

receiving, from a mobile device of a user, a request for accessing a user account;

in response to receiving the request, obtaining an image from a camera of the mobile device;

applying one or more image recognition algorithms to the captured image to extract a first set of features related to a first item in the captured image;

comparing the first set of features to stored features associated with a plurality of reference items designated for authenticating the user of the user account to determine that the first item in the captured image matches a first reference item in the plurality of reference items;

applying the one or more image recognition algorithms to the captured image to extract a second set of features related to a second item in the captured image;

comparing the second set of features to the stored features associated with the plurality of reference items designated for authenticating the user of the user account to determine that the second item in the captured image matches a second reference item in the plurality of reference items, wherein the first and second reference items are associated with a reference location;

determining a color of a third item in the captured image;

determining that a location of the mobile device corresponds to the reference location associated with the first and second reference items based at least in part on the color of the third item in the captured image;

determining, based on analyzing the captured image, that the first item and the second item are within a geographical boundary associated with the location of the mobile device;

in response to determining that the location of the mobile device corresponds to the reference location and that the first item and the second item are within the geographical boundary, granting the mobile device access to the user account according to a first access level;

retrieving additional descriptions of the first reference item, wherein the additional descriptions represent one or more characters or symbols appearing on the first reference item;

applying at least one of an optical character recognition algorithm or a pattern recognition algorithm to a portion of the captured image representing the first item to extract a third set of features related specifically to the first item, wherein the third set of features comprises at least one of a character or a symbol that appears on the first item;

determining whether the third set of features matches the additional descriptions associated with the first reference item; and

in response to determining that the third set of features matches the additional descriptions, granting the mobile device access to the user account according to a second access level that is less restrictive than the first access level.

16. The non-transitory machine-readable medium of claim 15, wherein the third set of features comprises a word.

17. The non-transitory machine-readable medium of claim 15, wherein each reference item in the plurality of reference items is assigned one or more access levels for accessing the user account, and wherein the first access level and the second access level are determined based on the one or more access levels assigned to at least one of the first reference item or the second reference item.

18. The non-transitory machine-readable medium of claim 15, wherein determining that the location of the mobile device corresponds to the reference location is further based on metadata extracted from the captured image.

19. The non-transitory machine-readable medium of claim 15, wherein determining that the location of the mobile device corresponds to the reference location is further based on global positioning system (GPS) coordinates obtained from a geo-location component of the mobile device.

20. The non-transitory machine-readable medium of claim 15, wherein the first access level designates a first spending limit with the user account and the second access level designates a second spending limit that is higher than the first spending limit.