IDENTIFYING CONSUMERS VIA FACIAL RECOGNITION TO PROVIDE SERVICES

• US 20160321633A1
• Filed: 04/29/2016
• Published: 11/03/2016
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Al Hashal, Jumana, Chandrasekaran, Sashikanth, Ho, Denise, Kalenichenko, Dmitry, Chitilian, Varouj, Zwiebel, Timothy Raymond
• Granted Patent: US 10,733,587 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G07F 9/001 Interfacing with vending machines using mobile or wearable devices
• CPC Class Codes: G07F 9/009 User recognition or proximity detection
• CPC Class Codes: G07F 9/026 for alarm, monitoring and auditing in vending machines or means for indication, e.g. when empty

Abstract

A user signs into an application via a user computing device at a merchant system location. The user computing device receives an identifier from a beacon device at the location to transmit to an account management system. The account management system transmits facial templates to the merchant camera device for users who are signed in to the application in range of the merchant beacon device. The user approaches a point of sale device to purchase a restricted product or service. The merchant camera device compares a captured facial image against the received facial templates to identify the user. A merchant POS device operator, after determining that the user is eligible for the restricted product or service based on account information of the user, selects a payment account of the user. The merchant POS device transmits transaction details to the account management system, which processes the transaction with an issuer system.

Claims

1. A computer-implemented method to identify users, comprising:

receiving, by a computing device and from a point of sale computing device, a request to identify a user for a transaction involving a restricted item;

capturing, by a camera on a computing device, a video feed of an environment external to the computing device;

extracting, by the computing device, a facial image of the user from the video feed;

generating, by the computing device, a facial template based on the facial image, the facial template comprising a computer code representation of the facial image;

retrieving, by the computing device, one or more stored facial templates associated with users located at a merchant location;

determining, by the computing device, a similarity between the generated facial template and each of the one or more stored facial templates;

identifying, by the computing device, a stored facial template that corresponds to the user based on a determination that the similarity between the generated facial template and the stored facial template exceeds a similarly threshold;

transmitting, by the computing device and to the point of sale computing device, an indication of the identity of the user, the point of sale computing device accessing account information associated with the identified user and indicating whether the identified user is eligible to complete the transaction involving the restricted item.

2. The method of claim 1, wherein the environment external to the computing device comprises a user located in proximity to a plurality point of sale computing devices.

3. The method of claim 1, wherein the restricted item comprises redemption of a coupon.

4. The method of claim 1, further comprising, processing, by the point of sale computing device, the transaction using payment account information associated with the identified user.

5. The method of claim 2, further comprising:

determining, by the computing device, a position of a face of the user and positions of each of the plurality of point of sale computing devices at the merchant location based on a frame of the video feed; and

assigning, by the computing device, the user to one of the plurality of point of sale computing devices, based on distances between the determined location of the face of the user and the determined position of each of the plurality of point of sale computing devices in the frame of the video feed,

wherein the indication of the identity of the user further comprises an identification that the user is assigned to the one of the plurality of point of sale computing devices and an identification of the one of the plurality of point of sale computing devices.

6. A computer-implemented method to identify users, comprising:

receiving, by one or more computing devices operated by an account management computing system and from a user computing device, an account identifier for a user and a merchant beacon identifier, wherein the user computing device receives the merchant beacon identifier from a merchant beacon device located at a merchant location;

retrieving, by the one or more computing devices operated by the account management computing system, a facial template associated with the user based on the account identifier;

identifying, by the one or more computing devices operated by the account management computing system, the merchant location associated with the merchant beacon identifier;

adding, by the one or more computing devices operated by the account management computing system, the facial template of the user and merchant location to a customer log, the customer log comprising facial templates of users currently at the merchant location;

transmitting, by the one or more computing devices operated by the account management computing system and to a merchant computing device at the merchant location, the customer log, the merchant computing device identifying the user by comparing the facial template from the customer log to a second facial template generated by the merchant computing device and from a facial image of the user received by the merchant computing device;

receiving, by the one or more computing devices operated by the account management computing system and from the merchant computing device, the facial template associated with an identified user;

determining, by the one or more computing devices operated by the account management computing system, an identity of the user based on the received facial template;

transmitting, by the one or more computing devices operated by the account management computing system and to the merchant computing device, the user identity and user account information indicating whether the user is eligible to complete a transaction involving a restricted purchase, the merchant computing device performing an operation based on the user identity the user account information.

7. The method of claim 6, further comprising:

determining, by the one or more computing devices operated by the account management computing system, that the user computing device is no longer in proximity of the merchant beacon device;

removing, by the one or more computing devices operated by the account management computing system, the facial template of the user and the merchant location from the customer log.

8. The method of claim 7, wherein the one or more computing devices operated by the account management computing system determines that the user computing device is no longer in proximity of the merchant beacon device when a subsequent transmission of the user account identifier and the merchant beacon identifier is not received within a threshold amount of time.

9. The method of claim 7, further comprising, receiving, from the merchant computing device, an assignment of the user to a point of sale computing device at the merchant location, the user identity being transmitted to the point of sale computing device at the merchant location.

10. A computer program product, comprising:

a non-transitory computer-readable medium having computer-readable program instructions embodied thereon that when executed by a computer cause the computer to identify users, the computer-readable program instructions comprising;

computer-readable program instructions to receive an input requesting a transaction involving a restricted purchase by a user;

computer-readable program instructions to transmit to a camera a request to identify the user;

computer-readable program instructions to receive from the camera an identify of the user;

computer-readable program instructions to transmit the identity of the user and a request for account information associated with the user to one or more computing devices operated by an account management computing system;

computer-readable program instructions to receive account information associated with the user from the one or more computing devices operated by an account management computing system, the account information comprising payment account information and eligibility information for restricted transactions;

computer-readable program instructions to determine not to process the transaction based on the eligibility information for restricted transactions and one or more eligibility criteria;

computer-readable program instructions to display an error message indicating that the user is ineligible to proceed with the transaction involving the restricted purchase.

11. The computer program product of claim 10, wherein the restricted purchase comprises an alcohol purchase, and wherein the one or more eligibility criteria comprise at least an eligibility criterion that the purchaser be greater than or equal to a predetermined age.

12. The computer program product of claim 10, wherein the restricted purchase comprises application of a discount to the transaction, and wherein the one or more eligibility criteria comprise at least an eligibility criterion that the user be greater than or equal to a predetermined age.

13. The computer program product of claim 10, wherein determining not to process the transaction based on the eligibility information for restricted transactions and one or more eligibility criteria, comprises:

computer-readable program instructions to determine one or more eligibility criteria required to the restricted purchase;

computer-readable program instructions to compare the eligibility information for restricted transactions to the one or more eligibility criteria; and

computer-readable program instructions to determine that the eligibility information for restricted transactions does not comprise at least one of the one or more eligibility criteria.

14. The computer program product of claim 10, wherein the eligibility information for restricted transactions comprises one or more of age information, demographic information, club membership information, or other relevant information to the purchase of restricted products or services.

15. The computer program product of claim 11, wherein the one or more eligibility criteria comprise one or more of an age requirement, demographic requirement, club membership requirement, or other relevant requirement to the purchase of restricted products or services.

16. The computer program product of claim 10, further comprising computer-readable program instructions to extract the one or more eligibility criteria, the one or more eligibility criteria comprising an identity of a restricted product in the requested transaction.

17. A system to identify users in a transaction, comprising:

a storage device; and

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to;

receive an input requesting a transaction involving a restricted purchase by a user;

transmit to a camera a request to identify the user;

receive from the camera an identify of the user;

transmit the identity of the user and a request for account information associated with the user to one or more computing devices operated by an account management computing system;

receive account information associated with the user from the one or more computing devices operated by an account management computing system, the account information comprising payment account information and eligibility information for restricted transactions;

determine to process the transaction based on the eligibility information for restricted transactions and one or more eligibility criteria;

receive an input of a selection of payment account information and an input to process the transaction; and

transmit the payment account information to an issuer computing system associated with the payment account information.

18. The system of claim 17, wherein determining, based on the eligibility information, to process the transaction comprises:

determine one or more eligibility criteria required to purchase the restricted product or service;

compare the eligibility information for restricted transactions to the one or more eligibility criteria; and

determine that the eligibility information for restricted transactions matches at least one of the one or more eligibility criteria.

19. The system of claim 17, wherein the eligibility information comprises one or more of age information, demographic information, club membership information, or other relevant information to the purchase of restricted products or services.

20. The system of claim 17, wherein the one or more eligibility criteria comprise one or more of an age requirement, demographic requirement, club membership requirement, or other relevant requirement to the purchase of restricted products or services.