IDENTIFYING CONSUMERS IN A TRANSACTION VIA FACIAL RECOGNITION

• US 20160321671A1
• Filed: 04/30/2015
• Published: 11/03/2016
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chandrasekaran, Sashikanth, Kalenichenko, Dmitry, Zwiebel, Timothy Raymond
• Granted Patent: US 9,619,803 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 20/202 Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR
• CPC Class Codes: G06Q 20/206 comprising security or operator identification provisions, e.g. password entry
• CPC Class Codes: G06Q 20/3224 Transactions dependent on location of M-devices
• CPC Class Codes: G06Q 20/326 Payment applications installed on the mobile devices
• CPC Class Codes: G06Q 20/40145 Biometric identity checks
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A merchant and a user register with a payment processing system, which establishes a facial template based on a user image. The user signs into a payment application via a user computing device, which receives an identifier from a merchant beacon device to transmit to the payment processing system. The payment processing system transmits facial templates to the merchant camera device for other users who are also signed in to the payment application in range of the merchant beacon device. The merchant camera device compares a captured facial image against the received facial templates to identify the user. A merchant POS device operator selects an account of the user. The merchant POS device transmits transaction details to the payment processing system, which processes the transaction with an issuer system. The payment processing system receives an approval of the transaction authorization request and transmits a receipt to the merchant POS device.

Claims

1. A computer-implemented method to identify users, comprising:

capturing, by a camera module of a computing device, a video feed of an environment external to the computing device;

extracting, by the computing device, a facial image of a user from the video feed;

generating, by the computing device, a facial template based on the facial image, wherein the facial template is a computer code representation of the facial image;

retrieving, by the computing device, one or more stored facial templates associated with users currently located at a merchant system location;

determining, by the computing device, a similarity between the generated facial template and each of the one or more stored facial templates;

identifying, by the computing device, a particular stored facial template as corresponding to the user based on the similarity between the generated facial template and the particular stored facial template exceeding a threshold;

transmitting, by the computing device, a response to a request to identify the user comprising an indication of the identity of the user.

2. The method of claim 1, further comprising receiving, by the computing device, a request to identify the user.

3. The method of claim 1, wherein the environment external to the computing device comprises a user located in proximity to one or more merchant system point of sale devices.

4. The method of claim 2, wherein the request to identify the user is received from a merchant system point of sale device, and wherein the response to the request to identify the user is transmitted to the merchant point of sale device.

5. The method of claim 4, further comprising, processing, by the merchant point of sale device, a transaction using payment account information associated with the identified user.

6. The method of claim 3, further comprising:

determining, by the computing device and based on a frame of the video feed, a position of a face of the user and positions of each of the one or more point of sale devices at the merchant location; and

assigning, by the computing device and based on distances between the face of the user and each of the one or more point of sale devices in the frame of the video feed, the user to a particular point of sale device of the one or more point of sale devices,

wherein the indication of the identity of the user further comprises an identifier of the particular point of sale device and an indication of the assignment of the user to the particular point of sale device.

7. A computer-implemented method to identify users, comprising:

receiving, by one or more computing devices and from a user computing device, an account identifier of a user and a merchant beacon identifier, wherein the user computing device receives the merchant beacon identifier from a merchant beacon device located at a location of a merchant system;

retrieving, by the one or more computing devices, a facial template associated with the user based on the account identifier of the user;

identifying, by the one or more computing devices, a merchant system location associated with the merchant beacon identifier;

adding, by the one or more computing devices, the facial template of the user and associated merchant system location to a current customer log comprising facial templates of users currently at the location of the merchant system;

transmitting, by the one or more computing devices and to a merchant computing device at the location of the merchant system, the current customer log, wherein the merchant computing device identifies the user based on identifying a facial template from the current customer log that is similar to a facial template generated from a facial image of the user received by the merchant computing device.

8. The method of claim 7, further comprising:

determining, by the one or more computing devices, that the user computing device is no longer in proximity of the merchant beacon device;

removing, by the one or more computing devices, the facial template of the user and associated merchant system location from the current customer log.

9. The method of claim 8, wherein the one or more computing devices determines that the user computing device is no longer in proximity of the merchant beacon device if a subsequent transmission of the user account identifier and the merchant beacon identifier is not received within a threshold amount of time from the user computing device.

10. The method of claim 7, further comprising:

receiving, by the one or more computing devices and from the computing device, a facial template associated with an identified user, a transaction total, and a merchant account identifier from the merchant computing device at the location of the merchant system;

determining, based on the received facial template, an account identifier associated with the user;

transmitting, by the one or more computing devices and to a point of sale device, one or more payment account information items associated with the user account;

receiving, by the one or more computing devices and from the point of sale device, an indication of a selection of a particular payment account information item for use in a transaction;

generating, by the one or more computing devices, a transaction authorization request comprising the transaction total, the merchant account identifier, and the particular payment account information item;

transmitting, by the one or more computing devices, the transaction authorization request to an issuer system associated with the particular payment account information item;

receiving, by the one or more computing devices and from the issuer system, data comprising an authorization of the transaction authorization request; and

transmitting, by the one or more computing devices and to the point of sale device for display on the point of sale device, a message regarding authorization of the transaction.

11. The method of claim 10, further comprising, receiving, from the computing device, an assignment of the identified user to a particular point of sale device at the location of the merchant system, wherein the one or more payment account information are transmitted to the particular point of sale device based on the assignment.

12. A computer program product, comprising:

a non-transitory computer-readable medium having computer-readable program instructions embodied thereon that when executed by a computer cause the computer to identify users, the computer-readable program instructions comprising;

computer-readable program instructions to receive a request to identify a user;

computer-readable program instructions to capture, by a camera module, a video feed of an environment external to the computer;

computer-readable program instructions to extract a facial image of the user from the video feed;

computer-readable program instructions to generate a facial template based on the facial image, wherein the facial template is a computer code representation of the facial image;

computer-readable program instructions to retrieve one or more stored facial templates associated with users currently located at a merchant system location;

computer-readable program instructions to determine a similarity between the generated facial template and each of the one or more stored facial templates;

computer-readable program instructions to identify a particular stored facial template as corresponding to the user based on the similarity between the generated facial template and the particular stored facial template exceeding a threshold; and

computer-readable program instructions to transmit a response to the request to identify the user comprising an indication of the identity of the user.

13. The computer program product of claim 12, wherein the environment external to the computer comprises a user located in proximity to one or more merchant system point of sale devices.

14. The computer program product of claim 12, wherein the request to identify the user is received from a merchant system point of sale device and wherein the response to the request to identify the user is transmitted to the merchant point of sale device.

15. The computer program product of claim 13, wherein the merchant point of sale device processes a transaction using payment account information associated with the identified user at a time after the request to identify the user is transmitted to the merchant point of sale device.

16. The computer program product of claim 13, further comprising:

computer-readable program instructions to determine, based on a frame of the video feed, a position of a face of the user and positions of each of one or more point of sale devices at the merchant location;

computer-readable program instructions to assign, based on distances between the face of the user and each of the one or more point of sale devices in the frame of the video feed, the user to a particular point of sale device of the one or more point of sale devices,

wherein the indication of the identity of the user further comprises an identifier of the particular point of sale device and an indication of the assignment of the user to the particular point of sale device.

17. A system to identify users in a transaction, comprising:

a storage device; and

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to;

receive, from a user computing device, an account identifier of a user and a merchant beacon identifier, wherein the user computing device receives the merchant beacon identifier via a network connection with a merchant beacon device located at a location of a merchant system;

retrieve a facial template associated with the user based on the account identifier of the user;

identify a merchant system location associated with the merchant beacon identifier;

add the facial template of the user and associated merchant system location to a current customer log comprising facial templates of users currently at the location of the merchant system; and

transmit, to a computing device at the location of the merchant system, the current customer log, wherein the computing device identifies the user based on identifying a facial template from the current customer log that is similar to a facial template generated from a received facial image of the user.

18. The system of claim 17, wherein the processor is further configured to execute computer-readable program instructions stored on the storage device to cause the system to:

determine that the user computing device no longer maintains a network connection with the merchant beacon device; and

remove the facial template of the user and associated merchant system location from the current customer log.

19. The system of claim 17, wherein determining that the user computing device no longer maintains the network connection with the merchant beacon device comprises determining that a subsequent transmission of the user account identifier and the merchant beacon identifier is not received within a threshold amount of time from the user computing device

20. The system of claim 17, wherein the processor is further configured to execute computer-readable program instructions stored on the storage device to cause the system to:

receive, from the computing device, a facial template associated with an identified user, a transaction total, and a merchant account identifier from the computing device at the location of the merchant system;

determine, based on the received facial template, an account identifier associated with the user;

transmit, to a point of sale device, one or more payment account information associated with the user account;

receive, from the point of sale device, an indication of a selection of a particular payment account information for use in a transaction;

generate a transaction authorization request comprising the transaction total, the merchant account identifier, and the particular payment account information;

transmit the transaction authorization request to an issuer system associated with the particular payment account;

receive, from the issuer system, data comprising an approval of the transaction authorization request; and

transmit, to the point of sale device for display on the point of sale device, a receipt comprising a summary of the transaction.