ASSOCIATING SHOPPERS TOGETHER

• US 20200134701A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: NCR Voyix Corporation
• Original Assignee: NCR Corporation
• Inventors: Zucker, Brent Vance, Lieberman, Adam Justin
• Granted Patent: US 11,176,597 B2
• Time in Patent Office: Days
• CPC Class Codes: G06K 7/10297 arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00
• CPC Class Codes: G06Q 20/12 specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06Q 20/34 using cards, e.g. integrated circuit [IC] cards or magnetic cards
• CPC Class Codes: G06Q 20/40145 Biometric identity checks
• CPC Class Codes: G06Q 30/0635 Processing of requisition or of purchase orders
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

A store having an automated checkout can automatically associate multiple shoppers with a single purchase. For example, a mother and her children can walk independently through the store and select items. Upon checkout, the items selected by the mother and children can all be charged to the mother'"'"'s credit card in a single purchase. The system and method discussed herein can capture images of multiple shoppers from one or more video streams of a store area, can associate the multiple shoppers with one another from the captured images when the multiple shoppers appear together in a check-in area of the store, can track the movement of the multiple shoppers within the store from the captured images, can track items selected by the multiple shoppers from the captured images, and can initiate a transaction that charges the selected items to an account associated with one of the multiple shoppers.

Claims

1. A method, comprising:

receiving, at a check-in location of a store, selection of an account to be charged for a purchase, the account being associated with a shopper;

identifying, with video surveillance of the store, the shopper as being one of a group of shoppers that are present in the check-in location of the store when the selection is received;

assigning a personal identifier to each one of the group of shoppers, wherein the personal identifier is unique to each one of the group of shoppers;

tracking based on the personal identifier for each one of the group of shoppers, with the video surveillance of the store, movement of each one of the group of shoppers throughout the store and items in the store selected for purchase by any of the shoppers in the group of shoppers; and

charging, through a frictionless check-out of the store, the selected account for purchase of the selected items.

2. The method of claim 1, wherein the video surveillance tracks the items in the store by tracking items that are handled by any of the group of shoppers.

3. The method of claim 2, wherein tracking the items in the store selected for purchase by any of the shoppers in the group of shoppers comprises:

sensing, with the video surveillance of the store, that a first shopper, of the group of shoppers, has picked up a first item from a shelf or rack; and

adding the first item to a virtual shopping cart.

4. The method of claim 3, wherein tracking the items in the store selected for purchase by any of the shoppers in the group of shoppers further comprises:

sensing, with the video surveillance of the store, that the first shopper has returned the first item to the shelf or rack; and

removing the first item from the virtual shopping cart.

5. The method of claim 1, wherein receiving the selection of the account comprises:

receiving input of data corresponding to a credit card number from the shopper.

6. The method of claim 5, wherein receiving the data corresponding to the credit card number comprises:

receiving a credit card swipe.

7. The method of claim 1, wherein receiving the selection of the account comprises:

reading a radiofrequency identification tag that corresponds to the shopper; and

retrieving, from a server, a stored credit number that corresponds to the shopper.

8. The method of claim 1, wherein receiving the selection of the account comprises:

scanning a visual code that corresponds to the shopper; and

retrieving, from a server, a stored credit number that corresponds to the shopper.

9. The method of claim 1, wherein receiving the selection of the account comprises:

scanning a biometric indicator that corresponds to the shopper; and

retrieving, from a server, a stored credit number that corresponds to the shopper.

10. The method of claim 1, wherein identifying the shopper as being one of the group of shoppers comprises:

receiving at least one image from the video surveillance system, the at least one image including the check-in location of the store; and

assigning a common cart identifier to each person identified in the at least one image, the common cart identifier identifying all the shoppers in the group.

11. The method of claim 1, wherein identifying the shopper as being one of the group of shoppers comprises:

forming cropped images of bodies of the identified shoppers in the group;

sending the cropped images to at least one processor configured to execute a deep-learning autoencoder; and

storing, on a storage device coupled to the at least one processor, a multi-dimensional feature vector in a nearest-neighbors database.

12. The method of claim 11, Wherein tracking the items in the store selected for purchase comprises:

sensing, with the video surveillance of the store, that a first item has been picked up;

sending, to the deep-learning autoencoder, a cropped image of the person who picked up the first item;

retrieving, from the deep-learning autoencoder, a feature vector corresponding to the sent cropped image;

calculating a Euclidean distance metric between the retrieved feature vector and feature vectors stored on the storage device; and

identifying, from the stored feature vector having the shortest Euclidean distance to the retrieved feature vector, a first shopper as the person who picked up the first item.

13. A system, comprising:

at least one video camera positioned to capture at least one video stream of a store;

a video interface configured to receive the at least one video stream; and

a processor coupled to the video interface and configured to execute computing instructions to perform data processing activities, the data processing activities comprising;

receiving, at a check-in location of the store, selection of an account to be charged for a purchase, the account being associated with a shopper;

identifying, from the at least one video stream, the shopper as being one of a group of shoppers that are present in the check-in location of the store when the selection is received;

assigning a personal identifier to each one of the group of shoppers, wherein the personal identifier is unique to each one of the group of shoppers;

tracking based on the personal identifier for each one of the group of shoppers, from the at least one video stream, movement of each one of the group of shoppers throughout the store and items in the store selected for purchase by any of the shoppers in the group of shoppers; and

charging, through a frictionless check-out of the store, the selected account for purchase of the selected items.

14. The system of claim 13, wherein tracking the items in the store comprises tracking items that are handled by any of the group of shoppers.

15. The system of claim 14, wherein tracking the items in the store comprises:

sensing, from the at least one video stream, that a first shopper, of the group of shoppers, has picked up a first item from a shelf or rack; and

adding the first item to a virtual shopping cart.

16. The system of claim 15, wherein tracking the items in the store further comprises:

sensing, from the at least one video stream, that the first shopper has returned the first item to the shelf or rack; and

removing the first item from the virtual shopping cart.

17. The system of claim 13, wherein receiving the selection of the account comprises:

receiving input of data corresponding to a credit card number from the shopper.

18. The system of claim 13, wherein identifying the shopper as being one of the group of shoppers comprises:

receiving at least one image from the at least one video stream, the at least one image including the check-in location of the store; and

assigning a common cart identifier to each person identified in the at least one image, the common cart identifier identifying all the shoppers in the group.

19. The system of claim 13, wherein identifying the shopper as being one of the group of shoppers comprises:

forming cropped images of bodies of the identified shoppers in the group;

sending the cropped images to a deep-learning autoencoder configured to execute on the processor; and

storing, on a storage device coupled to the processor, a multi-dimensional feature vector in a nearest-neighbors database.

20. A method, comprising:

receiving, at a check-in location of a store, selection of an account to be charged for a purchase, the account being associated with a shopper;

retrieving, from a server, a stored credit number that corresponds to the account;

identifying, with video surveillance of the store, the shopper as being one of a group of shoppers that are present in the check-in location of the store when the selection is received;

assigning a personal identifier to each one of the group of shoppers, wherein the personal identifier is unique to each one of the group of shoppers;

tracking based on the personal identifier for each one of the group of shoppers, with the video surveillance of the store, movement of each one of the group of shoppers throughout the store and items in the store selected for purchase by any of the shoppers in the group of shoppers; and

charging, through a frictionless check-out of the store, the retrieved credit card number for purchase of the selected items.