Methods for tracking shopping activity in a retail store having cashierless checkout

• US 10,681,519 B1
• Filed: 01/09/2019
• Issued: 06/09/2020
• Est. Priority Date: 07/25/2015
• Status: Active Grant

• Current Assignee: Albert S. Penilla, Gary M. Zalewski
• Original Assignee: Albert S. Penilla, Gary M. Zalewski
• Inventors: Zalewski, Gary M., Penilla, Albert S.
• Primary Examiner(s): Le, Thanh C
• Application Number:
  US16/244,081
• Time in Patent Office: 517 Days
• CPC Class Codes: G06Q 30/0635 Processing of requisition or of purchase orders
• CPC Class Codes: H02N 11/002 Generators
• CPC Class Codes: H04B 1/1607 Supply circuits converters H02M; filters therefor H02M1/14; voltage stabilisers G05F1/46
• CPC Class Codes: H04B 1/3833 Hand-held transceivers
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04W 4/50 Service provisioning or reconfiguring
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 76/10 Connection setup
• CPC Class Codes: H04W 76/14 Direct-mode setup
• CPC Class Codes: H04W 84/12 WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/02 Terminal devices
• CPC Class Codes: H04W 88/06 adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: Y02D 30/70 in wireless communication networks
• CPC Class Codes: Y04S 40/18 Network protocols supporting networked applications, e.g. including control of end-device applications over a network

Abstract

Methods and systems are provided. One method includes associating a plurality of sensor devices to a shelf having plurality of physical items. Each of the physical items is capable of being associated with one or more state data in response to user interaction. The method includes detecting user interaction by a user with one of the physical items. The user interactions cause a sensor device associated with the physical item to transmit a message over a network to an end node. The end node is configured to receive the message and process an action for the message. At least one physical item is offered for sale in a retail store and where the user interaction is associated to a user account. An indication of a status of the physical item as to whether the user has touched the physical item, lifted the physical item from the shelf, moved the physical item, or released the physical item in the retail store is associated to a log or status for the user account of the user.

Claims

1. A method, comprising:

associating a plurality of sensor devices to a shelf having plurality of physical items, each of the physical items capable of being associated with one or more state data in response to user interaction; and

detecting user interaction by a user with one of the physical items, the user interactions causes a sensor device associated with the physical item to transmit a message over a network to an end node;

wherein the end node is configured to receive the message and process an action for the message;

wherein at least one physical item is offered for sale in a retail store and where the user interaction is associated to a user account and an indication of a status of the physical item as to whether the user has touched the physical item, lifted the physical item from the shelf, moved the physical item, or released the physical item in the retail store.

2. The method of claim 1, wherein at least one sensor is a camera.

3. The method of claim 2, wherein the end node provides data for performing a cashier-less checkout using status of one or more physical items along with account information associated with the user.

4. The method of claim 2, wherein the method is executed by one or more computers associated with the physical items that enables the user to pick up one or more of said physical items offered for sale located in the retail store and leave the retail store without stopping at a human assisted checkout register.

5. The method of claim 4, further comprising,

detecting a physical shopping cart passing through an area of the retail store, and the physical shopping cart refers to the physical items picked up by the user or a physical shopping cart holding those physical items picked up by the user.

6. The method of claim 5, further comprising,

detecting an identity of the user before exiting the retail store,

wherein the area in the retail store in which the shopping cart passes is associated with completion of shopping or to an exit.

7. The method of claim 6, wherein detecting the identity of the user is performed at least in part by image processing, or by receiving identifying information from a device of the user, or via a QR code displayed on a display, or via biometric processing, or via near field communication, or via MAC address, or via a certificate or via a receiving identifying information of the user.

8. The method of claim 2, wherein the user interaction is represented in image input, or voice input, or fingerprint input, or image scan input, or eye scan input, or gesture input, or motion input, or signature input, or password input, or button press input, or button dial input, or sliding input, or pressing and sliding input, or lifting input, or pulling input, or depressing input, or pumping action input, or multiple press input, or tap input, or rub input, or gesturing and pressing input, or hand scan input, or area scan input, or thermal image capture input, or infrared image input, or combination of two or more thereof.

9. The method of claim 8, wherein machine learning is used to identify user interaction, and object detection is performed using an output of at least one camera.

10. The method of claim 9, wherein the sensor is configured with a processing entity to perform local image analysis, or perform local object detection or locally process a scene imaged by the sensor;

or

wherein data reflecting sensor output is received by a remote processing device linked through a wire-based or wireless network.

11. A method, comprising,

identifying presence of a mobile device associated with a user prior to exiting a store, the user having a user account that is accessible via the internet, the user account used for a cashier-less transaction to transact for shopping that is activated for the mobile device that is associated with the user and is identified as being present in the store;

detecting the exit of the user device from the store, and in response to detecting the exit of the user device from the store, initiating the cashier-less transaction for a good determined to be associated with an electronic shopping cart of the user;

wherein the good in the electronic shopping cart was detected to have been selected by interaction with the good by the user from a location of the store using one or more sensors located proximate to the location.

12. The method of claim 11, wherein detecting selection of the good is based on processing data collected from one or more sensors positioned in the store proximate to the good, and the good is configured to be added to the electronic shopping cart of the user account when the selection is confirmed based on said interaction with the good in one or more locations in the store, the interaction includes detected removal of the good from said one of said locations.

13. The method of claim 12, wherein said one or more sensors are cameras and the selection includes one of moving the good, picking the good, lifting the good or removing the good from the location, the location including a shelf in the store.

14. The method of claim 11, wherein said cashier-less transaction is used as an input to a metrics analysis process for determining sales performance of said good, the sales performance of said good is associated with the user account or user accounts of other cashier-less transactions.

15. The method of claim 11, wherein in addition to identifying presence of the mobile device, the method comprises,

identifying an identity of the user that is determined to be associated with the mobile device.

16. The method of claim 15, wherein identifying an identity of the user includes capturing one or more biometric inputs associated with the user, the biometric inputs include a camera image of the user, or a facial image of the user, a fingerprint of the user, a voice capture of the user, or gesture of the user, or an eye scan of the user, or a retina scan of the user, or a combination of two or more thereof.

17. The method of claim 11, further comprising,

identifying the user associated with the mobile device;

tracking movements of the user in the store; and

tracking movements of other users in the store to differentiate the user from said other users, such that said selection of the good is determined in part based on machine learning prediction.

18. The method of claim 11, further comprising,

identifying the user associated with the mobile device, the identifying is enabled by receiving inputs from the one or more sensors disposed in the store, the sensors including cameras for capturing images, or motion sensors, or biometric sensors, or combinations thereof.

19. The method of claim 11, further comprising,

processing a security operation to confirm the detected good selection matches a good leaving the store;

or

further comprising,

identifying the user associated with the mobile device;

tracking movements of the user in the store; and

tracking movements of other users in the store to differentiate the user from said other users, such that said selection of the good is determined to be by the user.

20. The method of claim 11, wherein said cashier-less transaction is performed automatically in response to detecting exit of the user from the store or is performed without a need of a store operated cashier.