Wireless coded communication (WCC) devices for tracking retail interactions with goods and association to user accounts

• US 9,911,290 B1
• Filed: 12/12/2016
• Issued: 03/06/2018
• Est. Priority Date: 07/25/2015
• Status: Active Grant

• Current Assignee: Albert S. Penilla, Gary M. Zalewski
• Original Assignee: Albert S. Penilla, Gary M. Zalewski
• Inventors: Zalewski, Gary M., Penilla, Albert S.
• Primary Examiner(s): West, Lewis
• Application Number:
  US15/376,636
• Time in Patent Office: 449 Days
• CPC Class Codes: G06Q 20/12 specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/327 Short range or proximity payments by means of M-devices
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G07G 1/0072 with means for detecting the weight of the article of which the code is read, for the verification of the registration weighing combined with recording means, e.g. in weighing scale, G01G19/415

Abstract

Devices, systems, and method are provided for tracking items in a store for processing a cashier-less purchase transaction. In one example, a method includes receiving sensor data regarding items associated with shelves of a store. The method also includes identifying a user entering the store. The identifying uses at least a device having wireless communication. The device has an application that is executed in associated with a user account of the user. Also, tracking the user movements while in the store. The tracked movements include detecting proximity of the user to a shelf having an item. And, detecting interaction data for the item on the shelf of the store by the user. The interaction data is configured to identify a type of said item and enable add said item to an electronic shopping cart of the user having said user account for enabling processing of said cashier-less purchase transaction. The method includes receiving data indicative of said user leaving an area of the store while the item is in the electronic shopping cart. Leaving is indicative that the user intends to purchase the item. An exit of the area includes one or more sensors for confirming that the user has left the area. The method includes processing an electronic charge to a payment service associated with the user account of the user for the item based on said confirming.

Claims

1. A method for tracking items in a store for processing cashier-less purchase transactions, comprising:

receiving, by a server, sensor data regarding items associated with shelves of a store;

receiving, by the server, interaction data of an item of a shelf of the store by a user, the interaction data is configured to identify a type of said item and add it to an electronic shopping cart of the user having a user account for processing said cashier-less purchase transactions;

receiving, by the server, data indicative of said user leaving an area of the store that is indicative that the user has purchased the item; and

processing, by the server, an electronic charge to a payment service of the user for the item upon confirming that said user left the area of the store,

receiving, by the server, proximity data of the user and a second user, the user and the second user configured to be proximate to each other near the shelf, the proximity data of the user and the second user being usable to distinguish the user and the second user;

wherein cameras disposed in the store function to produce part of the sensor data, and the cameras producing image data that is trackable to identify limbs of each of the user and the second user, and said limbs trackable in motion and used to verify whether the user or the second user is interacting with the item on the self.

2. The method of claim 1, wherein a volume or area in which the item is located is tracked to obtain said interaction data, the interaction data is used to determine movement or removal of said item from the shelf and determine if a take event has occurred with respect to the item and the item is added to said electronic shopping cart.

3. The method of claim 1, wherein machine learning and sensor fusion is used in the tracking of users and items and to make determinations that are validated.

4. The method of claim 1, wherein a confidence level of identifying said movement is increased by using multiple sensors, each sensor is provided a weighting that gives a respective sensor or sensors more or less significance in said identifying of the movement.

5. The method of claim 2, further comprising,

receiving, by a server, data indicative of a user device being present in the store, the data from the user device used to identify the user account of the user for enabling use of the electronic shopping cart for buying one or more items in the store via one of said cashier-less purchase transactions.

6. The method of claim 5, further comprising,

receiving, by a server, tracking data regarding movement of the user in the store, the movement of the user is indicative of browsing of said items on one or more of said shelves;

wherein the tracking includes tracking a hand of the user, the hand of the user caused by the user to touch, lift, move, and/or remove said item from the shelf, the hand tracking is further usable as said interaction data with the item.

7. The method of claim 6, further comprising,

sending, by the server, display information for rendering on a user display screen or on a display screen that is proximate to the shelf having the item for which the user is interacting, the display information being customized for the user based on information obtained from a profile of the user associated with the user account.

8. The method of claim 6, further comprising,

sending, by the server, audio data for outputting from said user device, or a cart, or a basket, or from a region of the store, the audio data being customized for the user based on interactions of the user or from a profile of the user associated with the user account.

9. The method of claim 8, wherein the audio data is transmitted parametrically using a modulated ultrasonic carrier wave that demodulates into an audio frequency upon striking an object, and where the transmission is directionally steered toward a user, or another shopper or group of shoppers.

10. The method of claim 1, further comprising,

receiving, by the server, eye or head gaze information of the user, the eye or head gaze information being indicative of actions taken by the user with the item for which the user is interacting, the eye or head gaze information being collected to determine product information that is of interest to the user, the server is configured to collect the product information that is of interest to the user and using said information to provide recommendations to the user regarding other items that may be of interest and/or to provide discount or promotional information for the item or other items.

11. The method of claim 1, further comprising,

receiving, by the server, first data indicative that a take event occurred for the item and second data indicative that the item having a tag communicates with a user device of the user, the user device of the user verifying that the item was subjected to the take even and was not returned prior to the user leaving the area of the store before processing the electronic charge.

12. The method of claim 1, wherein the interaction data includes one or more features that are descriptive of detected interaction, the features being processed as inputs to a model that classifies the interaction to be one of a take event or a returning event of the item from the shelf or back to the shelf.

13. The method of claim 12, wherein classifying using the model assists in predicting an action to be taken by the user or other users, the model is one that refines its classification over time based on prior interactions and assists in determining whether to add the item or other items to the electronic shopping cart of the user or other users.

14. The method of claim 12, wherein the model continually learns from said interaction data of one or more connected stores.

15. The method of claim 1, wherein a group of shoppers participate in the taking and returning of items to a single shopping account and wherein the classification of take and return events and the association of such events to a single account is done using one or more sensors.

16. The method of claim 1, wherein the item or additional item is pre-fetched by the store in advance of said user arriving at the store.

17. The method of claim 1, further comprising,

determining if the user and the second user, being located in proximity to each other, are unrelated or are part of a group of shoppers sharing said electronic shopping cart, and assigning one or more items in said electronic shopping cart to either the user or the second user based on said sensor data.

18. The method of claim 1, further comprising,

providing automated location guidance to the user or the second user while in the store, the automated location guidance includes instructions on where to move in the store to,

place the user or the second user in proximity to said item or another item that is desired by the user or the second user;

or

place the user in proximity to the second user or the second user in proximity to the user;

or

locate the user in proximity to the second user or the second user in proximity to the user;

or

locate the gaze of the user towards a direction.

19. The method of claim 1, wherein an indicator is provided to the user when the item is added to the electronic shopping cart, wherein the indicator is an output perceivable to the user or the second user when output by an electronic device;

wherein adding the item to the electronic shopping chart is related to a take event.

20. The method of claim 1, further comprising,

determining if a price of the item when in the electronic shopping chart of the user has a first price and when in the electronic shopping cart of the second user has a second price, the first price being different than the second price, based on,

presence of another item in the respective electronic shopping charts of the user or the second user;

or

a status of an account profile of the user or the second user;

or

a membership of the user or the second user to a subscription, or status or membership in connection with the store or a brand of the item or another item;

a status of an incentive associated with the item and the user or the second user;

or

a status of a discount or earned credit by the user or the second user in relation to the item or another item.

21. The method of claim 14, wherein said continual learning uses inputs of features from interactions in the store and/or interactions occurring in other stores that are connected to a shopping service that enables said cashier-less purchase transactions.

22. The method of claim 1, wherein one of said cashier-less purchase transactions include, after identifying the type of said item and adding it to the electronic shopping cart, further comprising,

enabling exiting of said store without further interaction with a clerk or computer interface of said store;

or

enabling exit of said store after interaction with a clerk of said store;

or

enabling exit of said store after interaction with a computer interface or kiosk of said store;

or

enabling exit of said store after user input is received from a mobile device of said user;

or

a combination of two or more of said enabled exists of said store.

23. The method of claim 1, further comprising,

identifying an gaze direction of the user, the gaze direction is associated with the item or another item or group of items;

providing feedback data to a device of the user identifying product information regarding the item or another item or group of items, the product information being associated with a time window after determining the gaze direction of the user, the product information including,

information regarding nutrition or allergy information regarding the item or said another item or group of items;

or

information regarding nutrition or allergy information regarding the item or said another item or group of items based on a predefined health condition of the user or the second user;

or

information regarding a discount available for the item or said another item or group of items available during the time window.

24. The method of claim 1, further comprising,

providing feedback data to a device of the user or associated with the user based on default settings or customized settings in the user account, the feedback data identifying product information regarding the item or another item or group of items, the product information including,

information regarding nutrition or allergy information regarding the item or said another item or group of items;

or

information regarding nutrition or allergy information regarding the item or said another item or group of items based on a predefined health condition of the user or the second user in said user account or another user account;

or

information regarding a discount available for the item or said another item or group of items available during a time window.

25. The method of claim 1, further comprising,

detecting a gesture made by the user in connection with the interaction data for the user with respect to the item,

the gesture is used to confirm or correct classification of the item as being a take event for said adding to the electronic shopping cart, or

the gesture invokes feedback data about the item based on default or customized settings in the user account, or

the gesture is used to provide guidance;

or

the gesture is used as a request to compare product information to default or customized pre-set criteria in the user account, wherein the default or customized pre-set criteria includes one or more of,

whether the item adheres to a diet;

or

whether the item is kosher;

or

whether the item is gluten free;

or

whether the item is one that is above or below a threshold of a particular nutrient or fat;

or

whether the item contains or does not contain a certain ingredient;

or

whether the item relates to shelf life;

or

whether the item relates to an expiration date;

or

whether the item relates to product requirements for keeping after opening;

or

whether the item relates to refrigeration;

or

a combination of two or more thereof.