Item management system using multiple scales

• US 10,262,293 B1
• Filed: 06/23/2015
• Issued: 04/16/2019
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Prater, Alexander Clark, Avramov, Ivan, Medrano, Ismael, Shapiro, Benjamin Lloyd, Shiee, Navid, Skaloud, Brett Allen
• Primary Examiner(s): Goyea, Olusegun
• Application Number:
  US14/747,553
• Time in Patent Office: 1,393 Days
• Field of Search: 705 1464, 705 23, 705323, 235375, 23546208, 702150, 384150, 382100
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

An inventory location such as a shelf may be used to stow different types of items. The shelf may be equipped with a plurality of lanes arranged parallel to one another. Items may rest upon two or more of the lanes. Each lane includes a weight sensor that provides weight data about a load on the lane. Based on the weight data and item data indicative of individual weights of those items stowed on the shelf, interaction data indicative of an activity such as a pick or place of an item may be determined. The interaction data may specify what item was picked or placed, quantity of that item that was picked or placed, and so forth. Data from other sensors, such as a camera, may be used to determine the interaction data.

Claims

1. A system comprising:

a first lane and a second lane that are adjacent to one another, each lane comprising;

a support structure to hold one or more items; and

at least one weight sensor to generate weight sensor data of a load on the support structure, wherein the load includes weight of the one or more items and the support structure;

an imaging sensor; and

a computing device comprising;

a memory, storing computer-executable instructions; and

a hardware processor to execute the computer-executable instructions to;

determine, using the imaging sensor, that a user is proximate to one or more of the first lane or the second lane;

determine, using the weight sensor data, a measured lane weight change for each of the first lane and the second lane;

determine, based on the measured weight change for the each of the first lane and the second lane, that the first lane and the second lane are part of a cluster;

determine, using the weight sensor data, a measured total weight change of the cluster;

determine, for each type of item stowed at the cluster;

an individual weight for the each type of item; and

a shelf location of the each type of item, indicative of placement of the each type of item with respect to one or more of the first lane or the second lane;

determine a plurality of hypotheses, each hypothesis indicative of different combinations of;

a predicted quantity change for one or more types of items stowed at the cluster,

a predicted lane weight change for the each of the first lane and the second lane, and

a predicted total weight change across the first lane and the second lane;

determine, for the each hypothesis in the plurality of hypotheses and for each of the lanes in the cluster, a difference between the predicted lane weight change and the measured lane weight change;

determine a score for the each hypothesis in the plurality of hypotheses based on a sum of the differences associated with that hypothesis;

determine, based on the score for the each hypothesis in the plurality of hypotheses, a change in quantity of the one or more items;

using the change in quantity of the one or more items, assess charges to an account associated with the user using a sale price of the one or more items; and

transmit billing information to the account associated with the user.

2. The system of claim 1, further comprising computer-executable instructions to:

determine the cluster comprising adjacent lanes, wherein each of the adjacent lanes has weight characteristic data indicative of a corresponding measured lane weight change greater than a first threshold during a specified time period.

3. The system of claim 1, further comprising computer-executable instructions to:

determine, for the each hypothesis in the plurality of hypotheses, a difference between the predicted total weight change and the measured total weight change; and

disregard those hypotheses having differences that exceed a threshold.

4. The system of claim 1, further comprising computer-executable instructions to:

limit the plurality of hypotheses to those hypotheses having the predicted quantity of changes that are less than or equal to a maximum storage capacity of items for the cluster; and

determine one or more of the predicted lane weight change or the predicted total weight change using one or more of;

the predicted quantity change,

the individual weight for the each type of item located at the cluster, or

the shelf location of the each type of item.

5. A system comprising:

a shelf comprising a plurality of lanes that are adjacent to one another, each lane comprising;

at least one weight sensor to generate weight sensor data of a load on the each lane; and

a support structure to hold an item;

an imaging sensor to generate image data; and

a computing device comprising;

a memory, storing computer-executable instructions; and

a hardware processor to execute the computer-executable instructions to;

determine, using the imaging data, that a user is proximate to the shelf;

determine weight characteristic data for one or more of the plurality of lanes based on the weight sensor data;

determine a cluster comprising adjacent lanes;

determine each of the adjacent lanes in the cluster has a measured lane weight change greater than a first threshold;

determine a measured total cluster weight change;

determine item data for one or more types of items associated with the shelf, wherein the one or more types of items are supported by the adjacent lanes in the cluster;

determine one or more hypotheses for the one or more types of items associated with the shelf, wherein each of the one or more hypotheses is indicative of one or more of;

a particular combination of the one or more types of items at the cluster,

a particular combination of a predicted quantity of the each of the one or more types of items at the cluster,

predicted lane weight changes for the each of the adjacent lanes in the cluster, and

a predicted total cluster weight that differs from the measured total cluster weight change by less than a second threshold;

determine a solution using the one or more hypotheses;

determine interaction data indicative of a change in quantity of the one or more types of items on the shelf based on the solution; and

assess charges to an account associated with the user based on the interaction data.

6. The system of claim 5, the shelf further comprising a shelf support, wherein the support structure is supported at least in part by the at least one weight sensor.

7. The system of claim 5, the at least one weight sensor comprising one or more of an electronic load cell supporting the support structure, an optical load cell supporting the support structure, an acoustic load cell supporting the support structure, or a force sensitive resistor atop the support structure.

8. The system of claim 5, the computer-executable instructions to determine the item data comprising computer-executable instructions to:

determine a shelf location identifier; and

retrieve from a data store the item data associated with the shelf location identifier.

9. The system of claim 5, further comprising computer-executable instructions to:

sum the measured lane weight changes for the each of the lanes in the cluster to determine the measured total cluster weight change.

10. The system of claim 9, further comprising computer-executable instructions to:

determine, for the each of the one or more hypotheses, a difference between the predicted lane weight change and the measured lane weight change for the each of the adjacent lanes in the cluster; and

determine, for the each hypothesis of the one or more hypotheses, a score based on a sum of the differences associated with that hypothesis; and

wherein the solution is based at least in part on the score.

11. The system of claim 10, further comprising computer-executable instructions to:

determine the score for each of a plurality of the one or more hypotheses is above a third threshold;

for the each of the plurality of the one or more hypotheses with the score above the third threshold, determine a first edge lane located at a first end of the cluster and a second edge lane located at a second end of the cluster;

determine a first neighboring lane adjacent to the first edge lane that is outside of the cluster;

determine a second neighboring lane adjacent to the second edge lane that is outside of the cluster; and

change the score for the each of the plurality of the one or more hypotheses satisfying one or more of the following conditions;

a measured lane weight of the first neighboring lane exceeds a fourth threshold;

a measured lane weight of the second neighboring lane exceeds the fourth threshold;

the predicted lane change weight for the first edge lane is greater than a fifth threshold and the measured lane weight of the first edge lane is below a sixth threshold;

or

the predicted lane change weight for the second edge lane is greater than the fifth threshold and the measured lane weight of the second edge lane is below the sixth threshold.

12. The system of claim 9, further comprising computer-executable instructions to:

for the each hypothesis of the one or more hypotheses, determine a first set of one or more lanes for which a predicted lane weight exceeds a third threshold;

determine a first sum of the measured lane weight changes for the one or more lanes in the first set;

determine a difference between the first sum and the predicted total cluster weight;

determine a second set of one or more lanes that are part of the cluster and not part of the first set;

determine a second sum of the measured lane weight changes for the one or more lanes in the second set;

determine a third sum of the difference and the second sum; and

determine a score for one or more of the one or more hypotheses based on the third sum.

13. The system of claim 9, further comprising computer-executable instructions to:

determine, for the each of the one or more hypotheses, a number of transactions, wherein a transaction comprises one or more of a pick to decrease a quantity of one of the one or more types of items supported by the shelf or a place to increase the quantity; and

determine a count of the number of transactions for the each of the one or more hypotheses; and

wherein the solution is based at least in part on the count.

14. The system of claim 5, further comprising computer-executable instructions to:

access the weight sensor data for a lane;

determine filtered weight sensor data from the weight sensor data using an exponential moving average function;

access a portion of the filtered weight sensor data having associated time values within a specified range of time values;

use a linear regression to fit a straight line to the portion of the filtered weight sensor data;

determine a slope value of the straight line;

determine one or more residuals, each residual comprising a difference between the filtered weight sensor data at a particular time and a value of the straight line at the particular time;

determine, based at least in part on one or more of the slope value of the straight line or the one or more residuals, a change in state of weight of the lane;

wherein the interaction data is determined responsive to the change in state; and

wherein the weight characteristic data is based on the filtered weight sensor data.

15. The system of claim 5, further comprising, on the support structure, one or more of a foam, a gel, or an elastomer.

16. A method comprising:

detecting that a user is proximate to a shelf, using an imaging sensor;

accessing weight sensor data indicative of weight as measured at each of a plurality of lanes that are adjacent to one another, wherein the plurality of lanes form the shelf that holds one or more items;

determining measured weight characteristic data using the weight sensor data;

determining lanes of the plurality of lanes included in a cluster based on the measured weight characteristic data;

determining each of the lanes in the cluster has a measured lane weight change greater than a first threshold;

determining a measured total cluster weight change;

accessing item data for the one or more items associated with the shelf;

accessing one or more hypotheses, wherein each of the one or more hypotheses is based at least in part on the item data and the each of the one or more hypotheses has a predicted total cluster weight that differs from the measured total cluster weight change by less than a second threshold;

determining a solution from the one or more hypotheses based on the measured weight characteristic data;

generating interaction data based on the solution; and

responsive to the generating the interaction data, generating billing information for an account associated with the user.

17. The method of claim 16, wherein the measured weight characteristic data is indicative of one or more of:

a measured lane weight of a particular lane;

the measured lane weight change of the particular lane from a first time to a second time; and

a weight distribution between two or more particular lanes.

18. The method of claim 16, wherein at least one of the one or more hypotheses includes:

a predicted quantity of the one or more items,

a predicted shelf location on the shelf of each of the one or more items, wherein the shelf location spans two or more lanes, and

a predicted weight characteristic based on the predicted quantity and the predicted shelf location for the one or more items.

19. The method of claim 16, further comprising:

summing the measured lane weight changes for the each of the lanes in the cluster to determine the measured total cluster weight change.

20. The method of claim 16, further comprising:

determining, for the each of the one or more hypotheses, a difference between a predicted lane weight change and the measured lane weight change for the each of the lanes in the cluster;

determining, for the each of the one or more hypotheses, a score based on a sum of the differences associated with that hypothesis; and

wherein the determining the solution is based at least in part on the score.

21. The method of claim 16, further comprising:

determining a subset of the one or more hypotheses, wherein each hypothesis in the subset has a score above a third threshold;

determining, for the each of the hypotheses in the subset, a first edge lane located at a first end of the cluster and a second edge lane located at a second end of the cluster;

determining a first neighboring lane adjacent to the first edge lane that is outside of the cluster;

determining a second neighboring lane adjacent to the second edge lane that is outside of the cluster; and

changing the score for the each of the one or more hypotheses satisfying one or more of the following conditions;

a measured lane weight of the first neighboring lane exceeds a fourth threshold;

a measured lane weight of the second neighboring lane exceeds the fourth threshold;

the predicted lane change weight for the first edge lane is greater than a fifth threshold and the measured lane weight of the first edge lane is below a sixth threshold;

or

the predicted lane change weight for the second edge lane is greater than the fifth threshold and the measured lane weight of the second edge lane is below the sixth threshold.

22. The method of claim 16, further comprising:

determining, for one or more of the one or more hypotheses;

a first set of the lanes in the cluster;

a first sum of the measured lane weight changes for the lanes in the first set;

the predicted total cluster weight;

a first difference between the first sum and the predicted total cluster weight;

determining a second set of the lanes that are part of the cluster and not part of the first set;

determining a second sum of the measured lane weight changes for the lanes in the second set;

determining a third sum of the first difference and the second sum;

determining a score for the one or more of the one or more hypotheses based on the third sum; and

wherein the determining the solution is based at least in part on the score.

23. The method of claim 16, further comprising:

determining a transaction count for one or more of the one or more hypotheses, wherein the transaction count is indicative of a number of transactions involving one or more of addition or removal of at least a portion of the one or more items on the shelf; and

wherein the determining the solution is based at least in part on the count.

24. A system comprising:

a shelf to hold one or more items, the shelf comprising a plurality of lanes that are adjacent to one another, each lane comprising;

at least one weight sensor to generate weight sensor data of a weight on the lane;

an imaging sensor to generate image data; and

a computing device comprising;

a memory, storing computer-executable instructions; and

a hardware processor to execute the computer-executable instruction to;

determine, using the image data, a user proximate to the shelf;

determine weight characteristic data for one or more of the plurality of lanes based on the weight sensor data;

determine one or more lanes are part of a cluster based on the weight characteristic data;

determine a measured lane weight change for each of the one or more lanes in the cluster;

determine a total cluster weight change;

determine one or more hypotheses associated with the one or more items on the shelf, wherein each of the one or more hypotheses comprises a combination of;

a predicted quantity change for the one or more items,

a predicted lane weight change for each of the one or more lanes in the cluster, and

a predicted total cluster weight change;

determine interaction data based on a solution selected from the one or more hypotheses; and

assess charges to an account associated with the user, based on the interaction data.

25. The system of claim 24, wherein the interaction data is indicative of a pick or a place of the one or more items on the shelf.

26. The system of claim 24, wherein the interaction data is indicative of a quantity of the one or more items removed from or added to the shelf.