Item management system using weight

• US 10,332,066 B1
• Filed: 03/30/2015
• Issued: 06/25/2019
• Est. Priority Date: 03/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Palaniappan, Ramanathan, Shiee, Navid, Ren, Xiaofeng, Goldstein, Michel Leonard, Kumar, Dilip, Gopal, Gopi Prashanth, Manyam, Ohil Krishnamurthy
• Primary Examiner(s): Franklin, Jamara A
• Application Number:
  US14/673,739
• Time in Patent Office: 1,548 Days
• Field of Search: 235383, 177 1, 177 2511- 2519, 705 28
• CPC Class Codes: G01G 19/42 for counting by weighing G01G19/387 takes precedence
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

An inventory location such as a shelf may be used to stow different types of items, with each type of item in a different partitioned area or section of the shelf. Weight data from weight sensors coupled to the shelf is used to determine a change in weight of the shelf and a change in the center-of-mass (“COM”) of the items on the shelf. Based on the weight data and item data indicative of what items are stowed in particular partitioned areas, activity such as a pick or place of an item and the partitioned area in which the activity occurred may be determined. Data from other sensors, such as a camera, may be used to confirm the occurrence of the activity, disambiguate the determination of the particular partitioned area, and so forth.

Claims

1. A system comprising:

a shelf to hold a type of item in one of a plurality of partitioned areas;

a plurality of weight sensors, each weight sensor coupled to the shelf at different points;

a camera with a field-of-view that includes at least a portion of the plurality of partitioned areas; and

a computing device comprising;

a memory, storing computer-executable instructions; and

a hardware processor configured to execute the computer-executable instructions to;

access first weight data acquired by the plurality of weight sensors at a first time;

access second weight data acquired by the plurality of weight sensors at a second time;

determine, using the first weight data, a first weight distribution of the shelf at the first time;

determine, using the second weight data, a second weight distribution of the shelf at the second time;

determine, using a difference between the first weight distribution and the second weight distribution, a location of weight change indicative of a location with respect to the shelf;

determine a partitioned area corresponding to the location of weight change;

determine measured weight change data indicative of a difference between a first total weight at the first time and a second total weight at the second time;

access item data indicative of weight of a single one of the type of item and placement of the type of item within the one of the plurality of partitioned areas;

determine a plurality of hypotheses with each hypothesis indicative of a predicted quantity of the type of item and one or more of;

a predicted weight change for the predicted quantity corresponding to the each hypothesis,

a predicted weight distribution based on the predicted weight change and the placement of the type of item within the one of the plurality of partitioned areas, or

a predicted change in weight distribution based on the predicted weight change and the placement of the type of item within a predicted partitioned area;

determine one or more selected hypotheses from the plurality of hypotheses for which;

the location of weight change is within the one of the plurality of partitioned areas indicated by the item data for the type of item, and

a difference between the measured weight change data and the predicted weight change is less than a threshold value;

access image data acquired by the camera;

determine, using the image data, activity data indicative of a hand at the one of the plurality of partitioned areas;

determine one of the one or more selected hypotheses for which the predicted partitioned area matches;

the one of the plurality of partitioned areas that the location of weight change is within, and

the one of the plurality of partitioned areas indicated in the activity data;

generate, based on the one of the one or more selected hypotheses, interaction data indicative of one or more of pick or place of a quantity of the type of item at the one of the plurality of partitioned areas; and

update, using the interaction data, a quantity stored in the item data.

2. The system of claim 1, further comprising computer-executable instructions to:

determine, using the image data, one or more of addition or removal of the type of item to the one of the plurality of partitioned areas; and

the determination of the one or more selected hypotheses is based on the one or more selected hypotheses having a predicted quantity consistent with the one or more of addition or removal of the type of item.

3. The system of claim 1, the determination of the first weight distribution of the shelf at the first time comprising computer-executable instructions to:

generate a first left side weight by summing first left weight data acquired at the first time from one or more of the plurality of weight sensors arranged proximate to a left side of the shelf;

generate a first right side weight by summing first right weight data acquired at the first time from one or more of the plurality of weight sensors arranged proximate to a right side of the shelf; and

the determination of the second weight distribution of the shelf at the second time comprising computer-executable instructions to;

generate a second left side weight by summing second left weight data acquired at the second time from the one or more of the plurality of weight sensors arranged proximate to the left side of the shelf; and

generate a second right side weight by summing second right weight data acquired at the second time from the one or more of the plurality of weight sensors arranged proximate to the right side of the shelf.

4. The system of claim 3, further comprising computer-executable instructions to determine the difference between the first weight distribution and the second weight distribution by:

subtracting the first left side weight obtained at the first time from the second left side weight obtained at the second time to generate a left side difference; and

subtracting the first right side weight obtained at the first time from the second right side weight obtained at the second time to generate a right side difference.

5. A system comprising:

an inventory location to hold one or more types of items;

a plurality of weight sensors coupled to the inventory location to generate weight data indicative of weight of a load, each of the weight sensors coupled to a different point of the inventory location;

a sensor; and

a computing device comprising;

a memory, storing computer-executable instructions; and

a hardware processor configured to execute the computer-executable instructions to;

determine a measured change in the weight of the load between a first time and a second time;

determine a measured change in weight distribution between the first time and the second time;

determine a plurality of hypotheses, each hypothesis indicative of different combinations of;

the one or more types of items, predicted quantity of each of the one or more types of items, and one or more of;

 

predicted change in weight, or  

predicted change in weight distribution;

access non-weight data acquired by the sensor;

determine activity data based on the non-weight data; and

determine one of the plurality of hypotheses based on the activity data and one or more of;

the measured change in the weight or the measured change in the weight distribution.

6. The system of claim 5, further comprising computer-executable instructions to:

access constraint data indicative of one or more of a minimum quantity of the one or more types of items or a maximum quantity of the one or more types of items; and

after the determination of the plurality of hypotheses, disregard from the determination of the one of the plurality of hypotheses those of the plurality of hypotheses having predicted quantities that are one or more less than the minimum quantity or more than the maximum quantity.

7. The system of claim 5, further comprising computer-executable instructions to:

disregard from the determination of the one of the plurality of hypotheses those of the plurality of hypotheses for which activity data differs from a predication in the hypotheses by at least a threshold value.

8. The system of claim 5, further comprising computer-executable instructions to:

update, using the predicted quantity indicated in the one of the plurality of hypotheses, data indicative of a quantity of the one or more types of items stowed at the inventory location.

9. The system of claim 5, wherein the non-weight data comprises images;

and the computer-executable instructions to determine the activity data further comprise one or more computer-executable instructions to;

compare a plurality of the images to determine a change between the plurality of the images, or

determine a contour of an object in at least a portion of the plurality of the images using an artificial neural network.

10. The system of claim 5, the sensor further comprising one or more of:

an optical sensor array comprising a plurality of optical sensors in a two-dimensional arrangement to detect a shadow cast by an object and generate the non-weight data comprising image data indicative of a shape of the shadow;

a camera with a field-of-view that includes at least a portion of the inventory location to generate the non-weight data comprising image data of the portion of the inventory location;

an instrumented auto-facing unit to generate the non-weight data indicative of a change in quantity held by the instrumented auto-facing unit;

or

a light curtain to generate touchpoint data and generate the non-weight data comprising a position of a touch relative to the inventory location.

11. The system of claim 5, wherein the activity data comprises information indicative of a count of items picked from or stowed to the inventory location;

further comprising computer-executable instructions to;

access the non-weight data comprising an image; and

processing the image using an artificial neural network to determine the count of items picked from or stowed to the inventory location.

12. The system of claim 5, the computer-executable instructions to determine the one of the plurality of hypotheses further comprising computer-executable instructions to:

determine, from the plurality of hypotheses, the one of the plurality of hypotheses having a least deviation from one or more of;

the measured change in the weight and the predicted change in the weight, or the measured change in the weight distribution and the predicted change in weight distribution.

13. The system of claim 5, the computer-executable instructions to determine the activity data further comprising computer-executable instructions to:

determine the measured change in the weight exceeds a threshold value; and

initiate the determination of the activity data in response to the determination that the measured change in the weight exceeds the threshold value.

14. The system of claim 5, further comprising computer-executable instructions to:

initiate the determination of the measured change in the weight and the determination of the measured change in the weight distribution responsive to a change in the activity data.

15. A computer implemented method executing on at least one processor, the computer implemented method comprising:

accessing non-weight data acquired by one or more non-weight sensors;

determining one or more measured weight characteristics of a load measured by a plurality of weight sensors;

accessing one or more hypotheses, each hypothesis including;

a predicted quantity of one or more items,

a predicted location within the load of each of the one or more items, and

a predicted weight characteristic based on the predicted quantity and the predicted location for the one or more items;

determining one of the one or more hypotheses based on the one or more measured weight characteristics and the non-weight data; and

generating interaction data based on the one of the one or more hypotheses.

16. The computer implemented method of claim 15, wherein the one or more measured weight characteristics comprise one or more of:

a change in measured weight between a first time and a second time, or

a change in measured weight distribution between the first time and the second time; and

wherein the predicted weight characteristic corresponding to the each hypothesis comprises one or more of;

a change in predicted weight based on predicted one or more of place or pick of predicted quantities of the one or more items, or

a change in predicted weight distribution based on the predicted one or more of place or pick of the predicted quantities of the one or more items from the predicted locations.

17. The computer implemented method of claim 15, wherein the non-weight data comprises one or more images;

and further comprising;

acquiring the one or more images using one or more of;

a camera, a three-dimensional sensor, or an optical sensor array.

18. The computer implemented method of claim 15, further comprising:

generating the non-weight data indicative of one or more of;

a location of an item on a platform supporting the load,

pick of the item from the platform, or

place of the item to the platform.

19. The computer implemented method of claim 15, further comprising:

accessing data indicative of;

physical dimensions of a platform supporting the load,

weight of an individual one of the one or more items associated with the load, and

locations for stowage of the one or more items on the platform;

designating a quantity of particular items;

calculating, for the quantity of particular items, a predicted weight distribution;

calculating, for the quantity of particular items, a predicted change in weight; and

storing, as the one or more hypotheses, the quantity of particular items as the predicted quantity, the predicted weight distribution, and the predicted change in weight.

20. The computer implemented method of claim 15, the determining the one or more measured weight characteristics of the load comprising:

generating a first weight by summing weight data acquired from one or more of the plurality of weight sensors arranged proximate to a first side of a platform supporting the load; and

generating a second weight by summing weight data acquired from one or more of the plurality of weight sensors arranged proximate to a second side of the platform supporting the load.

21. A computer implemented method comprising:

accessing non-weight data;

determining one or more weight characteristics of a load measured by a plurality of weight sensors;

accessing one or more hypotheses, each hypothesis including;

a predicted quantity of one or more items,

a predicted location within the load of each of the one or more items, and

a predicted weight characteristic; and

determining one of the one or more hypotheses based on the one or more weight characteristics and the non-weight data.

22. A computer implemented method executing on at least one processor, the computer implemented method comprising:

accessing non-weight data acquired by one or more non-weight sensors;

determining weight data acquired by one or more weight sensors;

accessing one or more hypotheses, each hypothesis including;

a predicted quantity of one or more types of items, and

predicted weight data based on the predicted quantity of the one or more types of items;

determining one of the one or more of the hypotheses based on the non-weight data and the weight data; and

generating interaction data based on the one of the one or more of the hypotheses.