Inventory event detection using residual heat information

• US 9,767,432 B1
• Filed: 06/20/2016
• Issued: 09/19/2017
• Est. Priority Date: 06/20/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Brazeau, Jeremiah David, Mendola, Sergio
• Primary Examiner(s): Kim, Ahshik
• Application Number:
  US15/187,535
• Time in Patent Office: 456 Days
• Field of Search: 235385, 705 28, 62129
• CPC Class Codes: G06K 19/06028 using bar codes
• CPC Class Codes: G06K 7/10861 sensing of data fields affixed to objects or articles, e.g. coded labels postal sorting B07C3/14, conveying articles B65G47/48
• CPC Class Codes: G06K 7/10891 the scanner to be worn on a finger or on a wrist
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

An inventory management system is described. The inventory management system may be configured to use thermal image data to determine that an item was successfully placed in a storage structure. As part of this process, thermal images may be processed to identify instances of temperature differences that correspond to the item. These temperature differences may be evidence of a recent touch by an operator or may be generated by temperature changing devices during item placement.

Claims

1. A system, comprising:

an inventory holder comprising a plurality of faces, a first face of the plurality of faces comprising a plurality of compartments configured to receive inventory items;

a temperature changing device configured to at least heat the inventory items;

a thermal imaging device configured to capture thermal images of at least the first face of the plurality of faces when the first face is oriented towards the thermal imaging device; and

a computing device configured to;

instruct warming, via the temperature changing device, of an inventory item to be stowed in a compartment of the inventory holder;

instruct stowing of the inventory item in the compartment;

instruct, in response to an input signal, the thermal imaging device to capture one or more thermal images of the first face;

identify, based at least in part on the one or more thermal images, a temperature difference between the inventory item stowed in the compartment of the inventory holder and a portion of the inventory holder surrounding the inventory item; and

determine, based at least in part on the temperature difference, that the inventory item was successfully stowed in the compartment of the inventory holder.

2. The system of claim 1, wherein the input signal comprises at least one of object detection information indicating that an area opposite the first face is unobstructed, scan information indicating that an operator has scanned the inventory item as stowed, button information indicating that the operator has pushed a button indicating that the inventory item has been placed, position information indicating that no obstructions are detected by a position detection device, or timer information indicating that a predetermined time has passed since the operator has taken a certain action with respect to the inventory item.

3. The system of claim 1, further comprising an inventory container in which is held the inventory item prior to the inventory item being stowed in the compartment of the inventory holder, the temperature changing device configured to heat the inventory item while the inventory item is in the inventory container.

4. The system of claim 1, wherein the temperature changing device is a first temperature changing device, the system further comprising a second temperature changing device, wherein the computing device is configured to instruct cooling of the first face of the inventory holder prior to instructing stowing of the inventory item in the compartment.

5. The system of claim 1, wherein the computing device is further configured to receive position information indicating a position on a light plane that was broken as part of the inventory item being stowed in the compartment, and wherein determining that the inventory item was successfully stowed in the compartment of the inventory holder is based at least in part on the position information.

6. A computer-implemented method, comprising:

instructing, by a computing device, application of a change in temperature to at least one of an item to be placed in a storage structure or the storage structure;

instructing, by the computing device, placement of the item in the storage structure;

instructing capture, by a thermal imaging device, of one or more thermal images depicting the storage structure and the item after the item has been placed in the storage structure, the item having a first temperature that is different than a second temperature of the storage structure; and

determining, by the computing device, that the item was successfully placed in the storage structure based at least in part on the one or more thermal images.

7. The computer-implemented method of claim 6, wherein the storage structure comprises at least one of (i) an inventory holder that comprises a plurality of compartments, the item being placed in a first compartment of the plurality of compartments, or (ii) an inventory exchange wall comprising a plurality of openings extending through the inventory exchange wall, and the one or more thermal images depict at least a first side of the inventory exchange wall.

8. The computer-implemented method of claim 6, wherein instructing application of a change in temperature comprises an operator contacting the item to cause the first temperature to be different from the second temperature.

9. The computer-implemented method of claim 8, wherein the operator contacts the item using a wearable device that causes at least a portion of the change in temperature.

10. The computer-implemented method of claim 6, wherein instructing application of a change in temperature comprises at least one of:

application of heat to the item by a first temperature changing device;

or

removal of heat from the storage structure by a second temperature changing device.

11. The computer-implemented method of claim 10, wherein at least one of the first temperature changing device or the second temperature changing device is included in a robotic arm that is instructed to place the item in the storage structure.

12. The computer-implemented method of claim 6, wherein determining that the item was successfully placed in the storage structure comprises:

identifying a residual heat region present in the one or more thermal images; and

determining, based at least in part on inventory information, that the residual heat region corresponds to the item, the inventory information generated at least in part in response to an action by an operator that placed the item.

13. The computer-implemented method of claim 12, wherein the residual heat region comprises a temperature difference between the item and a portion of the storage structure surrounding the item.

14. The computer-implemented method of claim 6, further comprising:

receiving position information indicating a position on a light plane that was broken as part of the item being placed in the storage structure; and

determining, based at least in part on the position information, a placed position of the item with respect to the storage structure; and

storing, in a database, information about the placed position.

15. The computer-implemented method of claim 6, wherein the one or more thermal images comprise a residual heat region corresponding to the item, the method further comprising:

accessing one or more other thermal images depicting the storage structure, the one or more other thermal images;

being captured by the thermal imaging device after one or more other items have been placed in the storage structure; and

comprising one or more other residual heat regions corresponding to the one or more other items; and

generating, based at least in part on the residual heat region and the one or more other residual heat regions, placement information that at least indicates an order that the item and the one or more other items were placed.

16. A computer-implemented method, comprising:

accessing one or more thermal images depicting a face of an inventory holder, the inventory holder comprising a plurality of compartments, the one or more thermal images captured by a thermal imaging device;

processing the one or more thermal images to identify a residual heat region present in the one or more thermal images, the residual heat region caused by application of a change in temperature during placement of an inventory item in the inventory holder;

determining a first position with respect to the plurality of compartments that corresponds to the residual heat region;

comparing the residual heat region and inventory information to confirm that the inventory item is located at the first position; and

storing, in a database, information about the inventory item being located at the first position.

17. The computer-implemented method of claim 16, further comprising accessing the inventory information indicating that the inventory item has been placed in the inventory holder, the inventory information being generated at least in part in response to an action by an operator that placed the inventory item.

18. The computer-implemented method of claim 16, further comprising generating a reconciliation task when it cannot be confirmed that the inventory item is located at the first position.

19. The computer-implemented method of claim 16, wherein the residual heat region comprises a temperature difference between the inventory item and a portion of the inventory holder surrounding the inventory item.

20. The computer-implemented method of claim 19, wherein an operator uses a wearable device to place the inventory item at the first position, the wearable device causing at least a portion of the temperature difference.