Verifying bin content in an automated materials handling facility

• US 9,120,621 B1
• Filed: 03/25/2014
• Issued: 09/01/2015
• Est. Priority Date: 03/25/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Graybill, Jules Cook, Curlander, James Christopher, Tappen, Marshall Friend, O'Brien, Barry James
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US14/225,326
• Time in Patent Office: 525 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

This disclosure describes a device and system for verifying the content of items in a bin of an inventory holder within a materials handling facility. In some implementations, a bin content verification apparatus may be positioned within the materials handling facility and configured to capture images of inventory holders that include bins as the inventory holders are moved past the apparatus by mobile drive units. The images may be processed to determine whether the content included in the bins has changed since the last time images of the bins were captured. A determination may also be made as to whether a change to the bin content was expected and, if so, if the determined change corresponds with the expected change.

Claims

1. A computer-implemented method for verifying content of a bin within a materials handling facility, comprising:

under control of one or more computing systems configured with executable instructions,

detecting an inventory holder within a defined distance of an image capture device, wherein;

the inventory holder is transported within the materials handling facility by a mobile drive unit; and

the inventory holder includes a bin;

causing an illumination element to illuminate the inventory holder;

causing the image capture device to capture an image of the inventory holder, wherein the image capture device is positioned so that the captured image will include a current bin image that includes a first representation of the bin and a first representation of content included in the bin;

determining a stored bin image, wherein the stored bin image includes a second representation of the bin and a second representation of content included in the bin;

processing the current bin image and the stored bin image to determine that a content change has occurred with respect to the content included in the bin;

determining an area of the current bin image corresponding to the determined content change;

determining a confidence score representative of a confidence that the determined content change corresponds to an expected content change; and

determining that the content included in the bin is accurate based at least in part on the confidence score exceeding a confidence threshold.

2. The computer-implemented method of claim 1, wherein the image capture device is positioned so that a field of view of the image capture device is directed in a downward angle into the bin.

3. The computer-implemented method of claim 1, wherein the illumination element and the image capture device are included in a bin content verification apparatus positioned adjacent to an operating station within the materials handling facility.

4. The computer-implemented method of claim 3, wherein the bin content verification apparatus further includes at least one of a computing component, a storage component, a distance determining element, or a communication component.

5. The computer-implemented method of claim 1, wherein detecting that the inventory holder is within the defined distance is determined at least in part by one or more of:

a ranging laser, an ultrasonic sensor, a pressure sensor, or visual identifier located on the inventory holder coming into focus within a field of view of the image capture device.

6. A system for verifying content included in a bin, the system comprising:

an apparatus, including;

a distance determining element configured to determine a distance between an inventory holder that includes the bin and the apparatus;

an illumination element oriented to illuminate the bin located in the inventory holder as the inventory holder reaches a first defined distance from the apparatus, as determined by the distance determining element;

an image capture device oriented to capture an image of a bin located in the inventory holder as the inventory holder reaches a second defined distance from the apparatus, as determined by the distance determining element;

a storage component in communication with the image capture device, wherein the storage component is configured to store the image when the image is captured; and

a computing system, including;

one or more processors; and

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least;

receive from the storage component the image of the bin;

correlate the image of the bin with a stored image of the bin;

determine, based at least in part on the image and the stored image, if at least one item of content included in the bin has changed;

determine if a change to the content included in the bin was expected; and

determine, based at least in part on the determination of whether at least one item of content included in the bin has changed and based at least in part on whether the change to the content included in the bin was expected, whether the content included in the bin is accurate.

7. The system as recited in claim 6, wherein the program instructions that when executed by the one or more processors further cause the one or more processors to determine a side of the inventory holder represented in the captured image.

8. The system as recited in claim 7, wherein the side of the inventory holder is determined based at least in part on a visual identifier included on the inventory holder.

9. The system as recited in claim 6, wherein the first defined distance and the second defined distance are a same distance.

10. The system as recited in claim 6, wherein:

the computing system is remote from the apparatus; and

the storage component is configured to be removed from the apparatus and connected to the computing system.

11. The system as recited in claim 6, wherein:

the computing system is remote from the apparatus; and

the apparatus includes a communication component configured to enable communication between the apparatus and the computing system; and

the computing system receives the image of the bin from the communication component.

12. The system as recited in claim 6, wherein:

the bin is determined based at least in part on a configuration of the inventory holder; and

the configuration of the inventory holder is determined based at least in part on an identification of a plurality of visual identifiers included in the image.

13. The system as recited in claim 12, wherein at least one of the plurality of visual identifiers is at least one of a barcode, a QR code, a bokode, a color, a shape, a character, an image, or a size.

14. A computer-implemented method for verifying content of a bin, comprising:

under control of one or more computing systems configured with executable instructions,

receiving a current image of at least a portion of an inventory holder, wherein the received image includes a representation of content included in a bin of the inventory holder;

processing the received image to determine a portion of the received image that includes the representation of content included in the bin;

correlating the portion of the received image with a stored image of the bin;

determining, based at least in part on the portion of the received image and the stored image, if at least one item of content included in the bin has changed;

determining if a change to the content included in the bin was expected; and

determining, based at least in part on the determination of whether at least one item of content included in the bin has changed and based at least in part on whether the change to the content included in the bin was expected, whether the content included in the bin is accurate.

15. The computer-implemented method of claim 14, further comprising:

determining that at least one item of content included in the bin has changed;

determining that a change to the content included in the bin was not expected; and

determining that the content included in the bin is not accurate.

16. The computer-implemented method of claim 14, wherein the inventory holder is configured to be moved by a mobile drive unit.

17. The computer-implemented method of claim 14, wherein the current bin image includes a representation of the bin while the bin is being transported by a mobile drive unit.

18. The computer-implemented method of claim 14, wherein processing the received image further includes:

determining a side of the inventory holder represented in the image; and

determining a bin configuration corresponding to the determined side of the inventory holder.

19. The computer-implemented method of claim 18, wherein at least one of determining a side of the inventory holder and determining a bin configuration is based at least in part on a visual identifier representative of the bin that is included in the image.

20. The computer-implemented method of claim 18, wherein determining a bin configuration corresponding with the determined side includes processing the image to determine at least one of a side of the inventory holder, a retention barrier of the inventory holder or a divider included in the inventory holder.