Agent identification and disambiguation

• US 10,528,638 B1
• Filed: 07/15/2019
• Issued: 01/07/2020
• Est. Priority Date: 08/13/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kumar, Dilip, Chinoy, Ammar, Raghavan, Sudarshan Narasimha, Vasudhevan, Shrivatsan, Kim, Jin Dong
• Primary Examiner(s): Cumbess, Yolanda R
• Application Number:
  US16/512,125
• Time in Patent Office: 176 Days
• Field of Search: None
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

This disclosure describes a system for disambiguating between multiple potential users that may have performed an item action (e.g., item removal or item placement) at an inventory location. For example, if there are three picking agents (users) standing near an inventory location and one of the agents removes an item (item action) from the inventory location, the example systems and processes described herein may utilize various inputs to disambiguate between the users and determine which of the potential users performed the item action.

Claims

1. A computing system, comprising:

one or more processors; and

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least;

receive, as a first user enters a materials handling facility and from a portable device of the first user, a unique identifier corresponding to the first user;

monitor a position of the first user as the first user progresses through the materials handling facility;

detect, while the first user is located in the materials handling facility, an item action, wherein the item action occurs at a first time and includes at least one of a removal of an item from an inventory location within the materials handling facility or a placement of the item at the inventory location;

determine the item involved in the item action;

determine, based at least in part on the monitored position of the first user, that the first user is adjacent inventory location at the first time;

determine a second user adjacent the inventory location at the first time;

determine a first probability that the first user performed the item action;

determine a second probability that the second user performed the item action; and

identify, based at least in part on the first probability being higher than the second probability, that the first user performed the item action.

2. The computing system of claim 1, wherein the program instructions that cause the one or more processors to monitor a position of the first user further include instructions that when executed by the one or more processors further cause the one or more processors to at least:

process images obtained from one or more imaging devices positioned within the materials handling facility to determine identifying information corresponding to the first user.

3. The computing system of claim 2, wherein the inventory location includes an identifier array configured to detect the unique identifier when the first user is adjacent the inventory location.

4. The computing system of claim 3, wherein the identifier array includes a plurality of antennas positioned in the inventory location to detect identifiers.

5. The computing system of claim 1, wherein the program instructions that cause the one or more processors to monitor a position of the first user further include instructions that when executed by the one or more processors further cause the one or more processors to at least:

process at least a first image obtained from one or more imaging devices within the materials handling facility to determine a visual identifier corresponding to the first user; and

process a plurality of images received from one or more imaging devices within the materials handling facility to determine the visual identifier present in the plurality of images; and

update the position of the first user based at least in part on the processing of the plurality of images to determine the visual identifier.

6. The computing system of claim 5, wherein:

the visual identifier is a temporary identifier that is associated with the first user while the first user is located in the materials handling facility.

7. A system for identifying a user, comprising:

a plurality of antennas positioned within a materials handling facility and configured to receive identifiers;

a plurality of cameras positioned within the materials handling facility and configured to generate images of users as users move within the materials handling facility; and

a computing system communicatively coupled to the plurality of antennas and the plurality of cameras and configured to at least;

receive, from at least one antenna of the plurality of antennas, and as a user enters the materials handling facility, a unique identifier corresponding to the user;

receive, from at least one of the plurality of cameras and while the user is located within the materials handling facility, images of the user;

process the images to monitor a position of the user while the user is located within the materials handling facility;

detect, while the user is located in the materials handling facility and at a first time, an item action corresponding to an item at an inventory location within the materials handling facility;

determine, based at least in part on the position of the user, that the user is adjacent the inventory location at the first time;

determine, based at least in part on the position of the user at the first time, that the user performed the item action; and

in response to a determination that the user performed the item action, associate the item action with the user.

8. The system of claim 7, further comprising an identifier reader communicatively coupled to at least some of the plurality of antennas and the computing system and configured to associate a received unique identifier with an antenna of the plurality of antennas that received the unique identifier.

9. The system of claim 7, wherein the item action is at least one of removing the item from the inventory location or placing the item at the inventory location.

10. The system of claim 7, wherein a time is associated with the received unique identifier identifying when the unique identifier was received by an antenna of the plurality of antennas.

11. The system of claim 7, wherein the computing system is further configured to at least:

determine a second user adjacent the inventory location at the first time;

determine a first probability that the user performed the item action;

determine a second probability that the second user performed the item action;

determine that the first probability is higher than the second probability; and

determine, based at least in part on a determination that the first probability is higher than the second probability, that the user performed the item action.

12. The system of claim 11, wherein the computing system configured to determine the first probability is further configured to at least:

determine the first probability based on one or more of;

a past behavior of the user,

items with which the user has previously performed item actions,

items included in an item list associated with the user,

items included in a tote associated with the user, or

a processing of an image of the user performing the item action.

13. The system of claim 12, wherein the processing of the image includes processing the image to determine at least one of:

a hand used to perform the item action,

a posture of the user,

a size of the user,

a shape of the user,

a movement of the user,

a gate of the user,

an orientation of the user relative to the item,

a skeletal structure of the user, or

a characteristic of the user.

14. The system of claim 7, wherein:

detection of the item action includes detecting at least one of a change in a weight at the inventory location, a presence of an item identifier corresponding to the item at the inventory location, or an absence of the item identifier corresponding to the item at the inventory location; and

the computing system is further configured to at least;

determine an identity of the item based at least in part on one or more of an item type associated with the inventory location, a change in a weight at the inventory location, a presence of an item identifier corresponding to the item at the inventory location, or an absence of the item identifier corresponding to the item at the inventory location.

15. The system of claim 7, wherein:

the unique identifier is received from a portable device in possession of the user; and

wherein the computing system is further configured to at least;

in response to a determination that the user performed the item action, send to the portable device and for presentation to the user at least one of information about the item action or information about the item.

16. The system of claim 7, wherein the computing system is further configured to at least:

process an image of the inventory location; and

compare the processed image with stored identifying information corresponding to the user to determine that the user is positioned adjacent the inventory location.

17. A computer-implemented method, comprising:

receiving, as a user enters a materials handling facility, a unique identifier corresponding to the user;

processing, while the user is located in the materials handling facility, images generated by imaging elements within the materials handling facility to determine a position of the user within the materials handling facility;

detecting an item action occurring at an inventory location at a first time;

determining, based at least in part on a position of the user at the first time, that the user is adjacent the inventory location at the first time;

determining, based at least in part on the position of the user at the first time, that the user performed the item action; and

sending, for presentation to the user, information about an item involved in the item action.

18. The computer-implemented method of claim 17, further comprising:

determining a plurality of users adjacent the inventory location at the first time;

determining, for each of the plurality of users, a probability that the user performed the item action; and

determining that the user has a highest probability.

19. The computer-implemented method of claim 17, wherein determining that the user performed the item action is further based at least in part on one or more of a past purchase history of the user, items related to the item, a relationship between the item and additional items selected by the user while in the materials handling facility, a behavior presented by the user while performing the item action, items included in an item list associated with the user, an appearance of the user, a movement of the user, or an orientation of the user.

20. The computer-implemented method of claim 17, wherein:

the unique identifier is received from a portable device associated with the user; and

the information is sent for presentation by the portable device.