Deep learning-based store realograms

• US 10,474,991 B2
• Filed: 01/24/2019
• Issued: 11/12/2019
• Est. Priority Date: 08/07/2017
• Status: Active Grant

• Current Assignee: Standard Cognition Corp.
• Original Assignee: Standard Cognition Corp.
• Inventors: Fisher, Jordan E., Fischetti, Daniel L., Suswal, Michael S., Locascio, Nicholas J.
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US16/256,355
• Publication Number:
  US 20190156273A1
• Time in Patent Office: 292 Days
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

Systems and techniques are provided for tracking inventory items in an area of real space. A plurality of cameras, or other sensors, produce respective sequences of images in corresponding fields of view in the real space. The field of view of each camera overlaps with the field of view of at least one other camera. The system is coupled to the plurality of cameras and uses the sequences of images produced by at least two cameras in the plurality of cameras to identify inventory events. The inventory event includes an item identifier, a location and a timestamp. A plurality of cells having coordinates in the area of real space are stored as a data set in the memory. The processing system calculates scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

Claims

1. A system for tracking inventory items in an area of real space, comprising:

a plurality of sensors, sensors in the plurality of sensors producing respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors; and

a processing system coupled to the plurality of sensors, the processing system including logic that uses the sequences of images produced by at least two sensors in the plurality of sensors to identify inventory events, and in response to the inventory events, to track locations of inventory items in the area of real space, logic to match locations of inventory items with coordinates of cells in a plurality of cells having coordinates in the area of real space, and to maintain a data representing inventory items matched with cells in the plurality of cells, and logic that calculates scores at a scoring time for inventory items having locations matching particular cells, the scores based on counts of inventory events.

2. The system of claim 1, wherein the inventory events include an item identifier, a put or take indicator, locations represented by positions in three dimensions of the area of real space, and a timestamp.

3. The system of claim 1, wherein the area of real space includes a plurality of inventory locations, and including logic to match inventory locations with the coordinates of cells in the plurality of cells.

4. The system of claim 1, the logic that calculates scores for cells uses sums of puts and takes of inventory items weighted by a separation between timestamps of the puts and takes and the scoring time.

5. The system of claim 1, the processing system including logic to render a display image representing cells in the plurality of cells and the scores for the cells.

6. The system of claim 1, the processing system including logic to select a set of inventory items for each cell based on the scores.

7. The system of claim 1, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, and including

a planogram in memory specifying a distribution of inventory items in inventory locations in the area of real space, and

the logic to maintain the data representing inventory items matched with cells in the plurality of cells includes, logic to determine misplaced items by comparing the data representing inventory matched with cells to the distribution of inventory items in the inventory locations specified in the planogram.

8. The system of claim 1, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, and including

the logic to maintain the data representing inventory items matched with cells in the plurality of cells includes, logic that determines, for a particular inventory item matched with particular cell, whether a count of the particular inventory item in that cell is below a threshold for re-stocking the inventory item on an inventory location correlating with the particular cell.

9. A method for tracking inventory items in an area of real space, the method including:

using a plurality of sensors to produce respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors;

identifying inventory events using the sequences of images produced by at least two sensors in the plurality of sensors;

tracking locations of inventory items in the area of real space in response to the identification of inventory events,

matching locations of inventory items with coordinates of cells in a plurality of cells having coordinates in the area of real space, and maintaining a data representing inventory items matched with cells in the plurality of cells; and

calculating scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

10. The method of claim 9, wherein the inventory events include an item identifier, a put or take indicator, locations represented by positions along three axes of the area of real space, and a timestamp.

11. The method of claim 9, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations.

12. The method of claim 9, further including calculating scores for cells using sums of puts and takes of inventory items weighted by a separation between timestamps of the puts and takes and the scoring time, and storing the scores in a memory.

13. The method of claim 9, further including rendering a display image representing cells in the plurality of cells and the scores for the cells.

14. The method of claim 9, further including selecting a set of inventory items for each cell based on the scores.

15. The method of claim 9, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, the method further including

specifying a distribution of inventory items in inventory locations in the area of real space as a planogram and storing the planogram in memory,

and

determining misplaced items by comparing the data representing inventory matched with cells to the distribution of inventory items in the inventory locations.

16. The method of claim 9, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, the method further including

determining for an inventory item matched with particular cell whether a count of the particular inventory item in that cell is below a threshold for re-stocking the inventory item on an inventory location correlating with the particular cell.

17. A non-transitory computer readable storage medium impressed with computer program instructions to track inventory items in an area of real space, the instructions, when executed on a processor, implement a method comprising:

using a plurality of sensors to produce respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors;

identifying inventory events using the sequences of images produced by at least two sensors in the plurality of sensors;

tracking locations of inventory items in the area of real space in response to the identification of inventory events;

matching locations of inventory items with coordinates of cells in a plurality of cells having coordinates in the area of real space, and maintaining a data representing inventory items matched with cells in the plurality of cells; and

calculating scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

18. The non-transitory computer readable storage medium of claim 17, wherein the inventory events include an item identifier, a put or take indicator, locations represented by positions along three axes of the area of real space, and a timestamp.

19. The non-transitory computer readable storage medium of claim 17, wherein the area of real space includes a plurality of inventory locations, and matching the coordinates of cells in the plurality of cells with inventory locations in the plurality of inventory locations.

20. The non-transitory computer readable storage medium of claim 17, implementing the method further comprising, calculating scores for cells using sums of puts and takes of inventory items weighted by a separation between timestamps of the puts and takes and the scoring time, and storing the scores in a memory.

21. The non-transitory computer readable storage medium of claim 17, implementing the method further comprising, including rendering a display image representing cells in the plurality of cells and the scores for the cells.

22. The non-transitory computer readable storage medium of claim 17, implementing the method further comprising, selecting a set of inventory items for each cell based on the scores.

23. The non-transitory computer readable storage medium of claim 17, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, implementing the method further comprising:

specifying a distribution of inventory items in inventory locations in the area of real space as a planogram and storing the planogram in memory,

and

determining misplaced items by comparing the data representing inventory matched with cells to the distribution of inventory items in the inventory locations.

24. The non-transitory computer readable storage medium of claim 17, wherein the area of real space includes a plurality of inventory locations, and the coordinates of cells in the plurality of cells correlate with inventory locations in the plurality of inventory locations, implementing the method further comprising:

determining for an inventory item matched with particular cell whether a count of the particular inventory item in that cell is below a threshold for re-stocking the inventory item on an inventory location correlating with the particular cell.