Method for tracking stock level within a store

• US 10,607,182 B2
• Filed: 11/09/2016
• Issued: 03/31/2020
• Est. Priority Date: 11/09/2015
• Status: Active Grant

• Current Assignee: Simbe Robotics, Inc.
• Original Assignee: Simbe Robotics, Inc.
• Inventors: Shah, Mirza Akbar, Bogolea, Bradley, Gee, Jeffrey
• Primary Examiner(s): Obeid, Fahd A
• Assistant Examiner(s): Walker, Michael Jared
• Application Number:
  US15/347,689
• Publication Number:
  US 20170193434A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

One variation of a method for tracking stock level within a store includes: dispatching a robotic system to image shelving structures within the store during a scan cycle; receiving images from the robotic system, each image recorded by the robotic system during the scan cycle and corresponding to one waypoint within the store; identifying, in the images, empty slots within the shelving structures; identifying a product assigned to each empty slot based on product location assignments defined in a planogram of the store; for a first product of a first product value and assigned to a first empty slot, generating a first prompt to restock the first empty slot with a unit of the first product during the scan cycle; and, upon completion of the scan cycle, generating a global restocking list specifying restocking of a set of empty slots associated with product values less than the first product value.

Claims

1. A method for tracking stock level within a store, the method comprising:

at a robotic system;

estimating a number of customers within the store;

in response to the number of customers exceeding a threshold number of customers, initiating a scan cycle in the store; and

navigating along a set of shelving structures within the store during the scan cycle;

receiving a set of images from the robotic system, each image in the set of images recorded by the robotic system during the scan cycle and depicting a shelving structure in the set of shelving structures;

in the set of images;

detecting a set of slots within the set of shelving structures; and

detecting absence of units of a set of products from a subset of slots in the set of slots, each product in the set of products assigned to a slot in the subset of slots by a planogram of the store;

for each product in the set of products, accessing a product value of the product representing a margin of the product and a current sale rate of the product;

for a first product in the set of products and assigned to a first slot in the subset of slots by the planogram, in response to a first product value of the first product exceeding a threshold product value, serving a first prompt to restock the first slot with units of the first product to an associate of the store at a first time prior to conclusion of the scan cycle;

for a second product in the set of products and assigned to a second slot in the subset of slots by the planogram, in response to a second product value of the second product falling below the threshold product value, designating the second slot for restocking in a global restocking list for the store;

for a third product in the set of products and assigned to a second slot in the subset of slots by the planogram, in response to a third product value of the third product falling below the threshold product value, designating the third slot for restocking in the global restocking list; and

at a second time following completion of the scan cycle, serving the global restocking list to a computing device associated with the store.

2. The method of claim 1, wherein serving the global restocking list to the computing device associated with the store comprises, following completion of the scan cycle by the robotic system and, during a scheduled restocking period in the store, distributing portions of the global restocking list across a set of computing devices affiliated with associates of the store.

3. The method of claim 2, further comprising:

at a time succeeding 1 the scheduled restocking period in the store, dispatching the robotic system to image the set of shelving structures during a second scan cycle;

receiving a second set of images from the robotic system, each image in the set of images recorded by the robotic system during the second scan cycle and depicting a shelving structure in the set of shelving structures;

in the second set of images;

detecting a second set of slots within the set of shelving structures; and

detecting types and quantities of products in the second set of slots;

detecting a set of deviations between the global restocking list and types and quantities products in the second set of slots; and

verifying completion of the scheduled restocking period in the store based on the set of deviations.

4. The method of claim 1:

further comprising;

defining an order for a set of waypoints within the store based on product values of products assigned to slots, within the set of shelving structures, in fields of view of cameras integrated into the robotic system when occupying each waypoint in the set of waypoints; and

serving the order for the set of waypoints to the robotic system for execution during the scan cycle;

wherein navigating along the set of shelving structures within the store during the scan cycle comprises navigating through the set of waypoints during the scan cycle; and

wherein receiving the set of images from the robotic system comprises receiving the set of images from the robotic system during the scan cycle, each image in the set of images recorded by the robotic system while proximal a waypoint in the set of waypoints.

5. The method of claim 4, wherein defining the order of the set of waypoints comprises:

for each shelving structure within the set of shelving structures, summing product values of products assigned to slots in the shelving structure to calculate a product value sum of the shelving structure;

prioritizing the set of shelving structures in order of product value sum; and

calculating the order for the set of waypoints that minimizes total distance traversed by the robotic system from waypoints defined along a first shelving structure associated with a greatest product value sum to waypoints defined along a last shelving structure associated with a lowest product value sum.

6. The method of claim 4, further comprising, in response to the robotic system detecting an object obstructing access to a next waypoint in the set of waypoints:

marking, in memory, the next waypoint as deferred;

calculating a revised waypoint order for remaining waypoints in the set of waypoints to enable the robotic system to physically avoid the object;

serving the revised waypoint order to the robotic system; and

triggering the robotic system to return to the next waypoint at a later time during the scan cycle.

7. The method of claim 4, wherein defining the order of the set of waypoints comprises:

for each waypoint in the set of waypoints, calculating a sum of product values of products assigned to slots within a field of view of the robotic system when occupying the waypoint to calculate a product value sum of the waypoint;

identifying a first subset of waypoints associated with product value sums exceeding a threshold aggregate product value;

identifying a second subset of waypoints associated with product value sums less than the threshold aggregate product value; and

defining the order of the set of waypoints specifying image capture at waypoints in the first subset of waypoints, followed by image capture at waypoints in the second subset of waypoints, followed by image capture at waypoints in the first subset of waypoints during the scan cycle.

8. The method of claim 1, further comprising, in response to serving the first prompt to the associate during the scan cycle, re-queuing the first slot in the store for reimaging by the robotic system prior to completion of the scan cycle.

9. The method of claim 1:

wherein receiving the set of images from the robotic system comprises;

over a first period of time, receiving a first subset of images recorded by the robotic system occupying predefined waypoints along a first shelving structure in the set of shelving structures; and

over a second period of time succeeding the first period of time, receiving a second subset of images recorded by the robotic system occupying waypoints along a second shelving structure in the set of shelving structures;

wherein identifying the set of slots within the set of shelving structures comprises;

stitching the first subset of images into a first composite image of the first shelving structure;

projecting a first set of slot boundaries defined in the planogram for the first shelving structure onto the first composite image;

stitching the second subset of images into a second composite image of the second shelving structure;

projecting a second set of slot boundaries defined in the planogram for the second shelving structure onto the second composite image; and

wherein detecting absence of units of the set of products from the subset of slots comprises;

identifying a first slot in the first composite image bounded by the first set of slot boundaries and excluding visual representation of a unit of the first product assigned to the first slot according to the planogram of the store; and

identifying a second slot in the second composite image bounded by the second set of slot boundaries and excluding visual representation of a unit of the second product assigned to the second slot according to the planogram of the store.

10. The method of claim 1:

wherein estimating the number of customers within the store comprises accessing a rate of sales in the store recorded by a point-of-sale system in the store; and

wherein initiating the scan cycle comprises initiating the scan cycle in response to the rate of sales exceeding a threshold rate.

11. The method of claim 1:

wherein detecting the set of slots within the set of shelving structures comprises;

in a first image, in the set of images and depicting a first shelving structure in the set of shelving structures, detecting a first slot in the first shelving structure; and

in a second image, in the set of images and depicting a second shelving structure in the set of shelving structures, detecting a second slot in the second shelving structure; and

in a third image, in the set of images and depicting a third shelving structure in the set of shelving structures, detecting a third slot in the third shelving structure; and

wherein detecting absence of units of the set of products from the subset of slots in the set of slots comprises;

detecting a first quantity of units of a first product present in the first slot in the first image, a first target quantity of units of the first product assigned to the first slot by a planogram of the store;

detecting a second quantity of units of a second product present in the second slot in the second image, a second target quantity of units of the second product assigned to the second slot by the planogram;

detecting a third quantity of units of a third product present in the third slot in the third image, a third target quantity of units of the third product assigned to the third slot by the planogram;

wherein accessing a product value of a product for each product in the set of products comprises;

calculating a first sale rate of the first product based on the first difference between the first quantity and the first target quantity;

calculating the first product value of the first product based on a first margin of the first product and the first sale rate of the first product;

calculating a second sale rate of the second product based on the second difference between the second quantity and the second target quantity;

calculating the second product value of the second product based on a second margin of the second product and the second sale rate of the second product;

calculating a third sale rate of the third product based on the third difference between the third quantity and the third target quantity; and

calculating the third product value of the third product based on a third margin of the third product and the third sale rate of the third product.

12. The method of claim 1:

wherein initiating the scan cycle in the store comprises initiating the scan cycle during a first time period intersecting a historical peak traffic period in the store for a current weekday;

wherein serving the first prompt to restock the first slot with units of the first product comprises serving the first prompt to restock the first slot with units of the first product to the associate during the first time period intersecting the peak historical traffic period in the store; and

wherein serving the global restocking list to the computing device associated with the store comprises serving the global restocking list to the computing device associated with the store during a second time period intersecting a historical low traffic period in the store for the current weekday.

13. A method for tracking stock level within a store, the method comprising:

at a robotic system;

estimating a number of customers within the store;

in response to the number of customers exceeding a threshold number of customers, initiating a scan cycle in the store; and

navigating along a set of shelving structures within the store during the scan cycle;

receiving a set of images from the robotic system, each image in the set of images recorded by the robotic system during the scan cycle and depicting a shelving structure in the set of shelving structures;

in a first image, in the set of images and depicting a first shelving structure in the set of shelving structures;

detecting a first slot in the first shelving structure; and

detecting a first quantity of units of a first product present in the first slot, a first target quantity of units of the first product assigned to the first slot by a planogram of the store;

in a second image, in the set of images and depicting a second shelving structure in the set of shelving structures;

detecting a second slot in the second shelving structure; and

detecting a second quantity of units of a second product present in the second slot, a second target quantity of units of the second product assigned to the second slot by the planogram;

in a third image, in the set of images and depicting a third shelving structure in the set of shelving structures;

detecting a third slot in the third shelving structure; and

detecting a third quantity of units of a third product present in the third slot, a third target quantity of units of the third product assigned to the third slot by the planogram;

calculating a first product value of the first product based on a first margin of the first product and a first difference between the first quantity and the first target quantity;

calculating a second product value of the second product based on a second margin of the second product and a second difference between the second quantity and the second target quantity;

calculating a third product value of the third product based on a third margin of the third product and a third difference between the third quantity and the third target quantity;

in response to the first product value exceeding a threshold product value, serving a first prompt to restock the first slot with units of the first product to an associate of the store at a first time prior to conclusion of the scan cycle;

in response to the second product value and the third product value of the third product falling below the threshold product value, designating the second slot and the third slot for restocking in a global restocking list for the store; and

at a second time following completion of the scan cycle, serving the global restocking list to a computing device associated with the store.

14. The method of claim 13, wherein serving the global restocking list to the computing device comprises serving the global restocking list to a set of computing devices assigned to a group of associates of the store following completion of the scan cycle and during a scheduled restocking period in the store.

15. The method of claim 13:

wherein estimating the number of customers within the store comprises accessing a rate of sales in the store recorded by a point-of-sale system in the store; and

wherein initiating the scan cycle in the store comprises initiating the scan cycle in response to the rate of sales exceeding a threshold rate.

16. The method of claim 13, wherein detecting the first quantity of units of the first product present in the first slot in the first image comprises, in the first image, detecting absence of units of the first product in the first slot.

17. The method of claim 13, wherein calculating the first product value of the first product comprises:

calculating a first sale rate of the first product based on the first difference between the first quantity and the first target quantity; and

calculating the first product value of the first product based on the first margin of the first product and the first sale rate of the first product.

18. The method of claim 13:

wherein initiating the scan cycle in the store comprises initiating the scan cycle during a first time period intersecting a historical peak traffic period in the store for a current weekday;

wherein serving the first prompt to restock the first slot with units of the first product comprises serving the first prompt to restock the first slot with units of the first product to the associate during the first time period intersecting the peak historical traffic period in the store; and

wherein serving the global restocking list to the computing device associated with the store comprises serving the global restocking list to the computing device associated with the store during a second time period intersecting a historical low traffic period in the store for the current weekday.

19. A method for tracking stock level within a store, the method comprising:

at a robotic system;

estimating a number of customers within the store;

in response to the number of customers exceeding a threshold number of customers, initiating a scan cycle in the store; and

navigating along a set of shelving structures within the store during the scan cycle;

receiving a set of images from the robotic system, each image in the set of images recorded by the robotic system during the scan cycle and depicting a shelving structure in the set of shelving structures;

in a first image, in the set of images and depicting a first shelving structure in the set of shelving structures;

detecting a first slot in the first shelving structure; and

detecting a first quantity of units of a first product present in the first slot, a first target quantity of units of the first product assigned to the first slot by a planogram of the store;

in a second image, in the set of images and depicting a second shelving structure in the set of shelving structures;

detecting a second slot in the second shelving structure; and

detecting a second quantity of units of a second product present in the second slot, a second target quantity of units of the second product assigned to the second slot by the planogram;

in a third image, in the set of images and depicting a third shelving structure in the set of shelving structures;

detecting a third slot in the third shelving structure; and

detecting a third quantity of units of a third product present in the third slot, a third target quantity of units of the third product assigned to the third slot by the planogram;

calculating a first product value of the first product based on a first popularity of the first product and a first difference between the first quantity and the first target quantity;

calculating a second product value of the second product based on a second popularity of the second product and a second difference between the second quantity and the second target quantity;

calculating a third product value of the third product based on a third popularity of the third product and a third difference between the third quantity and the third target quantity;

in response to the first product value exceeding a threshold product value, serving a first prompt to restock the first slot with units of the first product to an associate of the store at a first time prior to conclusion of the scan cycle;

in response to the second product value and the third product value of the third product falling below the threshold product value, designating the second slot and the third slot for restocking in a global restocking list for the store; and

at a second time following completion of the scan cycle, serving the global restocking list to a computing device associated with the store.

20. The method of claim 19, further comprising calculating the first popularity of the first product at the first slot based on quantities of units of the first product in the first slot detected in images recorded by the robotic system over a series of scan cycles preceding the scan cycle.