System and method for automatically restocking items on shelves using a pneumatic pipe system

• US 10,654,665 B2
• Filed: 05/04/2018
• Issued: 05/19/2020
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Inventors: Nemati, Behzad, Nazarian, Ehsan
• Primary Examiner(s): Cumbess, Yolanda R
• Application Number:
  US15/971,357
• Publication Number:
  US 20180319607A1
• Time in Patent Office: 746 Days
• Field of Search: None
• CPC Class Codes: B65G 1/1371 with data records
• CPC Class Codes: B65G 1/1376 the orders being assembled on a commissioning conveyor
• CPC Class Codes: B65G 2203/041 Camera
• CPC Class Codes: B65G 2812/16 Pneumatic conveyors
• CPC Class Codes: B65G 51/32 for despatch, delivery, and transit
• CPC Class Codes: B65G 51/46 according to separate signal systems
• CPC Class Codes: G05B 19/4182 manipulators and conveyor only
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G05B 19/4189 characterised by the transport system
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

Systems, methods and computer-readable media for automating the restocking of shelves process by sending a notification when a product on a shelf has reached, or will reach, an undesired level of emptiness. This is determined using imaging sensors, such as cameras, which can calculate how full or empty a respective shelf is and predict when the shelf will need to be restocked. When the restocking time arrives, the notification can be sent to automated systems, which automatically cause new products to be stocked on the shelf via a pneumatic pipe system.

Claims

1. A method comprising:

receiving, from a plurality of imaging sensors, real-time imaging data, the real-time imaging data including images of a plurality of products on a store shelf, the plurality of products having a plurality of product types, wherein the plurality of imaging sensors;

(1) detect motion of a human between an imaging sensor in the plurality of imaging sensors and the store shelf;

(2) detect an end of the motion; and

(3) wait a pre-determined amount of time after the end of the motion before recording the images;

storing the real-time imaging data in a database, wherein the database contains item-specific data associated with each product in the plurality of products, the item-specific data including dimensions, weight, and orientation information of the each product;

calculating, in real-time and for each product in the plurality of products, a current quantity of a product on the store shelf based on the real-time imaging data and the item-specific data;

calculating a current depletion rate for each product in the plurality of products based on the current quantity of the each product, a previous quantity of the each product, and a historical sales rate of the each product;

forecasting when an emptiness threshold for each product in the plurality of products will be reached based on the current depletion rate for the each product and the current quantity of the each product, to yield a plurality of forecasted replenishment times, each forecasted replenishment time in the plurality of forecasted replenishment times identifying when a respective product in the plurality of products will reach the emptiness threshold; and

at each forecasted replenishment time in the plurality of forecasted replenishment times;

generating an alert for restocking the respective product via a pneumatic pipe system; and

triggering the pneumatic pipe system to restock the respective product based on the alert.

2. The method of claim 1, wherein the predetermined time is in a range of between about 10-20 seconds.

3. The method of claim 1, wherein the forecasting further comprises using machine learning to forecast the plurality of forecasted replenishment times.

4. The method of claim 3, wherein the machine learning is updated on a periodic basis based on actual sales of the plurality of products.

5. The method of claim 1, wherein each imaging sensor in the plurality of imaging sensors comprises a camera that is positioned across an aisle from the store shelf.

6. The method of claim 1, wherein the pneumatic pipe system uses a diverter to place the respective product in a predetermined location on the store shelf.

7. The method of claim 1, wherein the plurality of imaging sensors are dormant until an aisle entrance motion detector detects a presence of an individual on an aisle containing the store shelf.

8. A restocking system, comprising:

a plurality of imaging sensors configured to capture imaging data of items on store shelves;

a database having item-specific data stored, the item-specific data including dimensions, weight, and orientation information of the items;

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

storing the imaging data in the database;

calculating a current quantity of a product on the store shelves based on the imaging data and the item-specific data;

calculating a current depletion rate for the product based on the current quantity of the product, a previous quantity of the product, and a historical sales rate of the product;

forecasting when an emptiness threshold for the product will be reached based on the current depletion rate and the current quantity of the product, to yield a forecasted replenishment time; and

at the forecasted replenishment time;

generating an alert for restocking the respective product via a pneumatic pipe system; and

triggering the pneumatic pipe system to restock the respective product based on the alert.

9. The restocking system of claim 8, wherein the database stores a predefined image of the store shelves in a full state.

10. The restocking system of claim 8, wherein the plurality of imaging sensors:

(1) detect motion of a human between an imaging sensor in the plurality of imaging sensors and the store shelf;

(2) detect an end of the motion; and

(3) wait a pre-determined amount of time after the end of the motion before recording the images.

11. The restocking system of claim 10, wherein the predetermined time is in a range of between about 10-20 seconds.

12. The restocking system of claim 9, wherein the forecasting further comprises using machine learning to forecast the plurality of forecasted replenishment times.

13. The restocking system of claim 12, wherein the machine learning is updated on a periodic basis based on actual sales of the plurality of products.

14. The restocking system of claim 9, wherein each imaging sensor in the plurality of imaging sensors comprises a camera that is positioned across an aisle from the store shelves.

15. The restocking system of claim 9, wherein the pneumatic pipe system uses a diverter to place the respective product in a predetermined location on the store shelf.

16. A non-transitory computer-readable storage medium having instructions stored which, when executed by a processor, cause the processor to perform operations comprising:

receiving, from a plurality of imaging sensors, real-time imaging data, the real-time imaging data including images of a plurality of products on a store shelf, the plurality of products having a plurality of product types, wherein the plurality of imaging sensors;

(1) detect motion of a human between an imaging sensor in the plurality of imaging sensors and the store shelf;

(2) detect an end of the motion; and

(3) wait a pre-determined amount of time after the end of the motion before recording the images;

storing the real-time imaging data in a database, wherein the database contains item-specific data associated with each product in the plurality of products, the item-specific data including dimensions, weight, and orientation information of the each product;

calculating, in real-time and for each product in the plurality of products, a current quantity of a product on the store shelf based on the real-time imaging data and the item-specific data;

calculating a current depletion rate for each product in the plurality of products based on the current quantity of the each product, a previous quantity of the each product, and a historical sales rate of the each product;

forecasting when an emptiness threshold for each product in the plurality of products will be reached based on the current depletion rate for the each product and the current quantity of the each product, to yield a plurality of forecasted replenishment times, each forecasted replenishment time in the plurality of forecasted replenishment times identifying when a respective product in the plurality of products will reach the emptiness threshold; and

at each forecasted replenishment time in the plurality of forecasted replenishment times;

generating an alert for restocking the respective product via a pneumatic pipe system; and

triggering the pneumatic pipe system to restock the respective product based on the alert.

17. The non-transitory computer-readable storage medium of claim 16, wherein the predetermined time is in a range of between about 10-20 seconds.

18. The non-transitory computer-readable storage medium of claim 16, wherein the forecasting further comprises using machine learning to forecast the plurality of forecasted replenishment times.

19. The non-transitory computer-readable storage medium of claim 18, wherein the machine learning is updated on a periodic basis based on actual sales of the plurality of products.

20. The non-transitory computer-readable storage medium of claim 16, wherein each imaging sensor in the plurality of imaging sensors comprises a camera that is positioned across an aisle from the store shelf.