MACHINE VISION TECHNOLOGY FOR SHELF INVENTORY MANAGEMENT

• US 20150262116A1
• Filed: 03/16/2014
• Published: 09/17/2015
• Est. Priority Date: 03/16/2014
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Katircioglu, Kaan K., Li, Ying
• Application Number:
  US14/215,006
• Publication Number:
  US 20150262116A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A47F 2010/025 using stock management systems
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A system, method and computer program product for maintaining shelf inventory data on a shelf. The system includes a camera for capturing shelf images of items on the shelf. An inventory database stores a product name, type, barcode, image and inventory data. An image-count correlation database stores historical product inventory images and product shelf inventory counts associated with the products in the historical images which are read from the product inventory database. A computer processor segments the shelf images into product inventory images, matches the inventory images with the historical images, and updates the shelf inventory data in the inventory database based on the product shelf inventory counts associated with the matched historical images in the image-count correlation database.

Claims

1. A system for maintaining shelf inventory data on a shelf, the system comprising:

a camera configured to capture a shelf image of a plurality of items on the shelf;

an inventory database storing data on a plurality of products, each product associated with a product type, a product image and a current product shelf inventory count;

an image-count correlation database storing a plurality of historical images for each product on the shelf and a product shelf inventory count associated with each of the historical images;

a computer processor configured to;

segment the shelf image into at least a first inventory image, the first inventory image containing only items of a single product type on the shelf;

match the first inventory image with one of the historical images in the image-count correlation database; and

update the inventory database based on the product shelf inventory count associated with the matched historical image in the image-count correlation database.

2. The system of claim 1, wherein the historical images include a plurality of inventory images of each product type, each of the inventory images are captured at different instances of time, such that each inventory image is associated with the product shelf inventory count at the time the inventory image was captured.

3. The system of claim 2, wherein the plurality of inventory images are captured and associated with the product shelf inventory count throughout a set calibration period.

4. The system of claim 2, wherein the plurality of inventory images are captured and associated with the product shelf inventory count continuously, such that the first inventory image and its associated product shelf inventory count are stored in the image-count correlation database after updating the inventory database.

5. The system of claim 1, further comprising a plurality of cameras configured to capture a plurality of shelf images, the plurality of shelf images segmented into a plurality of inventory images based on product types.

6. The system of claim 5, wherein the inventory images are segmented based on the locations of a plurality of identifier tags positioned on the shelf, such that each product on the shelf is associated with at least one identifier tag, wherein the identifier tags are recognized based on edge detection and text detection data extracted from the shelf image.

7. The system of claim 5, wherein the computer processor is further configured to segment the inventory images into a plurality of product images, each of the product images containing only one item, such that each product image is associated with one of the product types, and wherein the product images are utilized to identify the product types of items by matching a current product image with the product images in the inventory database.

8. The system of claim 7, wherein the product images are utilized to identify displaced items on the shelf by matching a current product image with a product image retrieved from the inventory database based on a barcode in an identifier tag of the product.

9. The system of claim 5, wherein the computer processor is configured to analyze the shelf images and inventory images based on at least one of edge detection data, image histogram data, text detection data, and barcode data.

10. The system of claim 1, wherein the inventory database includes product inventory data, the product inventory data including statistics on the rates of item removal and replacement of each product type.

11. A method for maintaining shelf inventory data on a shelf, the method comprising:

capturing a shelf image of a plurality of items on the shelf;

segmenting the shelf image into at least a first inventory image, the first inventory image containing only items of a single product type on the shelf;

matching the first inventory image with a historical image from an image-count correlation database, the image-count correlation database storing historical images of each product type on the shelf and a product shelf inventory count associated with each of the historical images; and

updating an inventory database based on the product shelf inventory count associated with the matched historical image in the image-count correlation database, the inventory database storing data for a plurality of products, each product associated with a product type, a product image and a current shelf inventory count.

12. The method of claim 11, wherein the inventory images are segmented based on the locations of a plurality of identifier tags positioned on the shelf, such that each product on the shelf is associated with at least one identifier tag, wherein the identifier tags are recognized based on edge detection and text detection data extracted from the shelf image.

13. The method of claim 11, further comprising:

segmenting the inventory images into a plurality of product images, each of the product images containing only one item, such that each product image is associated with the product types of the contained item; and

identifying the product types of items based on the product images by matching a current product image with product images in the inventory database.

14. The method of claim 11, further comprising populating the image-count correlation database with inventory images of each product type which are captured at different instances of time, such that each inventory image is associated with the product shelf inventory count at the time the inventory image was captured.

15. The method of claim 14, wherein the plurality of inventory images are captured and associated with the product shelf inventory count throughout a set calibration period.

16. The method of claim 14, wherein the population of the image-count correlation database is performed continuously, such that the first inventory image and its associated product shelf inventory count is stored in the image-count correlation database after updating the inventory database.

17. The method of claim 11, further comprising setting a shelf-stock flag if the product shelf inventory count is less than a set value.

18. A computer program product for maintaining shelf inventory data on a shelf, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code configured to;

capture a shelf image of a plurality of items on the shelf;

segment the shelf image into at least a first inventory image, the first inventory image containing only items of a single product type on the shelf;

match the first inventory image with a historical image from an image-count correlation database, the image-count correlation database storing historical images of each product type on the shelf and the product shelf inventory count associated with each of the historical images; and

update the shelf inventory data in the inventory database based on the product shelf inventory count associated with the matched historical image in the image-count correlation database.

19. The computer program product of claim 18, further comprising computer readable program code to populate the image-count correlation database with inventory images of each product type which are captured at different instances of time, such that each inventory image is associated with the product shelf inventory count at the time the inventory image was captured.

20. The computer program product of claim 18, further comprising computer readable program code to set a shelf-stock flag if the product shelf inventory count is less than a set value.