Method and system for estimating stock level of shelf

• US 10,360,532 B2
• Filed: 04/28/2017
• Issued: 07/23/2019
• Est. Priority Date: 09/23/2016
• Status: Active Grant

• Current Assignee: Wistron NeWeb Corporation (Wistron Corporation)
• Original Assignee: Wistron NeWeb Corporation (Wistron Corporation)
• Inventors: Chen, Wei-Hong, Wu, Yu-Sen, Wu, Po-Ching
• Primary Examiner(s): Iwarere, Oluseye
• Application Number:
  US15/582,083
• Publication Number:
  US 20180089613A1
• Time in Patent Office: 816 Days
• Field of Search: 382165, 705 28
• CPC Class Codes: G02B 5/208 for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

The disclosure is related to a method and a system for estimating a stock level of a shelf. An image processing technique is incorporated to conduct stock estimation that is used as a reference for replenishment decisions. In the method, a camera is used to capture a shelf image. The shelf image is divided into multiple sub-region images. Multiple binarized images with respect to the multiple sub-region images are obtained by applying a binarization method. A ratio of the binarization values with respect to the pixels within every sub-region is calculated. The method also includes comparing the ratio of the binarization values in at least two sub-regions with a reference ratio in the at least two sub-regions within a shelf reference image. The comparison is used to estimate the stock level, and also to determine whether the shelf is to be replenished.

Claims

1. A method for estimating stock level of a shelf, comprising:

using a camera unit that is mounted in a shelf to capture a shelf image of a multi-layer shelf at a specific angle;

obtaining the shelf image, being divided into multiple sub-regions;

performing a binarization computation upon the shelf image so as to obtain a binarized image of every sub-region, in which the binarization computation is performed to compare a pixel gray value for each pixel of each of the sub-regions with a gray threshold;

computing a first binarization value ratio for a first sub-region and a second binarization value ratio for a second sub-region, wherein the first sub-region and the second sub-region are among the sub-regions;

obtaining a first binarization reference ratio of the first sub-region and a second binarization reference ratio of the second sub-region based on a shelf reference image that is another shelf image of an empty shelf and is captured by the camera unit;

computing a first difference between the first binarization value ratio and the first binarization reference ratio of the first sub-region, so as to obtain a first stock decision parameter;

computing a second difference between the second binarization value ratio and the second binarization reference ratio of the second sub-region, so as to obtain a second stock decision parameter; and

estimating a stock level according to the first stock decision parameter and the second stock decision parameter.

2. The method as recited in claim 1, wherein the pixel with the pixel gray value smaller than the gray threshold is set to a first value, and the pixel with the pixel gray value larger than the gray threshold is set to a second value;

and the binarization value ratio or the binarization reference ratio of every sub-region is calculated by dividing the number of the first value or the number of the second value by a pixel number of every sub-region.

3. The method as recited in claim 2, wherein the binarized image of the sub-region is composed of a number of black pixels and white pixels, and the binarization value ratio or the binarization reference ratio indicates a ratio of number of black pixels in the sub-region to the pixel number of the sub-region, or another ratio of number of white pixels in the sub-region to the pixel number of the sub-region.

4. The method as recited in claim 1, wherein the shelf image is obtained by using the camera unit to capture image of the shelf when the shelf is illuminated by an infrared light;

and the camera unit includes a bandpass filter whose bandpass range and a wavelength range of the infrared light is 700 nm to 940 nm.

5. The method as recited in claim 1, wherein the multiple sub-regions of the shelf image are multiple parallel strip areas in one image.

6. The method as recited in claim 1, wherein the sub-regions respectively corresponding to the first stock decision parameter and the second stock decision parameter are continuous sub-regions.

7. The method as recite in claim 1, wherein the sub-regions respectively corresponding to the first stock decision parameter and the second stock decision parameter are discontinuous sub-regions.

8. The method as recited in claim 1, wherein the step for estimating the stock level using the stock decision parameters further comprises:

setting a decision threshold;

determining a starting point of items in stock on the shelf by a difference variation between the first stock decision parameter and the second stock decision parameter is larger than the decision threshold;

computing a ratio of the items placed in the sub-regions of the shelf according to the starting point of items in stock, and the ratio is rendered to estimate the stock level.

9. The method as recited in claim 8, further introducing an inventory threshold, a notice of replenishment is generated when the ratio is lower than the inventory threshold.

10. A system for estimating stock level in a shelf, comprising:

an image capture unit used to capture a shelf image;

a control unit, electrically connected to the image capture unit, used to control the image capture unit to capture the shelf image;

a memory unit, electrically connected to the control unit, used to store the shelf image captured by the image capture unit;

an instruction set stored in the memory unit, wherein the instruction set is executed by the control unit to perform a method for estimating stock level on the shelf comprising;

using the image capture unit that is mounted in the shelf to capture the shelf image of a multi-layer shelf at a specific angle;

obtaining the shelf image, being divided into multiple sub-regions;

performing a binarization computation upon the shelf image so as to obtain a binarized image of every sub-region, in which the binarization computation is performed to compare a pixel gray value for each pixel of each of the sub-regions with a gray threshold;

computing a first binarization value ratio for a first sub-region and a second binarization value ratio for a second sub-region, wherein the first sub-region and the second sub-region are among the sub-regions;

obtaining a first binarization reference ratio of the first sub-region and a second binarization reference ratio of the second sub-region based on a shelf reference image that is another shelf image of an empty shelf and is captured by the image capture unit;

computing a first difference between the first binarization value ratio and the first binarization reference ratio of the first sub-region, so as to obtain a first stock decision parameter;

computing a second difference between the second binarization value ratio and the second binarization reference ratio of the second sub-region, so as to obtain a second stock decision parameter; and

estimating a stock level according to the first stock decision parameter and the second stock decision parameter.

11. The system as recited in claim 10, wherein the image capture unit further captures a second shelf image from another angle of light, the control unit performs the binarization computation on the second shelf image for obtaining a third binarization value ratio of a third sub-region, and a fourth binarization value ratio of a fourth sub-region, wherein the control unit compares the third binarization value ratio with a third binarization reference ratio to generate a third stock decision parameter, compares the fourth binarization value ratio with a fourth binarization reference ratio to generate a fourth stock decision parameter, and estimates the stock level according to the first stock decision parameter, the second stock decision parameter, the third stock decision parameter, and the fourth stock decision parameter.

12. The system as recited in claim 11, wherein the first sub-region and the third sub-region are the same, and the second sub-region and the fourth sub-region are the same.

13. The system as recited in claim 11, wherein the pixel with the pixel gray value smaller than the gray threshold is set to a first value, and the pixel with the pixel gray value larger than the gray threshold is set to a second value;

and the binarization value ratio or the binarization reference ratio of every sub-region is calculated by dividing the number of the first value or the number of the second value by a pixel number of every sub-region.

14. The system as recited in claim 13 wherein the binarized image of the sub-region is composed of a number of black pixels and white pixels, and the binarization value ratio or the binarization reference ratio indicates a ratio of number of black pixels in the sub-region to the pixel number of the sub-region, or another ratio of number of white pixels in the sub-region to the pixel number of the sub-region.

15. The system as recited in claim 10, wherein the shelf image is obtained by using the image capture unit to capture image of the shelf when the shelf is illuminated by an infrared light;

and the image capture unit includes a bandpass filter whose bandpass range and a wavelength range of the infrared light is 700 nm to 940 nm.

16. The system as recited in claim 10, wherein the multiple sub-regions of the shelf image are multiple parallel strip areas in one image.

17. The system as recited in claim 10, wherein the sub-regions respectively corresponding to the first stock decision parameter and the second stock decision parameter are continuous sub-regions.

18. The system as recite in claim 10, wherein the sub-regions respectively corresponding to the first stock decision parameter and the second stock decision parameter are discontinuous sub-regions.

19. The system as recited in claim 10, wherein the step for estimating the stock level using the stock decision parameters further comprises:

setting a decision threshold;

determining a starting point of items in stock on the shelf by a difference variation between the first stock decision parameter and the second stock decision parameter is larger than the decision threshold;

computing a ratio of the items placed in the sub-regions of the shelf according to the starting point of items in stock, and the ratio is rendered to estimate the stock level.