Store planogram to image registrations

• US 20180068256A1
• Filed: 09/04/2016
• Published: 03/08/2018
• Est. Priority Date: 09/04/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: MARDER, MATTIAS
• Granted Patent: US 10,552,787 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Embodiments of the present invention may provide automated techniques for checking store shelves for compliance with planograms that can handle unknown arrangements in a uniform way, with little user involvement, and with relatively low processing complexity. For example, a computer-implemented method for determining compliance with a planogram may comprise receiving at least one image of a plurality of shelves containing objects, receiving at least one planogram representing desired positions of the objects on the shelves, wherein each row of the planogram corresponds to one shelf, adjusting positions of objects in the planogram to reflect a size of the shelves, determining a tolerance for inaccuracy in object position in the image using object dimension data, detecting object positions in the image, aligning each row of the planogram individually to the detected object positions, and comparing the detected object positions with the adjusted planogram positions to generate compliance information.

Claims

1. A computer-implemented method for determining compliance with a planogram comprising:

receiving at least one image of a plurality of shelves containing objects;

receiving at least one planogram representing desired positions of the objects on the shelves, wherein each row of the planogram corresponds to one shelf;

adjusting positions of objects in the planogram to reflect a size of the shelves;

determining a tolerance for inaccuracy in object position in the image using object dimension data;

detecting object positions in the image;

aligning each row of the planogram individually to the detected object positions; and

comparing the detected object positions with the adjusted planogram positions to generate compliance information.

2. The method of claim 1, wherein aligning each row of the planogram individually to the detected object positions comprises:

for each row in the planogram;

searching for detected objects;

for each detected object;

performing consensus voting on the object;

determining a detection confidence value for the object based on the tolerance for inaccuracy in object position; and

scaling the consensus vote according to the detection confidence value.

3. The method of claim 2, wherein aligning each row of the planogram individually to the detected object positions further comprises:

for detected objects having low detection confidence values, aligning the detected objects using predetermined adjustment values.

4. The method of claim 3, wherein the comparing comprises:

comparing the aligned detected objects having low detection confidence values and the detected objects having high detection confidence values with the adjusted planogram positions; and

generating compliance information representing a level of compliance of the detected object positions with the planogram.

5. The method of claim 4, wherein the generated compliance information comprises summary information.

6. The method of claim 4, wherein the generated compliance information comprises identifications of objects that are not positioned in accordance with the planogram.

7. A computer program product for determining compliance with a planogram, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

receiving at least one image of a plurality of shelves containing objects;

receiving at least one planogram representing desired positions of the objects on the shelves, wherein each row of the planogram corresponds to one shelf;

adjusting positions of objects in the planogram to reflect a size of the shelves;

determining a tolerance for inaccuracy in object position in the image using object dimension data;

detecting object positions in the image;

aligning each row of the planogram individually to the detected object positions; and

comparing the detected object positions with the adjusted planogram positions to generate compliance information.

8. The computer program product of claim 7, wherein aligning each row of the planogram individually to the detected object positions comprises:

for each row in the planogram;

searching for detected objects;

for each detected object;

performing consensus voting on the object;

determining a detection confidence value for the object based on the tolerance for inaccuracy in object position; and

scaling the consensus vote according to the detection confidence value.

9. The computer program product of claim 8, wherein aligning each row of the planogram individually to the detected object positions further comprises:

for detected objects having low detection confidence values, aligning the detected objects using predetermined adjustment values.

10. The computer program product of claim 9, wherein the comparing comprises:

comparing the aligned detected objects having low detection confidence values and the detected objects having high detection confidence values with the adjusted planogram positions; and

generating compliance information representing a level of compliance of the detected object positions with the planogram.

11. The computer program product of claim 10, wherein the generated compliance information comprises summary information.

12. The computer program product of claim 10, wherein the generated compliance information comprises identifications of objects that are not positioned in accordance with the planogram.

13. A system for determining compliance with a planogram, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

receiving at least one image of a plurality of shelves containing objects;

receiving at least one planogram representing desired positions of the objects on the shelves, wherein each row of the planogram corresponds to one shelf;

adjusting positions of objects in the planogram to reflect a size of the shelves;

determining a tolerance for inaccuracy in object position in the image using object dimension data;

detecting object positions in the image;

aligning each row of the planogram individually to the detected object positions; and

comparing the detected object positions with the adjusted planogram positions to generate compliance information.

14. The system of claim 13, wherein aligning each row of the planogram individually to the detected object positions comprises:

for each row in the planogram;

searching for detected objects;

for each detected object;

performing consensus voting on the object;

determining a detection confidence value for the object based on the tolerance for inaccuracy in object position; and

scaling the consensus vote according to the detection confidence value.

15. The system of claim 14, wherein aligning each row of the planogram individually to the detected object positions further comprises:

for detected objects having low detection confidence values, aligning the detected objects using predetermined adjustment values.

16. The system of claim 15, wherein the comparing comprises:

comparing the aligned detected objects having low detection confidence values and the detected objects having high detection confidence values with the adjusted planogram positions; and

generating compliance information representing a level of compliance of the detected object positions with the planogram.

17. The system of claim 16, wherein the generated compliance information comprises summary information.

18. The system of claim 16, wherein the generated compliance information comprises identifications of objects that are not positioned in accordance with the planogram.