Planogram extraction based on image processing

• US 8,189,855 B2
• Filed: 08/31/2007
• Issued: 05/29/2012
• Est. Priority Date: 08/31/2007
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Opalach, Agata, Fano, Andrew, Linaker, Fredrik, Groenevelt, Robert Bernard (Robin)
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): SHAH, UTPAL D
• Application Number:
  US11/849,165
• Publication Number:
  US 20090059270A1
• Time in Patent Office: 1,733 Days
• Field of Search: 382/100, 382/209, 382/217, 382/218
• US Class Current: 382/100
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Image analysis techniques, including object recognition analysis, are applied to images obtained by one or more image capture devices deployed within inventory environments. The object recognition analysis provides object recognition data (that may include one or more recognized product instances) based on stored product (training) images. In turn, a variety of functionalities may be enabled based on the object recognition data. For example, a planogram may be extracted and compared to a target planogram, or at least one product display parameter for a product can be determined and used to assess presence of the product within the inventory environment, or to determine compliance of display of the product with a promotional objective. In yet another embodiment, comparisons may be made within a single image or between multiple images over time to detect potential conditions requiring response. In this manner, efficiency and effectiveness of many previously manually-implemented tasks may be improved.

Claims

1. A method comprising:

receiving, by an image analysis server, an image of at least one product within an inventory environment;

performing, by the image analysis server, object recognition analysis on the image based on stored product images provide product recognition data; and

providing, by a planogram server in communication with the image analysis server, an extracted planogram, based on the product recognition data, indicating a current state visually representing actual locations of the at least one product within the inventory environment by mapping the actual locations of the at least one product within the inventory environment to corresponding locations within the extracted planogram.

2. The method of claim 1, further comprising:

identifying, by the image analysis server, an image location of a product support device within the image; and

mapping, by the planogram server, the location of the product support device in the image into an actual location of the shelf within the inventory environment,

wherein the extracted planogram further illustrates the actual location of the product support device.

3. The method of claim 1, wherein the product recognition data includes at least one recognized product instance, and wherein extracted planogram includes for each of the at least one recognized product instance, a corresponding image of the recognized product instance.

4. The method of claim 2, further comprising mapping, by the planogram server, the image location to the actual location based on position information of an image capture device within the inventory environment used to capture the image.

5. The method of claim 1, further comprising:

sending, by the image analysis server, a request for the image to an image capture device within the inventory environment.

6. The method of claim 1, further comprising:

comparing, by the planogram server, the extracted planogram to a target planogram.

7. The method of claim 6, wherein the target planogram is either of a previously extracted planogram based on a previous image and an optimal planogram.

8. The method of claim 6, further comprising:

determining, by the planogram server, a cost associated with differences between the extracted planogram and the target planogram.

9. A non-transitory, processor-readable medium having stored thereon executable instructions that, when executed, cause a processing device to:

receive an image of at least one product within an inventory environment;

perform object recognition analysis on the image based on stored product images to provide product recognition data; and

provide an extracted planogram, based on the product recognition data, indicating a current state visually representing actual locations of the at least one product within the inventory environment by mapping the actual locations of the at least one product within the inventory environment to corresponding locations within the extracted planogram.

10. The processor-readable medium of claim 9, having stored thereon executable instruction that, when executed, further cause the processing device to:

identify an image location of a product support device within the image; and

map the location of the product support device in the image into an actual location of the product support device within the inventory environment,

wherein the extracted planogram further illustrates the actual location of the product support device.

11. The processor-readable medium of claim 10, having stored thereon executable instructions that, when executed, further cause the processing device to include, in the extracted planogram, a corresponding image for each of at least one recognized product instance in the product recognition data.

12. The processor-readable medium of claim 10, having stored thereon executable instruction that, when executed, further cause the processing device to map the image location to the actual location based on position information of an image capture device within the inventory environment used to capture the image.

13. The processor-readable medium of claim 9, having stored thereon executable instruction that, when executed, further cause the processing device to:

send a request for the image to an image capture device within the inventory environment.

14. The processor-readable medium of claim 10, having stored thereon executable instruction that, when executed, further cause the processing device to:

compare the extracted planogram to a target planogram.

15. The processor-readable medium of claim 14, wherein the target planogram is either of a previously extracted planogram based on a previous image and an optimal planogram.

16. The processor-readable medium of claim 14, having stored thereon executable instruction that, when executed, further cause the processing device to:

determine cost associated with differences between the extracted planogram and the target planogram.

17. A system comprising:

an image capture device configured and disposed to capture an image of at least one product within an inventory environment; and

an automatic planogram generator, in communication with the image capture device, operative to identify at least one candidate product based on a configuration profile listing products expected within a field of view of the image capture device, to perform object recognition analysis on the image based on stored product images corresponding to the at least one candidate product to provide product recognition data and to provide an extracted planogram, based on the product recognition data, indicating a current state visually representing actual locations of the at least one product within the inventory environment by mapping the actual locations of the at least one product within the inventory environment to corresponding locations within the extracted planogram.

18. An apparatus, comprising:

an image analysis server, operative to receive an image of at least one product within an inventory environment, perform object recognition analysis on the image based on stored product images to provide product recognition data; and

a planogram server, in communication with the image analysis server, operative to provide an extracted planogram, based on the product recognition data, indicating a current state visually representing actual locations of the at least one product within the inventory environment by mapping the actual locations of the at least one product within the inventory environment to corresponding locations within the extracted planogram.

19. The apparatus of claim 18, wherein the image analysis server is further operative to identify an image location of a product support device within the image and the planogram server is further operative to map the location of the product support device in the image into an actual location of the shelf within the inventory environment, wherein the extracted planogram further illustrates the actual location of the product support device.

20. The apparatus of claim 18, wherein the product recognition data includes at least one recognized product instance, and wherein the extracted planogram includes for each of the at least one recognized product instance, a corresponding image of the recognized product instance.

21. The apparatus of claim 19, wherein the planogram server is further operative to map the image location to the actual location based on position information of an image capture device within the inventory environment used to capture the image.

22. The apparatus of claim 18, wherein the image analysis server is further operative to send a request for the image to an image capture device within the inventory environment.

23. The apparatus of claim 18, wherein the planogram server is further operative to compare the extracted planogram to a target planogram.

24. The apparatus of claim 23, wherein the target planogram is either of a previously extracted planogram based on a previous image and an optimal planogram.

25. The apparatus of claim 23, wherein the planogram server is further operative to determine a cost associated with differences between the extracted planogram and the target planogram.