System and method for retail store promotional price tag detection

• US 10,002,344 B2
• Filed: 10/17/2016
• Issued: 06/19/2018
• Est. Priority Date: 10/17/2016
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Wu, Wencheng, Venable, Dennis L., Rizzolo, Charles D., Wade, Thomas F., Hann, Daniel S., Shen, Ethan, Williams, Lee Anne
• Primary Examiner(s): Labaze, Edwyn
• Application Number:
  US15/295,498
• Publication Number:
  US 20180107968A1
• Time in Patent Office: 610 Days
• Field of Search: 235385, 235375, 235487, 705 22, 705 28
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06K 19/06028 using bar codes
• CPC Class Codes: G06K 19/07758 arrangements for adhering the record carrier to further objects or living beings, functioning as an identification tag
• CPC Class Codes: G06K 7/1413 1D bar codes
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 1/6002 Corrections within particular colour systems

Abstract

A system to detect and maintain retail store promotional price tags (PPTs) includes a heuristic PPT description extractor module, a heuristic rule deriver module, a store shelf image acquisition system, a barcode locator and recognizer module, and a heuristic PPT classifier module. The heuristic PPT description extractor module extracts heuristic descriptions of PPTs. The heuristic rule deriver module derives a set of heuristic parameters for the PPTs. The barcode locator and recognizer module analyzes images acquired by the store shelf image acquisition system to localize and recognize barcodes. The heuristic PPT description classifier module extracts heuristic attributes from the images acquired by the store shelf image acquisition system using the set of PPT parameters supplied by the heuristic rule deriver module.

Claims

1. A retail store promotional price tag (PPT) detection and classification system comprising:

a store shelf image acquisition module configured to acquire images of a plurality of product display units located in a retail store, the plurality of product display units including a plurality of products arranged on one or more of the product display fixtures, and a plurality of PPTs attached to the product display fixtures, each PPT including one or more printed colors and product-related data including a barcode associated with a respective product proximately located on the product display fixture near the PPT;

a device characterization module configured to generate one or more color profiles associated with transforming a plurality of colors associated with the acquired image of the printed PPTs from an image acquisition color space to an original color space associated with a digital representation of the PPTs;

a PPT detection and classifier module configured to extract one or more heuristic attributes including color from the acquired images using the one or more color profiles and apply one or more predefined heuristic rules to the heuristic attributes to detect and classify each PPT detected in the acquired images as one of a plurality of PPT types;

a mobile base including an operatively associated navigation component configured to determine a location of the mobile base within the retail store;

an image capture assembly mounted on the mobile base, the assembly including a plurality of image capture devices for acquiring the images of the plurality of product display units; and

a master control unit configured to execute control unit instructions for processing the images of the plurality of product display units at a sequence of locations during a movement of the mobile base in the retail store and thereafter generating a store profile indicating locations of the products and/or PPTs throughout the retail store.

2. The retail store PPT detection and classification system according to claim 1, wherein the color profiles are sparse color profiles.

3. The retail store PPT detection and classification system according to claim 1, wherein the color profiles include characterizing a color transformation associated with one or more of a PPT digital template to a printing device, a printed PPT to an image capture device, and a scanner to an image capture device.

4. The retail store PPT detection and classification system according to claim 1, further comprising:

a heuristic rule generation module configured to generate the predefined PPT detection and classification rules and attributes based on a plurality of PPT templates.

5. The retail store PPT detection and classification system according to claim 1, further comprising:

a barcode localization and recognition module configured to localize and recognize the barcodes associated with each of the plurality of PPTs.

6. The retail store PPT detection and classification system according to claim 1, wherein the plurality of image capture devices are vertically spaced and the master control unit translates vertically spaced acquired images based on a computed spatial profile of each of the image capture devices.

7. The retail store PPT detection and classification system according to claim 1, wherein the master control unit generates a packaging order and/or printing order for a set of replacement PPTs based on the generated store profile.

8. The retail store PPT detection and classification system according to claim 1, wherein the mobile base acquires the images of the plurality of product display units during one of a continuous movement mode of the mobile base, and a stop and go movement mode of the mobile base.

9. The retail store PPT detection and classification system according to claim 1, wherein the product display units include one or more of vertically spaced shelves and peg boards.

10. A retail store promotional price tag (PPT) detection and classification system comprising:

a store shelf image acquisition model configured to acquire images from a plurality of image capture devices of a plurality of product display units located in a retail store, the plurality of product display units including a plurality of products arranged on one or more of the product display fixtures, and a plurality of printed PPTs attached to the product display fixtures, each PPT including product-related data including a barcode associated with a respective product proximately located on the product display fixture near the PPT;

a device characterization module configured to generate one or more spatial profiles associated with transforming the acquired images to compensate for distortions associated with each image capture device and alignment of the acquired images;

an image stitching module configured to access the one or more spatial profiles and stitch the acquired images to generate a panorama image;

a PPT detection and classifier module configured to extract one or more heuristic attributes from the panorama image and apply one or more predefined heuristic rules to the heuristic attributes to detect and classify each PPT detected as one of a plurality of PPT types;

a mobile base including an operatively associated navigation component configured to determine a location of the mobile base within the retail store;

an image capture assembly mounted on the mobile base, the assembly including the plurality of image capture devices for acquiring the images of the plurality of product display units; and

a master control unit configured to execute control unit instructions for processing the images of the plurality of product display units at a sequence of locations during a movement of the mobile base in the retail store and thereafter generating a store profile indicating locations of the products and/or PPTs throughout the retail store.

11. The retail store PPT detection and classification system according to claim 10, further comprising:

a heuristic rule generation module configured to generate the predefined heuristic rules and associated heuristic attributes based on a plurality of PPT templates.

12. The retail store PPT detection and classification system according to claim 10, further comprising:

a barcode localization and recognition module configured to localize and recognize the barcodes associated with each of the plurality of PPTs.

13. The retail store PPT detection and classification system according to claim 10, wherein the plurality of image capture devices are vertically spaced and the master control unit translates vertically spaced acquired images based on the one or more spatial profiles.

14. The retail store PPT detection and classification system according to claim 10, wherein the master control unit generates a packaging order and/or printing order for a set of replacement PPTs based on the generated store profile.

15. The retail store PPT detection and classification system according to claim 10, wherein the mobile base acquires the images of the plurality of product display units during one of a continuous movement mode of the mobile base, and a stop and go movement mode of the mobile base.

16. The retail store PPT detection and classification system according to claim 10, wherein the product display units include one or more of vertically spaces shelves and peg boards.

17. The retail store profile PPT detection and classification system according to claim 10, wherein the image stitching module is configured to perform model based stitching of the acquired images to generate the panorama image.

18. The retail store profile PPT detection and classification system according to claim 10, wherein the device characterization module is further configured to generate one or more color profiles associated with transforming a plurality of colors associated with the acquired images of the printed PPTs from an image acquisition color space to an original color space associated with a digital representation of the PPTs.

19. A retail store profile generation system comprising:

a mobile base including an operatively associated navigation component configured to determine a location of the mobile base within a retail store, the retail store including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures, a plurality of product arranged on one or more of the product display fixtures, and a plurality of tags attached to the product display fixtures, each tag including product-related data including a barcode associated with a respective product proximately located on the product display fixture near the tag;

an image capture assembly mounted on the mobile base, the assembly including a plurality of vertically spaced image capture devices for acquiring images of the product display units;

a master control unit including an operatively associated memory which stores instructions and a processor for executing the instructions, the control unit instructions including instructions for processing the images acquired by the plurality of image capture devices at a sequence of locations during a movement of the mobile base in the store, thereafter extracting the product-related data from the images of the product display units and generating a store profile indicating locations of the products and/or tags throughout the store based on the extracted product-related data, a spatial characterization of the image capture assembly, a color characterization associated with transforming a plurality of colors associated with the acquired images from an image acquisition color space to an original color space associated with a digital representation of the tags, and the locations of the mobile base at the sequence of locations provided by the navigation component during an interval of time that the images were acquired.

20. The retail store profile generation system according to claim 19, wherein the master control unit is configured to generate a model-based stitched panorama image based on the spatial characterization including a total vertical height associated with a portion of a product display unit, and detect and classify detected tags from the panorama image based on one or more extracted heuristic attributes including color processed according to the color characterization and applying one or more predefined heuristic rules to the heuristic attributes.

21. The retail store profile generation system according to claim 20, wherein the plurality of tags are promotional price tags.