INTERACTIVE PRODUCT AUDITING WITH A MOBILE DEVICE

• US 20170255891A1
• Filed: 09/30/2015
• Published: 09/07/2017
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Morate, Diego Garcia, Garcia, Antonio Hurtado
• Granted Patent: US 10,796,262 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 10/06393 Score-carding, benchmarking or key performance indicator [KPI] analysis
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Interactive product auditing with a mobile device is described. Example methods disclosed herein include performing, with an auditing device, image recognition based on a first set of candidate patterns accessed by the auditing device to identify a first product in a first region of interest of a segmented image. The disclosed example methods also include prompting, with the auditing device, a user to enter input associated with a first grid of the first region of interest displayed on a display, the first grid including the first product. The disclosed example methods further include determining, with the auditing device, a second set of candidate patterns to use to identify a second product in a second region of interest of the segmented image, the second set of candidate patterns determined based on the user input and a group of products identified in a neighborhood of the first region of interest.

Claims

1. An interactive product auditing method comprising:

performing, with a processor of an auditing device, image recognition based on a first set of candidate patterns accessed by the auditing device to identify a first product depicted in a first region of interest of a segmented image;

prompting, with the processor of the auditing device, a user to enter a user input associated with a first grid of the first region of interest displayed on a display of the auditing device, the first grid including the first product; and

determining, with the processor of the auditing device, a second set of candidate patterns to use to identify a second product in a second region of interest of the segmented image, the second set of candidate patterns being determined based on the user input and a group of products identified in the segmented image in a neighborhood of the first region of interest.

2. The method of claim 1, further including:

displaying, on the display of the auditing device, an image-based result for the first and second products identified in the first and second regions of interest; and

modifying the image-based results based on the user input.

3. The method of claim 2, wherein modifying the image based results includes changing an identification of at least one of the first product or the second product, or changing a segment of the segmented image.

4. The method of claim 1, further including;

displaying, on the display of the auditing device, key performance indicators providing data related to the first and second products;

displaying a prompt to the user for a second user input while the key performance indicators are displayed on the display of the auditing device; and

modifying at least one of the key performance indicators based on the second user input.

5. The method of claim 4, wherein the key performance indicators include at least one of respective shares of shelf space or respective positions of the first and second products in the segmented image.

6. The method of claim 1, further including at least one of determining a location of the first and second products in the segmented image, determining respective numbers of facings of the first and second products in the segmented image, detecting a grid in the first region of interest that is empty, or detecting a grid in the first region of interest that is associated with an out-of-stock product.

8. The method of claim 1, wherein the first set of candidate patterns is selected based on at least one of a store being audited, a product category, or an auditor performing the audit.

9. The method of claim 1, wherein the second set of candidate patterns is further determined based on at least one of a product category or a store.

7. (canceled)

10. An auditing device comprising:

a display;

an input/output interface;

a product identifier to perform image recognition based on a first set of candidate patterns accessed by the auditing device to identify a first product depicted in a first region of interest of a segmented image;

a results analyzer to prompt, via the display, a user to enter a user input, via an input/output interface, associated with a first grid of the first region of interest displayed on the display, the first grid including the first product; and

a candidate pattern selector to determine a second set of candidate patterns to use to identify a second product in a second region of interest of the segmented image, the second set of candidate patterns being determined based on the user input and a group of products identified in the segmented image in a neighborhood of the first region of interest.

11. The auditing device of claim 10, wherein the results analyzer of the auditing device is further to:

display, via the display, an image-based result for the first and second products identified in the first and second regions of interest; and

enable the user to modify the image-based results.

12. The auditing device of claim 11, wherein the results analyzer is to modify the image based results by changing an identification of at least one of the first product or the second product, or changing a segment of the segmented image.

13. The auditing device of claim 10, further including a key performance indicator definer to:

determine key performance indicators providing data related to the first and second products;

display the key performance indicators displayed on the display; and

prompt for a second user input to modify at least one of the key performance indicators.

14. The auditing device of claim 13, wherein the key performance indicators include at least one of respective shares of shelf space or respective positions of the first and second products in the segmented image.

15. The auditing device of claim 13, wherein the key performance indicator definer is further to:

determine a location of the first and second products in the segmented image; and

determine respective numbers of facings of the first and second products in the segmented image.

16. The auditing device of claim 10, wherein an image segmentor is further to:

detect a grid in the first region of interest that is empty; and

detect a grid in the first region of interest that is associated with an out-of-stock product.

18. The auditing device of claim 10, wherein the candidate pattern selector is further to select the first set of candidate patterns based on at least one of a store being audited, a product category, or a user performing the audit.

19. The auditing device of claim 10, wherein the candidate pattern selector is further to select the second set of candidate patterns based on at least one of a product category or a store.

17. (canceled)

20. A tangible computer readable storage medium comprising instructions that, when executed, causes a processor of an auditing device to at least:

perform image recognition based on a first set of candidate patterns accessed by the auditing device to identify a first product depicted in a first region of interest of a segmented image;

prompt a user to enter a user input associated with a first grid of the first region of interest displayed on a display of the auditing device, the first grid including the first product; and

determine a second set of candidate patterns to use to identify a second product in a second region of interest of the segmented image, the second set of candidate patterns being determined based on the user input and a group of products identified in the segmented image in a neighborhood of the first region of interest.

21. The machine-readable instructions of claim 20, wherein the instructions, when executed, further cause the processor to:

display, on the display of the auditing device, an image-based result for the first and second products identified in the first and second regions of interest; and

modify the image-based results based on the user input.

23. The machine-readable instructions of claim 20, wherein the instructions, when executed, further cause the processor to:

display, on the display of the auditing device, key performance indicators providing data related to the first and second products;

display a prompt to the user for a second user input while the key performance indicators are displayed on the display of the auditing device; and

modify at least one of the key performance indicators based on the second user input.

22. (canceled)

24-55. -55. (canceled)