IMAGE RECOGNITION TO SUPPORT SHELF AUDITING FOR CONSUMER RESEARCH

• US 20130051667A1
• Filed: 08/31/2011
• Published: 02/28/2013
• Est. Priority Date: 08/31/2011
• Status: Active Grant

• Current Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Original Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Inventors: Deng, Kevin Keqiang, Xie, Wei
• Granted Patent: US 8,908,903 B2
• Time in Patent Office: Days
• US Class Current: 382/168
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 20/36 Indoor scenes

Abstract

Image recognition methods, apparatus and articles or manufacture to support shelf auditing for consumer research are disclosed herein. An example method disclosed herein comprises comparing an input image depicting a shelf to be audited with a set of reference images depicting reference shelves displaying respective sets of reference items, identifying a first reference image from the set of reference images that has been determined to match the input image, and determining an initial audit result for the shelf depicted in the input image based on a first set of reference items associated with the first reference image.

Claims

1. A method for shelf auditing, the method comprising:

comparing, using a processor, an input image depicting a shelf to be audited with a set of reference images depicting reference shelves displaying respective sets of reference items;

identifying a first reference image from the set of reference images that has been determined to match the input image; and

determining, using the processor, an initial audit result for the shelf depicted in the input image based on a first set of reference items associated with the first reference image.

2. A method as defined in claim 1 wherein determining the initial audit result for the shelf depicted in the input image is performed if a comparison of the input image and the first reference image satisfies a matching criterion, and further comprising, if the comparison does not satisfy the matching criterion, indicating that the shelf is a new shelf for which determination of the initial audit result was unsuccessful.

3. A method as defined in claim 1 wherein the set of reference images is associated with a respective set of reference image signatures, and further comprising:

determining an input image signature for the input image;

comparing the input image signature to the set of reference image signatures; and

determining that the first reference image matches the input image when a first reference image signature associated with the first reference image is determined to match the input image signature.

4. A method as defined in claim 3 wherein the input image signature comprises an input image histogram, and the set of reference image signatures comprises a set of reference image histograms.

5. A method as defined in claim 3 the first reference image signature is determined to match the input image signature when the first reference image signature and the input image signature have at least one of a minimum distance or a maximum correlation value relative to the other reference image signatures in the set of reference image signatures

6. A method as defined in claim 1 further comprising:

determining a first region of the input image that differs from a corresponding first region of the first reference image; and

modifying the initial audit result for the shelf depicted in the input image based on an item depicted in the first region of the input image.

7. A method as defined in claim 6 wherein modifying the initial audit result comprises identifying the item depicted in the first region of the input image and registering the item as being associated with the shelf depicted in the input image.

8. A method as defined in claim 6 further comprising:

determining a second region of the input image that differs from a corresponding second region of the first reference image; and

further modifying the initial audit result for the shelf depicted in the input image based on a second item depicted in the second region of the input image.

9. A method as defined in claim 6 wherein determining the first region of the input image that differs from the corresponding first region of the first reference image comprises:

determining a difference image representing differences between pixels of the input image and respective pixels of the first reference image; and

determining the first region of the input image to correspond to a first region of the difference image in which a neighborhood of pixels meets or exceeds a difference threshold.

10. A method as defined in claim 6 further comprising processing the first region of the input image with a template representative of the item to identify another instance of the item in the first region of the input image.

11. A tangible article of manufacture storing machine readable instructions which, when executed, cause a machine to at least:

compare an input image depicting a shelf to be audited with a set of reference images depicting reference shelves displaying respective sets of reference items;

identify a first reference image from the set of reference images that has been determined to match the input image; and

determine an initial audit result for the shelf depicted in the input image based on a first set of reference items associated with the first reference image.

12. A tangible article of manufacture as defined in claim 11 wherein the set of reference images is associated with a respective set of reference image signatures, and the machine readable instructions, when executed, further cause the machine to:

determine an input image signature for the input image; and

compare the input image signature to the set of reference image signatures; and

determining that the first reference image matches the input image when a first reference image signature associated with the first reference image is determined to match the input image signature.

13. A tangible article of manufacture as defined in claim 11 wherein the machine readable instructions, when executed, further cause the machine to:

determine a first region of the input image that differs from a corresponding first region of the first reference image; and

modify the initial audit result for the shelf depicted in the input image based on an item depicted in the first region of the input image.

14. A tangible article of manufacture as defined in claim 13 wherein to modify the initial audit result, the machine readable instructions, when executed, cause the machine to identify the item depicted in the first region of the input image and register the item as being associated with the shelf depicted in the input image.

15. A tangible article of manufacture as defined in claim 11 wherein the machine readable instructions, when executed, further cause the machine to:

determine a difference image representing differences between pixels of the input image and respective pixels of the first reference image; and

determine the first region of the input image to correspond to a first region of the difference image in which a neighborhood of pixels meets or exceeds a difference threshold.

16. An apparatus to perform shelf auditing, the apparatus comprising:

a reference database to store a set of reference images depicting reference shelves displaying respective sets of reference items, the set of reference images and the respective sets of reference items associated with the respective reference images being obtained from prior audits of the reference shelves;

a reference image identifier to;

compare an input image depicting a shelf to be audited with the set of reference images; and

identify a first reference image from the set of reference images that has been determined to match the input image and that satisfies a matching criterion; and

an image auditor to determine an initial audit result for the shelf depicted in the input image based on a first set of reference items associated with the first reference image, the initial audit result to indicate that the first set of reference items is displayed by the shelf depicted in the input image.

17. An apparatus as defined in claim 16 wherein the reference database is also to store a set of reference image signatures associated respectively with the set of reference images, and the reference image identifier comprises:

a signature generator to generate an input image signature for the input image; and

a signature comparator to;

compare the input image signature to the set of reference image signatures; and

determine that the first reference image matches the input image when a first reference image signature associated with the first reference image is determined to match the input image signature.

18. An apparatus as defined in claim 16 further comprising a difference determiner to determining a first region of the input image that differs from a corresponding first region of the first reference image, and wherein the image auditor is to modify the initial audit result for the shelf depicted in the input image based on an item depicted in the first region of the input image.

19. An apparatus as defined in claim 18 wherein to modify the initial audit result, the image auditor is to identify the item depicted in the first region of the input image and register the item as being associated with the shelf depicted in the input image.

20. An apparatus as defined in claim 18 wherein the difference determiner is to:

determine a difference image representing differences between pixels of the input image and respective pixels of the first reference image; and

determining the first region of the input image to correspond to a first region of the difference image in which a neighborhood of pixels meets or exceeds a difference threshold.