Method and apparatus for image processing to avoid counting shelf edge promotional labels when counting product labels

• US 9,697,429 B2
• Filed: 07/15/2016
• Issued: 07/04/2017
• Est. Priority Date: 06/12/2013
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Patel, Ankur R., Super, Boaz J.
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US15/211,103
• Publication Number:
  US 20160328618A1
• Time in Patent Office: 354 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/23211 with adaptive number of clusters
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An example disclosed method includes receiving an image containing shelf edge content displayed on a shelf edge; determining a location of the shelf edge content and an apparent orientation of the shelf edge content; determining, based on the location of the shelf edge content and the apparent orientation of the shelf edge content, a region in the image corresponding to the shelf edge; and providing an output based on the image that identifies the region in the image corresponding to the shelf edge.

Claims

1. A method for processing images, comprising:

receiving, at a shelf edge detection component having a processor, an image containing shelf edge content;

determining, by the processor, locations of instances of the shelf edge content by identifying interest point matches between the image and a reference image;

determining, by the processor, regions in the image corresponding to shelf edges by;

identifying a geometric pattern in the image based on the interest point matches; and

forming clusters of instances of the geometric pattern; and

estimating, by the processor, a number of shelf edges in the image based on the clusters.

2. The method of claim 1, wherein identifying the geometric pattern comprises applying a geometric model fitting and verification algorithm to the interest point matches.

3. The method of claim 1, further comprising fitting an upper and a lower line for each of the clusters, wherein the upper and lower lines correspond to upper and lower boundaries of each of the shelf edges, and wherein a space between each upper and lower line pair is the region identified as a shelf edge.

4. The method of claim 3, wherein fitting the upper and lower lines comprises splitting corners associated with a cluster into first and second sets, with the first set corresponding to the upper boundary and the second set corresponding to the lower boundary.

5. The method of claim 4, wherein splitting the corners is based on point ordering of each geometric pattern, where first and second points of each geometric pattern are grouped in the first set and third and fourth points of each geometric pattern are grouped in the second set.

6. The method of claim 5, where the point ordering comprises numbering first, second, third, and fourth corners of each geometric pattern in a clockwise order.

7. The method of claim 1, wherein forming the clusters comprises selecting a first kernel bandwidth in a horizontal direction of the image that is larger than a second kernel bandwidth in a vertical direction of the image.

8. The method of claim 1, wherein forming the clusters comprises associating corners of instances of the geometric pattern based on minimizing a distance measure between vertical co-ordinates of the corners and centers of the clusters.

9. The method of claim 1, further comprising providing an output by at least one of obscuring the regions or providing metadata that identifies the regions.

10. The method of claim 1, further comprising providing output by providing the output to a product label recognition engine that counts product labels of product items in the image.

11. The method of claim 1, wherein reference image contains an image of a known shelf edge promotional item.

12. A non-transitory computer-readable storage device comprising machine-readable instructions that, when executed, cause a machine to at least:

receive an image containing shelf edge content;

determine locations of instances of the shelf edge content by identifying interest point matches between the image and a reference image;

determine regions in the image corresponding to shelf edges by;

identifying a geometric pattern in the image based on the interest point matches; and

forming clusters of instances of the geometric pattern; and

estimate a number of shelf edges in the image based on the clusters.

13. The non-transitory computer-readable storage device of claim 12, wherein the instructions, when executed, cause the machine to identify the geometric pattern by applying a geometric model fitting and verification algorithm to the interest point matches.

14. The non-transitory computer-readable storage device of claim 12, wherein the instructions, when executed, cause the machine to fit an upper and a lower line for each of the clusters, wherein the upper and lower lines correspond to upper and lower boundaries of each of the shelf edges, and wherein a space between each upper and lower line pair is the region identified as a shelf edge.

15. The non-transitory computer-readable storage device of claim 14, wherein the instructions, when executed, cause the machine to fit the upper and lower lines by splitting corners associated with a cluster into first and second sets, with the first set corresponding to the upper boundary and the second set corresponding to the lower boundary.

16. The non-transitory computer-readable storage device of claim 12, wherein forming the clusters comprises selecting a first kernel bandwidth in a horizontal direction of the image that is larger than a second kernel bandwidth in a vertical direction of the image.

17. The non-transitory computer-readable storage of claim 12, wherein forming the clusters comprises associating corners of instances of the geometric patter based on minimizing a distance measure between vertical co-ordinates of the corners and centers of the clusters.

18. An apparatus, comprising:

memory including machine-readable instructions; and

a processor to execute the machine-readable instructions to perform operations including;

receiving an image containing shelf edge content;

determining locations of instances of the shelf edge content by identifying interest point matches between the image and a reference image;

determining regions in the image corresponding to shelf edges by;

identifying a geometric pattern in the image based on the interest point matches; and

forming clusters of instances of the geometric pattern; and

estimate a number of shelf edges in the image based on the clusters.