Identifying consumer products in images

• US 10,136,103 B2
• Filed: 11/23/2015
• Issued: 11/20/2018
• Est. Priority Date: 11/23/2015
• Status: Active Grant

• Current Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Original Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Inventors: Booth, Robert Reed
• Primary Examiner(s): Summers, Geoffrey E
• Application Number:
  US14/948,489
• Publication Number:
  US 20170147903A1
• Time in Patent Office: 1,093 Days
• Field of Search: 382103, 382181, 382190, 382195
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods identify consumer products in images. Known consumer products are captured as grayscale or color images. They are converted to binary at varying thresholds. Connected components in the binary images identify image features according to pixels of a predetermined size, shape, solidity, aspect ratio, and the like. The image features are stored and searched for amongst image features similarly extracted from unknown images of consumer products. Identifying correspondence between the features of the images lends itself to identifying or not known consumer products.

Claims

1. A method for identifying consumer products in images, comprising:

capturing first images of known consumer products;

converting each of the first images to a plurality of binary first images of varying binary thresholds;

identifying a first set of image features from the plurality of binary first images;

receiving second images of unknown consumer products;

converting each of the second images to a plurality of binary second images of the varying binary thresholds;

identifying a second set of image features from the plurality of binary second images;

finding a subset of image features from each of the first set of image features and the second set of image features that substantially match;

devising triangles between the subset of image features in said each of the plurality of binary first images and of the plurality of binary second images; and

determining if the triangles are similar to identify or not possible matches of the known consumer products within the second images, thus identifying or not known consumer products at a location from whence the second images were captured,

wherein each image feature is a set of connected points having a centroid and wherein each centroid is used in devising the triangles, wherein each of the first and second images have pluralities of pixels defined in bits and the subset of image features reside in some pixels but not all of the pluralities of pixels, wherein the finding of the subset of image features further includes XORing the bits of two of the subset of image features one each from the first and second images and counting the numbers of 1 bits, wherein a match between the two of the subset of image features exists if the number of the 1 bits does not exceed ten percent of the total number of the bits in the XORing operation.

2. The method of claim 1, further including determining line lengths between each of the centroids of the image features in the subset of image features.

3. The method of claim 2, further including comparing the line lengths between said each of the first images and the second images.

4. The method of claim 1, further including restricting a pixel distance between the subset of image features during the finding of the subset of image features.

5. The method of claim 1, further including determining whether the subset of image features reside in a corresponding portion of the first and second images.

6. The method of claim 1, further including determining whether the subset of image features has a corresponding orientation in the first and second images.

7. The method of claim 1, further including determining whether the subset of image features correspond mostly to black or white image features.

8. The method of claim 1, further including determining an aspect ratio in pixels for each image feature of the subset of image features in the first and second images.

9. A method for identifying consumer products in images, comprising:

storing first images of known consumer products;

converting each of the first images to a plurality of binary first images of varying binary thresholds;

identifying a first set of image features from the plurality of binary first images that meet a predefined set of criteria;

receiving second images of unknown consumer products;

converting each of the second images to a plurality of binary second images of the varying binary thresholds;

identifying a second set of image features from the plurality of binary second images that meet the predefined set of criteria;

finding three image features from each of the plurality of binary first and second images that correspond to one another;

devising triangles between the three image features in said each of the plurality of binary first and the second images; and

comparing the triangles for similarity to identify or not possible matches of the known consumer products within the second images, thus identifying or not known consumer products at a location from whence the second images were captured,

wherein each image feature of the three image features is a set of connected points having a centroid and wherein each triangle during the devising the triangles is devised by using a distance between a first centroid of a first image feature and a second centroid of a second image feature, wherein each of the first and second images have pluralities of pixels defined in bits and the three image features reside in some pixels but not all of the pluralities of pixels, wherein the finding of the three image features corresponding to one another further includes XORing the bits of two of the three image features one each from the first and second images and counting the numbers of 1 bits, wherein a match between the two of the three image features exists if the number of the 1 bits does not exceed ten percent of the total number of the bits in the XORing operation.

10. The method of claim 9, further including determining line lengths between each of the centroids of the three image features.

11. The method of claim 10, further including comparing the line lengths between said each of the first and second images.

12. The method of claim 9, further including restricting a pixel distance between the three image features during the finding of the three image features.

13. A method for identifying consumer products in images, comprising:

receiving first images of known consumer products;

converting each of the first images to a plurality of binary first images of varying binary thresholds;

identifying a first set of image features from the plurality of binary first images that meet a predefined set of criteria;

receiving second images of unknown consumer products;

converting each of the second images to a plurality of binary second images of the varying binary thresholds;

identifying a second set of image features from the plurality of binary second images that meet the predefined set of criteria;

finding three image features in each of the first and the second images that correspond to one another;

devising triangles between the three image features in said each of the first and the second images; and

determining if the triangles are similar to identify or not possible matches of the known consumer products within the second images, thus identifying or not known consumer products at a location from whence the second images were captured,

wherein each image feature of the three image features is a set of connected binary pixels having a centroid pixel defining a center of a shape of each image feature and wherein each centroid pixel is used during the devising of the triangles, wherein each of the first and second images have pluralities of pixels defined in bits and the three image features reside in some pixels but not all of the pluralities of pixels, wherein the finding of the three image features corresponding to one another further includes XORing the bits of two of the three image features one each from the first and second images and counting the numbers of 1 bits, wherein a match between the two of the three image features exists if the number of the 1 bits does not exceed ten percent of the total number of the bits in the XORing operation.

14. The method of claim 13, wherein the identifying the first and the second set of image features from the binary first and second images, respectively, includes identifying connected components in the binary first and binary second images, respectively.

15. The method of claim 14, further including filtering the connected components to exclude from the three image features in said each of the first and second images the connected components not meeting a predefined size, a predefined aspect ratio, a predefined shape or a predefined limit of solidity.

16. The method of claim 14, further including standardizing a size of each image feature of the three image features in said each of the first images and the second images.

17. The method of claim 14, further including storing the first images of the known consumer products.