UPC substitution fraud prevention

• US 8,068,674 B2
• Filed: 09/04/2007
• Issued: 11/29/2011
• Est. Priority Date: 09/04/2007
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Evolution Robotics Retail Incorporated (Datalogic ADC Incorporated)
• Inventors: Goncalves, Luis
• Primary Examiner(s): Repko; Jason M
• Assistant Examiner(s): Thomas; Mia M
• Application Number:
  US11/849,503
• Publication Number:
  US 20090060259A1
• Time in Patent Office: 1,547 Days
• Field of Search: 382100-103, 382/113, 382/162, 382/167, 382/190, 382/192, 382/194, 382/183, 382/212, 382/219, 382/209, 382/312, 382/313, 382/321, 382/276, 235/462.01, 235/383, 713/176
• US Class Current: 382/194
• CPC Class Codes: A47F 9/047 for recording self-service articles without cashier or assistant
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G07G 1/0045 with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G07G 1/12 electronically operated digital data processing aspects G06Q20/00

Abstract

A system and method for detecting fraudulent identification tags, such as Universal Product Codes (UPC) applied to goods to be purchased is disclosed. Images of the goods to be purchased and corresponding feature models are stored in a database. When a customer desires to purchase an item containing a UPC, a scanned image of the item about to be purchased may be acquired and a feature model of the scanned image may be created. The system may retrieve from the database the image and feature model previously stored for the item associated with the just-scanned UPC. A variety of image processing techniques may be used to compare the scanned and database images and/or the scanned and database feature models. In one embodiment, these image processing techniques may include determining a geometric transformation that maps the features of the scanned image onto the features of a database model.

Claims

1. A method of confirming the identity of an item including an identifier, the method comprising:

acquiring an image of the item including the identifier;

extracting a first set of geometric point features from the image of the item;

reading identification code data from the identifier included with the item;

retrieving from a database a second set of geometric point features based on the identification code data, the database comprising sets of geometric point features associated with a plurality of known items;

comparing via a processor the first set of geometric point features from the image with the second set of geometric point features retrieved based on the identification code data; and

determining whether the item in the image corresponds to the identification code data read from the identifier included with the item based on the comparing of the first set of geometric point features with the second set of geometric point features.

2. The method of claim 1, wherein the identifier is a universal product code (UPC) barcode or a radio-frequency identification (RFID) tag.

3. The method of claim 1, wherein the first set of geometric point features and second set of geometric point features are scale-invariant feature transform (SIFT) features.

4. The method of claim 1, wherein the second set of geometric point features corresponds to a feature model of one of the plurality of known items in the database.

5. The method of claim 1, wherein the step of acquiring the image of the item is performed in a point of sale system.

6. The method of claim 1, wherein the step of comparing the first set of geometric point features with the second set of geometric point features comprises:

generating a geometric transform for mapping one or more of the geometric point features of the first set of geometric point features to one or more of the geometric point features of the second set of geometric point features.

7. The method of claim 6, wherein generating the geometric transform minimizes an error associated with the mapping of said one or more of the geometric point features of the first set of geometric point features to said one or more of the geometric point features of the second set of geometric point features.

8. The method of claim 7, wherein the geometric transform is an affine transform or homography.

9. The method of claim 1, wherein the step of comparing further comprises:

generating an image correlation between the acquired image of the item and an image of at least one of the plurality of known items.

10. The method of claim 9, wherein the step of generating an image correlation comprises:

determining an overlapping region between the acquired image of the item and the image of at least one of the plurality of known items; and

determining a correlation value for the overlapping region.

11. A system for confirming the identity of an item including an identification code, the system comprising:

an optical sensor for capturing an image of the item including the identification code;

a feature extractor coupled to the optical sensor for extracting geometric point features from the image;

a database comprising a plurality of feature models, each feature model comprising one or more geometric point features, and each feature model associated with at least one of a plurality of known items; and

a processor configured to;

a) compare the extracted geometric point features to the geometric point features of a feature model stored in the database and selected based on the identification code included with the item; and

b) determine whether the item in the image corresponds to the identification code included with the item based on comparison of the extracted geometric point features to the geometric point features of the feature model selected based on the identification code.

12. The system of claim 11, wherein the optical sensor further comprises a code scanner for scanning the identification code included with the item.

13. The system of claim 12, wherein the code scanner comprises a laser incorporated in the optical sensor.

14. The system of claim 12, wherein the optical sensor comprises a two-dimensional imager.

15. The system of claim 14, wherein the code scanner comprises a processor configured to read the identification code based on the image of the item captured by the two-dimensional imager.

16. The system of claim 12, wherein the code scanner is substantially fixed in a horizontal or vertical position, whereby the identification code is read when the item is passed by the code scanner.

17. The system of claim 12, wherein the code scanner is a handheld scanner adapted to be pointed at or waived over the identification code.

18. The system of claim 17, wherein the handheld scanner comprises a distance indicator configured to signal to a user whether to move the handheld scanner closer to the item or further away from the item.

19. The system of claim 18, wherein the handheld scanner is configured to signal the user to move the handheld scanner away from the item if only a portion of the item is in a field of view of the optical sensor.

20. The system of claim 11, wherein the extracted geometric point features are scale-invariant feature transform (SIFT) features.

21. The system of claim 20, wherein the feature extractor is further configured to:

generate Difference-of-Gaussians images from the image of the item;

locate pixel amplitude extrema in the Difference-of-Gaussians images;

assign an orientation to each of the extracted geometric point features at each extremum; and

generate a feature descriptor for each of the extracted geometric point features at the assigned orientation.

22. A method of preventing fraudulent purchases of products including identification codes, comprising:

scanning an identification code included with a product using a point of sale system;

acquiring an image of the product;

generating geometric feature data from the acquired image of the product;

retrieving feature data associated with merchandise based on the scanned identification code;

identifying via a processor matching features between the generated geometric feature data and the retrieved feature data, wherein a number of matching features is identified; and

generating an alert if the number of identified matching features is below a predetermined threshold.

23. The method of claim 22, wherein the step of identifying matching features comprises:

comparing scale-invariant feature transformation vectors associated with the generated geometric feature data with scale-invariant transformation vectors associated with the retrieved feature data;

identifying pairs of features with a minimal difference in scale-invariant feature transformation vectors;

determining if the minimal difference in scale-invariant feature transformation vectors is below a second threshold; and

selecting pairs of features as matching pairs if the minimal difference in scale-invariant feature transformation vectors is below the second threshold.

25. The method of claim 22, wherein the generated geometric feature data correspond to generated geometric features having acquired image coordinates and the retrieved feature data correspond to stored geometric features of one or more stored images of a known product of the merchandise, the stored geometric features having stored image coordinates, and wherein each of the matching features corresponds to a pair of a generated geometric feature and an associated stored geometric feature that match, the method further comprising:

generating a geometric transform for ones of the pairs of the generated geometric features and the associated stored geometric features, the geometric transform for implementing a mapping between the acquired image coordinates of the generated geometric features and the stored image coordinates of the associated stored geometric features, and the geometric transform providing a residual projection error of the mapping;

comparing the residual projection error of the mapping to an error threshold to determine an accuracy of the geometric transform;

if the residual projection error is less than the error threshold, computing an image correlation value corresponding to a correlation between the acquired image of the product and said one or more stored images of the known product of the merchandise;

comparing the image correlation value to a correlation threshold; and

generating the alert if the image correlation value is below the correlation threshold.

24. A method of verifying the identity of an item including an identifier, the method comprising:

acquiring an image of the item including the identifier;

reading identification code data from the identifier included with the item;

extracting a first set of geometric point features from the image of the item;

comparing via a processor the first set of geometric point features from the image with geometric point features from a database, the geometric point features from the database being associated with a plurality of known items;

retrieving identification code data from the database based on a match between the first set of geometric point features and geometric point features from the database that are associated with one of the plurality of known items; and

determining whether the identification code data read from the identifier included with the item matches the identification code data retrieved from the feature database.