Fraudulent activity detection at a barcode scanner by verifying visual signatures

• US 9,870,565 B2
• Filed: 01/07/2015
• Issued: 01/16/2018
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: NCR Voyix Corporation
• Original Assignee: Joshua Migdal, Malay Kundu
• Inventors: Migdal, Joshua, Kundu, Malay
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/591,883
• Publication Number:
  US 20150193780A1
• Time in Patent Office: 1,105 Days
• CPC Class Codes: G06Q 30/018 Certifying business or products

Abstract

System and method for detecting a fraudulent activity at a barcode scanner is disclosed. The method issues an alert when the fraudulent activity is confirmed by comparing the visual signature of the item being transacted over the checkout terminal to the model visual signature. The model visual signature is obtained by averaging the collection of visual signature of the item gathered over a period of time. A human validation via a remote processor is employed to confirm the fraudulent activity verified by a computer.

Claims

1. A method for detecting a fraudulent activity at a checkout terminal, the checkout terminal comprising a computer, the method operated via the computer, comprising the steps of:

detecting an item identifier number of an item by scanning a barcode with a barcode scanner, the barcode being affixed to the item, wherein the barcode scanner is in communication with the computer;

capturing an image of the item with a camera, the camera positioned to take the image of the item, the camera being in communication with the computer, wherein the image comprises the barcode and a surrounding of the barcode;

obtaining a visual signature from the image;

obtaining a model visual signature of the item by creating a probabilistic model of a group of visual signatures collected from the item;

determining a similarity between the visual signature and the model visual signature to verify the fraudulent activity, by comparing the visual signature obtained from the image to the model visual signature associated with the item identifier number, the model visual signature representing an expected visual signature of the item, wherein the model visual signature is stored at a storage unit being accessible to the computer;

resetting the group of visual signatures with a second group of visual signatures, when the similarity indicates the fraudulent activity more than a predetermined number of occasions, wherein the second group of visual signatures is collected from the item; and

obtaining the model visual signature by creating a probabilistic model of the second group of visual signatures.

2. The method of claim 1 wherein the barcode scanner is at least one of an optical-based scanner and a laser-based scanner, the optical-based scanner comprising the camera.

3. The method of claim 1 wherein the step of obtaining a visual signature from the image comprises:

detecting at least one interesting point from the image using an interest operator, the at least one interesting point being located in the surrounding of the barcode.

4. The method of claim 3 wherein the interest operator is selected from the group consisting of:

a corner detector;

an edge detector; and

an area of high texture.

5. The method of claim 1 further comprising the step of:

verifying the similarity against a threshold value, wherein the threshold value sets a maximum allowed difference in the similarity; and

issuing an alert when the similarity exceeds the threshold value.

6. The method of claim 5 wherein the alert is selected from the group consisting of:

displaying an alert message on a display of the checkout terminal;

issuing a visual alarm;

issuing an audible alarm; and

halting an operation of the checkout terminal.

7. The method of claim 1 further comprising the steps of:

obtaining a snippet of video comprising at least one image of the item with the camera;

sending the snippet of video to a remote processor for a human validation of the fraudulent activity; and

receiving the human validation from the remote processor.

8. The method of claim 7 wherein the snippet of video comprises the image captured by the camera.

9. The method of claim 7 wherein the snippet of video is sent to the remote processor based on a confidence score, the confidence score representing a degree of difference between the visual signature and the model visual signature.

10. The method of claim 1 wherein the step of capturing an image of the item comprises:

correcting the image into a canonical view, wherein the canonical view is obtained by applying a transform to compensate for an image distortion of the image.

11. The method of claim 10 wherein the transform is identified by comparing a size and shape of the barcode in the image to a predetermined reference size and shape of the barcode, the predetermined reference size and shape being stored at a storage unit being accessible to the computer.

12. The method of claim 11 wherein the size and shape of the barcode in the image is determined by at least one of:

fitting a bounding box around the barcode;

identifying at least two corner points of the barcode and relations between the at least two corner points; and

calculating a parameter of a plane on which the barcode resides.

13. The method of claim 10 where in the transform is selected from the group consisting of:

scale transform, rotational transform, projective transform, and barrel transform.

14. The method of claim 1 wherein the step of determining a similarity between the visual signature and a model visual signature comprises:

collecting a group of visual signatures from the item; and

creating a probabilistic model of the group of visual signatures to obtain the model visual signature of the item.

15. The method of claim 1 wherein the probabilistic model is at least one of:

a nonparametric probabilistic model; and

an average of the group of visual signatures.

16. The method of claim 1 wherein the group of visual signatures of the item is updated by adding the visual signature obtained from the image to the group of visual signatures.

17. The method of claim 14 wherein the step of determining a similarity between the visual signature and a model visual signature further comprises:

resetting the group of visual signatures when the similarity indicates the fraudulent activity of the item more than a predetermined number of occasions, wherein the group of visual signatures is reset by collecting a second group of visual signatures from the item; and

creating a probabilistic model of the second group of visual signatures to obtain the model visual signature of the item.

18. A checkout system for detecting a fraudulent activity, comprising:

a computer;

a barcode scanner in communication with the computer, the barcode scanner positioned to detect an item identifier number of an item from a barcode, the barcode being affixed to the item;

a camera in communication with the computer, the camera positioned to capture an image of the item, wherein the image comprises the barcode and a surrounding of the barcode; and

the checkout system in communication with a storage unit via a network, wherein the checkout system is configured to;

obtain a visual signature from the image;

obtain a model visual signature of the item by creating a probabilistic model of a group of visual signatures collected from the item;

determine a similarity between the visual signature and the model visual signature to verify the fraudulent activity, by comparing the visual signature obtained from the image to the model visual signature associated with the item identifier number, the model visual signature representing an expected visual signature of the item, wherein the model visual signature is stored at the storage unit;

reset the group of visual signatures with a second group of visual signatures, when the similarity indicates the fraudulent activity more than a predetermined number of occasions, wherein the second group of visual signatures is collected from the item; and

obtain the model visual signature by creating a probabilistic model of the second group of visual signatures.

19. The checkout system of claim 18 wherein the barcode scanner is at least one of an optical-based scanner and a laser-based scanner, the optical-based scanner comprising the camera.

20. The checkout system of claim 18 further comprising an alert system, wherein the alert system issues an alert when the similarity exceeds a threshold value, wherein the threshold value sets a maximum allowed difference in the similarity.

21. The checkout system of claim 20 wherein the alert is selected from the group consisting of:

displaying an alert message on a display of the checkout terminal;

issuing a visual alarm;

issuing an audible alarm; and

halting an operation of the checkout terminal.

22. The checkout system of claim 18 further comprising a remote processor in communication with the computer via the network, wherein the remote processor is configured to:

receive a snippet of video obtained by the camera for a human validation of the fraudulent activity, the snippet of video comprising at least one image of the item; and

send the human validation to the computer.

23. The checkout system of claim 18 wherein the checkout system is configured to obtain the visual signature from the image by detecting at least one interesting point from the image using an interest operator, the at least one interesting point being located in the surrounding of the barcode.

24. The checkout system of claim 18 wherein the probabilistic model is at least one of:

a nonparametric probabilistic model; and

an average of the group of visual signatures.