Extracting card data with wear patterns

• US 9,235,771 B2
• Filed: 11/26/2013
• Issued: 01/12/2016
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rowley, Henry Allan, Kumar, Sanjiv, Wang, Xiaohang, Bissacco, Alessandro, Rodrigues, Jose Jeronimo Moreira, Papineni, Kishore Ananda
• Primary Examiner(s): Park, Soo
• Application Number:
  US14/091,093
• Publication Number:
  US 20150003667A1
• Time in Patent Office: 777 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06Q 20/227 characterised in that multiple accounts are available, e.g. to the payer
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/34 using cards, e.g. integrated circuit [IC] cards or magnetic cards
• CPC Class Codes: G06Q 20/367 involving electronic purses or money safes
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2253 Recognition of characters printed with magnetic ink G06V30/2247 takes precedence
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G07F 7/0893 the card reader reading the card in a contactless manner

Abstract

Embodiments herein provide computer-implemented techniques for allowing a user computing device to extract financial card information using optical character recognition (“OCR”). Extracting financial card information may be improved by applying various classifiers and other transformations to the image data. For example, applying a linear classifier to the image to determine digit locations before applying the OCR algorithm allows the user computing device to use less processing capacity to extract accurate card data. The OCR application may train a classifier to use the wear patterns of a card to improve OCR algorithm performance. The OCR application may apply a linear classifier and then a nonlinear classifier to improve the performance and the accuracy of the OCR algorithm. The OCR application uses the known digit patterns used by typical credit and debit cards to improve the accuracy of the OCR algorithm.

Claims

1. A computer-implemented method for training systems to recognize wear patterns on cards, comprising:

receiving, by one or more computing devices, an image of a card from a camera, the card having digits thereon that are worn according to a wear pattern;

performing, by the one or more computing devices, a classification algorithm the image;

providing for display, by the one or more computing devices, results of the performance of the classification algorithm along with a revision indication control to provide revisions of the results of the performance of the classification algorithm;

receiving, by the one or more computing devices, a revision of the results of the performance of the classification algorithm, wherein the revision comprises an identification of a wear pattern of the digits and the revision of the results was input using the revision indication control;

categorizing, by the one or more computing devices, the wear pattern of the digits indicated by the revision; and

creating, by the one or more computing devices, a wear pattern data transformation for a category of the wear pattern based at least in part on the revised results.

2. The method of claim 1, wherein the image of the card is simulated.

3. The method of claim 1, wherein the revisions are provided by an operator of the one or more computing devices.

4. The method of claim 1, wherein an application of the wear pattern data transformation to a classification of a subsequent card corrects errors caused by wear patterns on the subsequent card.

5. A computer-implemented method for using wear pattern transformations, comprising:

receiving, by one or more computing devices, an image of a card from a camera;

performing, by the one or more computing devices, an optical character recognition algorithm on the image;

determining, by the one or more computing devices, a first confidence level of a result of the application of the optical character recognition algorithm to the data;

applying, by the one or more computing devices, a first wear pattern data transformation to the image;

performing, by the one or more computing devices, the optical character recognition algorithm to the transformed image;

determining, by the one or more computing devices, a second confidence level of a result of the application of the optical character recognition algorithm to the transformed image;

determining that the second confidence level is higher than the first confidence level; and

applying, by the one or more computing devices, the first wear pattern data transformation to an image of a second physical card based upon a determination that the second confidence level is higher than the first confidence level.

6. The method of claim 5, wherein the first wear pattern data transformation is a support vector machine.

7. The method of claim 5, wherein the first wear pattern transformation is applied to any subsequent card image associated with an account of a user associated with the physical card.

8. The method of claim 5, further comprising:

applying, by the one or more computing devices, additional wear pattern data transformations to the image;

determining, by the one or more computing devices, a confidence level of a result of the applications of the optical character recognition algorithm to the data after an application of each of the additional wear pattern data transformations;

determining, by the one or more computing devices, the wear pattern data transformation that is associated with a highest confidence level; and

applying, by the one or more computing devices, the wear pattern data transformation that is associated with the highest confidence level to an image of a second card.

9. The method of claim 5, wherein the card is a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.

10. A computer program product, comprising:

a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by a computer cause the computer to extract financial card information with relaxed alignment, comprising;

computer-executable program instructions to receive an image of a card from a camera;

computer-executable program instructions to perform an optical character recognition algorithm on the image;

computer-executable program instructions to determine a first confidence level of a result of the application of the optical character recognition algorithm to the data;

computer-executable program instructions to apply a first wear pattern data transformation to the image;

computer-executable program instructions to perform the optical character recognition algorithm to the transformed image;

computer-executable program instructions to determine a second confidence level of a result of the application of the optical character recognition algorithm to the transformed image;

computer-executable program instructions to determine that the second confidence level is higher than the first confidence level; and

computer-executable program instructions to select the result of the application of the optical character recognition algorithm to the transformed image based at least in part on the determination that the second confidence level is higher than the first confidence level.

11. The computer program product of claim 10, wherein the first wear pattern data transformation is a support vector machine.

12. The computer program product of claim 10, wherein the first wear pattern transformation is applied to any subsequent card image associated with an account of a user associated with the physical card.

13. The computer program product of claim 10, the computer-executable program instructions further comprising:

computer program instructions to apply additional wear pattern data transformations to the image;

computer program instructions to determine a confidence level of a result of the applications of the optical character recognition algorithm to the data after an application of each of the additional wear pattern data transformations;

computer program instructions to determine the wear pattern data transformation that is associated with a highest confidence level; and

computer program instructions to apply the wear pattern data transformation that is associated with the highest confidence level to an image of a second card.

14. The computer program product of claim 10, wherein the card is a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.

15. A system for training systems to recognize wear patterns on cards, the system comprising:

a storage device;

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to;

receive an image of a card from a camera, the card having digits thereon that are worn according to a wear pattern;

perform a linear classification algorithm on the image;

provide for display results of the performance of the linear classification algorithm along with a revision indication control to provide revisions of the results of the performance of the linear classification algorithm;

receive a revision of the results of the performance of the linear classification algorithm, wherein the revision comprises an identification of a wear pattern of the digits and the revision of the results was input using the revision indication control;

categorize the wear pattern of the digits indicated by the revision; and

create a wear pattern data transformation for a category of the wear pattern based at least in part on the revised results.

16. The system of claim 15, wherein the image of the card is simulated.

17. The system of claim 15, wherein the revisions are provided by an operator of the one or more computing devices.

18. The system of claim 15, wherein an application of the wear pattern data transformation to a classification of a subsequent card corrects errors caused by wear patterns on the subsequent card.

19. The system of claim 15, the processor executing further application code instructions that are stored in the storage device and that cause the system to:

receive a subsequent image of a subsequent card from a camera;

perform an optical character recognition algorithm on the subsequent image;

determine a first confidence level of a result of the application of the optical character recognition algorithm to the data;

apply a first wear pattern data transformation to the subsequent image;

perform the optical character recognition algorithm to the transformed subsequent image;

determine a second confidence level of a result of the application of the optical character recognition algorithm to the transformed subsequent image;

determine that the second confidence level is higher than the first confidence level; and

select the result of the application of the optical character recognition algorithm to the transformed subsequent image.

20. The system of claim 15, wherein the card is a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.