Hierarchical classification in credit card data extraction

• US 9,213,907 B2
• Filed: 10/21/2013
• Issued: 12/15/2015
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kumar, Sanjiv, Rowley, Henry Allan, Wang, Xiaohang, Rodrigues, Jose Jeronimo Moreira
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US14/059,071
• Publication Number:
  US 20150003748A1
• Time in Patent Office: 785 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06Q 20/227 characterised in that multiple accounts are available, e.g. to the payer
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/34 using cards, e.g. integrated circuit [IC] cards or magnetic cards
• CPC Class Codes: G06Q 20/367 involving electronic purses or money safes
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2253 Recognition of characters printed with magnetic ink G06V30/2247 takes precedence
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G07F 7/0893 the card reader reading the card in a contactless manner

Abstract

Embodiments herein provide computer-implemented techniques for allowing a user computing device to extract financial card information using optical character recognition (“OCR”). Extracting financial card information may be improved by applying various classifiers and other transformations to the image data. For example, applying a linear classifier to the image to determine digit locations before applying the OCR algorithm allows the user computing device to use less processing capacity to extract accurate card data. The OCR application may train a classifier to use the wear patterns of a card to improve OCR algorithm performance. The OCR application may apply a linear classifier and then a nonlinear classifier to improve the performance and the accuracy of the OCR algorithm. The OCR application uses the known digit patterns used by typical credit and debit cards to improve the accuracy of the OCR algorithm.

Claims

1. A computer-implemented method to extract card information, comprising:

receiving, by one or more computing devices, an image of a card from a camera;

identifying, by the one or more computing devices, a first area of the image, the first area being selected as a potential location of a digit on the card in the image and of a size that will encompass not more than a single complete digit, the potential location and the size of the first area being identified from a comparison of the image to a database of card layouts stored on the one or more computing devices;

performing, by the one or more computing devices, a linear classification algorithm on data encompassed by the first area;

determining, by the one or more computing devices, a confidence level of a first result of the application of the linear classification algorithm to the first area, wherein the confidence level of the first result indicates a likelihood that the first area encompasses the single complete digit;

determining, by the one or more computing devices, that the first area does not encompasses a single complete digit upon determining that the confidence level of the first result is under a configured threshold;

identifying, by the one or more computing devices, a second area of the image, the second area being in a different location from the first area and of a size that will encompass not more than a single complete digit;

performing, by the one or more computing devices, a linear classification algorithm on data encompassed by the second area;

determining, by the one or more computing devices, a confidence level of a second result of the application of the linear classification algorithm to the second area indicating that the second area encompasses a single complete digit, wherein the confidence level of the second result indicates a likelihood that the second area encompasses the single complete digit;

determining, by the one or more computing devices, that the second area encompasses the single complete digit upon determining that the confidence level of the second result is over a configured threshold; and

performing, by the one or more computing devices, an optical character recognition algorithm on the second area upon a determination that the second area encompasses the single complete digit.

2. The method of claim 1, wherein the second area is shifted horizontally from the first area.

3. The method of claim 1, wherein the second area is shifted vertically from the first area.

4. The method of claim 1, wherein the linear classification algorithm is a support vector machine.

5. The method of claim 1, wherein the card comprises one or more of a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.

6. A computer program product, comprising:

a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by a computer cause the computer to extract card information, the computer-executable program instructions comprising;

computer-executable program instructions to receive an image of a card from a camera;

computer-executable program instructions to identify a first area of the image, the first area being selected as a potential location of a digit on the card in the image and of a size that will encompass not more than a single complete digit, the potential location and the size of the first area being identified from a comparison of the image to a database of card layouts stored on the non-transitory computer-readable storage device;

computer-executable program instructions to perform a linear classification algorithm on data encompassed by the first area;

computer-executable program instructions to determine a confidence level of a first result of the application of the linear classification algorithm to the first area wherein the confidence level of the first result indicates a likelihood that the first area encompasses the single complete digit;

computer-executable program instructions to determine that the first area does not encompasses a single complete digit upon determining that the confidence level of the first result is under a configured threshold;

computer-executable program instructions to identify a second area of the image, the second area being in a different location from the first area and of a size that will encompass not more than a single complete digit;

computer-executable program instructions to perform a linear classification algorithm on data encompassed by the second area;

computer-executable program instructions to determine a confidence level of a second result of the application of the linear classification algorithm to the second area indicating that the second area encompasses a single complete digit, wherein the confidence level of the second result indicates a likelihood that the second area encompasses the single complete digit;

computer-executable program instructions to determine that the second area encompasses the single complete digit upon determining that the confidence level of the second result is over a configured threshold; and

computer-executable program instructions to perform an optical character recognition algorithm on the second area upon a determination that the second area encompasses the single complete digit.

7. The computer program product of claim 6, wherein the second area is shifted horizontally from the first area.

8. The computer program product of claim 6, wherein the second area is shifted vertically from the first area.

9. The computer program product of claim 6, wherein the linear classification algorithm is a support vector machine.

10. The computer program product of claim 6, wherein the card comprises one or more of a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.

11. A system to extract card information, comprising:

a storage device;

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to;

receive an image of a card from a camera;

identify a first area of the image, the first area being selected as a potential location of a digit on the card in the image and of a size that will encompass not more than a single complete digit, the potential location and the size of the first area being identified from a comparison of the image to a database of card layouts stored on the storage device;

perform a linear classification algorithm on data encompassed by the first area;

determine a confidence level of a first result of the application of the linear classification algorithm to the first area, wherein the confidence level of the first result indicates a likelihood that the first area encompasses the single complete digit;

determine that the first area does not encompasses a single complete digit upon determining that the confidence level of the first result is under a configured threshold;

identify a second area of the image, the second area being in a different location from the first area and of a size that will encompass not more than a single complete digit;

perform a linear classification algorithm on data encompassed by the second area;

determine a confidence level of a second result of the application of the linear classification algorithm to the second area indicating that the second area encompasses a single complete digit;

determine that the second area encompasses a single complete digit upon determining that the confidence level of the second result is over a configured threshold; and

perform an optical character recognition algorithm on the second area upon a determination that the second area encompasses the single complete digit.

12. The system of claim 11, wherein the second area is shifted horizontally from the first area.

13. The system of claim 11, wherein the second area is shifted vertically from the first area.

14. The system of claim 11, wherein the card comprises one or more of a credit card, a debit card, an identification card, a loyalty card, an access card, or a stored value card.