Method and system for recognizing information on a card

• US 10,210,415 B2
• Filed: 03/01/2016
• Issued: 02/19/2019
• Est. Priority Date: 06/03/2013
• Status: Active Grant

• Current Assignee: Alipay.Com Co., Ltd. (Ant Group Co., Ltd.)
• Original Assignee: Alipay.Com Co., Ltd. (Ant Group Co., Ltd.)
• Inventors: Li, Yang, Chen, Guo
• Primary Examiner(s): Niu, Feng
• Application Number:
  US15/058,060
• Publication Number:
  US 20160247037A1
• Time in Patent Office: 1,085 Days
• Field of Search: None
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields

Abstract

Embodiments of the present application relate to a method for recognizing information, a system for recognizing information, and a computer program product for recognizing information. A method for recognizing information is provided. The method includes locating a card zone for each frame within a card image frame sequence comprising a plurality of frames, locating an information zone within each card zone, dividing each information zone into at least one character zone, de-blurring a character zone corresponding to a same region across all the frames in the card image frame sequence, and recognizing character string information based on the de-blurred character zone.

Claims

1. A method for recognizing information, comprising:

locating, using a processor, at least one card zone for each frame within a card image frame sequence comprising a plurality of frames;

locating, using the processor, at least one information zone within each card zone of the at least one card zone;

dividing, using the processor, the at least one information zone into a plurality of character zones, the plurality of character zones including a first character zone and a second character zone, the first character zone and the second character zone corresponding to different positions within the at least one information zone;

de-blurring, using the processor, the first character zone and the second character zone, comprising;

for first character zones of the plurality of character zones corresponding to a first same region across the plurality of frames in the card image frame sequence;

calculating clarities of the first character zones, wherein a clarity of a character zone is calculated by;

determining intensity values of a plurality of pixels in the character zone based on horizontal gradient features and vertical gradient features of the plurality of pixels, comprising;

 

determining an arrangement of matrix elements of a matrix corresponding to an arrangement of the plurality of pixels; and

 

summing values of the matrix elements of the matrix in a direction to obtain the intensity values of the plurality of pixels, the direction being horizontal or vertical; and

determining the clarity of the character zone based on intensity values of the plurality of pixels in the character zone, wherein the determining of the clarity of the character zone comprises;

 

calculating a mean of the intensity values of the plurality of pixels in the character zone; and

 

determining the mean of the intensity values as the clarity of the character zone; and

identifying, as a first candidate character zone, a first character zone having a highest clarity;

determining whether the first candidate character zone of a first image satisfies a predetermined threshold;

in response to a determination that the first candidate character zone satisfies a predetermined threshold, selecting the first image as the selected image for subsequent recognition processing;

for second character zones of the plurality of character zones corresponding to a second same region across the plurality of frames in the card image frame sequence;

calculating clarities of the second character zones; and

identifying, as a second candidate character zone, a second character zone having a highest clarity;

in response to a determination that the second candidate character zone of the first image fails to satisfy the predetermined threshold;

acquiring a next card image frame sequence to de-blur the second character zone; and

recognizing, using the processor, character string information based on the de-blurred first candidate character zone and the de-blurred second candidate character zone.

2. The method as described in claim 1, further comprising:

prior to the locating of the at least one card zone for the each frame within the card image frame sequence;

storing, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquiring the card image frame sequence from the cache.

3. The method as described in claim 2, further comprising:

prior to the locating of the at least one card zone for each frame within the card image frame sequence;

storing, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquiring the card image frame sequence from the cache, comprising;

selecting, from the video stream or the plurality of single-frame images in the cache, various frames in the card image frame sequence at intervals of a predetermined number of frames.

4. The method as described in claim 1, further comprising:

verifying whether the character string information complies with a predetermined rule for a card;

in the event that the character string information complies with the predetermined rule, displaying the character string information as card information to a user to be used in confirmation; and

in the event that the character string information does not comply with the predetermined rule, acquiring a next card image frame sequence to be recognized.

5. The method as described in claim 1, wherein the next card image frame sequence is different from the card image frame sequence.

6. A system for recognizing information, comprising:

at least one processor configured to;

locate at least one card zone for each frame within a card image frame sequence comprising a plurality of frames;

locate at least one information zone within each card zone of the at least one card zone;

divide each information zone of the at least one information zone into a plurality of character zones, the plurality of character zones including a first character zone and a second character zone, the first character zone and the second character zone corresponding to different positions within the at least one information zone;

de-blur the first character zone and the second character zone, comprising to;

for first character zones of the plurality of character zones corresponding to a first same region across the plurality of frames in the card image frame sequence;

calculate clarities of the first character zones, wherein a clarity of a character zone is calculated by;

 

determining intensity values of a plurality of pixels in the character zone based on horizontal gradient features and vertical gradient features of the plurality of pixels, comprising;

 

determining an arrangement of matrix elements of a matrix corresponding to an arrangement of the plurality of pixels; and

 

summing values of the matrix elements of the matrix in a direction to obtain the intensity values of the plurality of pixels, the direction being horizontal or vertical; and

 

determining the clarity of the character zone based on intensity values of the plurality of pixels in the character zone, wherein the determining of the clarity of the character zone comprises;

 

calculating a mean of the intensity values of the plurality of pixels in the character zone; and

 

determining the mean of the intensity values as the clarity of the character zone; and

identify, as a first candidate character zone, a first character zone having a highest clarity;

determine whether the first candidate character zone of a first image satisfies a predetermined threshold;

in response to a determination that the first candidate character zone satisfies a predetermined threshold, regard the first image as the selected image for subsequent recognition processing;

for second character zones of the plurality of character zones corresponding to a second same region across the plurality of frames in the card image frame sequence;

calculate clarities of the second character zones; and

identify, as a second candidate character zone, a second character zone having a highest clarity;

in response to a determination that the second candidate character zone of the first image fails to satisfy the predetermined threshold;

acquire a next card image frame sequence to de-blur the second character zone; and

recognize character string information based on the de-blurred first candidate character zone and the de-blurred second candidate character zone; and

a memory coupled to the at least one processor and configured to provide the at least one processor with instructions.

7. The system as described in claim 6, wherein the at least one processor is further configured to:

prior to the locating of the at least one card zone for the each frame within the card image frame sequence;

store, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquire the card image frame sequence from the cache.

8. The system as described in claim 6, wherein:

the at least one processor is further configured to;

prior to the locating of the at least one card zone for each frame within the card image frame sequence;

store, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquire the card image frame sequence from the cache, comprising to;

select, from the video stream or the plurality of single-frame images in the cache, various frames in the card image frame sequence at intervals of a predetermined number of frames.

9. The system as described in claim 6, wherein the at least one processor is further configured to:

verify whether the character string information complies with a predetermined rule for a card;

in the event that the character string information complies with the predetermined rule, display the character string information as card information to a user to be used in confirmation; and

in the event that the character string information does not comply with the predetermined rule, acquire a next card image frame sequence to be recognized.

10. A system for recognizing information, comprising:

a processor; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to;

acquire a card image frame sequence comprising a plurality of frames;

locate at least one card zone for each frame within a card image frame sequence;

locate at least one information zone within each card zone of the at least one card zone;

divide each information zone of the at least one information zone into a plurality of character zones, the plurality of character zones including a first character zone and a second character zone, the first character zone and the second character zone corresponding to different positions within the at least one information zone;

de-blur the first character zone and the second character zone, comprising to;

for first character zones of the plurality of character zones corresponding to a first same region across the plurality of frames in the card image frame sequence;

calculate clarities of the first character zones, wherein a clarity of a character zone is calculated by;

 

determining intensity values of a plurality of pixels in the character zone based on horizontal gradient features and vertical gradient features of the plurality of pixels, comprising to;

 

determine an arrangement of matrix elements of a matrix corresponding to an arrangement of the plurality of pixels; and

 

sum values of the matrix elements of the matrix in a direction to obtain the intensity values of the plurality of pixels, the direction being horizontal or vertical; and

 

determining the clarity of the character zone based on intensity values of the plurality of pixels in the character zone, wherein the determining of the clarity of the character zone comprises;

 

calculating a mean of the intensity values of the plurality of pixels in the character zone; and

 

determining the mean of the intensity values as the clarity of the character zone; and

identify, as a first candidate character zone, a first character zone having a highest clarity;

determine whether the first candidate character zone of a first image satisfies a predetermined threshold;

in response to a determination that the first candidate character zone satisfies a predetermined threshold, select the first image as the selected image for subsequent recognition processing;

for second character zones of the plurality of character zones corresponding to a second same region across the plurality of frames in the card image frame sequence;

calculate clarities of the second character zones; and

identify, as a second candidate character zone, a second character zone having a highest clarity;

in response to a determination that the second candidate character zone of the first image fails to satisfy the predetermined threshold;

acquire a next card image frame sequence to de-blur the second character zone; and

recognize character string information based on the de-blurred first candidate character zone and the de-blurred second candidate character zone; and

display the recognized character string information.

11. The system as described in claim 10, wherein the processor is further configured to:

prior to the locating of the at least one card zone for the each frame within the card image frame sequence;

store, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquire the card image frame sequence from the cache.

12. The system as described in claim 10, wherein:

the processor is further configured to;

prior to the locating of the at least one card zone for each frame within the card image frame sequence;

store, in a cache, a video stream or a plurality of single-frame images taken of a card face; and

acquire the card image frame sequence from the cache, comprising to;

select, from the video stream or the plurality of single-frame images in the cache, various frames in the card image frame sequence at intervals of a predetermined number of frames.

13. The system as described in claim 10, wherein the processor is further configured to:

verify whether the character string information complies with a predetermined rule for a card;

in the event that the character string information complies with the predetermined rule, display the character string information as card information to a user to be used in confirmation; and

in the event that the character string information does not comply with the predetermined rule, acquire a next card image frame sequence to be recognized.

14. A computer program product for recognizing information, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

locating at least one card zone for each frame within a card image frame sequence comprising a plurality of frames;

locating at least one information zone within each card zone of the at least one card zone;

dividing each information zone of the at least one information zone into a plurality of character zones, the plurality of character zones including a first character zone and a second character zone, the first character zone and the second character zone corresponding to different positions within the at least one information zone;

de-blurring the first character zone and the second character zone, comprising;

for first character zones of the plurality of character zones corresponding to a first same region across the plurality of frames in the card image frame sequence;

calculating clarities of the first character zones, wherein a clarity of a character zone is calculated by;

determining intensity values of a plurality of pixels in the character zone based on horizontal gradient features and vertical gradient features of the plurality of pixels, comprising;

 

determining an arrangement of matrix elements of a matrix corresponding to an arrangement of the plurality of pixels; and

 

summing values of the matrix elements of the matrix in a direction to obtain the intensity values of the plurality of pixels, the direction being horizontal or vertical; and

determining the clarity of the character zone based on intensity values of the plurality of pixels in the character zone, wherein the determining of the clarity of the character zone comprises;

 

calculating a mean of the intensity values of the plurality of pixels in the character zone; and

 

determining the mean of the intensity values as the clarity of the character zone; and

identifying, as a first candidate character zone, a first character zone having a highest clarity;

determining whether the first candidate character zone of a first image satisfies a predetermined threshold;

in response to a determination that the first candidate character zone satisfies a predetermined threshold, selecting the first image as the selected image for subsequent recognition processing;

for second character zones of the plurality of character zones corresponding to a second same region across the plurality of frames in the card image frame sequence;

calculating clarities of the second character zones; and

identifying, as a second candidate character zone, a second character zone having a highest clarity;

in response to a determination that the second candidate character zone of the first image fails to satisfy the predetermined threshold;

acquiring a next card image frame sequence to de-blur the second character zone; and

recognizing character string information based on the de-blurred first candidate character zone and the de-blurred second candidate character zone.