System and method for recognizing text information in object

• US 9,418,304 B2
• Filed: 02/07/2012
• Issued: 08/16/2016
• Est. Priority Date: 06/29/2011
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Koo, Hyung-Il, You, Kisun, Cho, Hyun-Mook
• Primary Examiner(s): Krasnic, Bernard
• Application Number:
  US13/367,764
• Publication Number:
  US 20130004076A1
• Time in Patent Office: 1,652 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

A method for recognizing a text block in an object is disclosed. The text block includes a set of characters. A plurality of images of the object are captured and received. The object in the received images is then identified by extracting a pattern in one of the object images and comparing the extracted pattern with predetermined patterns. Further, a boundary of the object in each of the object images is detected and verified based on predetermined size information of the identified object. Text blocks in the object images are identified based on predetermined location information of the identified object. Interim sets of characters in the identified text blocks are generated based on format information of the identified object. Based on the interim sets of characters, a set of characters in the text block in the object is determined.

Claims

1. A method for recognizing a set of characters in an object, the method comprising:

receiving, at a client device, a plurality of images of an object, each image of the plurality of images including a text block;

extracting a pattern associated with the object from a particular image of the plurality of images of the object;

identifying a character format based on the extracted pattern;

determining, at the client device, a boundary of the object based on the extracted pattern;

identifying, at the client device, the text block in each image of the plurality of images of the object based on the boundary of the object; and

determining, at the client device, a set of characters in the object based in part on the identified text blocks and based in part on the character format.

2. The method of claim 1, wherein determining the set of characters includes:

for each image of the plurality of images of the object, generating an interim set of characters for the text block identified in the corresponding image of the plurality of images; and

inferring the set of characters based on the interim sets of characters.

3. The method of claim 2, further comprising, for each interim set of characters, determining a confidence level based on a sharpness of the interim set of characters, wherein inferring the set of characters includes inferring the set of characters based in part on the confidence level of each interim set of characters.

4. The method of claim 2, wherein each interim set of characters includes characters at character positions, wherein a first character in the set of characters is inferred based on a number of occurrences of the first character at a first character position in each interim set of characters.

5. The method of claim 2, wherein a first character in the set of characters is inferred when a number of occurrences of the first character at a first character position in each interim set of characters exceeds a threshold value.

6. The method of claim 2, further comprising receiving format information at the client device from a server regarding the set of characters in the object, and wherein, for each image of the plurality of images of the object, generating the interim set of characters is further based on the received format information.

7. The method of claim 1, wherein determining the boundary of the object further comprises:

detecting the boundary of the object in each image of the plurality of images of the object by determining boundary features, wherein the boundary features include a corner, an edge, or both; and

verifying the boundary based on size information of the object, wherein the size information includes a length of the object, a width of the object, an aspect ratio of the object, or a combination thereof.

8. The method of claim 1, further comprising receiving another image of the plurality of images of the object at the client device if a detected boundary of the object in a particular image of the plurality of images is not verified based on size information of the object.

9. The method of claim 1, further comprising:

identifying a type of the object based on the extracted pattern; and

identifying a configuration of the object based on the type of the object.

10. The method of claim 1, wherein one or more of the plurality of images of the object include a video frame of the object.

11. The method of claim 1, wherein the client device is a cell phone, and further comprising displaying the set of characters on a display of the cell phone.

12. The method of claim 1, further comprising receiving configuration information at the client device from a server, wherein the configuration information comprises location information of the text block with respect to the boundary, wherein the text block in each image of the plurality of images is identified based further on the location information, and wherein each image of the plurality of images is captured at a different time.

13. The method of claim 1, further comprising:

for each image of the plurality of images of the object, generating an interim set of characters for the text block identified in the image; and

for each interim set of characters, determining a confidence level based on a sharpness of the interim set of characters, wherein determining the set of characters in the object is further based on the confidence level of each interim set of characters.

14. The method of claim 1, further comprising:

for each image of the plurality of images of the object, generating an interim set of characters for the text block identified in the image; and

for each interim set of characters, determining a confidence level based on a similarity value of the interim set of characters, wherein the similarity value indicates a similarity to reference characters, and wherein determining the set of characters in the object is further based on the confidence level of each interim set of characters.

15. The method of claim 1, wherein the boundary is determined further based on size information of the object, the size information determined based on the extracted pattern.

16. The method of claim 15, wherein identifying the text block in each image of the plurality of images of the object is based further on location information that indicates a location of the text block with respect to the boundary.

17. A device for recognizing a set of characters in an object, the device comprising:

an image capturing unit configured to capture a plurality of images of an object, each image of the plurality of images including a text block;

an object recognition unit configured to extract a pattern associated with the object from a particular image of the plurality of images of the object and configured to identify a character format based on the extracted pattern;

a boundary detector configured to detect a boundary of the object based on the extracted pattern;

a masking unit configured to identify the text block in each image of the plurality of images of the object based on the boundary of the object; and

a temporal filter configured to determine a set of characters in the object based in part on the identified text blocks and based in part on the character format.

18. The device of claim 17, further comprising an optical character reader (OCR) engine configured to, for each image of the plurality of images of the object, generate an interim set of characters by performing optical character recognition on the text block identified in the corresponding image of the plurality of images of the object using the character format, wherein the character format identifies a number of characters in the set of characters and a type of each character in the set of characters.

19. The device of claim 18, wherein the temporal filter is further configured to infer the set of characters based on a confidence level of each interim set of characters.

20. The device of claim 19, wherein a first character in the set of characters is inferred based on a number of occurrences of the first character at a first character position in each interim set of characters.

21. The device of claim 20, wherein the first character in the set of characters is inferred when the number of occurrences of the first character exceeds a threshold value.

22. The device of claim 18, wherein the OCR engine is further configured to:

receive format information related to the set of characters in the object, wherein the format information includes the character format; and

for each image of the plurality of images of the object, generate the interim set of characters based on the format information.

23. The device of claim 17, wherein the boundary detector is further configured to detect the boundary of the object in each image of the plurality of images of the object based on edge detection and to verify the boundary detected in each image of the plurality of images based on size information of the object, wherein the masking unit is further configured to identify the text block in each image of the plurality of images of the object based on location information of the text block, and wherein the size information and the location information are determined based on the extracted pattern.

24. The device of claim 17, wherein the image capturing unit is further configured to receive another image of the plurality of images of the object if a detected boundary of the object in a particular image of the plurality of images is not verified based on size information of the object.

25. The device of claim 17, wherein the object recognition unit is further configured to identify a type of the object based on the extracted pattern and to identify a configuration of the object based on the type of the object.

26. The device of claim 25, wherein the configuration of the object indicates a layout of the object, a size of the object, location information of the text block with respect to the boundary, language information, or appearance information.

27. The device of claim 17, wherein the plurality of images of the object includes a sequence of images, and wherein the character format indicates a number of characters and an arrangement of the characters.

28. The device of claim 17, wherein the image capturing unit is configured to capture each image of the plurality of images at a different time, and wherein the temporal filter is configured to determine the set of characters using each interim set of characters.

29. The device of claim 17, wherein the object comprises a credit card, a driver'"'"'s license, or a passport.

30. The device of claim 17, wherein the object recognition unit is further configured to

compare the extracted pattern to stored patterns to determine a configuration associated with the object, the stored patterns accessible to the object recognition unit.

31. The device of claim 17, wherein the extracted pattern includes a logo or a set of features, and wherein the logo or the set of features are located outside of the text block in each image of the plurality of images of the object.

32. The device of claim 17, wherein the object includes a reference feature unique to the object, wherein the reference feature is located outside of the text block in each image of the plurality of images of the object, and wherein the extracted pattern corresponds to at least a portion of the reference feature.

33. The device of claim 17, further comprising a rectification unit configured to:

receive configuration information from a server;

receive the plurality of images from the boundary detector; and

rectify the plurality of images based on the configuration information and the boundary of the object to produce rectified images, wherein the rectified images are provided to the masking unit.

34. The device of claim 33, wherein the configuration information comprises coordinates of the text block with respect to the boundary, an aspect ratio of the object, a length of the object, a width of the object, or a combination thereof, and wherein the masking unit is further configured to identify the text block in each image of the plurality of images based further on the coordinates of the text block.

35. The device of claim 33, wherein the rectification unit is configured to rectify a particular image of the plurality of images by changing from an original value of an aspect ratio, a size, or an orientation of the particular image to a rectified value.

36. A non-transitory computer-readable medium comprising instructions for recognizing a set of characters in an object, the instructions causing a processor to perform operations comprising:

receiving a plurality of images of an object, each image of the plurality of images including a text block;

extracting a pattern associated with the object from a particular image of the plurality of images of the object;

identifying a character format based on the extracted pattern;

determining a boundary of the object based on the extracted pattern;

identifying the text block in each image of the plurality of images of the object based on the boundary of the object; and

determining a set of characters in the object based in part on the identified text blocks and based in part on the character format.

37. The non-transitory computer-readable medium of claim 36, wherein determining the set of characters includes:

for each identified text block, generating an interim set of characters by performing optical character recognition on the text block using the character format, wherein the character format identifies a number of characters in the set of characters and a type of each character in the set of characters; and

inferring the set of characters based on each interim set of characters.

38. The non-transitory computer-readable medium of claim 37, wherein the set of characters is inferred based on a confidence level of each interim set of characters.

39. The non-transitory computer-readable medium of claim 38, wherein a first character in the set of characters is inferred based on a number of occurrences of the first character at a first character position in each interim set of characters.

40. The non-transitory computer-readable medium of claim 39, wherein the first character in the set of characters is inferred when the number of occurrences of the first character exceeds a threshold value.

41. The non-transitory computer-readable medium of claim 37, wherein generating the interim set of characters includes:

receiving format information regarding the set of characters in the object, wherein the format information includes the character format; and

for each image of the plurality of images of the object, generating the interim set of characters based on the format information.

42. The non-transitory computer-readable medium of claim 36, wherein the operations further comprise verifying the boundary determined in each image of the plurality of images based on size information of the object.

43. The non-transitory computer-readable medium of claim 36, wherein the operations further comprise determining a type of the object based on the extracted pattern.

44. An apparatus for recognizing a set of characters in an object, the apparatus comprising:

means for capturing a plurality of images of an object, each image of the plurality of images including a text block;

means for extracting a pattern associated with the object from a particular image of the plurality of images of the object and for identifying a character format based on the extracted pattern;

means for determining a boundary of the object based on the extracted pattern;

means for identifying the text block in each image of the plurality of images of the object based on the boundary of the object; and

means for determining a set of characters in the object based in part on the identified text blocks and based in part on the character format.

45. The apparatus of claim 44, further comprising means for generating, for each image of the plurality of images, an interim set of characters for the text block identified in the corresponding image of the plurality of images of the object, wherein the means for determining the set of characters is further configured to infer the set of characters based on each interim set of characters.

46. The apparatus of claim 45, wherein a first character in the set of characters is inferred based on a number of occurrences of the first character at a first character position in each interim set of characters.

47. The apparatus of claim 45, wherein a first character in the set of characters is inferred when a number of occurrences of the first character at a first character position in each interim set of characters exceeds a threshold value.

48. The apparatus of claim 44, further comprising means for verifying the boundary determined in each image of the plurality of images based on size information of the object.