Method of handling complex variants of words through prefix-tree based decoding for Devanagiri OCR

• US 9,262,699 B2
• Filed: 03/14/2013
• Issued: 02/16/2016
• Est. Priority Date: 07/19/2012
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Barman, Kishor K., Baheti, Pawan Kumar, Krishna Kumar, Raj Kumar
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US13/828,060
• Publication Number:
  US 20140023274A1
• Time in Patent Office: 1,069 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/293 of characters other than Kanji, Hiragana or Katakana

Abstract

An electronic device and method identify a block of text in a portion of an image of real world captured by a camera of a mobile device, slice sub-blocks from the block and identify characters in the sub-blocks that form a first sequence to a predetermined set of sequences to identify a second sequence therein. The second sequence may be identified as recognized (as a modifier-absent word) when not associated with additional information. When the second sequence is associated with additional information, a check is made on pixels in the image, based on a test specified in the additional information. When the test is satisfied, a copy of the second sequence in combination with the modifier is identified as recognized (as a modifier-present word). Storage and use of modifier information in addition to a set of sequences of characters enables recognition of words with or without modifiers.

Claims

1. A method to decode text in real world images, the method comprising:

receiving a natural image of a scene of real world captured by a camera of a mobile device, the natural image comprising text pixels in one or more text regions and further comprising non-text pixels;

detecting in the natural image, a text region comprising a first sequence of characters of a predetermined script in which a predetermined language is expressed;

selecting a second sequence of characters from a predetermined set of sequences, based on the second sequence of characters matching the first sequence of characters, wherein at least one of the sequences in the predetermined set is associated with information on location of one or more modifiers;

after the selecting and responsive to the second sequence being associated with the information, analyzing the natural image to determine whether at least one pixel satisfies a predetermined test associated with said information;

adding a modifier to a specific character in a copy of the second sequence of characters, when the predetermined test is satisfied; and

identifying said first sequence of characters detected in said text region, as a word in the predetermined language comprising the copy of said second sequence of characters with the modifier;

wherein one or more of the receiving, the detecting, the selecting, the analyzing, the adding, and the identifying are performed by at least one processor coupled to a memory.

2. The method of claim 1 further comprising:

checking for presence of a line of pixels of a common binary value in the text region comprising the first sequence of characters.

3. The method of claim 2 wherein:

the at least one pixel is checked for location in the natural image above the line of pixels, when a longitudinal direction of a bounding box of the text region is horizontal.

4. The method of claim 3 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word formed by the first sequence of characters; and

the at least one pixel in the natural image is comprised in a group of pixels that correspond to a vowel maatra located to a right side of the text region, when the longitudinal direction of the text region is horizontal.

5. The method of claim 4 wherein:

the vowel maatra is one of matraas or .

6. The method of claim 2 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word formed by the first sequence of characters; and

the predetermined test identifies a DOT maatra located above the shiro-rekha.

7. The method of claim 1 wherein:

the checking comprises determining a center of mass of a group of pixels including the at least one pixel.

8. The method of claim 1 further comprising:

determining the predetermined test based on the second sequence of characters; and

storing in the memory the second sequence of characters as another word recognized in the predetermined language, when the second sequence of characters is found to be not associated with any test, by the determining.

9. At least one non-transitory computer readable storage media comprising a plurality of instructions to be executed by at least one processor to decode text in real world images, the plurality of instructions comprising:

first instructions to receive a natural image of a scene of real world captured by a camera of a mobile device, the natural image comprising text pixels in one or more text regions and further comprising non-text pixels;

second instructions to detect in the natural image, a text region comprising a first sequence of characters of a predetermined script in which a predetermined language is expressed;

third instructions to select a second sequence of characters from a predetermined set of sequences, based on the second sequence of characters matching the first sequence of characters, wherein at least one of the sequences in the predetermined set is associated with information on location of one or more modifiers;

fourth instructions, to be executed after the third instructions and responsive to the second sequence being associated with the information, to analyze the natural image to determine whether at least one pixel satisfies a predetermined test associated with said information;

fifth instructions to add a modifier to a specific character in a copy of the second sequence of characters, when the predetermined test is satisfied; and

sixth instructions to identify said first sequence of characters detected in said text region, as a word in the predetermined language comprising the copy of said second sequence of characters with the modifier.

10. The at least one non-transitory computer readable storage media of claim 9 wherein the plurality of instructions further comprises:

a group of instructions to further check for presence of a line of pixels of a common binary value in a text region comprising the first sequence of characters.

11. The at least one non-transitory computer readable storage media of claim 10 wherein:

the at least one pixel is checked for location in the natural image above the line of pixels, when a longitudinal direction of a bounding box of the text region is horizontal.

12. The at least one non-transitory computer readable storage media of claim 11 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the at least one pixel is comprised in a group of pixels that correspond to a vowel maatra located to a right side of the bounding box, when the longitudinal direction of the bounding box is horizontal.

13. The at least one non-transitory computer readable storage media of claim 12 wherein:

the vowel maatra is one of matraas or .

14. The at least one non-transitory computer readable storage media of claim 10 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the predetermined test identifies a DOT maatra located above the shiro-rekha.

15. The at least one non-transitory computer readable storage media of claim 9 wherein:

the fourth instructions, when executed determining a center of mass of a group of pixels including the at least one pixel.

16. The at least one non-transitory computer readable storage media of claim 9 wherein the plurality of instructions further comprises:

seventh instructions to determine the predetermined test based on the second sequence of characters;

wherein the second sequence of characters is stored as another word recognized in the predetermined language, when the second sequence of characters is found to be not associated with any test, by execution of the seventh instructions.

17. A mobile device to decode text in real world images, the mobile device comprising:

a camera;

a memory operatively connected to the camera to receive at least a natural image of a scene therefrom, the natural image comprising text pixels in one or more text regions and further comprising non-text pixels;

at least one processor operatively connected to the memory to execute a plurality of instructions stored in the memory;

wherein the plurality of instructions cause the at least one processor to;

detect in the natural image, a text region comprising a first sequence of characters of a predetermined script in which a predetermined language is expressed;

select a second sequence of characters from a predetermined set of sequences, based on the second sequence of characters matching the first sequence of characters, wherein at least one of the sequences in the predetermined set is associated with information on location of one or more modifiers;

after selection of the second sequence of characters and responsive to the second sequence being associated with the information, analyzing the natural image to determine whether at least one pixel satisfies a predetermined test associated with said information;

add a modifier to a specific character in a copy of the second sequence of characters, when the predetermined test is satisfied; and

identify said first sequence of characters detected in said text region, as a word in the predetermined language comprising the copy of said second sequence of characters with the modifier.

18. The mobile device of claim 17 wherein the at least one processor is further configured to:

further check for presence of a line of pixels of a common binary value in a text region comprising the first sequence of characters.

19. The mobile device of claim 18 wherein:

the at least one pixel is checked for location above the line of pixels, when a longitudinal direction of a bounding box of the text region is horizontal.

20. The mobile device of claim 19 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the at least one pixel is comprised in a group of pixels that correspond to a vowel maatra located to a right side of the bounding box, when the longitudinal direction of the bounding box is horizontal.

21. The mobile device of claim 18 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the predetermined test identifies a DOT maatra located above the shiro-rekha.

22. The mobile device of claim 17 wherein the at least one processor is configured to:

determine a center of mass of a group of pixels including the at least one pixel.

23. An apparatus to decode text in real world images, the apparatus comprising:

a memory storing a natural image of a scene outside the apparatus, the natural image comprising text pixels in one or more text regions and further comprising non-text pixels;

means for detecting in the natural image, a text region comprising a first sequence of characters of a predetermined script in which a predetermined language is expressed;

means for selecting a second sequence of characters from a predetermined set of sequences, based on the second sequence of characters matching the first sequence of characters, wherein at least one of the sequences in the predetermined set is associated with information on location of one or more modifiers;

means, operable after the means for selecting and responsive to the second sequence being associated with the information, for analyzing the natural image to determine whether at least one pixel satisfies a predetermined test associated with said information;

means for adding a modifier to a specific character in a copy of the second sequence of characters, when the predetermined test is satisfied; and

means for identifying said first sequence of characters detected in said text region, as a word in the predetermined language comprising the copy of said second sequence of characters with the modifier.

24. The apparatus of claim 23 further comprising:

means for further checking for presence of a line of pixels of a common binary value in the text region comprising the first sequence of characters.

25. The apparatus of claim 24 wherein:

the at least one pixel is checked for location in the natural image above the line of pixels, when a longitudinal direction of a bounding box of the text region is horizontal.

26. The apparatus of claim 25 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the at least one pixel is comprised in a group of pixels that correspond to a vowel maatra located to a right side of the bounding box, when the longitudinal direction of the bounding box is horizontal.

27. The apparatus of claim 26 wherein:

the vowel maatra is one of matraas or .

28. The apparatus of claim 24 wherein:

the predetermined script is Devanagari;

the line of pixels is comprised in a shiro-rekha of the word; and

the predetermined test identifies a DOT maatra located above the shiro-rekha.

29. The apparatus of claim 23 further comprising:

means for determining a center of mass of a group of pixels including the at least one pixel.