OBJECT EXTRACTION IN COLOUR COMPOUND DOCUMENTS

• US 20100157340A1
• Filed: 12/14/2009
• Published: 06/24/2010
• Est. Priority Date: 12/18/2008
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Liu, Ping, Chen, Yu-Ling, McDonell, Trevor Lee
• Granted Patent: US 8,351,691 B2
• Time in Patent Office: Days
• US Class Current: 358/1.900
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

Disclosed is a computer implemented method of text extraction in colour compound documents. The method connects similarly coloured pixels of an image of a colour compound document into connected components (CCs); classifies each CC as either text or non-text; refines the text CC classification for each text CC using global colour context statistics; groups text CCs into text blocks; recovers misclassified non-text CCs into a nearby text block; and removes extraneous CCs from each text block using local colour context statistics to thereby provide the extracted text in the text blocks. Also disclosed is a computer implemented method of locating graphics objects in a colour compound document image. The method connects similarly coloured pixels of said image into connected components (CCs) and placing the CCs in an enclosure tree; classifies (330,730) each CC into one of a plurality of classes wherein at least one class (862) represents salient graphics components; identifies (1140) a graphics container (441) to perform semantic analysis for each CC of said class representing salient graphics components; profiles (1170) descendents of said graphics container in said tree to obtain semantic context statistics; and decides (1710) whether the graphics container contains a whole or part of a graphics object based on said semantic context statistics.

Claims

1. A computer implemented method of identifying at least one of a photo region and a graphics region in a colour bitmap image of a compound document, said method comprising:

(a) connecting similarly coloured pixels of said image into connected components and placing the connected components in an enclosure tree, the enclosure tree forming a hierarchy of ancestor and descendent connected components where an ancestor connected component encloses a descendent connected component wherein each node of the enclosure tree is a connected component;

(b) classifying each connected component into one of a plurality of classes wherein at least one class represents non-text connected components;

(c) selecting a connected component from one of the non-text connected components of the enclosure tree;

(d) calculating context statistics for the selected connected component based on descendent and touching sibling connected components of the selected connected component;

(e) identifying whether the selected connected component contains at least one of a photo region and a graphics region based on the context statistics; and

(f) storing the selected connected component identified as either graphics or photo object to memory.

2. A method according to claim 1 wherein said selecting step comprises:

(ca) selecting a non-text connected component with either a thin characteristic or low pixel density compared to a bounding box area of the non-text connected component;

(cb) selecting a child connected component of said non-text connected component as the selected connected component if said non-text connected component encloses the child connected component with a total pixel count greater than a first threshold based on an enclosed pixel count of said non-text connected component;

(cc) selecting the non-text connected component as the selected connected component if said non-text connected component encloses more than one child connected component and none of the child connected components have a total pixel count greater than the first threshold;

(cd) selecting a touching connected component of said non-text connected component as the selected connected component if the touching connected component has children and a total pixel count greater than a second threshold based on a pixel count of the bounding box of said non-text connected component.

3. A method according to claim 1 wherein the calculating context statistics step calculates, from descendent CCs, at least one statistic selected from the group of statics consisting of:

(i) number of pixels belonging to a graphics object;

(ii) number of TEXT child CCs;

(iii) number of TEXT grandchild CCs;

(iv) number of NOISE CCs;

(v) sum of Total Pixel Count of NOISE CCs;

(vi) number of solid like CCs;

(vii) sum of Total Pixel Count of solid like CCs;

(viii) sum of Descendent Count of all child CCs;

(ix) sum of Enclosed Pixel Count of all child CCs; and

(x) number of graphics interior CCs.

4. A method according to claim 1 wherein the calculating context statistics calculates, from touching CCs, at least one (FIG. 16, 1170) statistic of either a busy state of surrounding connected components or a colour difference range to touching siblings of the selected connected component.

5. A method according to claim 1 wherein the calculating context statistics step further comprises:

calculating for said selected connected component at least one (FIG. 16, 1170) statistic selected from the group consisting of;

(i) a size and density histogram of TEXT classified children; and

(ii) a fragmentation state.

6. A method according to claim 1 wherein the identifying step further comprises:

classifying the selected connected component as a good text container if a size and density histogram of TEXT classified children has each of the following characteristics;

(i) a number of TEXT classified children is greater than a third predetermined threshold;

(ii) a mean of Self Pixel Count of TEXT classified children is greater than a forth predetermined threshold;

(iii) a variance of Self Pixel Count of TEXT classified children is less than a fifth predetermined threshold.

7. A method according to claim 1 wherein a decision on whether the selected connected component contains a photo region depends on said size and density histogram of TEXT classified children having no characteristics of a text paragraph.

8. A method according to claim 1 wherein said selected connected component identified as a graphics region is optimised by the steps of:

[1760]

(ga) consolidating grouped graphics descendants;

(gb) consolidating touching childless CCs; and

(ge) consolidating flat CCs that are fully contained by a grouped graphics group.

9. A method according to claim 1 wherein selected connected component identified as a photo region is optimised by the steps of

(ha) consolidating a parent CC if the parent CC is not a good text container;

(hb) consolidating any touching sibling non-text CCs if consolidating any of the touching sibling non-text CCs will increase a pixel density of a bounding box area of said photo region; and

(hc) consolidating any touching sibling text CCs if any of the touching sibling text CCs are fully enclosed by the bounding box area of said photo region.

10. A computer readable storage medium having a computer program recorded thereon, the program being executable by computer apparatus to identify at least one of a photo region and a graphics region in a colour bitmap image of a compound document, said program comprising:

code for connecting similarly coloured pixels of said image into connected components and placing the connected components in an enclosure tree, the enclosure tree forming a hierarchy of ancestor and descendent connected components where an ancestor connected component encloses a descendent connected component wherein each node of the enclosure tree is a connected component;

code for classifying each connected component into one of a plurality of classes wherein at least one class represents non-text connected components;

code for selecting a connected component from one of the non-text connected components of the enclosure tree;

code for calculating context statistics for the selected connected component based on descendent and touching sibling connected components of the selected connected component;

code for identifying whether the selected connected component contains at least one of a photo region and a graphics region based on the context statistics; and

code for storing the selected connected component identified as either graphics or photo object to memory.

11. Computer apparatus, said apparatus comprising a processor and a memory coupled to the processor, the memory having recorded thereon a program for identifying at least one of a photo region and a graphics region in a colour bitmap image of a compound document, the image being stored in the memory, the program comprising:

code means executable by the processor for connecting similarly coloured pixels of said image into connected components and placing the connected components in an enclosure tree, the enclosure tree forming a hierarchy of ancestor and descendent connected components where an ancestor connected component encloses a descendent connected component wherein each node of the enclosure tree is a connected component;

code means executable by the processor for classifying each connected component into one of a plurality of classes wherein at least one class represents non-text connected components;

code means executable by the processor for selecting a connected component from one of the non-text connected components of the enclosure tree;

code means executable by the processor for calculating context statistics for the selected connected component based on descendent and touching sibling connected components of the selected connected component;

code means executable by the processor for identifying whether the selected connected component contains at least one of a photo region and a graphics region based on the context statistics; and

code means executable by the processor for storing in the memory the selected connected component identified as either graphics or photo object.

12. A copier comprising:

a scanner for scanning a compound document to form a colour bitmap image;

a processor coupled to the scanner;

a memory coupled to the processor for storing the colour bitmap image, said memory having recorded thereon a program executable by the processor for identifying at least one of a photo region and a graphics region in the colour bitmap image of the compound document, the program comprising;

code means for connecting similarly coloured pixels of said image into connected components and placing the connected components in an enclosure tree, the enclosure tree forming a hierarchy of ancestor and descendent connected components where an ancestor connected component encloses a descendent connected component wherein each node of the enclosure tree is a connected component;

code means for classifying each connected component into one of a plurality of classes wherein at least one class represents non-text connected components;

code means for selecting a connected component from one of the non-text connected components of the enclosure tree;

code means for calculating context statistics for the selected connected component based on descendent and touching sibling connected components of the selected connected component;

code means for identifying whether the selected connected component contains at least one of a photo region and a graphics region based on the context statistics; and

code means for storing in the memory the selected connected component identified as either graphics or photo object.