Method and apparatus to convert bitmapped images for use in a structured text/graphics editor

• US 20030142106A1
• Filed: 01/25/2002
• Published: 07/31/2003
• Est. Priority Date: 01/25/2002
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Saund, Eric, Moran, Thomas P., Mahoney, James V., Cass, Todd A., Larner, Daniel
• Granted Patent: US 7,139,004 B2
• Time in Patent Office: Days
• US Class Current: 345/582
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

In accordance with one aspect of the present invention, disclosed is an image analysis and conversion method and system, where bitmapped ink images are converted to structured object representations of the bitmapped images, which may be read and edited by a structured text/graphics editor.

Claims

1. An image analysis and conversion method comprising:

receiving a bitmapped image; and

converting the bitmapped image into structured object representations of the bitmapped image, which are editable by a structured text/graphics editor.

2. The method according to claim 1 wherein the receiving step includes,

scanning a non-electronic generated image into the bitmapped image.

3. The method according to claim 1 wherein the bitmapped image received in the receiving step is an originally generated electronic bitmapped image.

4. The method according to claim 1 wherein the converting step includes,

altering the bitmapped image into multiple alternative interpretations.

5. The method according to claim 4 wherein the altering of the bitmapped image into multiple alternative interpretations includes,

altering the bitmapped image into informal structured object representations that are editable by the structured text/graphics editor, and

altering the bitmapped image into formal structured object representations that are editable by the structured text/graphics editor.

6. The method according to claim 1 wherein the step of converting the bitmapped image into structured object representations of the bitmapped image includes configuring the structured object representations to represent an electronic slide of the structured text/graphics editor.

7. The method according to claim 1, wherein the converting step includes forming of an Alternative graph.

8. The method according to claim 7 wherein forming of an Alternative Graph includes:

forming for each of a number of closed curvilinear paths, a first intermediate data structure, and placing in the first intermediate data structure a list of alternative interpretations of the closed curvilinear paths;

forming for each of a number of open curvilinear paths, a second intermediate structure, and placing in the second intermediate data structure a list of alternative interpretations of the open paths;

forming for text of a number of text groups, a third intermediate data structure, and placing in the third intermediate data structure, a list of alternative interpretations of the text;

determining spatial containment relations, for each intermediate data structure representing the closed curvilinear paths;

testing all other intermediate data structures and determining whether an object is spatially contained within the bounds of each of the closed curvilinear paths and, if so, denoting this relation in a contained-by slot of the intermediate data structure;

selecting all structures for which a contained-by slot of the intermediate data structures is empty;

creating an Alternative node for the Alternative Graph for each intermediate data structure for which the contained-by slot is empty; and

for each Alternative node, performing the steps of,

traversing a collection of objects related to the Alternative node,

counting at each intermediate data structure encountered a number of alternative interpretations of the objects,

defining, as N, a maximum of this count over all descendents of the Alternative node,

creating N nodes of a type, CHOICE,

linking the CHOICE type nodes to the Alternative node, and

constructing, for each choice branch of the Alternative node, a tree of descendent and nested objects.

9. The method according to claim 1 wherein the step of converting the bitmapped image to the structured object representations includes generating multiple structured object representations of the bitmapped image, the multiple structured object representations, representing at least a first image representation having formal structured object representations, and a second image representation containing informal structured object representations.

10. The method according to claim 1 wherein the editing by the structured text/graphics editor permits movement of structured object representations by at least one of, individual objects, a sub-group of all the structured objects, or as an overall group of the structured object representations.

11. The method according to claim 1 wherein the bitmapped image is converted into the structured objects representations of the bitmapped image through the use of an Alternative Graph.

12. The method according to claim 11 wherein forming of an Alternative Graph includes:

forming for each of a number of closed curvilinear paths, a first intermediate data structure, and placing in the first intermediate data structure a list of alternative interpretations of the closed curvilinear paths;

forming for each of a number of open curvilinear paths, a second intermediate structure, and placing in the second intermediate data structure a list of alternative interpretations of the open paths;

forming for text of a number of text groups, a third intermediate data structure, and placing in the third intermediate data structure, a list of alternative interpretations of the text;

determining spatial containment relations, for each intermediate data structure representing the closed curvilinear paths;

testing all other intermediate data structures and determining whether an object is spatially contained within the bounds of each of the closed curvilinear paths and, if so, denoting this relation in a contained-by slot of the intermediate data structure;

selecting all structures for which a contained-by slot of the intermediate data structures is empty;

creating an Alternative node for the Alternative Graph for each intermediate data structure for which the contained-by slot is empty; and

for each Alternative node, performing the steps of,

traversing a collection of objects related to the Alternative node,

counting at each intermediate data structure encountered a number of alternative interpretations of the objects,

defining, as N, a maximum of this count over all descendents of the Alternative node,

creating N nodes of a type, CHOICE,

linking the CHOICE type nodes to the Alternative node, and

constructing, for each choice branch of the Alternative node, a tree of descendent and nested objects.

13. An image analysis and conversion system comprising:

an image receiving station for receiving a bitmapped image; and

a converter configured to convert the bitmapped image into structured object representations of the bitmapped image which is editable by a structured text/graphics editor.

14. The system according to claim 13 wherein when the converter converts the bitmapped image to the structured object representations of the bitmapped image, the structured object representations are in the form of at least one of formal structured objects and informal structured objects.

15. The image analysis and conversion system according to claim 14 wherein the converter includes,

an image processing and segmentation module, which processes the bitmap image to separate classes of primitive image objects, (i) small connected components at the scale of text characters, (ii) substantially straight curvilinear line fragments uncomplicated by junctions, called stroke-fragments, and (iii) large scale pictorial objects;

a text identification and layout analysis module which identifies groupings of bitmap and stroke objects representing runs of text characters forming text lines and columns, to generate a list of text group structures, each containing two or more alternative interpretations of the text groups;

a line art analysis module, which identifies groupings of stroke objects, to generate a list of line art group objects, each containing two or more alternative interpretations of the line art objects;

a construction of alternative graph module, which takes as input the lists of text and line art groups including alternative interpretations for each group, and constructs an Alternative Graph; and

an export to structured graphics editor module, which traverses the alternative graph and creates presentation slides with text and line art objects, and a hierarchy of groups.

16. On a screen display of an electronic device operating a structured text/graphics editor, an image representation comprising:

structured object representations of a converted bitmapped image, the structured object representations correlating to perceptually salient areas of the bitmapped image, wherein the structured object representations are editable by the structured text/graphics editor to allow a user to generate alternative interpretations of the bitmapped image.

17. The image representation according to claim 16 wherein the structured object representations are informal structured object representations, and wherein the informal structured object representations are editable to formal structured object representations.

18. The image representation according to claim 17 wherein the alternative interpretations permit a mixing of formal structured object representations and informal structured object representations in a single image representation displayed on the computer screen.

19. The image representation according to claim 16 wherein a first structured object representation is spatially contained within a second structured object representation.

20. The image representation according to claim 16 wherein a new structured object representation is added to existing structured object representations.

21. The image representation according to claim 16 wherein the structured object representations define a text block structure.

22. The image representation according to claim 21 wherein the text block structure includes a display of text parameters including at least one of text layout, text font, bullets, underlines and dummy characters.

23. The image representation according to claim 22 wherein the dummy characters are replaceable with target characters.

24. The image representation according to claim 16 wherein distinct alternative interpretations may be displayed.

25. The image representation according to claim 24 wherein display of the distinct alternative interpretations is accomplished by the use of at least one of underlays, bubble or balloon images, coloring, shading transparency/translucency, defocusing, and pop-up windows.