Method and apparatus to convert digital ink images for use in a structured text/graphics editor

• US 8,875,016 B2
• Filed: 10/17/2006
• Issued: 10/28/2014
• Est. Priority Date: 01/25/2002
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Saund, Eric, Moran, Thomas P., Larner, Daniel, Mahoney, James V., Cass, Todd A.
• Primary Examiner(s): HUYNH, THU V
• Application Number:
  US11/550,233
• Publication Number:
  US 20070065013A1
• Time in Patent Office: 2,933 Days
• Field of Search: 345/619, 345/582, 382/186, 382/187, 715/268, 715/256
• US Class Current: 715/268
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

In accordance with one aspect of the present invention, disclosed is an image analysis and conversion method and system, where digital ink images are converted to structured object representations of the digital ink images, capable of being edited by a structured text/graphics editor.

Claims

1. An image analysis and conversion method comprising:

receiving a digital ink image having defined perceptually salient structures by an electronic device configured to perform the receiving;

converting the digital ink image into multiple structured object representations of the digital ink image by the electronic device the multiple structured object representations correlating to ones of the defined perceptually salient structures of the digital ink image; and

altering at least one of the structured object representations into multiple simultaneously existing structured alternative interpretations of the at least one structured object representations of the digital ink image by the electronic device, each of the alternative interpretations being viewable by a user and being plausible intended outputs of the user.

2. The method according to claim 1, wherein each of the multiple structured object representations is editable by a structured text/graphics editor.

3. The method according to claim 2, wherein the step of converting the digital ink image into multiple structured object representations of the digital ink image includes configuring the multiple structured object representations to represent an electronic slide for the structured text/graphics editor.

4. The method according to claim 2, wherein the altering multiple structured object representations of the digital ink image into multiple simultaneously existing structured alternative interpretations of the digital ink image includes,

altering the multiple structured object representations of the digital ink image into informal simultaneously existing structured object representations that are editable by the structured text/graphics editor;

and

altering the multiple structured object representations of the digital ink image into formal simultaneously existing structured object representations that are editable by the structured text/graphics editor.

5. The method according to claim 2, wherein the structured text/graphics editor permits movement of each of the multiple structured object representations as at least one of an individual structured object representation, a member of a sub-group of all the multiple structured object representations, and a member of an overall group of the multiple structured object representations.

6. The method according to claim 1, wherein the converting step includes forming an Alternative Graph, wherein the Alternative Graph is configured for the generation of the multiple simultaneously existing structured alternative interpretations of the digital ink image, wherein the Alternative Graph is configured to permit movement of each of the multiple structured object representations as at least one of an individual structured object representation, a member of a sub-group of all the multiple structured object representations, and a member of an overall group of the multiple structured object representations.

7. The method according to claim 1, wherein the digital ink image is converted into the multiple structured object representations of the digital ink image through the use of an Alternative Graph.

8. The image analysis and conversion method according to claim 1, wherein the converting includes automatically defining the multiple structured object representations.

11. The method of claim 1, further including,

selecting one of the multiple simultaneously existing structured alternative interpretations from among all of the multiple simultaneously existing structured alternative interpretations as the intended output, at a point in a time after multiple alterations have been made to the digital ink image.

12. The method of claim 1, wherein the altering permits transfer of formal interpretations and of informal interpretations of the structured alternative interpretations of the digital ink image from one representative presentation to another representative presentation of an alternate formality, at will.

9. An image analysis and conversion system comprising:

an electronic device;

an image receiving input portion of the electronic device designed to receive a digital ink image; and

a converter portion of the electronic device configured to convert the digital ink image into structured object representations of the digital ink image, each of the structured object representations correlating to a perceptually salient structure of the digital ink image defined by the converter, wherein each of the structured object representations is editable by a structured text/graphics editor and the converter is further configured to alter the digital ink image into multiple simultaneously existing structured alternative interpretations of the digital ink image, each of the alternative interpretations being viewable by a user and being plausible intended outputs of the user, wherein the converter is configured to permit transfer of formal interpretations and informal interpretations of the structured alternative interpretations of the digital ink image from one representative presentation to another representative presentation of an alternate formality, at will.

10. The system according to claim 9, wherein when the converter converts the digital ink image to the structured object representations of the digital ink image, the structured object representations are in a form of at least one of formal structured objects and informal structured objects, and wherein the converter includes,

an image processing and segmentation module, which identifies classes of primitive image objects, including at least one of (i) small connected components at the scale of text characters, and (ii) substantially straight curvilinear line fragments uncomplicated by junctions, called stroke objects;

a text identification and layout analysis module, which identifies groupings of stroke objects representing runs of text characters forming text lines and columns, to generate a list of text group structures, each containing two or more alternative interpretations of the text groups;

a line art analysis module, which identifies groupings of stroke objects, to generate a list of line art group objects, each containing two or more alternative interpretations of the line art objects;

a construction of alternative graph module, which takes as input the lists of text and line art groups including alternative interpretations for each group, and constructs an Alternative Graph; and

an export to structured graphics editor module, which traverses the Alternative Graph and creates presentation slides with text and line art objects, and a hierarchy of groups.

13. The system of claim 9, wherein the converter is further configured to permit selection of one of the multiple simultaneously existing structured alternative interpretations from among all of the multiple simultaneously existing structured alternative interpretations as the intended output, at a point in time after multiple alterations have been made to the digital ink image.

14. An image analysis and conversion method, wherein the method is embodied and executed on a processor having memory means, the method comprising:

receiving a digital ink image having defined perceptually salient structures;

automatically converting the digital ink image into multiple structured object representations of the digital ink image, wherein each of the multiple structured object representations correlates to a perceptually salient structure of the digital ink image;

altering at least one of the multiple structured object representations into multiple simultaneously existing structured alternative interpretations of the at least one of the multiple structured object representations; and

altering the multiple structured object representations into multiple simultaneously existing structured alternative interpretations of the digital ink image, wherein at least one of the multiple simultaneously existing structured alternative interpretations of the digital ink image includes at least one of the multiple simultaneously existing structured alternative interpretations of the at least one of the multiple structured object representations, wherein each of the multiple simultaneously existing structured alternative interpretations of the digital ink image are plausible intended outputs of a user.

15. The image analysis and conversion method according to claim 14, wherein the altering of the multiple structured objects representations into multiple simultaneously existing structured alternative interpretations of the digital ink image includes forming an Alternative Graph, wherein the Alternative Graph is configured for the generation of the multiple simultaneously existing structured alternative interpretations of the digital ink image.

16. The image analysis and conversion method according to claim 14, wherein the altering of the multiple structured object representations into multiple simultaneously existing structured alternative interpretations of the digital ink image includes,

altering the multiple structured object representations into informal simultaneously existing structured alternative interpretations of the digital ink image;

and

altering the multiple structured object representations into formal simultaneously existing structured alternative interpretations of the digital ink image.

17. The image analysis and conversion method according to claim 14, wherein a user is permitted to transfer formal interpretations and informal interpretations of the structured alternative interpretations of the digital ink image from one representative presentation to another representative presentation of an alternate formality, at will.

18. The image analysis and conversion method according to claim 14, further including,

selecting one of the multiple simultaneously existing structured alternative interpretations from among all of the multiple simultaneously existing structured alternative interpretations, at a time after alterations have been made to the digital ink image.

19. The image analysis and conversion method according to claim 14, wherein at least one of the multiple structured object representations includes a curve.