System and method for editing electronic images

• US 6,903,751 B2
• Filed: 03/22/2002
• Issued: 06/07/2005
• Est. Priority Date: 03/22/2002
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Saund, Eric, Popat, Ashok C., Moran, Thomas P., Larner, Daniel L., Mahoney, James V., Fleet, David J.
• Primary Examiner(s): Razavi, Michael
• Assistant Examiner(s): Lee, Hwa C.
• Application Number:
  US10/104,805
• Publication Number:
  US 20030179214A1
• Time in Patent Office: 1,173 Days
• Field of Search: 345/614, 345/625, 345/442, 345/650, 345/619
• US Class Current: 345/619
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A graphical input and display system for creating and manipulating electronic images includes input devices permitting a user to manipulate elements of electronic images received from various image input sources. A processor, connected to the system, receives requests for various image editing operations and also accesses a memory structure. The system memory structure includes a user interaction module, which allows a user to enter new image material or select and modify existing image material to form primary image objects, as well as a grouping module, which maintains an unrestricted grouping structure, an output module, and data memory.

Claims

1. A graphical input and display system for creating and manipulating an electronic image, comprising:

user input means for manipulating elements of the electronic image;

image input means for entering the electronic image onto a display means;

a processor, connected for receiving requests for image editing operations and for accessing a memory structure;

program memory means comprising;

user interaction module for entering new, unrestricted image material or selecting and modifying existing image material to form primary image objects;

grouping module for creating visible image elements and maintaining an unrestricted, arbitrary grouping structure, and setting a priority and ordering for composite objects, wherein said priority ordering includes at least one member selected from the group consisting of rearrangement based on algorithms that determine smallest-to-largest spatial extent, measures of perceptual salience, and the context of selection criteria recently exercised by the user;

data output module for presenting a view of the graphical image; and

data memory means comprising;

data input data structure; and

data output data structure.

2. The graphical input and display system according to claim 1, further comprising a data preparation module.

3. The graphical input and display system according to claim 2, wherein said data preparation module determines whether an electronic image is to be treated as a photographic image.

4. The graphical input and display system according to claim 2, wherein said data preparation module determines whether to treat all objects in the electronic image as text and line art.

5. The graphical input and display system according to claim 2, wherein said data preparation module automatically distinguishes foreground pixels from background pixels and as a result sets background pixels to transparent.

6. The graphical input and display system according to claim 2, wherein said data preparation module determines whether to apply pre-segmentation to the electronic image.

7. The graphical input and display system according to claim 1, further comprising a hyperlink module for creating hyperlinks between regions on documents.

8. The graphical input and display system according to claim 1, further comprising a parsed image data structure.

9. The graphical input and display system according to claim 8, wherein said parsed image data structure comprises image region classes of text and line art, continuous/photographic, and halftone regions.

10. The graphical input and display system according to claim 1, further comprising a segmentation module for automatically decomposing image material into primary image objects, including a plurality of elemental bitmap objects and curvilinear bitmap objects.

11. The graphical input and display system according to claim 1, wherein said grouping module reconstitutes the grouping structure when grouping structure is destroyed through operation on not less than one bitmap object.

12. The graphical input and display system according to claim 1, wherein said grouping module permits the creation of new groups.

13. The graphical input and display system according to claim 1, wherein said grouping module invokes automatic recognition routines to identify perceptually meaningful groups to be represented by not less than one composite object.

14. The graphical input and display system according to claim 1, wherein said grouping module infers a user'"'"'s intent as to the kind of perceptual object of most interest to said user.

15. The graphical input and display system according to claim 1, wherein said user interaction module exchanges typed text for selected image objects.

16. The graphical input and display system according to claim 15, wherein exchanging typed text for selected image objects comprises maintaining the grouping relationships present between bitmap objects and other objects in the graphical image.

17. A method for organizing a source electronic image entered on a display device into image objects comprising:

selecting arbitrary existing image material;

establishing primary image objects;

modifying the image material; and

reconstituting an unrestricted grouping structure, including setting a priority and ordering for composite objects, wherein said priority ordering includes at least one member selected from the group consisting of rearrangement based on algorithms that determine smallest-to-largest spatial extent, measures of perceptual salience, and the context of selection criteria recently exercised by the user, wherein reconstituting said unrestricted grouping structure comprises;

identifying composite objects supported by a plurality of primary image objects;

moving not less than one primary image object;

determining whether the new location of the moved primary image objects destroys group structure;

forming a new composite object with those primary image objects not moved;

creating a new composite object for primary image objects that were moved; and

creating a new grouping structure.

18. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising distinguishing foreground pixels of text and line art from background pixels.

19. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising creating not less than one primary image object representing the preprocessed source image, in which background pixels are set as transparent.

20. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising creating hyperlinks between regions containing image material.

21. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, wherein said selecting existing image material comprises inputting a closed path enclosing the desired image material.

22. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, wherein said selecting existing image material comprises splitting at least one primary object to form fragmented primary image objects.

23. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, wherein creating a new composite object comprises:

gathering fragmented primary objects and remaining unfragmented primary objects enclosed by said closed selection path to form enclosed primary objects;

creating a provisional composite object representing the group of enclosed primary objects;

removing affected primary objects from a primary object list;

adding fragmented primary objects to said primary object list; and

creating support links to enclosed primary objects.

24. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, wherein establishing a new primary image object by merging not less than two primary image objects comprises:

selecting at least one primary image object;

determining the bounding box of the union of the selected primary image objects;

creating a new blank bitmap corresponding to the size of said bounding box;

copying foreground material from all selected primary image objects into said new bitmap;

creating a new primary image object of said new bitmap;

removing selected primary image objects from a primary object list; and

adding new primary object to said primary object list.

25. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, wherein creating new composite objects comprises:

locating composite objects supported by all selected primary objects to form filly supported composite objects;

replacing support links from said selected primary objects with support links to a new primary object for each said filly supported composite object;

locating composite objects supported by a subset of selected primary objects to form partially supported composite objects; and

removing all support links to primary objects for each partially supported composite object.

26. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising:

segmenting image regions into text and line art, continuous tone/photographic, and halftone subregions;

creating separate bitmap objects for said continuous tone and said halftone regions;

deleting said separate bitmap objects from the source image;

breaking the processed source image into a plurality of elemental bitmap objects to form primary objects; and

identifying collections of bitmap objects into composite objects, or groups, within a lattice grouping structure.

27. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising invoking automatic recognition routines to identify perceptually meaningful groups that should be represented by composite objects.

28. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising automatically determining whether a graphical image is to be treated as a photographic image.

29. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising determining whether to treat all elements of the graphical image as text and line art.

30. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising distinguishing foreground pixels from background pixels and setting background pixels to transparent.

31. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising determining whether to apply pre-segmentation to the graphical image.

32. The method for organizing a source electronic image entered on a display device into image objects according to claim 17, further comprising exchanging typed text for selected image objects.

33. The method for organizing a source electronic image entered on a display device into image objects according to claim 32, wherein exchanging typed text for selected image objects comprises maintaining grouping relationships present between the bitmap objects and other objects in the image.

34. The method for organizing a source electronic image entered on a display device into image objects according to claim 33, further comprising establishing not less than one alterative formality relation between not less than one composite objects and not less than one text string bitmap object.

35. An article of manufacture comprising a computer usable medium having computer readable program code embodied in said medium which, when said program code is executed by said computer causes said computer to perform method steps for editing and manipulating an electronic image entered onto a display, said method steps comprising:

decomposing the electronic image into primary image objects;

organizing said primary image objects into unrestricted, arbitrary groups of primary image objects, such that each said primary image object belongs to one or more groups and each group contains not less than one primary image object;

creating new primary image objects; and

reorganizing all or part of said groups of primary image objects into one or more new groups of primary image objects in response to user manipulation of not less than one primary image object, including setting a priority and ordering for said new groups of primary image objects, wherein said priority ordering includes at least one member selected from the group consisting of rearrangement based on algorithms that determine smallest-to-largest spatial extent, measures of perceptual salience, and the context of selection criteria recently exercised by the user.

36. A memory for storing data for access by a program being executed on a computer for creating and manipulating data representing an electronic image, said memory comprising:

a lattice data structure stored in said memory for providing an unrestricted, arbitrary grouping structure defining the relationships between primary image objects and composite objects, including;

a plurality of primary objects stored in said memory, said primary objects being bitmap objects or curvilinear objects, said primary objects arranged in at least one primary object list;

a plurality of composite objects, each composite object including at least one primary object, said composite objects arranged in at least one composite object list, wherein said composite objects are given a priority and ordering, wherein said priority ordering includes at least one member selected from the group consisting of rearrangement based on algorithms that determine smallest-to-largest spatial extent, measures of perceptual salience, and the context of selection criteria recently exercised by the user; and

a plurality of hyperlinks linking said primary objects with either or both destination and source objects.

37. A memory for storing data for access by a program being executed on a computer for creating and manipulating data representing an electronic image according to claim 36, wherein said curvilinear objects are derived from at least one of either the electronic image data or input provided by a user.