Method of editing static digital combined images comprising images of multiple objects

• US 9,179,035 B2
• Filed: 07/18/2012
• Issued: 11/03/2015
• Est. Priority Date: 07/19/2011
• Status: Active Grant

• Current Assignee: HP Printing Korea Co., Ltd. (HP Inc.)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kurilin, Ilya Vasilievich, Lee, Ho-keun, Safonov, Ilya Vladimirovich
• Primary Examiner(s): Reinier, Barbara
• Assistant Examiner(s): WAIT, CHRISTOPHER
• Application Number:
  US13/551,887
• Publication Number:
  US 20130021629A1
• Time in Patent Office: 1,203 Days
• Field of Search: 382/173
• US Class Current: 1/1
• CPC Class Codes: H04N 1/3873 defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming
• CPC Class Codes: H04N 1/3878 Skew detection or correction

Abstract

A method of editing a combined image which includes a plurality of object images. This editing method includes displaying an outline rectangle which forms outlines of one or more object images within the combined image, and when a predetermined first operation is performed on an object image selected from among the object images located within the outline rectangle, segmenting the selected object image and another object image located within the outline rectangle, and alternatively, when a predetermined second operation is performed on an object image selected from among the object images located within the outline rectangle, combining the selected object image with another object image located within an outline rectangle other than the outline rectangle. After the segmenting or combining is performed, renewing displaying of the outline rectangle according the segmenting or combining result.

Claims

1. A method of editing a combined image comprising a plurality of object images, the method comprising:

determining coordinates of vertexes of an outline rectangle which forms outlines of an object image within the combined image, and displaying the outline rectangle according to the determined coordinates of vertexes;

when a predetermined first operation is performed on an object image selected from among the object images located within the outline rectangle by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle, segmenting the selected object image and another object image located within the outline rectangle;

when a predetermined second operation is performed on an object image within the outline rectangle selected from among the object images by performing the drag operation on the selected object image into another outline rectangle other than the outline rectangle, combining the selected object image with another object image located within the another outline rectangle; and

renewing displaying of the outline rectangle according the segmenting or combining,

wherein the determining the coordinates of the vertexes includes scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

2. The method according to claim 1, further comprising:

determining coordinates of vertexes of an outline rectangle which forms outlines of the object image in the combined image,

wherein the displaying displays an outline rectangle which forms outlines of the object image within the combined image according to the determined coordinates of vertexes.

7. The method according to claim 1, wherein:

the predetermined first operation is an operation of moving the selected object image to a free location of the combined image which is not occupied by the outline rectangle which includes the selected object image, and

the predetermined second operation is an operation of moving the selected object image inside the another outline rectangle other than the outline rectangle which includes the selected object image.

8. The method according to claim 7, wherein the moving operation is performed by a drag and drop operation.

3. A method of editing a combined image comprising a plurality of object images, the method comprising:

displaying an outline rectangle which forms outlines of an object image within the combined image;

when a predetermined first operation is performed on an object image selected from among the object images located within the outline rectangle, segmenting the selected object image and another object image located within the outline rectangle;

when a predetermined second operation is performed on an object image selected from among the object images located within the outline rectangle, combining the selected object image with another object image located within an outline rectangle other than the outline rectangle;

renewing displaying of the outline rectangle according the segmenting or combining; and

determining coordinates of vertexes of an outline rectangle which forms outlines of the object image in the combined image,

wherein the displaying displays an outline rectangle which forms outlines of the object image within the combined image according to the determined coordinates of vertexes, and

wherein the determining coordinates of vertexes of an outline rectangle comprises;

scaling the combined image with a predetermined resolution;

converting the scaled combined image into a binary image;

detecting foreground and background areas in the binary image;

classifying the area according to a predetermined category;

combining the foreground areas into groups based on the classification result; and

calculating coordinates of vertexes of the outline rectangle on each group combined.

4. The method according to claim 3, wherein the converting the scaled combined image into a binary image comprises:

applying a gamma revision to the scaled combined image;

converting an image where the gamma revision is applied into a half-tone image;

detecting an edge in the half-tone image;

performing critical-value processing of the half-tone image;

performing OR logical calculation on each pixel of the half-tone images critical-value processed half-tone images corresponding to each pixel of the half-tone image which includes the detected edge;

deleting an area of which its size does not exceed a predetermined critical value in the combined binary image; and

deleting an area adjacent to the detected edge in the combined binary image.

5. The method according to claim 3, wherein the classifying the area according to the predetermined category comprises:

detecting a straight line portion on each side of the detected foreground area;

estimating a parameter of the detected straight line portion;

calculating mutual alignment of the detected straight line portion;

if the calculated mutual alignment corresponds to a rectangle shape, classifying the foreground areas into foreground areas having rectangle shapes;

if the calculated mutual alignment corresponds to a straight line shape, classifying the foreground areas into foreground areas having straight line shapes;

if a distance between the foreground areas having rectangle shapes does not exceed a predetermined value, classifying a portion of a background located between the foreground areas having rectangle shapes into a background area located between areas having rectangle shapes; and

classifying areas that are not classified by the above classification into foreground areas having shapes other than rectangle shapes.

6. The method according to claim 5, wherein the combining the foreground areas into groups based on the classification result comprises:

analyzing whether a crossing occurred by a descending order of sizes of rectangles in order to detect another area which crosses the foreground areas classified into the foreground areas having rectangle shapes;

if the another crossed area is not an area classified into a background area located between the areas having rectangle shapes, as a result of the analysis, combining the another crossed area with the area having rectangle shapes into one group;

updating coordinates of vertexes of the outline rectangle according to the combination;

analyzing a difference of distance and skew angle of the foreground areas classified into areas having rectangle shapes and the foreground areas classified into areas having straight line shapes;

if the difference of distance and skew angle do not exceed a predetermined value, as a result of the analysis, combining the areas having straight line shapes and the areas having rectangle shapes into one group;

updating coordinates of vertexes of the outline rectangle according to the combination;

if there is searched a foreground area having a shape other than rectangle shape of which a distance from the foreground areas classified into the areas having rectangle shapes is smaller than a predetermined value, combining the foreground area having a shape other than a rectangle shape with the areas having rectangle shapes into one group;

updating coordinates of vertexes of the outline rectangle according to the combination;

searching for a foreground area having a shape other than a rectangle shape which is not grouped with the areas having rectangle shapes; and

if there is searched a foreground area having a shape other than rectangle shape which is not grouped with the areas having rectangle shapes, as a result of the search, and a size of the area is bigger than a predetermined value, calculating coordinates of vertexes of the outline rectangle on the area.

9. A user terminal comprising:

a display unit which displays a GUI (Graphic User Interface) for editing a combined image which includes a plurality of object images;

a user interface unit which receives an input of a user manipulation on the GUI; and

a control unit which, when a predetermined first operation is performed on an object image selected from among object images located within an outline rectangle by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle, segments the selected object and another object image located within the outline rectangle, and when a predetermined second operation is performed on an object image within the outline rectangle selected from among object images by performing the drag operation on the selected object image into another outline rectangle other than the outline rectangle, combines the selected object image with another object image located within the another outline rectangle, and renews displaying of the outline rectangle according to the segmenting or combining;

wherein the outline rectangle is an identifier which forms outlines of the object image within the combined image, and

wherein coordinates of vertexes of the outline rectangle are determined by scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

10. The user terminal according to claim 9, wherein the predetermined first operation is an operation of moving the selected object image to a free location of the combined image which is not occupied by the outline rectangle which includes the selected object image, and

the predetermined second operation is an operation of moving the selected object image inside the another outline rectangle other than the outline rectangle which includes the selected object image.

11. The user terminal according to claim 10, wherein the moving operation is performed by a drag and drop operation.

12. An image forming apparatus comprising:

a scanning unit which generates a combined image which includes a plurality of object images corresponding to a read manuscript;

a display unit which displays a GUI (Graphic User Interface) for editing the generated combined image;

a user interface unit which receives an input of a user manipulation on the GUI;

a control unit which, when a predetermined first operation is performed on an object image selected from among object images located within an outline rectangle by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle, segments the selected object and another object image located within the outline rectangle, and when a predetermined second operation is performed on an object image within the outline rectangle selected from among object images by performing the drag operation on the selected object image into another outline rectangle other than the outline rectangle, combines the selected object image with another object image located within the another outline rectangle, and renews displaying of the outline rectangle according to the segmenting or combining;

wherein the outline rectangle is an identifier which forms outlines of the object image within the combined image, and

wherein coordinates of vertexes of the outline rectangle are determined by scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

13. The image forming apparatus according to claim 12, wherein the predetermined first operation is an operation of moving the selected object image to a free location of the combined image which is not occupied by the outline rectangle which includes the selected object image, and

the predetermined second operation is an operation of moving the selected object image inside the another outline rectangle other than the outline rectangle which includes the selected object image.

14. The image forming apparatus according to claim 13, wherein the moving operation is performed by a drag and drop operation.

15. A non-transient computer readable recording medium which stores a program code to perform an editing method in an apparatus for editing a combined image comprising a plurality of object images, the editing method comprising:

determining coordinates of vertexes of an outline rectangle which forms outlines of an object image within the combined image, and displaying the outline rectangle according to the determined coordinates of vertexes;

when a predetermined first operation is performed on an object image selected from among object images located within the outline rectangle by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle, segmenting the selected object image and another object image located within the outline rectangle;

when a predetermined second operation is performed on an object image within the outline rectangle selected from among object images by performing the drag operation on the selected object image into another outline rectangle other than the outline rectangle, combining the selected object image with another object image located within the another outline rectangle; and

renewing displaying of the outline rectangle according the segmenting or combining,

wherein the determining the coordinates of the vertexes includes scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

16. A method of editing a combined image comprising a plurality of object images, the method comprising:

determining coordinates of vertexes of an outline rectangle which forms outlines of an object image within the combined image, and displaying the outline rectangle according to the determined coordinates of vertexes;

segmenting a selected object image from other object images located within the outline rectangle when a predetermined first operation is performed on the selected object image by performing a drag operation on the selected object from an internal portion to an outside of the outline rectangle;

combining a selected object image within the outline rectangle with another object image located within another outline rectangle other than the outline rectangle when a predetermined second operation is performed on the selected object image by performing the drag operation on the selected object image into the another outline rectangle; and

updating the displaying to display the outline rectangle including the performed segmenting or combining operation,

wherein the determining the coordinates of the vertexes includes scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

17. A method of editing a combined image comprising a plurality of object images, the method comprising:

displaying an outline rectangle to be manipulated;

segmenting a selected object image from other object images within a displayed outline rectangle when a segmenting request is received by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle;

combining a selected object image within a displayed outline rectangle with another object image within another outline rectangle when a combining request is received by performing the drag operation on the selected object image into the another outline rectangle; and

updating the displaying of the outline rectangle to illustrate the segmenting or combining result,

wherein coordinates of vertexes of the outline rectangle are determined by scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

18. A non-transient computer readable recording medium which stores a program code to perform an editing method, the method comprising:

determining coordinates of vertexes of an outline rectangle which forms outlines of an object image within the combined image, and displaying an outline rectangle according to the determined coordinates of vertexes;

segmenting a selected object image from other object images located within the outline rectangle when a predetermined first operation is performed on the selected object image by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle;

combining a selected object image within the outline rectangle with another object image located within another outline rectangle other than the outline rectangle when a predetermined second operation is performed on the selected object image by performing the drag operation on the selected object image into the another outline rectangle; and

updating the displaying to display the outline rectangle including the performed segmenting or combining operation,

wherein the determining the coordinates of the vertexes includes scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.

19. A non-transient computer readable recording medium which stores a program code to perform an editing method, the method comprising:

displaying an outline rectangle to be manipulated;

segmenting a selected object image from other object images within a displayed outline rectangle when a segmenting request is received by performing a drag operation on the selected object from an internal portion of the outline rectangle to an outside of the outline rectangle;

combining a selected object image within a displayed outline rectangle with another object image within another outline rectangle when a combining request is received by performing the drag operation on the selected object image into the another outline rectangle; and

updating the displaying of the outline rectangle to illustrate the segmenting or combining result,

wherein coordinates of vertexes of the outline rectangle are determined by scaling the combined image with a predetermined resolution prior to converting the scaled combined image into a binary image, and calculating the coordinates of the vertexes using groups determined based on classifying areas of the binary image according to a predetermined category.