Method and apparatus for producing a composed output resembling an image

• US 6,587,599 B1
• Filed: 07/14/2000
• Issued: 07/01/2003
• Est. Priority Date: 02/24/1998
• Status: Expired due to Term

• Current Assignee: Resource Consortium Limited, LLC
• Original Assignee: Xmlcities
• Inventors: Huang, Shih-Chi
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US09/615,658
• Time in Patent Office: 1,082 Days
• Field of Search: 382/162, 382/187, 382/163, 382/164, 382/165, 382/284, 382/296, 382/298, 382/280, 345/103, 345/594, 345/597, 345/600, 345/619, 345/626, 345/530, 345/700, 345/629, 345/681, 348/239, 348/586, 348/588
• US Class Current: 382/284
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

The disclosed system has particular applications to image archives and reediting. Given an image of an article having graphic objects including texts and pictures, the disclosed system first distinguishes graphic objects from the background by segmenting the image so as to produce an object image comprising a foreground, wherein the foreground corresponds to the graphic objects in the image. With respect to the foreground, the disclosed system “repaints” those spaces previously occupied by the graphic objects, referred to as object spaces, with the background color in the image as such the background continuity is preserved when the graphic objects are rendered and subsequently superimposed onto the repainted background. To account for various backgrounds, the disclosed system uses a point-wise patching approach followed by a one-dimensional patching approach if the object spaces contain two different colors. In addition, a two-dimensional patching approach is implemented to account for printing effects such as half-toning and mosaic patterns.

Claims

1. A method for producing a composed output resembling an image, said method comprising:

receiving said image comprising graphic objects;

producing an object image from said image, wherein said object image comprises a background and a foreground, said foreground corresponding to said graphic objects in said image, said background corresponding to an area unoccupied by said graphic objects, and both of said foreground and background are extracted from said image;

generating a patched image from said object image by filling in spaces occupied by said foreground with colors of said background;

receiving rendered objects corresponding to said foreground;

reformatting said rendered objects in a way preferable to a user; and

superimposing said rendered objects onto said patched image to produce the composed output.

2. The method as recited in claim 1, wherein said superimposing said rendered objects onto said patched image comprises placing said rendered objects onto the patched image with respect to original positions of said foreground to produce the composed output image with said rendered objects replacing said graphic objects.

3. The method as recited in claim 2, wherein said rendered objects are generated from said graphic objects in said acquired image with respect to said foreground in said object image so that said composed output image containing said rendered objects resembling said graphic objects.

4. The method as recited in claim 3,

wherein said acquired image comprises vector pixels; and

wherein said acquiring said image comprises loading said acquired image into a working memory for subsequent manipulations of said vector pixels.

5. The method as recited in claim 4, wherein said producing said object image comprises:

transforming said vector pixels, correspondingly and respectively, to scalar pixels; and

binarizing said scalar pixels to generate a binarized image.

6. The method as recited in claim 5, wherein said producing said object image further comprises segmenting said binarized image into said object image.

7. The method as recited in claim 6, wherein said producing said object image further comprises separating interactively said graphic objects into disjoined graphic objects if more than one of said graphic objects are ambiguously joined together when displayed.

8. The method as recited in claim 7, wherein said generating said patched image comprising:

selecting a reference color; and

patching said object spaces in said acquired image with said reference color with respect to said foreground to produce said patched image.

9. The method as recited in claim 8, wherein said generating said patched image further comprising measuring a color difference in said patched image by comparing patched said object spaces with areas, cross-referenced by said background in said object image, in said acquired image.

10. The method as recited in claim 9, wherein said generating said patched image further comprising:

selecting a new reference color if the comparison in said comparing said patched object spaces with said areas in said acquired image is greater than a threshold; and

patching said object spaces in the acquired image with said new reference color with respect to said foreground to produce a new patched image.

11. The method as recited in claim 7, wherein said generating said patched image further comprising:

measuring a color difference in said patched image by comparing said object spaces with areas, cross-referenced by said background in said object image, in said acquired image;

finding a reference color recursively to minimize said color difference; and

patching said object spaces in said acquired image with said reference color with respect to said foreground to produce said patched image.

12. The method as recited in claim 11, wherein said generating said patched image further comprising:

converting one of said object spaces, patched with said reference color, that repeatedly causes said color difference greater than a threshold to a luminance component and two chrominance components thereof;

applying Fourier Transform to said luminance component and said two chrominance components to produce a spectral luminance and two chrominance components, respectively;

filtering said spectral luminance and said two chrominance components, respectively, to produce a filtered spectral luminance component and two filtered spectral chrominance components; and

applying inverse Fourier Transform to said filtered spectral luminance component and said two filtered spectral chrominance components, respectively, to produce a filtered luminance component and two filtered chrominance components.

13. The method as recited in claim 12, wherein said generating said patched image further comprising converting said filtered luminance component and said two chrominance components back to said image.

14. The method as recited in claim 1 further comprising:

selecting a chosen object from said object image; and

receiving rendered chosen object generated from said acquired image with respect to said chosen object.

15. The method as recited in claim 14 further comprising superimposing said rendered chosen object onto said patched image to produce a composed output image.

16. The method as recited in claim 1 further comprising:

selecting interactively a regional image including selected objects from the acquired image;

generating a regional object image from said regional image;

generating a regional patched image from said regional image with respect to said regional object image;

receiving rendered selected objects generated from said regional image with respect to said regional object image; and

superimposing said rendered objects onto said regional patched image to produce a composed regional output.

17. The method as recited in claim 16 further comprising superimposing said composed regional output onto the acquired image according to said regional image to produce a composed output.

18. An apparatus for producing a composed output resembling in an image, said apparatus comprising:

a working memory containing said image comprising graphic objects, each of said graphic objects occupying a respective object space;

a display screen;

an object engine generating an object image entirely from said image, said object image containing a foreground and a background, said foreground corresponding to said graphic objects, and said background corresponding to areas unoccupied by said graphic objects, wherein both of said foreground and background are extracted from said image, and said object image is displayable on said display screen so that a user can verify if said foreground in said object image includes correctly all of said graphic objects;

a patch generator generating a patched Image from said object Image by filling in spaces occupied by said foreground;

said patched image displayable on said display screen so that the user can determine if said spaces are filled properly;

an object rendering engine producing rendered objects from said graphic objects in said Image with respect to said object image; and

a composer superimposing said rendered objects onto said patched image to produce the composed image that has said rendered objects in place of said graphic objects, respectively.

19. The apparatus as recited in claim 18 wherein said object image is in binary format.

20. The apparatus as recited in claim 19 wherein said image comprises vector pixels.

21. The apparatus as recited in claim 20, wherein said object engine comprises:

a color-to-gray converter converting, correspondingly and respectively, said vector pixels to scalar pixels; and

a binarizer binarizing said scalar pixels to produce a binarized image.

22. The apparatus as recited in claim 21, wherein said patch generator comprises:

a point-wise patching generator patching each of said object spaces using a color chosen from said image with respect to said background in said object image from said object engine.

23. The apparatus as recited in claim 22, wherein said patch generator further comprises:

a color difference calculator determining a color difference between patched said object spaces and areas in said image cross-referenced by said background.

24. The apparatus as recited in claim 18, wherein said patch generator comprises:

a one-dimensional patching generator producing said patched image by patching object spaces with a reference color adaptively chosen to minimize a color difference between said patched color spaces and areas, cross-referenced by said foreground in said object image, in said image.

25. The apparatus as recited in claim 24, wherein said patch generator further comprises:

a two-dimensional patching generator producing a filtered image from said patched image.

26. The apparatus as recited in claim 25, wherein said two-dimensional patching generator comprises:

a color separator separating said object image into a luminance component and two chrominance components;

a Fourier transformer receiving said luminance component and said two chrominance components and outputting a spectral luminance component and two chrominance components, sequentially and respectively;

a filter filtering said spectral luminance component and said two chrominance components to produce a filtered spectral luminance component and two filtered spectral chrominance components; and

a Fourier inverse transformer, coupling to said filter, transforming said filtered spectral luminance component and said two filtered spectral chrominance components to a filtered luminance component and two filtered chrominance components, respectively.

27. The apparatus as recited in claim 26, wherein said two-dimensional patching generator further comprises:

a color transformer, coupling to said Fourier inverse transformer, transforming said filtered luminance component and said two filtered chrominance components to a color image.

28. The apparatus as recited in claim 27, wherein said two-dimensional patching generator further comprises:

an adder adding said color image to said patched image from said one-dimensional patching generator.