Word-to-word selection on images

• US 6,640,010 B2
• Filed: 11/12/1999
• Issued: 10/28/2003
• Est. Priority Date: 11/12/1999
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Dance, Christopher R., Taylor, Stuart A., Seeger, Mauritius, Newman, William M.
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/438,891
• Publication Number:
  US 20030185448A1
• Time in Patent Office: 1,446 Days
• Field of Search: 382/175, 382/176, 382/177, 382/179, 382/180, 382/186, 382/187, 382/188, 382/189, 382/229, 382/289, 382/290, 382/310, 382/311, 382/321, 382/322, 382/323, 358/453, 358/462, 358/474, 707/500, 707/531, 235/454
• US Class Current: 382/229
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields

Abstract

An image processing technique for selecting a text region from an image is described. Character and formatting information for each word in the image is used to determine an active region for each word in the image. For a preferred embodiment of the present invention, the character and formatting information is derived during optical character recognition (OCR). A first and last word within a selected text region is identified based on at least one active region associated with at least one word within the selected text region. Using the first and last words within the selected text region, all words within the selected text region are identified. An image of the selected text region may be displayed. Text contained within the selected text region may be copied to an application program.

Claims

1. A method of selecting a text region from an image, comprising:

(a) accessing from a memory device character and formatting information for each word in the image;

the formatting information including word bounding box, text line boundary, and text region boundary information;

(b) determining an active region associated with each word in the image based on the character and formatting information;

the active regions identifying regions in the image between word bounding boxes, text line boundaries, and text region boundaries within text regions of the image;

(c) identifying a first word and a last word in a selected text region based on at least one active region associated with at least one word within the selected text region; and

(d) identifying all words within the selected text region based on the first and last words in the selected text region.

2. The method of claim 1, comprising, prior to (a):

(e) generating the character information from image data for each word in the image;

(f) determining the formatting information for each word in the image; and

(g) storing the character and formatting information for each word in the image in the memory device.

3. The method of claim 1, wherein (c) comprises identifying the first and last words based on a first active region and a second active region, the first active region associated with the first word in the selected text region and the second active region associated with the last word in the selected text region.

4. The method of claim 1, further comprising

(e) determining formatting Information for each word in the image by determining word bounding box, text line boundary, and text region boundary information for each word in the image;

and

wherein (c) comprises;

(c1) determining boundaries of the active region for each of the words based upon at least the word bounding box and one of the text line boundary and the text region boundary information for each of the words;

(c2) matching at least one designated image coordinate point to an image coordinate point associated with the at least one active region;

wherein each active region determined for each word in the Image represents a plurality of image coordinate points.

5. The method of claim 1, comprising after (b):

(e) displaying the image;

(f) receiving user input designating at least one image coordinate point within the selected text region; and

(g) identifying the at least one active region associated with the at least one image coordinate point designated by the user;

wherein each active region determined for each word in the image represents a plurality of image coordinate points.

6. The method of claim 1, further comprising:

(e) performing optical character recognition (OCR) on the image to generate OCR results, the OCR results including text and formatting information for each word in the image;

(f) generating an image of the selected text region based on the character information and the formatting information for each word within the selected text region; and

(g) displaying the image of the selected text region.

7. The method of claim 1, further comprising:

(e) storing the text contained within the selected text region in a clipboard; and

(f) transferring the text stored in the clipboard to an application program.

8. The method of claim 1, further comprising:

(e) copying the text contained within the selected text region to an application program.

9. The method of claim 2, further comprising, prior to (e):

(h) determining that the image is warped;

(i) estimating a warp function; and

(j) applying the warp function to the image to dewarp the image.

10. The method of claim 9, further comprising, after (d):

(k) applying the warp function to an image of the selected text region to warp an image of the selected text region.

11. The method of claim 1, further comprising repeating steps (c) and (d).

12. The method of claim 7, further comprising repeating steps (c) through (f).

13. The method of claim 8, further comprising repeating steps (c) through (e).

14. A method of selecting a text region from an image, comprising:

(a) performing optical character recognition (OCR) on the image to generate OCR results, the OCR results including text and formatting information for each word in the image;

the formatting information including word bounding box, text line boundary, and text region boundary information;

(b) determining an active region associated with each word in the image based on the OCR results;

the active regions identifying regions in the image between word bounding boxes text line boundaries, and text region boundaries within text regions of the image;

(c) identifying a first word and a last word in a selected text region based on at least one active region associated with at least one word within the selected text region; and

(d) identifying all words within the selected text region based on the first and last words in the selected text region.

15. The method of claim 14, further comprising:

(e) after (b);

(e1) displaying the image,

(e2) receiving user input designating at least one image coordinate point within the selected text region,

(e3) identifying the at least one active region associated with the at least one image coordinate point designated by the user; and

(f) after (d);

generating an image of the selected text region;

wherein each active region determined for each word in the image represents a plurality of image coordinate points; and

wherein (a) comprises determining word bounding box, text line boundary, and text region boundary information for each of the words, and

wherein (b) comprises determining boundaries of the active region for each of the words based upon at least the word bounding box, and one of the text line boundary and the text region boundary information for each of the words.

16. The method of claim 14, further comprising, prior to (a):

(e) determining that the image is warped;

(f) estimating a warp function; and

(g) applying the warp function to the image to dewarp the image, and

further comprising, after (d);

(h) applying the warp function to an image of the selected text region to warp the image of the selected text region.

17. A method of selecting a text region from an image, comprising:

(a) converting image data into coded text for each word in the image;

(b) determining formatting information for each of the words in the image;

the formatting information including word bounding box, text line boundary, and text region boundary information;

(c) determining an active region associated with each word based on the formatting information, each active region representing a plurality of image coordinate points associated with one of the plurality of words;

the active regions identifying regions in the image between word bounding boxes, text line boundaries, and text region boundaries within text regions of the image;

(d) identifying a first image coordinate point designating a beginning of a selected text region and a second image coordinate point designating an end of the selected text region;

(e) determining the active region corresponding to each of the first and second image coordinate points;

(f) determining the word associated with each of the active regions corresponding to the first and second image coordinate points; and

(g) identifying all the words within the selected text region.

18. The method of claim 17, further comprising:

(h) displaying an image of the selected text region.

19. The method of claim 17, further comprising:

(h) saving the text contained within the selected text region to a clipboard;

(i) transferring the text stored in the clipboard to an application program.

20. The method of claim 19, further comprising:

(j) repeating steps (d) through (i).

21. The method of claim 17, wherein (b) comprises determining word bounding box, text line boundary, and text region boundary information for each of the words, and wherein (c) comprises determining boundaries of the active region for each of the words based upon at least the word bounding box, and one of the text line boundary and the text region boundary information for each of the words.

22. The method of claim 17, wherein (e) further comprises identifying a first active region which includes a first image coordinate point that matches the identified first image coordinate point, and identifying a second active region which includes a second image coordinate point that matches the identified second image coordinate point.

23. The method of claim 17, further comprising, prior to (a):

(h) determining that the image is warped;

(i) estimating a warp function; and

(j) applying the warp function to the image to dewarp the image;

and further comprising, after (g);

(k) applying the warp function to an image of the selected text region to warp the image of the selected text region.

24. An article of manufacture for selecting a text region from an image, the article of manufacture comprising computer usable media including computer readable instructions embedded therein that causes a computer to perform:

(a) accessing from a memory device character and formatting information for each word in the image;

the formatting information including word bounding box, text line boundary, and text region boundary information;

(b) determining an active region associated with each of the words in the image based on the character information and formatting information for each of the words in the image;

the active regions identifying regions in the image between word bounding boxes, text line boundaries, and text region boundaries within text regions of the image;

(c) identifying a first word and a last word in a selected text region based on at least one active region associated with at least one of the words within the selected text region; and

(d) identifying all words within the selected text region based on the first and last words in the selected text region.

25. The article of manufacture of claim 24, further causing the computer to perform:

(e) generating character information from image data for each word in the image;

(f) determining formatting information for each word in the image; and

(g) storing the character and formatting information for each word in the image.

26. The article of manufacture of claim 24, further causing the computer to perform:

(e) after (b);

(e1) displaying the image;

(e2) receiving user input designating at least one image coordinate point from the displayed image; and

(e3) identifying the at least one active region associated with the at least one image coordinate point designated by the user; and

(f) after (d);

generating an image of the selected text region;

wherein each active region determined for each word in the image represents a plurality of image coordinate points; and

wherein (a) comprises determining word bounding box, text line boundary, and text region boundary information for each of the words, and

wherein (b) comprises determining boundaries of the active region for each of the words based upon at least the word bounding box, and one of the text line boundary and the text region boundary information for each of the words.

27. The article of manufacture of claim 24, further causing the computer to perform:

(e) displaying an image of the selected text region.

28. The article of manufacture of claim 24, further causing the computer to perform:

(e) storing the text contained within the selected text region in a clipboard; and

(f) transferring the image of the selected text region to an application program.

29. The article of manufacture of claim 24, further causing the computer to perform:

(e) copying the text contained within the selected text region to an application program.

30. The article of manufacture of claim 24, further causing the computer to perform:

(e) determining that the image is warped;

(f) estimating a warp function; and

(g) applying the warp function to the image to dewarp the image.

31. The article of manufacture of claim 30, further causing the computer to perform:

(h) applying the warp function to the selected text region to warp the image of the selected text region.

32. An image processing system, comprising:

a character recognition module that provides character information for each word in an image;

a formatting module that provides formatting information for each of the words in the image;

the formatting Information including word bounding box, text line boundary, and text region boundary information;

an active region module that provides active region information associated with each of the words in the image based on the character information and the formatting information for each of the words;

the active regions identifying regions in the image between word bounding boxes, text line boundaries, and text region boundaries of the image;

a word selection module that provides word selection information indicating which words are included within a selected text region based on the active region information associated with at least one word with the selected text region; and

an image generating module that produces an image of the selected text region based on the word selection information.

33. The image processing system of claim 32, further comprising:

a warp module that produces a warp function for dewarping the image and warping the image of the selected text region.

34. The image processing system of claim 32, further comprising:

a display for displaying the image; and

user I/O for receiving user input designating at least one image coordinate point within the selected text region;

wherein the formatting module determines word bounding box, text line boundary, and text region boundary information for each of the words;

wherein the active region module;

(a) determines boundaries of the active region for each of the words based upon at least the word bounding box, and one of the text line boundary and the text region boundary information for each of the words; and

(b) identifies the at least one active region associated with the at least one image coordinate point designated by the user;

each active region determined for each word in the image represents a plurality of image coordinate points.