Generator for document with HTML tagged table having data elements which preserve layout relationships of information in bitmap image of original document

• US 5,893,127 A
• Filed: 11/18/1996
• Issued: 04/06/1999
• Est. Priority Date: 11/18/1996
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Information Systems Incorporated (Canon Inc.)
• Inventors: Tyan, Ching-Yu, Huang, Hung Khei, Niki, Toru
• Primary Examiner(s): Burwell, Joseph R.
• Application Number:
  US08/751,676
• Time in Patent Office: 869 Days
• Field of Search: 395/774, 395/762, 395/779, 395/780, 395/781, 395/785, 395/788, 382/176, 382/173, 382/175, 382/180, 707/513, 707/518, 707/509
• US Class Current: 715/209
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

Automatic generation of hypertext markup language (HTML) files based on bitmap image data, which faithfully preserves layout information of an original document from which the bitmap data was obtained. Generally, multi-column document layouts result in automatic generation of HTML files that use HTML "table tags" to display each of the different columns. More particularly, a bitmap image is obtained such as by scanning or retrieval of a pre-existing image, and the bitmap image is segmented into blocks. The location of each block is determined, each block is analyzed in preparation for insertion of appropriate data into an HTML file, and layout analysis is performed to identify layout relationships between the blocks based on the relative locations of the blocks in the bitmap image. Based on the layout relationships, a block type is determined for each block, column span and row span data for each block is determined, blocks are re-ordered if needed, and an HTML file is generated in which blocks are tagged as data elements in a row of an HTML "table tag" based on block type and based on column and row span information for the block.

Claims

1. A method for generating a hypertext markup language (HTML) file based on a bitmap image, comprising:

obtaining at least two horizontally adjacent image blocks in separate vertical columns of the bitmap image, including an image block for plural character images and an image block for a bitmap image;

generating an image file which stores the bitmap image; and

automatically generating, from the image blocks, an HTML file in which blocks corresponding to the two horizontally adjacent image blocks obtained from the bitmap image are tagged as data elements in a row of an HTML tagged table, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

2. A method according to claim 1, wherein an HTML code is assigned to each block in accordance with a position of the block relative to other elements within the bitmap image.

3. A method according to claim 1, wherein table tags are generated and inserted into the HTML file in said generating step.

4. A method according to claim 1, further comprising the step of character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

5. A method for generating a hypertext markup language (HTML) file based on a bitmap image, comprising:

a segmenting step of segmenting the bitmap image to obtain image blocks, including an image block for plural character images and an image block for a bitmap image;

a determining step of determining where in the bitmap image each of the image blocks is located;

an identifying step of identifying positional relationships between the image blocks based on thier relative locations;

a first generating step of generating an image file which stores the bitmap image; and

a second step of automatically generating an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an HTML tagged table, the tags being determined in accordance with the identified positional relationships, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

6. A method according to claim 5, wherein the identifying step includes forming vertical and horizontal groupings of the blocks.

7. A method according to claim 5, wherein the identifying step includes forming a hierarchical structure.

8. A method according to claim 7, wherein the step of forming the hierarchical structure includes forming horizontal and vertical groupings of elements, the elements consisting of previously formed groupings and ungrouped blocks.

9. A method according to claim 8, wherein horizontal and vertical groupings are formed by testing two elements at a time, a new grouping being formed when a rectangle can be formed which encloses the two tested elements and no other elements.

10. A method according to claim 9, wherein the step of forming vertical and horizontal groupings includes alternating between forming all possible vertical groupings and forming all possible horizontal groupings.

11. A method according to claim 9, wherein two identified elements are tested for vertical grouping only if the two identified elements are not horizontally adjacent to each other.

12. A method according to claim 9, wherein two identified elements are tested for horizontal grouping only if the two identified elements are not vertically adjacent to each other.

13. A method according to claim 8, wherein the identifying step includes ordering the blocks based on a sequence in which the vertical and horizontal groupings are formed.

14. A method according to claim 5, wherein the identifying step includes a step of ordering the blocks based on the identified positional relationships.

15. A method according to claim 5, wherein table tags are generated and inserted into the HTML file in said generating step.

16. A method according to claim 5, further comprising the step of character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

17. A method for generating a hypertext markup language (HTML) file based on bitmap image data, comprising the steps of:

segmenting the bitmap image into image blocks, including an image block for plural character images and an image block for a bitmap image;

analyzing layout relationships between the image blocks based on the relative locations of the image blocks in the bitmap image;

a block-type determination step of determining block type of each image block based on the layout relationships obtained in said analyzing step;

a span determination step of determining column span and row span information for each image block that spans more than one column and/or row;

a first generating step of generating an image file which stores the bitmap image; and

a second generating step of automatically generating an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an HTML tagged table, the tags being determined based on the block type obtained in said block-type determination step and based on the column span and row span information obtained in said span determination step, wherein the block in the RTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image tile.

18. A method according to claim 17, wherein said analyzing step includes iterative steps of combining blocks into vertical and horizontal groupings.

19. A method according to claim 17, wherein said block-type determination step includes the step of assigning to each block a type selected from the group consisting of:

a single block, a combined block, a joint block, and a multiple block.

20. A method according to claim 19, wherein said step of assigning is based on proximity of each block to adjacent blocks and also based on layout information obtained is said analyzing step.

21. A method according to claim 17, wherein analyzing step groups blocks into vertical and horizontal groupings, and wherein said span determination step includes the step of determining column span and row span by counting the number of blocks within vertical groupings and the number of blocks within horizontal groupings.

22. A method according to claim 17, further comprising the step of character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

23. An apparatus for generating a hypertext markup language (HTML) file based on a bitmap image, comprising:

a memory which stores process steps that are executable by a processor; and

a processor which executes the process steps stored in the memory to (1) obtain at lest two horizontally adjacent image blocks in separate vertical columns of the bitmap image, including an image block for plural character images and an image block for a bitmap image, (2) generate an image file which stores the bitmap image, and (3) automatically generate an HTML file in which blocks corresponding to the two horizontally adjacent image blocks obtained by the obtaining means are tagged as data elements in a row of an HTML tagged table, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

24. An apparatus according to claim 23, wherein the processor assigns an HTML code to each block in accordance with a position of the block relative to other elements within the bitmap image.

25. An apparatus according to claim 23, wherein table tags are generated and inserted into the HTML file by said processor.

26. An apparatus according to claim 23, further comprising character recognition means for character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

27. An apparatus for generating a hypertext markup language (HTML) file based on a bitmap image, comprising:

a memory which stores process steps that are executable by a processor; and

a processor which executes the process steps stored in the memory (1) to segment the bitmap image to obtain image blocks, including an image block for plural character images and an image block for a bitmap image, (2) to determine where in the bitmap image each of the image blocks is located, (3) to identify positional relationships between the image blocks based on their relative locations, (4) to generate an image file which stores the bitmap image, and (5) to automatically generate an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an RTML tagged table, the tags being determined in accordance with the identified positional relationships, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

28. An apparatus according to claim 27, wherein when the processor identifies positional relationships between the blocks, the processor forms vertical and horizontal groupings of the blocks.

29. An apparatus according to claim 27, wherein when the processor identifies positional relationships between the blocks, the processor forms a hierarchical structure.

30. An apparatus according to claim 29, wherein the processor forms the hierarchical structure by forming horizontal and vertical groupings of elements, the elements consisting of previously formed groupings and ungrouped blocks.

31. An apparatus according to claim 30, wherein the processor forms the horizontal and vertical groupings by testing two elements at a time, a new grouping being formed when a rectangle can be formed which encloses the two tested elements and no other elements.

32. An apparatus according to claim 31, wherein the processor forms the vertical and horizontal groupings by alternating between forming all possible vertical groupings and forming all possible horizontal groupings.

33. An apparatus according to claim 31, wherein the processor tests two identified elements for vertical grouping only if the two identified elements are not horizontally adjacent to each other.

34. An apparatus according to claim 31, wherein the processor tests two identified elements for horizontal grouping only if the two identified elements are not vertically adjacent to each other.

35. An apparatus according to claim 30, wherein the processor identifies the positional relationships between the blocks by ordering the blocks based on a sequence in which the vertical and horizontal groupings are formed.

36. An apparatus according to claim 27, wherein the processor identifies the positional relationships between the blocks by ordering the blocks based on the identified positional relationships.

37. An apparatus according to claim 27, wherein table tags are generated and inserted into the HTML file by said processor.

38. An apparatus according to claim 27, further comprising character recognition means for character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

39. An apparatus for generating a hypertext markup language (HTML) file based on bitmap image data, comprising:

a memory for storing the bitmap image data and process steps executable by a processor; and

a processor which executes the process steps stored in the memory (1) to segment the bitmap image into image blocks, including an image block for plural character images and an image block for a bitmap image, (2) to analyze layout relationships between the image blocks based on the relative locations of the image blocks in the bitmap image, (3) to determine a block type of each image block based on the layout relationships obtained by the processor, (4) to determine column span and row span information for each image block that spans more than one column and/or row, (5) to generate an image file which stores the bitmap image; and

(6) to automatically generate an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an HTML tagged table, the tags being determined based on the block type determined by the processor and the column span and row span information determined by the processor, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

40. An apparatus according to claim 39, wherein the processor analyzes layout relationships between the blocks by combining the blocks into vertical and horizontal groupings.

41. An apparatus according to claim 39, wherein the processor determines a block type of each block by assigning to each block a type selected from the group consisting of:

a single block, a combined block, a joint block, and a multiple block.

42. An apparatus according to claim 41, wherein the processor assigns a block type to each block based on proximity of each block to adjacent blocks and also based on layout information obtained by the processor.

43. An apparatus according to claim 39, wherein the layout relationship analysis step executed by the processor includes grouping the blocks into vertical and horizontal groupings, and wherein the processor determines column span and row span information for each block by counting the number of blocks within vertical groupings and the number of blocks within horizontal groupings.

44. An apparatus according to claim 39, further comprising character recognition means for character-recognizing the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

45. Computer-executable process steps stored on a computer-readable medium, the computer executable process steps to generate a hypertext markup language (HTML) file based on a bitmap image, the process steps comprising:

an obtaining step to obtain at least two horizontally adjacent image blocks in separate vertical columns of the bitmap image, including an image block for plural characters and an image block for a bitmap image;

a generating step to generate an image file which stores the bitmap image; and

an automatic generating step to automatically generate an HTML file in which blocks corresponding to the two horizontally adjacent image blocks obtained from the bitmap image are tagged as data elements in a row of an HTML tagged table, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

46. Computer-executable process steps according to claim 45, wherein an HTML code is assigned to each block in accordance with a position of the block relative to other elements within the bitmap image.

47. Computer-executable process steps according to claim 45, wherein table tags are generated and inserted into the HTML file in said generating step.

48. Computer-executable process steps according to claim 45, further comprising a process step to character-recognize the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

49. Computer-executable process steps stored on a computer-readable medium, the computer executable process steps to generate a hypertext markup language (HTML) file based on a bitmap image, the computer-executable process steps comprising:

a segmenting step to segment the bitmap image to obtain image blocks, including an image block for plural character images and an image block for a bitmap image;

a determining step to determining where in the bitmap image each of the image blocks is located;

an identifying step to identify positional relationships between the image blocks based on their relative locations;

a first generating step to generate an image file which stores the bitmap image; and

a second generating step to automatically generate an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an HTML tagged table, the tags being determined in accordance with the identified positional relationships, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

50. Computer-executable process steps according to claim 49, wherein the identifying step includes forming vertical and horizontal groupings of the blocks.

51. Computer-executable process steps according to claim 49, wherein the identifying step includes forming a hierarchical structure.

52. Computer-executable process steps according to claim 51, wherein the step of forming the hierarchical structure includes forming horizontal and vertical groupings of elements, the elements consisting of previously formed groupings and ungrouped blocks.

53. Computer-executable process steps according to claim 52, wherein horizontal and vertical groupings are formed by testing two elements at a time, a new grouping being formed when a rectangle can be formed which encloses the two tested elements and no other elements.

54. Computer-executable process steps according to claim 53, wherein the step of forming vertical and horizontal groupings includes alternating between forming all possible vertical groupings and forming all possible horizontal groupings.

55. Computer-executable process steps according to claim 53, wherein two identified elements are tested for vertical grouping only if the two identified elements are not horizontally adjacent to each other.

56. Computer-executable process steps according to claim 53, wherein two identified elements are tested for horizontal grouping only if the two identified elements are not vertically adjacent to each other.

57. Computer-executable process steps according to claim 52, wherein the identifying step includes ordering the blocks based on a sequence in which the vertical and horizontal groupings are formed.

58. Computer-executable process steps according to claim 49, wherein the identifying step includes a step to order the blocks based on the identified positional relationships.

59. Computer-executable process steps according to claim 49, wherein table tags are generated and inserted into the HTML file in said generating step.

60. Computer-executable process steps according to claim 49, further comprising a process step to character-recognize the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.

61. Computer-executable process steps stored on a computer-readable medium, the computer executable process steps to generate a hypertext markup language (HTML) file based on bitmap image data, the computer-executable process steps comprising:

a segmenting step to segment the bitmap image into image blocks, including an image block for plural character images and an image block for a bitmap image;

an analyzing step to analyze layout relationships between the image blocks based on the relative locations of the image-blocks in the bitmap image;

a block-type determination step to determine a block type of each image block based on the layout relationships obtained in said analyzing step;

a span determination step to determine column span and row span information for each image block that spans more than one column and/or row;

a first generating step to generate an image file which stores the bitmap image; and

a second generating step to automatically generate an HTML file in which blocks corresponding to the image blocks obtained from the bitmap image are tagged as being data elements in an HTML tagged table, the tags being determined based on the block type obtained in said block-type determination step and based on the column span and row span information obtained in said span determination step, wherein the block in the HTML file corresponding to the image block for the bitmap image includes an HTML link to the generated image file.

62. Computer-executable process steps according to claim 61, wherein said analyzing step includes iterative steps to combine blocks into vertical and horizontal groupings.

63. Computer-executable process steps according to claim 61, wherein said block-type determination step includes a step to assign to each block a type selected from the group consisting of:

a single block, a combined block, a joint block, and a multiple block.

64. Computer-executable process steps according to claim 63, wherein said step of assigning is based on proximity of each block to adjacent blocks-and also based on layout information obtained is said analyzing step.

65. Computer-executable process steps according to claim 61, wherein analyzing step groups blocks into vertical and horizontal groupings, and wherein said span determination step includes a step to determine column span and row span by counting the number of blocks within vertical groupings and the number of blocks within horizontal groupings.

66. Computer-executable process steps according to claim 61, further comprising a process step to character-recognize the character images in the image block for plural character images, so as to obtain character codes for such character images, wherein the block in the HTML file corresponding to the image block for the plural character images includes the character codes for the character images.