Semantic document smartnails

• US 20060136491A1
• Filed: 12/22/2004
• Published: 06/22/2006
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Berkner, Kathrin, Miao, Jinghao
• Granted Patent: US 7,330,608 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A method and apparatus is disclosed herein for generating visualizations of documents based on semantic information. In one embodiment, the method comprises selecting a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments and determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.

Claims

1. A method comprising:

selecting a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments;

determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.

2. The method defined in claim 1 further comprising performing a semantic document analysis on a set of document image segments of a document, wherein the plurality of image segments is selected from the set of document image segments based on the semantic importance information.

3. The method defined in claim 2 wherein performing semantic document analysis comprises extracting keyword attributes from each of the document image segments.

4. The method defined in claim 3 wherein extracting keyword attributes from each of the image segments comprises performing semantic document information characterization on each of the document image segments.

5. The method defined in claim 4 wherein the semantic document information characterization is based on keyword font information.

6. The method defined in claim 5 wherein the keyword font information comprises one or more selected from a group consisting of a size of a keyword font, a type of keyword font, and whether the keyword is capitalized.

7. The method defined in claim 4 wherein the semantic document information characterization is based on keyword position information.

8. The method defined in claim 7 wherein the keyword position information comprises one or more selected from a group consisting of whether the keyword is located at a beginning of an image segment or at an end of the document image segment.

9. The method defined in claim 4 wherein the semantic document information characterization is based on keyword position information and keyword font information.

10. The method defined in claim 3 wherein the keyword attributes comprise a keyword position indicator, a keyword font type indictor and a document image segment capitalization indicator.

11. The method defined in claim 3 wherein the keyword attributes comprise one or more of the size of a bounding box for a word and the point size of a word.

12. The method defined in claim 3 further comprising generating information for each document image segment indicative of semantic importance in the document.

13. The method defined in claim 12 wherein the semantic importance is based on a keyword query.

14. The method defined in claim 12 wherein the semantic importance is based on an automatic keyword extraction.

15. The method defined in claim 12 wherein the semantic importance is based on an automatic categorization.

16. The method defined in claim 12 wherein the semantic importance is based on key paragraphs.

17. The method defined in claim 12 wherein the information indicative of semantic importance comprises a semantic importance value.

18. The method defined in claim 17 further comprising selecting the document image segments based on the semantic importance value.

19. The method defined in claim 1 further comprising:

generating the semantic importance information;

generating visual importance information for each document image segment;

merging the visual importance information and the visual importance information for each document image segment to produce merged importance information for each document image segment;

selecting the plurality of document image segments from a set of document image segments of a document based on the merged importance information.

20. The method defined in claim 19 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

21. The method defined in claim 19 further comprising generating a capitalization indicator for each image segment, and wherein merging semantic importance information and visual importance information for each image segment comprises merging the semantic importance information, visual importance information and the capitalization indicator to produce merged importance information for each document image segment.

22. The method defined in claim 21 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

23. The method defined in claim 1 further comprising reformatting the plurality of document image segments to display the document image segments on a target canvas.

24. The method defined in claim 1 further comprising reformatting the plurality of document image segments to display the document image segments on a target canvas to create a keyword dependent layout.

25. The method defined in claim 1 further comprising segmenting the document into the document image segments based on line interspaces and font characteristics.

27. The article of manufacture defined in claim 25 wherein the method further comprises performing a semantic document analysis on a set of document image segments of a document, wherein the plurality of document image segments is selected from the set of document image segments based on the semantic importance information.

28. The article of manufacture defined in claim 27 wherein performing semantic document analysis comprises extracting keyword attributes from each of the document image segments.

29. The article of manufacture defined in claim 28 wherein extracting keyword attributes from each of the document image segments comprises performing semantic document information characterization on each of the document image segments.

30. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword font information.

31. The article of manufacture defined in claim 30 wherein the keyword font information comprises one or more selected from a group consisting of a size of a keyword font, a type of keyword font, and whether the keyword is capitalized.

32. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword position information.

33. The article of manufacture defined in claim 32 wherein the keyword position information comprises one or more selected from a group consisting of whether the keyword is located at a beginning of a document image segment or at an end of the document image segment.

34. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword position information and keyword font information.

35. The article of manufacture defined in claim 28 wherein the keyword attributes comprise a keyword position indicator, a keyword font type indictor and an image segment capitalization indicator.

36. The article of manufacture defined in claim 28 wherein the keyword attributes comprise one or more of the size of a bounding box for a word and the point size of a word.

37. The article of manufacture defined in claim 28 wherein the method further comprises generating information for each document image segment indicative of semantic importance in the document.

38. The article of manufacture defined in claim 37 wherein the semantic importance is based on a keyword query.

39. The article of manufacture defined in claim 37 wherein the semantic importance is based on automatic keyword extraction.

40. The article of manufacture defined in claim 37 wherein the semantic importance is based on automatic categorization.

41. The article of manufacture defined in claim 37 wherein the semantic importance is based on key paragraphs.

42. The article of manufacture defined in claim 37 wherein the information indicative of semantic importance comprises a semantic importance value.

43. The article of manufacture defined in claim 42 wherein the method further comprises selecting the document image segments based on the semantic importance value.

26. An article of manufacture having one or more recordable media having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

selecting a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments;

determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.

44. The article of manufacture defined in claim 26 wherein the method further comprises:

generating the semantic importance information;

generating visual importance information for each document image segment;

merging semantic importance information and visual importance information for each document image segment to produce merged importance information for each document image segment;

selecting the plurality of image segments from a set of document image segments of a document based on the merged importance information.

45. The article of manufacture defined in claim 44 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

46. The article of manufacture defined in claim 44 wherein the method further comprises generating a capitalization indicator for each document image segment, and wherein merging semantic importance information and visual importance information for each document image segment comprises merging the semantic importance information, visual importance information and the capitalization indicator to produce merged importance information for each document image segment.

47. The article of manufacture defined in claim 46 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

48. The article of manufacture defined in claim 26 wherein the method further comprises reformatting the plurality of document image segments to display the document image segments on a target canvas.

49. The article of manufacture defined in claim 26 wherein the method further comprises segmenting the document into the document image segments based on line interspaces and font characteristics.

50. An apparatus comprising:

an analyzer to select a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments;

merging unit to determine a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.

51. A method comprising:

performing semantic document analysis on a document;

extracting semantic information from the document based on results of the semantic document analysis;

a merging extracted semantic information with extracted visual information; and

creating a layout for a visual representation of the document based on results of merging.

52. An article of manufacture having one or more recordable media having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

performing semantic document analysis on a document;

extracting semantic information from the document based on results of the semantic document analysis;

merging extracted semantic information with extracted visual information; and

creating a layout for a visual representation of the document based on results of merging.

53. A method comprising:

generating a plurality of image segments having a plurality of attributes that include a semantic importance indication information for each of the plurality of image segments;

performing a plurality of operations on one or more image segments of the plurality of image segments selected based on associated semantic importance information to generate processed image segments; and

displaying the processed image segments on a target canvas.

54. An article of manufacture having one or more recordable media having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

generating a plurality of image segments having a plurality of attributes that include a semantic importance indication information for each of the plurality of image segments;

performing a plurality of operations on one or more image segments of the plurality of image segments selected based on associated semantic importance information to generate processed image segments; and

displaying the processed image segments on a target canvas.

55. A method comprising:

choosing a set of areas in a document for inclusion in a target display based on semantic importance information associated with each area in the set of areas;

determining whether the set of areas chosen for inclusion in a target display is to be updated;

converting absolutely or relatively positioned areas to absolute locations on the target display if the set of areas is not to be updated; and

generating an output list of instructions to operate on the document.

56. An article of manufacture having one or more recordable media having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

choosing a set of areas in a document for inclusion in a target display based on semantic importance information associated with each area in the set of areas;

determining whether the set of areas chosen for inclusion in a target display is to be updated;

converting absolutely or relatively positioned areas to absolute locations on the target display if the set of areas is not to be updated; and

generating an output list of instructions to operate on the document.