Semantic document smartnails

• US 7,330,608 B2
• Filed: 12/22/2004
• Issued: 02/12/2008
• Est. Priority Date: 12/22/2004
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Miao, Jinghao, Berkner, Kathrin
• Primary Examiner(s): Mariam; Daniel
• Application Number:
  US11/023,142
• Publication Number:
  US 20060136491A1
• Time in Patent Office: 1,147 Days
• Field of Search: 382/176, 382/181, 382/282, 382/284, 382/291, 382/298, 382/305, 382/313, 358/451, 707/103.R, 345/676, 715/517, 715/526, 348/561, 348/581
• US Class Current: 382/298
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A method and apparatus is disclosed herein for generating visualizations of documents based on semantic information. In one embodiment, the method comprises selecting a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments and determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.

Claims

1. A method comprising:

selecting a plurality of document image segments from a set of document image segments based on a plurality of attributes that includes a semantic importance information and a visual importance information for each of the plurality of document image segments; and

determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes that include the semantic importance information and visual importance information, and display constraints.

2. The method defined in claim 1 further comprising performing a semantic document analysis on a set of document image segments of a document, wherein the plurality of image segments is selected from the set of document image segments based on the semantic importance information.

3. The method defined in claim 2 wherein performing semantic document analysis comprises extracting keyword attributes from each of the document image segments.

4. The method defined in claim 3 wherein extracting keyword attributes from each of the image segments comprises performing semantic document information characterization on each of the document image segments.

5. The method defined in claim 4 wherein the semantic document information characterization is based on keyword font information.

6. The method defined in claim 5 wherein the keyword font information comprises one or more selected from a group consisting of a size of a keyword font, a type of keyword font, and whether the keyword is capitalized.

7. The method defined in claim 4 wherein the semantic document information characterization is based on keyword position information.

8. The method defined in claim 7 wherein the keyword position information comprises one or more selected from a group consisting of whether the keyword is located at a beginning of an image segment or at an end of the document image segment.

9. The method defined in claim 4 wherein the semantic document information characterization is based on keyword position information and keyword font information.

10. The method defined in claim 3 wherein the keyword attributes comprise a keyword position indicator, a keyword font type indictor and a document image segment capitalization indicator.

11. The method defined in claim 3 wherein the keyword attributes comprise one or more of the size of a bounding box for a word and the point size of a word.

12. The method defined in claim 3 further comprising generating information for each document image segment indicative of semantic importance in the document.

13. The method defined in claim 12 wherein the semantic importance is based on a keyword query.

14. The method defined in claim 12 wherein the semantic importance is based on an automatic keyword extraction.

15. The method defined in claim 12 wherein the semantic importance is based on an automatic categorization.

16. The method defined in claim 12 wherein the semantic importance is based on key paragraphs.

17. The method defined in claim 12 wherein the information indicative of semantic importance comprises a semantic importance value.

18. The method defined in claim 17 further comprising selecting the document image segments based on the semantic importance value.

19. The method defined in claim 1 further comprising:

generating the semantic importance information;

generating visual importance information for each document image segment;

merging the semantic importance information and the visual importance information for each document image segment to produce merged importance information for each document image segment;

selecting the plurality of document image segments from a set of document image segments of a document based on the merged importance information.

20. The method defined in claim 19 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

21. The method defined in claim 19 further comprising generating a capitalization indicator for each image segment, and wherein merging semantic importance information and visual importance information for each image segment comprises merging the semantic importance information, visual importance information and the capitalization indicator to produce merged importance information for each document image segment.

22. The method defined in claim 21 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

23. The method defined in claim 1 further comprising reformatting the plurality of document image segments to display the document image segments on a target canvas.

24. The method defined in claim 1 further comprising reformatting the plurality of document image segments to display the document image segments on a target canvas to create a keyword dependent layout.

25. The method defined in claim 1 further comprising segmenting the document into the document image segments based on line interspaces and font characteristics.

26. An article of manufacture having one or more computer-readable medium having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

selecting a plurality of document image segments from a set of document image segments based on a plurality of attributes that includes a semantic importance information and a visual importance information for each of the plurality of document image segments;

determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes that include the semantic importance information and visual importance information, and display constraints.

27. The article of manufacture defined in claim 26 wherein the method further comprises performing a semantic document analysis on a set of document image segments of a document, wherein the plurality of document image segments is selected from the set of document image segments based on the semantic importance information.

28. The article of manufacture defined in claim 27 wherein performing semantic document analysis comprises extracting keyword attributes from each of the document image segments.

29. The article of manufacture defined in claim 28 wherein extracting keyword attributes from each of the document image segments comprises performing semantic document information characterization on each of the document image segments.

30. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword font information.

31. The article of manufacture defined in claim 30 wherein the keyword font information comprises one or more selected from a group consisting of a size of a keyword font, a type of keyword font, and whether the keyword is capitalized.

32. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword position information.

33. The article of manufacture defined in claim 32 wherein the keyword position information comprises one or more selected from a group consisting of whether the keyword is located at a beginning of a document image segment or at an end of the document image segment.

34. The article of manufacture defined in claim 29 wherein the semantic document information characterization is based on keyword position information and keyword font information.

35. The article of manufacture defined in claim 28 wherein the keyword attributes comprise a keyword position indicator, a keyword font type indictor and an image segment capitalization indicator.

36. The article of manufacture defined in claim 28 wherein the keyword attributes comprise one or more of the size of a bounding box for a word and the point size of a word.

37. The article of manufacture defined in claim 28 wherein the method further comprises generating information for each document image segment indicative of semantic importance in the document.

38. The article of manufacture defined in claim 37 wherein the semantic importance is based on a keyword query.

39. The article of manufacture defined in claim 37 wherein the semantic importance is based on automatic keyword extraction.

40. The article of manufacture defined in claim 37 wherein the semantic importance is based on automatic categorization.

41. The article of manufacture defined in claim 37 wherein the semantic importance is based on key paragraphs.

42. The article of manufacture defined in claim 37 wherein the information indicative of semantic importance comprises a semantic importance value.

43. The article of manufacture defined in claim 42 wherein the method further comprises selecting the document image segments based on the semantic importance value.

44. The article of manufacture defined in claim 26 wherein the method further comprises:

generating the semantic importance information;

generating visual importance information for each document image segment;

merging semantic importance information and visual importance information for each document image segment to produce merged importance information for each document image segment;

selecting the plurality of image segments from a set of document image segments of a document based on the merged importance information.

45. The article of manufacture defined in claim 44 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

46. The article of manufacture defined in claim 44 wherein the method further comprises generating a capitalization indicator for each document image segment, and wherein merging semantic importance information and visual importance information for each document image segment comprises merging the semantic importance information, visual importance information and the capitalization indicator to produce merged importance information for each document image segment.

47. The article of manufacture defined in claim 46 wherein the semantic importance information comprises semantic importance values and the visual importance information comprises visual importance values.

48. The article of manufacture defined in claim 26 wherein the method further comprises reformatting the plurality of document image segments to display the document image segments on a target canvas.

49. The article of manufacture defined in claim 26 wherein the method further comprises segmenting the document into the document image segments based on line interspaces and font characteristics.

50. An apparatus comprising:

an analyzer to select a plurality of document image segments from a set of document image segments based on a plurality of attributes that includes semantic importance information and visual importance information for each of the plurality of document image segments;

a merging unit to determine a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes that include semantic importance information and visual importance information, and display constraints.

51. A method comprising:

performing semantic document analysis on a document based on input information;

extracting semantic information from the document based on results of the semantic document analysis;

merging extracted semantic information with extracted visual information;

creating a layout that includes a plurality of document image segments selected from a set of document image segments, dependent on the input information, for a visual representation of the document based on results of merging; and

generating the visual representation using the layout, including extracting the plurality of segments of the document and positioning the plurality of segments in the visual representation based on the extracted semantic information.

52. The method defined in claim 51 wherein the input information comprises query information in a user query, and further wherein the layout comprises a query dependent layout.

53. The method defined in claim 51 wherein the document has a first representation and the visual representation is a second representation different than the first representation.

54. An article of manufacture having one or more computer-readable medium having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

performing semantic document analysis on a document;

extracting semantic information from the document based on results of the semantic document analysis;

merging extracted semantic information with extracted visual information;

creating a layout that includes a plurality of document image segments selected from a set of document image segments, dependent on the input information, for a visual representation of the document based on results of merging; and

generating the visual representation using the layout, including extracting the plurality of segments of the document and positioning the plurality of segments in the visual representation based on the extracted semantic information.

55. The article of manufacture defined in claim 54 wherein the input information comprises query information in a user query, and further wherein the layout comprises a query dependent layout.

56. The article of manufacture defined in claim 54 wherein the document has a first representation and the visual representation is a second representation different than the first representation.

57. A method comprising:

generating a plurality of image segments from a document having a first representation, the plurality of image segments having a plurality of attributes that include a semantic importance indication information and visual importance indication information for each of the plurality of image segments;

performing a plurality of operations on one or more image segments of the plurality of image segments selected based on associated semantic importance information and visual importance information to generate processed image segments; and

displaying the processed image segments on a target canvas as a second representation, different than the first representation, the second representation including semantically important content from the document determined based on the semantic importance information and visual importance information.

58. The method defined in claim 57 further comprising determining a position of at least one of the plurality of image segments based on associated attribute information, including semantic importance information.

59. An article of manufacture having one or more computer-readable medium having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

generating a plurality of image segments from a document having a first representation, the plurality of image segments having a plurality of attributes that include a semantic importance indication information and a visual importance information for each of the plurality of image segments;

performing a plurality of operations on one or more image segments of the plurality of image segments selected based on associated semantic and visual importance information to generate processed image segments; and

displaying the processed image segments on a target canvas as a second representation, different than the first representation, the second representation including semantically important content from the document determined based on the semantic importance information and visual importance information.

60. The article of manufacture defined in claim 59 wherein the method further comprises determining a position of at least one of the plurality of image segments based on associated attribute information, including semantic importance information.

61. A method comprising:

choosing a set of areas in a document for inclusion in, and prior to display on, a target display based on semantic importance information and visual importance information associated with each area in the set of areas;

determining whether the set of areas chosen for inclusion in a target display is to be updated;

converting absolutely or relatively positioned areas to absolute locations on the target display if the set of areas is not to be updated; and

generating an output list of instructions to operate on the document, the output list of instructions specifying the set of areas chosen and the absolute locations created or a result of converting absolutely or positioned areas.

62. An article of manufacture having one or more computer-readable medium having instructions stored therein which, when executed by a system, cause the system to perform a method comprising:

choosing a set of areas in a document for inclusion in, and prior to display on, a target display based on semantic importance information and visual importance information associated with each area in the set of areas;

determining whether the set of areas chosen for inclusion in a target display is to be updated;

converting absolutely or relatively positioned areas to absolute locations on the target display if the set of areas is not to be updated; and

generating an output list of instructions to operate on the document, the output list of instructions specifying the set of areas chosen and the absolute locations created or a result of converting absolutely or positioned areas.