Method and an apparatus for visual summarization of documents

• US 6,895,552 B1
• Filed: 05/31/2000
• Issued: 05/17/2005
• Est. Priority Date: 05/31/2000
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Lee, Dar-Shyang, Balabanovic, Marko
• Primary Examiner(s): Shah, Sanjiv
• Application Number:
  US09/585,713
• Time in Patent Office: 1,812 Days
• Field of Search: 715/517, 715/513, 707/102, 707/104, 707/7, 345/635, 382/305, 358/403
• US Class Current: 715/209
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 715/967 Visual or iconic programming

Abstract

A system and a method for visually summarizing a document comprising a display, a processor coupled to the display, and a memory coupled to the processor. Stored in the memory is a routine, which when executed by the processor, causes the processor to generate display data. The routine causes the processor to generate data through extracting at least one visual feature from a document having a plurality of pages, ranking the pages in a document, selecting a page for representing a document according to the visual feature, and displaying the selected page as display data.

Claims

1. A method comprising:

extracting at least one visual feature from a document, the document having a plurality of pages, wherein at least one visual feature is weighted based on one of gradations and a location of the visual feature;

ranking each of the pages in the document based on the at least one visual feature, wherein the ranking is performed based on a similarity relationship between content of the respective page and the at least one visual feature; and

displaying page icons representing the pages of the document in an order based on a result of the ranking.

2. The method of claim 1 wherein the visual feature is one of a picture, a text block, a character size, a character style, and a color, wherein each type of the visual features is assigned with a weight value representing a weight of the respective type used in the ranking.

3. The method of claim 1 further comprising performing a saliency evaluation on each of the pages within the document to identify one or more pages that are significantly different from a remainder of the pages, using the at least one visual feature.

4. The method of claim 3 wherein the visual feature is represented in a vector form, and wherein performing saliency evaluation further comprises:

determining a centroid representing an average visual feature vector of the pages;

subtracting the determined centroid from a visual feature vector of each individual of the pages to reduce common characteristics of the visual feature vectors of the pages.

5. The method of claim 3, wherein pages having a significantly higher saliency are displayed as a full image within the respective page icon while a remainder of the pages are displayed partially.

6. The method of claim 1 wherein the visual feature is used as a distance measure between a first and a second consecutive pages within the document, the distance measure representing a visual difference between the first and second consecutive pages.

7. The method of claim 6, wherein the first and second consecutive pages of the document are displayed in overlap, and wherein an amount of overlapping is determined based on the distance measure represented by the visual features of the first and second consecutive pages.

8. The method of claim 1, further comprising clustering of a plurality of pages of the document into one or more clusters based on visual similarities of the pages with respect to the at least one visual feature.

9. The method of claim 8, further comprising:

selecting an exemplar page from each cluster to form exemplar pages of the document, the exemplar page of each cluster having a visual feature vector closest to a center of the respective cluster; and

displaying the exemplar page of the document using one or more page icons.

10. The method of claim 8, wherein a page having a most saliency among the pages of a respective cluster is displayed in full image, while a remainder of the pages in the respective cluster is displayed partially.

11. The method of claim 1 further comprising using visual features to reveal a transition from a first page to a second page of a document.

12. The method of claim 1, wherein ranking of the pages includes a correction mechanism.

13. The method of claim 1, wherein a scheme showing one of a plurality of pages in a document and a plurality of documents is by one of a linear display, a line of icons, and as a stack.

14. A computer system comprising:

a display;

a processor coupled to the display; and

a memory coupled to the processor and having stored therein a routine, which when executed by the processor, causes the processor to generate display data through;

extracting at least one visual feature from a document, the document having a plurality of pages, wherein at least one visual feature is weight based on one of gradations and a location of the visual feature;

ranking the pages in the document according to the at least one visual feature, wherein the ranking is performed based on a similarity relationship between content of the respective page and the at least one visual feature,

selecting a page for representing a document according to a result of the ranking, and

displaying the selected page as the display data.

15. The computer system of claim 14 wherein the visual feature is one of a picture, a text block, a character size, a character style, and a color, wherein each type of the visual features is assigned with a weight value representing a weight of the respective type used in the ranking.

16. The computer system of claim 14 further comprising performing a saliency evaluation on each of the pages within the document to identify one or more pages that are significantly different from a remainder of the pages, using the at least one visual feature.

17. The computer system of claim 16 wherein the visual feature is represented in a vector form, and wherein performing saliency evaluation further comprises:

determining a centroid representing an average visual feature vector of the pages;

subtracting the determined centroid from a visual feature vector of each individual of the pages to reduce common characteristics of the visual feature vectors of the pages.

18. The computer system of claim 14 wherein the visual feature is used as a distance measure between a first and a second consecutive pages within the documents, the distance measure representing a visual difference between the first and second consecutive pages.

19. The computer system of claim 14, wherein generating display data further comprises clustering of a plurality of pages of the document into one or more clusters based on visual similarities of the pages with respect to the at least one visual feature.

20. The computer system of claim 14, wherein a plurality of pages are selected and generating display data further comprises using visual features to reveal a transition from a first page to a second page of a document.

21. The computer system of claim 14, wherein ranking the pages includes a correction mechanism.

22. The computer system of claim 14, wherein a scheme showing one of a plurality of pages in a document and a plurality of documents is by one of a linear display, a line of icons, and as a stack.

23. An article of manufacture having at least one machine readable storage media containing executable program instructions which when executed by a digital processing system caused the digital processing system to:

extract at least one visual feature from a document, the document having a plurality of pages, wherein at least one visual feature is weighted based on one of gradations and a location of the visual feature,

rank pages in the document based on said at least one visual feature, wherein the ranking is performed based on a similarity relationship between content of the respective page and the at least one visual feature,

select the pages for representing a document based on a result of the ranking, and

display selected pages.

24. The machine readable storage media of claim 23, wherein the visual feature is one of a picture, a text block, a character size, a character style, and a color, wherein each type of the visual features is assigned with a weight value representing a weight of the respective type used in the ranking.

25. The machine readable storage media of claim 23, further comprising performing a saliency evaluation on each of the pages within the document to identify one or more pages that are significantly different from a remainder of the pages, using the at least one visual feature.

26. The machine readable storage media of claim 23 wherein the visual feature is represented in a vector form, and wherein performing saliency evaluation further comprises:

determining a centroid representing an average visual feature vector of the pages;

subtracting the determined centroid from a visual feature vector of each individual of the pages to reduce common characteristics of the visual feature vectors of the pages.

27. The machine readable storage media of claim 23, wherein the visual feature is used as a distance measure between a first and a second consecutive pages within the document, the distance measure representing a visual difference between the first and second consecutive pages.

28. The machine readable storage media of claim 23, further comprising clustering of a plurality of pages of the document into one or more clusters based on visual similarities of the pages with respect to the at least one visual feature.

29. The machine readable storage media of claim 23, further comprising using visual features to reveal a transition from a first page to a second page of a document.

30. The machine readable storage media of claim 23, wherein ranking the pages includes a correction mechanism.

31. The machine readable storage media of claim 23, wherein a scheme showing one of a plurality of pages in a document and a plurality of documents is by one of a linear display, a line of icons, and as a stack.

32. A method comprising:

extracting at least one visual feature from a document, the document having a plurality of Pages, wherein at least one visual feature is weighted based on one of gradations and a location of the visual feature;

grouping the pages of the document based on a similarity relationship between the respective page and the at least one visual feature;

selecting representative pages from groups that are significantly different from a remainder of the pages of the respective group; and

displaying the representative pages for representing the pages of the respective group.