System and method for document collection, grouping and summarization

• US 20050203970A1
• Filed: 03/04/2005
• Published: 09/15/2005
• Est. Priority Date: 09/16/2002
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Evans, Dave, Hatzivassiloglou, Vasileios, Klavans, Judith, Schiffman, Barry, Nenkova, Ani, Barzilay, Regina, McKeown, Kathleen R.
• Granted Patent: US 8,176,418 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A system for generating a summary of a plurality of documents and presenting the summary information to a user is provided which includes a computer readable document collection containing a plurality of related documents stored in electronic form. Documents can be pre-processed to group documents into document clusters. The document clusters can also be assigned to predetermined document categories for presentation to a user. A number of multiple document summarization engines are provided which generate summaries for specific classes of multiple documents clusters. A summarizer router is employed to determining a relationship of the documents in a cluster and select one of the document summarization engines for use in generating a summary of the cluster. A single event engine is provided to generate summaries of documents which are closely related temporally and to a specific event. A dissimilarity engine for multiple document summary generation is provided which generates summaries of document clusters having documents with varying degrees of relatedness. A user interface is provided to display categories, cluster titles, summaries, related images.

Claims

1. A system for generating a summary of a plurality of documents comprising:

a computer readable document collection containing a plurality of related documents stored in electronic form therein;

a plurality of document summarization engines; and

a router, the router determining a relationship of at least a subset of the documents in the collection and selecting one of the plurality of document summarization engines for generating a summary of the subset of documents based on the relationship.

2. The system for generating a summary of a plurality of documents of claim 1, wherein the plurality of document summarization engines include a single event engine, a biography engine and a multi-event engine.

3. The system for generating a summary of a plurality of documents of claim 2, wherein the router selects the single event engine if a predetermined number of documents in the subset of the collection are generated within a predetermined time period.

4. The system for generating a summary of a plurality of documents of claim 2, wherein the router selects the biography engine if the documents in the collection are not generated within a predetermined time period and the number of capitalized words and the number of personal pronouns each exceed a predetermined threshold value.

5. The system for generating a summary of a plurality of documents of claim 1, further comprising a document collection program, the document collection program being configured to search the content of a predetermined set of content sources to gather documents for the document collection.

6. The system for generating a summary of a plurality of documents of claim 5, wherein the content sources are websites of content providers on the Internet.

7. A computer-based method of presenting content, such as news articles, to a user, comprising:

gathering a plurality of articles from a plurality of content providers;

determining clusters of at least a portion of the articles;

selecting one of a plurality of multiple document summarization engines for each cluster of articles;

generating a summary for each cluster of articles; and

displaying at least one summary to a user.

8. The computer-based method of presenting content, such as news articles, to a user of claim 7, further comprising generating labels for the cluster summaries related to the cluster content.

9. The computer-based method of presenting content, such as news articles, to a user of claim 8, wherein the step of generating labels further comprises extracting a set of keywords from the articles in the clusters.

10. The computer-based method of presenting content, such as news articles, to a user of claim 8, further comprising sorting each of the summaries into one of a plurality of categories.

11. The computer-based method of presenting content, such as news articles, to a user of claim 10, wherein the step of displaying the summary includes generating a display on a graphical user interface, the graphical user interface presenting the summaries organized in accordance with the categories and labels of cluster summaries.

12. The computer-based method of presenting content, such as news articles, to a user of claim 11, further comprising the step of displaying images associated with the summaries.

13. The computer-based method of presenting content, such as news articles, to a user of claim 12, wherein the step of gathering articles further comprises selecting images that are associated with the articles and wherein the selected images are the images displayed in association with the summaries.

14. The computer-based method of presenting content, such as news articles, to a user of claim 12, wherein articles are in the form of HTML content and the step of selecting images associated with the articles includes determining whether the image is within a common HTML cell as the content.

15. The computer-based method of presenting content, such as news articles, to a user of claim 14, wherein the step of selecting images associated with the articles includes determining whether the computer readable file format of the image file is representative of an image associated with the content.

16. The computer-based method of presenting content, such as news articles, to a user of claim 15, wherein the step of selecting images associated with the articles includes rejecting image files having an address of the image file that is indicative of advertising content.

17. The computer-based method of presenting content, such as news articles, to a user of claim 7, wherein the step of gathering articles further comprises the step of extracting the news articles from other content.

18. The computer-based method of presenting content, such as news articles, to a user of claim 7, wherein the step of gathering articles further comprises selecting images that are associated with the articles.

19. The computer-based method of presenting content, such as news articles, to a user of claim 18, wherein articles are in the form of HTML content and the step of selecting images associated with the articles includes determining whether the image is within a common HTML cell as the content.

20. The computer-based method of presenting content, such as news articles, to a user of claim 18, wherein the step of selecting images associated with the articles includes determining whether the computer readable file format of the image file is representative of an image associated with the content.

21. The computer-based method of presenting content, such as news articles, to a user of claim 20, wherein the step of selecting images associated with the articles includes rejecting image files having an address of the image file that is indicative of advertising content.

22. The computer-based method of presenting content, such as news articles, to a user of claim 7, wherein the plurality of summarization engines include a single event engine and at least one dissimilarity engine for multiple document summarization.

23. The computer-based method of presenting content, such as news articles, to a user of claim 7, wherein the step of selecting one of a plurality of summarization engines includes using a temporal relationship to determine whether the documents in a cluster relate to a single event.

24. A system for document collection, grouping and summarization comprising:

a document collection engine, the document collection engine being configured to be operatively coupled to a computer network to access a plurality of content providers;

a computer readable document collection database, the document collection database being operatively coupled to the document collection engine;

a cluster processing engine operatively coupled to the document collection database, the cluster processing engine grouping at least a portion of the documents in the document collection database into clusters having a plurality of related documents;

a plurality of document summarization engines;

a summarization router, the summarization router being interposed between the cluster processing engine and the plurality of document summarization engines, the summarization router determining a relationship among the documents in the clusters and selecting one of the plurality of document summarization engines for generating a summary of the cluster based on the relationship; and

a graphical user interface, the graphical user interface being operatively coupled to the cluster processing engine and the plurality of summarization engines and providing a display including cluster summaries and cluster information.

25. The system for document collection, grouping and summarization of claim 24, wherein the plurality of document summarization engines include a single event engine, a biography engine and a multi-event engine.

26. The system for document collection, grouping and summarization comprising claim 25, wherein the summarization router selects the single event engine if a predetermined number of documents in the subset of the collection are generated within a predetermined time period.

27. The system for document collection, grouping and summarization of claim 26, wherein the router selects the biography engine if the documents in the collection are not generated within a predetermined time period and the number of capitalized words and the number of personal pronouns each exceed a predetermined threshold value.

28. The system for document collection, grouping and summarization of claim 24, wherein the document collection engine is configured to search the content of a predetermined set of content sources.

29. The system for document collection, grouping and summarization comprising of claim 28, wherein the content sources are websites of content providers on the Internet.

30. The system for document collection, grouping and summarization of claim 24, wherein the cluster processing engine establishes cluster labels for each of the clusters and wherein the cluster information displayed by the graphical user interface includes the cluster labels.

31. The system for document collection, grouping and summarization of claim 30, wherein the cluster processing engine sorts the clusters into one of a plurality of predetermined document categories, and wherein cluster information displayed by the graphical user interface includes the predetermined categories.

32. The system for document collection, grouping and summarization of claim 31, wherein the graphical user interface presents a display including the predetermined categories and cluster labels.

33. The system for document collection, grouping and summarization of claim 32, wherein one of the document collection engine or cluster processing engine identifies image files associated with documents that are collected and wherein the graphical user interface displays the image files with the summary associated with the document cluster.

34. A method of generating a summary of multiple documents in a document cluster, comprising:

for each document in the cluster, determining a score for each sentence based on features of the sentences;

selecting a subset of sentences based on a weighted sentence score;

merging the selected sentences in accordance with a predetermined order for the selected sentences; and

removing selected sentences which are duplicative.

35. The method of generating a summary according to claim 34, further comprising the steps of determining variants of named entity references in the summary and performing a named entity substitution by inserting a first variant for the first occurrence of the reference and inserting a second variant for the remaining references.

36. The method of generating a summary according to claim 34, wherein the features of the sentences include a score based on lead values.

37. The method of generating a summary according to claim 34, wherein the features of the sentences include a score based on verb specificity.

38. The method of generating a summary according to claim 34, wherein the features of the sentences include a score based on noun-verb concept sets.

39. The method of generating a summary according to claim 34, wherein the features of the sentences include a score based on lead values, verb specificity, and noun-verb concept sets.

40. The method of generating a summary according to claim 34, wherein the features of the sentences include a score based on at least one heuristic relationship selected from the set including sentence location, publication date, target, sentence length, other entity, pronoun, and role.

41. Computer readable media encoded with instructions for a computer processor to perform the following steps:

gathering a plurality of articles from a plurality of content providers;

determining clusters of at least a portion of the articles;

selecting one of a plurality of multiple document summarization engines for each cluster of articles;

generating a summary for each cluster of articles; and

displaying at least one summary to a user.