System and method for document collection, grouping and summarization

• US 8,176,418 B2
• Filed: 03/04/2005
• Issued: 05/08/2012
• Est. Priority Date: 09/16/2002
• Status: Expired due to Fees

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: McKeown, Kathleen R., Barzilay, Regina, Evans, Dave, Hatzivassiloglou, Vasileios, Klavans, Judith, Nenkova, Ani, Schiffman, Barry
• Primary Examiner(s): HUYNH, THU V
• Application Number:
  US11/071,968
• Publication Number:
  US 20050203970A1
• Time in Patent Office: 2,622 Days
• Field of Search: 715/255, 715/254, 715/229, 715/234, 703/203, 703/205, 707/203, 707/205, 707/638, 707/648
• US Class Current: 715/254
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A system for generating a summary of a plurality of documents and presenting the summary information to a user is provided which includes a computer readable document collection containing a plurality of related documents stored in electronic form. Documents can be pre-processed to group documents into document clusters. The document clusters can also be assigned to predetermined document categories for presentation to a user. A number of multiple document summarization engines are provided which generate summaries for specific classes of multiple documents clusters. A summarizer router is employed to determining a relationship of the documents in a cluster and select one of the document summarization engines for use in generating a summary of the cluster. A single event engine is provided to generate summaries of documents which are closely related temporally and to a specific event. A dissimilarity engine for multiple document summary generation is provided which generates summaries of document clusters having documents with varying degrees of relatedness. A user interface is provided to display categories, cluster titles, summaries, related images.

Claims

1. A system for generating a summary of a plurality of documents comprising:

a computer processor;

a computer readable document collection containing a plurality of related documents stored in electronic form therein;

a plurality of forms of multiple document summarization engines including a single event engine, a biography engine and a multi-event engine operating on the computer processor; and

a router, the router determining a temporal relationship of at least a subset of the documents in the collection and selecting one of the plurality of forms of multiple document summarization engines for generating a summary of the subset of documents based on the temporal relationship, wherein the router selects the biography engine if the documents in the collection are not generated within a predetermined time period and the number of capitalized words and the number of personal pronouns each exceed a predetermined threshold value.

2. The system for generating a summary of a plurality of documents of claim 1, wherein the router selects the single event engine if a predetermined number of documents in the subset of the collection are generated within a predetermined time period.

3. The system for generating a summary of a plurality of documents of claim 1, further comprising a document collection program, the document collection program being configured to search the content of a predetermined set of content sources to gather documents for the document collection.

4. The system for generating a summary of a plurality of documents of claim 3, wherein the content sources are websites of content providers on the Internet.

5. A system for document collection, grouping and summarization comprising:

a computer processor;

a document collection engine, the document collection engine being configured to be operatively coupled to a computer network to access a plurality of content providers;

a computer readable document collection database, the document collection database being operatively coupled to the document collection engine;

a cluster processing engine operatively coupled to the document collection database, the cluster processing engine grouping at least a portion of the documents in the document collection database into clusters having a plurality of related documents;

a plurality of forms of multiple document summarization engines including a single event engine, a biography engine and a multi-event engine operating on the computer processor;

a summarization router, the summarization router being interposed between the cluster processing engine and the plurality of forms of multiple document summarization engines, the summarization router determining a temporal relationship among the documents in the clusters and selecting one of the plurality of forms of multiple document summarization engines for generating a summary of the cluster based on the temporal relationship, wherein the summarization router selects the single event engine if a predetermined number of documents in the subset of the collection are generated within a predetermined time period and the router selects the biography engine if the documents in the collection are not generated within a predetermined time period and the number of capitalized words and the number of personal pronouns each exceed a predetermined threshold value; and

a graphical user interface, the graphical user interface being operatively coupled to the cluster processing engine and the plurality of summarization engines and providing a display including cluster summaries and cluster information.

6. The system for document collection, grouping and summarization of claim 5, wherein the document collection engine is configured to search the content of a predetermined set of content sources.

7. The system for document collection, grouping and summarization comprising of claim 6, wherein the content sources are websites of content providers on the Internet.

8. The system for document collection, grouping and summarization of claim 5, wherein the cluster processing engine establishes cluster labels for each of the clusters and wherein the cluster information displayed by the graphical user interface includes the cluster labels.

9. The system for document collection, grouping and summarization of claim 8, wherein the cluster processing engine sorts the clusters into one of a plurality of predetermined document categories, and wherein cluster information displayed by the graphical user interface includes the predetermined categories.

10. The system for document collection, grouping and summarization of claim 9, wherein the graphical user interface presents a display including the predetermined categories and cluster labels.

11. The system for document collection, grouping and summarization of claim 10, wherein one of the document collection engine or cluster processing engine identifies image files associated with documents that are collected and wherein the graphical user interface displays the image files with the summary associated with the document cluster.