Automatic correlation method for generating summaries for text documents

• US 20020052901A1
• Filed: 08/31/2001
• Published: 05/02/2002
• Est. Priority Date: 09/20/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yang, Li Ping, Guo, Zhi Li
• Granted Patent: US 7,017,114 B2
• Time in Patent Office: Days
• US Class Current: 715/531
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A method and program product to generate summaries for text documents. A user can also specify a query, topic, and terms that he/she is interested in. This method determines the importance of each sentence by using the linguistic salience of the word to the user profile, the similarity among the word, the query and topic provided by a user and the sum of scores of the sentence comprising the word. After computing the score for each word, this method computes the score for each sentence in the set of sentences according to the score of words composing it and the position of the sentence in a section and a paragraph.

Claims

1. An automatic method for generating summaries for text documents, comprising steps of:

generating a set of sentences for a set of documents by document discourse analysis and a set of words by morphologic process;

initializing a score for each word in the set of words and for each sentence in the set of sentences;

computing the score for each word in the set of words according to the score of sentences containing it and the correlation degree between the word and the user information;

computing the score for each sentence in the set of sentences according to the score of words composing it and the position of the sentence in a section and a paragraph; and

if the sum of scores of the words and the sum of scores of the sentences change apparently, go back to the step of computing the word score;

otherwise continuing;

outputting the top-ranked sentences as the summary of the set of documents, the top-ranked words as the keywords list of the set of documents.

2. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the linguistic salience of the word to the user profile.

3. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the similarity among the word, the query and topic provided by a user.

4. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the similarity among the word and the terms in the titles of the documents.

5. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the ratio of its occurrence number in the document to its occurrence number in the set of documents.

6. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the ratio of the number of the documents including the word to the number of documents in the set of documents.

7. An automatic method according to claim 1, wherein the step of computing the score for each word comprises:

computing the score for each word in the set of words according to the weighted-average of at least two of;

the linguistic salience of the word to the user profile;

the similarity among the word, the query and topic provided by a user;

the similarity among the word and the terms in the titles of the documents;

the ratio of its occurrence number in the document to its occurrence number in the set of documents; and

the ratio of the number of the documents comprising the word to the number of documents in the set of documents.

8. A computer program product for automatically generating summaries for text documents, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:

computer program code means for generating a set of sentences for a set of documents by document discourse analysis and a set of words by morphologic process;

computer program code means for initializing a score for each word in the set of words, and each sentence in the set of sentences;

computer program code means for computing the score for each word in the set of words according to the score of sentences containing it and the correlation degree between the word and the user information;

computer program code means for computing the score for each sentence in the set of sentences according to the score of words composing it and the position of the sentence in a section and a paragraph;

computer program code means for determining if the sum of scores of the words and the sum of scores of the sentences exhibit an apparent change; and

computer program code means for outputting the top-ranked sentences as the summary of the set of documents, the top-ranked words as the keywords list of the set of documents.

9. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the linguistic salience of the word to the user profile.

10. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the similarity among the word, the query and topic provided by a user.

11. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the similarity among the word and the terms in the titles of the documents.

12. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the ratio of its occurrence number in the document to its occurrence number in the set of documents.

13. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the ratio of the number of the documents comprising the word to the number of documents in the set of documents.

14. A computer program product for automatically generating summaries according to claim 8, wherein the computer program code means for computing the score for each word comprises:

computer program code means for computing the score for each word in the set of words according to the weighted-average of at least two of;

the linguistic salience of the word to the user profile,

the similarity among the word, the query and topic provided by a user,

the similarity among the word and the terms in the titles of the documents,

the ratio of its occurrence number in the document to its occurrence number in the set of documents, and

the ratio of the number of the documents including the word to the number of documents in the set of documents.