Method for producing a document summary

• US 20080104506A1
• Filed: 10/30/2006
• Published: 05/01/2008
• Est. Priority Date: 10/30/2006
• Status: Abandoned Application

• Current Assignee: Atefeh Farzindar
• Original Assignee: Atefeh Farzindar
• Inventors: Farzindar, Atefeh
• Application Number:
  US11/589,142
• Publication Number:
  US 20080104506A1
• Time in Patent Office: Days
• US Class Current: 715/254
• CPC Class Codes: G06F 16/345 Summarisation for human users

Abstract

A method for producing a document summary from a document. The method includes:

associating with the document a specific category from a set of predetermined categories;

performing a thematic segmentation of the document to produce a segmented document, the segmented document including a plurality of text segments;

associating with each text segment from the plurality of text segments a theme selected from a set of predetermined themes; and

summarizing the segmented document to produce the document summary by processing each text segment from the plurality of text segments to either

• • select at least one summary textual unit from the text segment, the at least one summary textual unit including at least one word and being a textual unit considered important in summarizing the document; or
  • extract no textual unit from the text segment.
    The summary textual units are used to form the document summary. The thematic segmentation is dependent on the category to which the document is associated and the summary textual units are selected for each text segment depending on the theme with which the text segment is associated.

Claims

1. A method for producing a document summary from a document, said document including a plurality of words and being segmentable into a plurality of text segments, each text segment including at least one word, said document being classifiable as belonging to a category selected from a set of predetermined categories and each text segment being classifiable as belonging to a theme selected from a set of predetermined themes, said method comprising:

associating with said document a specific category from said set of predetermined categories;

performing a thematic segmentation of said document to produce a segmented document, said segmented document including said plurality of text segments;

associating with each text segment from said plurality of text segments a theme selected from said set of predetermined themes; and

summarizing said segmented document to produce said document summary by processing each text segment from said plurality of text segments to either

select at least one summary textual unit from said text segment, said at least on summary textual unit including at least one of said word, said at least one summary textual unit being a textual unit considered important in summarizing said document;

or

extract no textual unit from said text segment;

said summary textual units being used to form said document summary;

wherein said thematic segmentation is dependent on said category to which said document is associated and said summary textual units are selected for each text segment depending on said theme with which said text segment is associated.

2. A method as defined in claim 1, wherein associating said document with a specific category includes computing for each category from said set of predetermined categories a respective document categorization score indicative of a likelihood that said document is classifiable in said category, said document categorization score being computed from said document, said specific category being a category from said set of predetermined categories for which said document categorization score associated therewith is maximal.

3. A method as defined in claim 2, wherein computing said document categorization scores includes computing a document statistic of said document and comparing said document statistic with a set of predetermined statistics, each predetermined statistic being

associated with a respective predetermined category from said set of predetermined category;

and

representative of documents that are classifiable in said respective predetermined category.

4. A method as defined in claim 3, wherein said document statistic is obtained using a support vector machine method.

5. A method as defined in claim 2, wherein computing said document categorization scores includes

applying a set of predetermined categorization rules to said document, the application of each predetermined categorization rule to said document resulting in the computation of a respective categorization rule score;

and

combining said categorization rule scores to obtain said document categorization scores.

6. A method as defined in claim 2, wherein computing said document categorization scores includes combining a statistical score and a heuristic score, each of said statistical and heuristic scores being computed from said document.

7. A method as defined in claim 2, wherein said set of predetermined categories is a hierarchical set of categories.

8. A method as defined in claim 1, further comprising dividing said document into said plurality of text segments.

9. A method as defined in claim 8, wherein associating with each text segment from said plurality of text segments said theme selected from said set of predetermined themes includes computing for each text segment from said plurality of text segments a set of segment categorization scores, each segment categorization score from said set of segment categorization scores being associated with a respective theme from said set of predetermined themes and being indicative of a likelihood that said text segment is classifiable in said theme with which said segment categorization score is associated, each of said text segment being associated with a theme from said set of predetermined themes for which said segment categorization score associated therewith is maximal.

10. A method as defined in claim 9, wherein computing said segment categorization scores includes computing a segment statistic of said text segment and comparing said segment statistic with a set of predetermined segment statistics, each predetermined segment statistic being

associated with a respective predetermined theme from said set of predetermined themes;

and

representative of segments that are classified in said respective predetermined theme for document classified in said specific category.

11. A method as defined in claim 10, wherein

said document includes at least one section identified by a section heading present in said document, each of said sections including at least one paragraph, each of said paragraphs including at least one sentence, each of said sentences including at least one word;

each of said text segment includes at least one paragraph;

each of said segment statistic depends on a least one factor from the set consisting of;

a section in which said at least one paragraph is included, a position of said at least one paragraph in said document, a presence of a predetermined group of words in said at least one paragraph and linguistic information derived from words included in said at least one paragraph included in said text segment.

12. A method as defined in claim 1, wherein

said document includes at least one section identified by a section heading present in said document, each of said sections including at least one paragraph, each of said paragraphs including at least one sentence, each of said sentences including at least one word;

summarizing said segmented document to produce said document summary includes computing for each sentence of said document a respective sentence score indicative of a likelihood that said sentence is important in summarizing said document.

13. A method as defined in claim 12, wherein computing said sentence scores for each sentence includes computing a sentence statistic of said sentence.

14. A method as defined in claim 13, wherein said sentence statistic depends on at least one factor selected from the set consisting of:

a position of said sentence in said document, a position of a paragraph in which said sentence is included in said section in which said paragraph is included;

a frequency of words included in said sentence as compared with a frequency with which said words are included in said document, an expected frequency with which said words included in said sentence are expected to be included in documents categorized in said specific category and in themes associated with said paragraph in which said sentence is included, a frequency of textual units included in said sentence as compared with a frequency with which said textual units are included in said document, and an expected frequency with which textual units included in said sentence are expected to be included in documents categorized in said specific category and in themes associated with said paragraph in which said sentence is included.

15. A method as defined in claim 14, wherein computing said sentence score includes, for each sentence,

computing a heuristic sentence score from said sentence by applying a set of predetermined heuristic sentence rules to said sentence, each heuristic sentence rule being associated with a sentence rule score;

combining said sentence rule scores to obtain said heuristic sentence score; and

combining said heuristic sentence score and said sentence statistic to obtain said sentence score.

16. A method as defined in claim 15, wherein said document summary includes sentences from said document having a sentence score higher than a threshold score, said threshold score being selected so that said summary document is smaller than a predetermined size.

17. A method as defined in claim 16, wherein said threshold score is selected individually for each of said predetermined themes so that said sentences selected to be part of said document summary for each of said predetermined themes represent a predetermined fraction of said document.

18. A method as defined in claim 1, further comprising filtering said document to remove words satisfying a predetermined word rejection criterion.

19. A method as defined in claim 1, wherein summarizing said document includes replacing in said document expressions included in a list of predetermined expressions by respective predetermined abbreviations.

20. A method as defined in claim 1, further comprising translating said document summary.

21. A method as defined in claim 20, wherein translating said document is performed using translation rules which depend on said specific category.

22. A method as defined in claim 1, wherein said document is a court judgment.

23. A computer readable storage medium containing a program element for execution by a computing device, said program element being able to produce a document summary from a document, said document including a plurality of words and being segmentable into a plurality of text segments, each text segment including at least one word, said document being classifiable as belonging to a category selected from a set of predetermined categories and each text segment being classifiable as belonging to a theme selected from a set of predetermined themes, said program element comprising:

an input module operative for receiving the document;

a categorization module operative for associating with said document a specific category from said set of predetermined categories;

a segmentation module operative for

performing a thematic segmentation of said document to produce a segmented document, said segmented document including said plurality of text segments; and

associating with each text segment from said plurality of text segments a theme selected from said set of predetermined themes;

a summarization module operative for summarizing said segmented document to produce said document summary by processing each text segment from said plurality of text segments to either

select at least one summary textual unit from said text segment, said at least on summary textual unit including at least one of said word, said at least one summary textual unit being a textual unit considered important in summarizing said document;

or

extract no textual unit from said text segment;

said summary textual units being used to form said document summary; and

an output module operative for releasing the summarized document;

wherein said thematic segmentation is dependent on said category to which said document is associated and said summary textual units are selected for each text segment depending on said theme with which said text segment is associated.