System and method for document section segmentation

• US 7,818,308 B2
• Filed: 09/07/2007
• Issued: 10/19/2010
• Est. Priority Date: 10/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Heyvaert, Stefaan, MacPherson, Melissa, Parkes, Cornelia, Carus, Alwin B.
• Primary Examiner(s): Corrielus; Jean M
• Application Number:
  US11/851,871
• Publication Number:
  US 20080059498A1
• Time in Patent Office: 1,138 Days
• Field of Search: 707/3, 707/6, 707/705, 707/736, 707/737, 707/755, 707/756
• US Class Current: 707/705
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A system and method for facilitating the processing and the use of documents by providing a system for categorizing document section headings under a set of canonical section headings. In the method for categorizing section headings, there may be a process of training a database and matching methods to categorize different but equivalent document section headings under canonical headings and categories. Once trained, the system may match and categorize the document sections with little to no supervision of the categorization for large sets of documents.

Claims

1. An automated computer implemented method for categorizing document section headings comprising the steps of:

determining a set of canonical section headings from a set of documents;

establishing a data set containing said canonical section headings and information associating at least one of the canonical section headings with at least one other section heading that is different from the at least one of the canonical section headings but corresponds to the at least one of the canonical section headings;

extracting at least one section heading from another document;

transforming the extracted section heading into a plurality of n-grams, and;

associating the extracted section heading with a particular one of the canonical section headings in the data set if said plurality of n-grams have a predetermined level of similarity to said particular one of the canonical section headings.

2. The method according to claim 1, where the canonical section headings include empirical data consisting of headings taken from existing documents.

3. The method according to claim 1, where the canonical section headings include words that are normalized to the word stems.

4. The method according to claim 3, where the stemmed forms are filtered for non-content or stop words.

5. The method according to claim 3, where the stemmed forms include synonyms or hypernyms.

6. At least one computer readable storage medium encoded with instructions that, when executed, perform a method for categorizing document section headings comprising acts of:

determining a set of canonical section headings from a set of documents;

establishing a data set containing said canonical section headings and information associating at least one of the canonical section headings with at least one other section heading that is different from the at least one of the canonical section headings but corresponds to the at least one of the canonical section headings;

extracting at least one section heading from another document;

transforming the extracted section heading into a plurality of n-grams, and;

associating the extracted section heading with a particular one of the canonical section headings in the data set if said plurality of n-grams have a predetermined level of similarity to said particular one of the canonical section headings.

7. The at least one computer readable storage medium according to claim 6, where the canonical section headings include empirical data consisting of headings taken from existing documents.

8. The at least one computer readable storage medium according to claim 6, where the canonical section headings include words that are normalized to the word stems.

9. The at least one computer readable storage medium according to claim 8, where the stemmed forms are filtered for non-content or stop words.

10. The at least one computer readable storage medium according to claim 8, where the stemmed forms include synonyms or hypernyms.

11. A system comprising:

at least one processor programmed to;

determine a set of canonical section headings from a set of documents;

establish a data set containing said canonical section headings and information associating at least one of the canonical section headings with at least one other section heading that is different from the at least one of the canonical section headings but corresponds to the at least one of the canonical section headings;

extract at least one section heading from another document;

transform the extracted section heading into a plurality of n-grams, and;

associate the extracted section heading with a particular one of the canonical section headings in the data set if said plurality of n-grams have a predetermined level of similarity to said particular one of the canonical section headings.

12. The system according to claim 11, where the canonical section headings include empirical data consisting of headings taken from existing documents.

13. The system according to claim 11, where the canonical section headings include words that are normalized to the word stems.

14. The system according to claim 13, where the stemmed forms are filtered for non-content or stop words.

15. The system according to claim 13, where the stemmed forms include synonyms or hypernyms.