Method and system for document classification based on document structure and written style

• US 20090300046A1
• Filed: 05/29/2008
• Published: 12/03/2009
• Est. Priority Date: 05/29/2008
• Status: Active Grant

• Current Assignee: Rania Abouyounes
• Original Assignee: Rania Abouyounes
• Inventors: Abouyounes, Rania
• Granted Patent: US 8,082,248 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A document classification method and system based on document structure and style. The classification method and system categorizes document alphabetical words into complex and non-complex words, categorizes document linguistic sentences into subjective and non-subjective sentences and categorizes document images into descriptive and non-descriptive. The categorization is further used to calculate a complexity, subjectivity and descriptive-images classification of a document. This classification system can be used by a web search engine to filter, sort or tag a set of document references based on user selection.

Claims

1. A method of determining the classification of a document for the purpose of searching, comprising:

a) receiving the textual content of said document in the form of linguistic sentences and alphabetical words;

b) receiving meta-data on images of said document including each image size, title or description;

c) categorizing said linguistic sentences into subjective and non-subjective sentences;

d) categorizing said alphabetical words into complex and non-complex words;

e) categorizing said images into descriptive and non-descriptive images;

f) calculating the document subjectivity classification as the count of said subjective sentences or the ratio of subjective sentences to non-subjective sentences or total sentences in said document;

g) calculating the document complexity classification as the count of complex alphabetical words or the ratio of complex alphabetical words to non-complex words or total words; and

h) calculating the document descriptive-images classification as the count of descriptive-images.

2. The method in accordance with claim 1, wherein categorizing said linguistic sentences into subjective and non-subjective sentences comprises determining if said sentence contains at least one subjective pronoun and counting said sentence as a subjective sentence.

3. The method in accordance with claim 2, wherein a subjective pronoun is a property of the said document language or said sentence language and for an English document or English sentence said subjective pronouns are “

I, WE, HE, SHE, THEY, YOU and IT”

whereby “

YOU” and

“

IT”

are subjective pronouns when they are located at the beginning of an English sentence.

4. The method in accordance with claim 1, wherein categorizing said alphabetical words into complex and non-complex words comprises counting the number of syllables in each alphabetical word and counting said word as complex if the number of syllables is over a threshold value.

5. The method in accordance with claim 4, wherein a syllable in an alphabetical word is a linguistic function that is made up of a syllable nucleus usually a vowel optionally between two consonants whereby most languages support this linguistic function.

6. The method in accordance with claim 1, wherein categorizing said images into descriptive and non-descriptive images comprises counting an image as a descriptive-image when an image has a title or a description and its size is within a specified image size.

7. The method in accordance with claim 6, wherein the descriptive-images classification is the count of descriptive-images in said document.

8. The method in accordance with claim 1, wherein all three said document classifications are numerical values that can be stored and associated with each said document.

9. A document classification system that determines the subjectivity, complexity and descriptive-images classification of a document, comprising:

a memory; and

a document classification circuit or routine that;

a) receives linguistic sentences, alphabetical words and images meta-data of said document;

b) categorizes said received linguistic sentences into subjective and non-subjective sentences and based on this categorization calculates the subjectivity classification of said document;

c) categorizes said received alphabetical words into complex and non-complex words and based on this categorization calculates the complexity classification of said document; and

d) categorizes said images into descriptive and non-descriptive images and based on this categorization calculates the descriptive-images classification.

10. The document classification determining system of claim 9, wherein the document classification circuit or routine categorizes said linguistic sentences into subjective and non-subjective sentences by determining if each sentence contains at least one subjective pronoun and counting said sentence as a subjective sentence, wherein a subjective pronoun is a property of the document language or the sentence language and for an English language document or sentence said subjective pronouns are “

I, WE, HE, SHE, THEY, YOU and IT”

whereby “

YOU” and

“

IT”

are regarded as subjective pronouns when they are located at the beginning of an English sentence.

11. The document classification determining system of claim 9, wherein the document classification circuit or routine categorizes said alphabetical words into complex and non-complex by counting the number of syllables of each alphabetical word and counting said words as complex if the number of syllables is over a threshold value, wherein a syllable in a word is a linguistic function that is made up of a syllable nucleus usually a vowel optionally between two consonants whereby most languages support this linguistic function.

12. The document classification determining system of claim 9, wherein the document classification circuit or routine categorizes said images into descriptive and non-descriptive images by counting an image as a descriptive-image when an image has a title or a description and its size is within a specified image size.

13. The document classification determining system of claim 9, wherein the document classification circuit or routine determines the document subjectivity, complexity and descriptive-images classifications as numeric values by applying these steps:

a) calculating document subjectivity classification as the count of said subjective sentences or as the ratio of subjective sentences to non-subjective sentences or to the total number of sentences in said document;

b) calculating said document complexity classification as the count of complex alphabetical words or as the ratio of complex alphabetical words to non-complex words or to the total number of words in said document; and

c) calculating said document descriptive-images classification as the count of descriptive-images.

14. The document classification determining system of claim 9, wherein the document classification circuit or routine stores each of the three classification numerical values and additional classification tags to denote one or more of the three said document classifications.

15. The document classification determining system of claim 14, wherein the said classification tags can be either the explicit calculated classification numerical values or symbolic representations of said numerical values in the form of one or more pictures or symbolic characters that can graphically distinguish between low, equal or high numerical values when said document classifications are displayed or stored.