AUTOMATIC DOCUMENT CLASSIFICATION USING TEXT AND IMAGES

• US 20010042085A1
• Filed: 09/30/1998
• Published: 11/15/2001
• Est. Priority Date: 09/30/1998
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: CULLEN, JOHN F., HULL, JONATHAN J., PEAIRS, MARK
• Granted Patent: US 7,039,856 B2
• Time in Patent Office: Days
• US Class Current: 715/526
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/5846 using extracted text

Abstract

A method and apparatus for automatic document classification using text and images. The present invention provides a method and apparatus for automatic document classification based on text and image. A new document is analyzed based on textual content as well as visual appearance. The new document is automatically stored in one or more mirror directories in which the new document would most likely be stored by the user of the device if the new document were placed manually. Determination of the most likely directories is based on an analysis of multiple documents stored by the user in various directories. The mirror directories are components of a mirror directory structure, which is a copy of a pre-existing directory structure, such as the user'"'"'s hard drive. By storing the new document automatically, the user is relieved of the duty of manually selecting a directory for the new document.

Claims

1. A method for document classification comprising:

analyzing textual and graphical properties of an electronic document;

generating a classification of the document based on the textual and graphical properties; and

storing the electronic document in a pre-existing document hierarchy based on the classification.

2. The method defined in

claim 1

where the document hierarchy comprises a hierarchy of documents mirroring in a similar fashion an organization in a pre-existing memory storing documents.

3. The method defined in

claim 2

when the pre-existing memory comprises a hard disk.

4. The method of

claim 1

, wherein analyzing textual properties of an electronic document comprises:

determining characteristic words of the document;

determining a frequency for each characteristic word; and

building a frequency table based on the frequency associated with each characteristic word.

5. The method of

claim 1

, wherein analyzing graphical properties of an electronic document comprises:

determining a point set corresponding to the electronic document, wherein points of the point set correspond to end points of lines;

determining a density of points within the point set;

generating a document profile based, at least in part, on the density of points within the point set.

6. The method of

claim 1

, wherein generating a classification of the document based on the textual and graphical properties comprises combining results from the textual and graphical analysis using a Borda Count.

7. The method defined in

claim 1

further comprising building the pre-existing directory structure by building hierarchy of documents based on a user'"'"'s hard drive.

8. The method defined in

claim 1

further comprising building the pre-existing directory structure by extracting graphical and text features from documents in a directory-based memory to obtain a

11. The machine-readable medium of

claim 5

, wherein the sequences of instructions that cause the processor to analyze graphical properties of an electronic document further comprise sequences of instructions that cause the processor to:

determine a point set corresponding to the electronic document, wherein points of the point set correspond to end points of lines;

determine a density of points within the point set;

generate a document profile based, at least in part, on the density of points within the point set.

12. The machine-readable medium of

claim 5

, wherein the sequences of instructions that cause the processor to generate a classification of the document based on the textual and graphical properties further comprises sequences of instructions that cause the processor to combine results from the textual and graphical analysis using a Borda Count.

9. A software product including a machine-readable medium having stored thereon sequences of instructions, which, when executed by a processor, cause the processor to:

analyze textual and graphical properties of an electronic document;

generate a classification of the document based on the textual and graphical properties; and

store the electronic document in a pre-existing directory structure based on the classification.

10. The machine-readable medium of

claim 9

, wherein the sequences of instructions that cause the processor to analyze textual properties of an electronic document further comprise sequences of instructions that cause the processor to:

determine characteristic words of the document;

determine a frequency for each characteristic word; and

build a frequency table based on the frequency associated with each characteristic word.

13. A method for document classification comprising:

analyzing documents in a pre-existing document directory structure to determine an organization of the pre-existing document directory structure;

generating a mirror directory structure based on the pre-existing document directory structure; and

placing a document in the mirror directory structure based on the organization of the pre-existing document directory structure, results of textual analysis of the document, and results of graphical analysis of the document.

14. The method of

claim 13

, wherein analyzing the pre-existing document directory structure to determine the organization further comprises:

recursively descending the pre-existing document directory structure;

generating a list of directories in the pre-existing document directory structure;

examining files in directories of the pre-existing document directory structure to determine content; and

examining the content of the files to determine the organization of the directories in the pre-existing document directory structure.

15. The method of

claim 13

wherein the pre-existing document directory structure is a hard disk directory structure.

16. The method of

claim 13

wherein generating a mirror directory structure based on the pre-existing document directory structure comprises generating a document directory structure having a set of directories and relationships equivalent to the pre-existing document directory structure.

17. The method of

claim 13

wherein placing a document in the mirror directory structure based on the organization of the pre-existing document directory structure comprises:

determining a primary directory in the pre-existing document directory structure in which the document should be placed based on the organization of the pre-existing document directory structure; and

storing the document in a primary corresponding directory in the mirror directory structure that corresponds to the primary directory in the pre-existing document directory structure.

18. The method of

claim 17

further comprising:

determining a secondary directory in the pre-existing document directory in which the document should be placed based on the organization of the pre-existing document directory structure; and

storing the document in a corresponding secondary directory in the mirror directory structure that corresponds to the secondary directory in the pre-existing document directory structure.

19. A computer-readable medium having stored thereon sequences of instructions which, when executed by a processor, cause the processor to:

analyze a pre-existing document directory structure to determine an organization of the pre-existing document directory structure;

generate a mirror directory structure based on the pre-existing document directory structure; and

place a document in the mirror directory structure based on the organization of the pre-existing document directory structure.

20. The computer-readable medium of

claim 19

, wherein the sequences of instructions that cause the processor to analyze a pre-existing document directory structure to determine an organization of the pre-existing document directory structure further comprise sequences of instructions that cause the processor to:

recursively descending the pre-existing document directory structure;

generating a list of directories in the pre-existing document directory structure;

examining files in directories of the pre-existing document directory structure to determine content; and

examining the content of the files to determine the organization of the directories in the pre-existing document directory structure.

21. The computer-readable medium of

claim 19

, wherein the sequences of instructions that cause the processor to generate a mirror directory structure based on the pre-existing document directory structure further comprise sequences of instructions that cause the processor to generate a document directory structure having a set of directories and relationships equivalent to the pre-existing document directory structure.

22. The computer-readable medium of

claim 19

, wherein the sequences of instructions that cause the processor to place a document in the mirror directory structure based on the organization of the pre-existing document directory structure further comprise sequences of instructions that cause the processor to:

determine a primary directory in the pre-existing document directory structure in which the document should be placed based on the organization of the pre-existing document directory structure; and

store the document in a primary corresponding directory in the mirror directory structure that corresponds to the primary directory in the pre-existing document directory structure.

23. The computer-readable medium of

claim 22

further comprising sequences of instructions that cause the processor to:

determine a secondary directory in the pre-existing document directory in which the document should be placed based on the organization of the pre-existing document directory structure; and

store the document in a corresponding secondary directory in the mirror directory structure that corresponds to the secondary directory in the pre-existing document directory structure.

24. An apparatus comprising:

means for analyzing a pre-existing document directory structure to determine an organization of the pre-existing document directory structure;

means for generating a mirror directory structure based on the pre-existing document directory structure; and

means for placing a document in the mirror directory structure based on the organization of the pre-existing document directory structure.

25. The apparatus of

claim 24

, wherein means for analyzing the pre-existing document directory structure to determine the organization further comprises:

means for recursively descending the pre-existing document directory structure;

means for generating a list of directories in the pre-existing document directory structure;

means for examining files in directories of the pre-existing document directory structure to determine content; and

means for examining the content of the files to determine the organization of the directories in the pre-existing document directory structure.

26. The apparatus of

claim 24

, wherein means for generating a mirror directory structure based on the pre-existing document directory structure comprises means for generating a document directory structure having a set of directories and relationships equivalent to the pre-existing document directory structure.

27. The apparatus of

claim 24

, wherein means for placing a document in the mirror directory structure based on the organization of the pre-existing document directory structure comprises:

means for determining a primary directory in the pre-existing document directory structure in which the document should be placed based on the organization of the pre-existing document directory structure; and

means for storing the document in a primary corresponding directory in the mirror directory structure that corresponds to the primary directory in the pre-existing document directory structure.

28. The apparatus of

claim 27

further comprising:

means for determining a secondary directory in the pre-existing document directory in which the document should be placed based on the organization of the pre-existing document directory structure; and

means for storing the document in a corresponding secondary directory in the mirror directory structure that corresponds to the secondary directory in the pre-existing document directory structure.

29. A document processing system comprising:

a document scanning device;

a document storage device coupled to the document scanning device, wherein the document storage device is organized as a document directory structure having multiple directories, and further wherein the document storage device has a mirror directory structure having multiple directories organized based on the document directory structure; and

a processor coupled to the document scanning device and to the document storage device, wherein the processor analyzes a document scanned by the document scanning device to determine a directory in the document directory structure in which the document should be placed and stores the document in a corresponding directory in the mirror directory structure.

30. The document processing system of

claim 29

wherein the processor determines a secondary director in the document directory structure in which the document should be placed and stored the document is a corresponding secondary directory in the mirror directory structure.

31. The document processing system of

claim 29

wherein the processor analyzes files stored in the document directory structure to determine content and generates an organizational description of the document directory structure based on the analysis.

32. The document processing system of

claim 29

wherein the document is analyzed based on image and textual content.