Document clustering method and system

• US 20040249787A1
• Filed: 01/29/2004
• Published: 12/09/2004
• Est. Priority Date: 03/22/2000
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Mehmet Oguz Sayal, Meichun Hsu, Parvathi Chundi, Umeshwar Dayal
• Inventors: Hsu, Meichun, Dayal, Umeshwar, Sayal, Mehmet Oguz, Chundi, Parvathi
• Granted Patent: US 7,181,678 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Document clustering method and system utilizing both th logbased clustering method and the content-based clustering method are disclosed. The method includes the steps of generating log-based document clusters and combining vectors from the log-based document clusters with individual document clusters for content-based clustering analysis. The log-based document clusters are generated by accessing the retrieval session log, clustering the retrieval sessions, and combining the documents opened during each of the sessions of session clusters.

Claims

1-16. -16. cancel.

17. A method for clustering documents, including generating clusters with user perspective comprising:

receiving session logs;

performing log-based clustering on the session logs to generate session clusters;

representing each session cluster as a log-based document suitable for content based clustering;

receiving a plurality of documents that includes a first document that was accessed in one session and a second document that was not accessed in the sessions;

replacing the first document with a log-based document associated with the session cluster that includes the first document; and

performing content based clustering on at least the first document and the second document to generate clusters with user perspective.

18. The method of claim 17 wherein representing each session cluster as a log-based document suitable for content based clustering includes modifying each document referenced in the session cluster so that a Euclidean distance between the documents is the same.

19. The method of claim 17, wherein each session log comprises a query used to retrieve documents.

20. The method of claim 17, wherein each session log comprises a number of documents found to satisfy a query.

21. The method of claim 17, wherein each session log comprises a list of documents opened by a user.

22. The method of claim 17, wherein each session log comprises a length of time that a document was opened.

23. A method for clustering documents comprising:

generating a hybrid matrix of vectors comprising a first vector representing a first document and a second vector representing a log-based document cluster; and

clustering the documents using the hybrid matrix.

24. The method of claim 23 wherein a second vector is used in place of a second document within the hybrid matrix wherein the second document forms a portion of the log-based document cluster.

25. The method of claim 23 wherein clustering the documents using the hybrid matrix is performed using a content-based clustering technique.

26. The method recited in claim 23 wherein generating the hybrid matrix comprises:

accessing retrieval session logs;

clustering retrieval sessions into session clusters;

generating, a log-based document cluster for each session cluster by combining all documents opened during any retrieval session of the session cluster;

generating a log-based document cluster vector for each of the log-based document clusters;

replacing each document in the log-based document cluster with the log-based document cluster vector;

generating an individual document vector for each document not opened during any retrieval session; and

combining the log-based document cluster vector and the individual document cluster vector.

27. The method of claim 26 wherein the step of clustering retrieval sessions into session clusters comprises the steps of:

generating a Boolean session vector for each retrieval session;

forming a matrix of the Boolean session vectors; and

applying a clustering algorithm to the matrix of the Boolean session vectors.

28. A system for clustering documents, the system comprising;

a storage for storing retrieval session logs; and

a processor connected to the storage, configured to cluster the retrieval sessions into session clusters, generate, for each session cluster, a log-based document cluster, generate a log-based document cluster vector for each of the log-based document clusters, generate an individual document vector for each document not opened during any retrieval session, cluster the documents using the log-based document cluster vectors and individual document vectors.

31. A data processing system having session logs and documents, the system comprising:

a processor for executing program instructions; and

a media readable by the processor having a document clustering module having a plurality of instructions, that when executed by the processor, performs log-based clustering on the session logs to generate session clusters, converts the session clusters into a form suitable for content-based clusters, performs content-based clustering on the documents and session clusters in a form suitable for content-based clustering to generate document clusters with users'"'"' perspective.

34. The system of claim 31 wherein the media is one of a floppy disk, compact disc, a volatile memory, and a non-volatile memory.

35. A machine readable memory device encoded with a data structure for clustering documents, the data structure having entries for a log-based document cluster vector generated from a log-based document cluster, and an individual document vector corresponding to a vector generated from a first document, the first document not belonging to any log based document cluster.