Agent-based method for distributed clustering of textual information

• US 20060080311A1
• Filed: 10/12/2004
• Published: 04/13/2006
• Est. Priority Date: 10/12/2004
• Status: Active Grant

• Current Assignee: UT-Battelle LLC
• Original Assignee: UT-Battelle LLC
• Inventors: Potok, Thomas E., Reed, Joel W.
• Granted Patent: US 7,805,446 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A computer method and system for storing, retrieving and displaying information has a multiplexing agent (20) that calculates a new document vector (25) for a new document (21) to be added to the system and transmits the new document vector (25) to master cluster agents (22) and cluster agents (23) for evaluation. These agents (22, 23) perform the evaluation and return values upstream to the multiplexing agent (20) based on the similarity of the document to documents stored under their control. The multiplexing agent (20) then sends the document (21) and the document vector (25) to the master cluster agent (22), which then forwards it to a cluster agent (23) or creates a new cluster agent (23) to manage the document (21). The system also searches for stored documents according to a search query having at least one term and identifying the documents found in the search, and displays the documents in a clustering display (80) of similarity so as to indicate similarity of the documents to each other.

Claims

1. A computer method for storing information in a computer system having at least first and second computers for retrieval and display based on similarity of information, the method comprising:

a first-tier program module operating on a first computer for determining a new document vector to characterize a new document for comparison of a similarity of the new document to other documents stored in the computer system;

the first-tier program module transmitting the new document vector to a second-tier program module operating on a second computer in the computer system;

the second-tier program module transmitting a similarity value to said first-tier program module which represents a comparison of the new document vector to at least one composite vector characterizing a similarity of a plurality of documents stored in the computer system; and

based on said similarity value received from said second-tier program module, said first-tier program module determining whether said new document should be transmitted to said second-tier program module for storage in the computer system.

2. The computer method of claim 1, wherein the second-tier program module transmits the document vector to a plurality of third-tier program modules operating on the second computer in the computer system;

the third-tier program modules each storing a composite vector representing the similarity of a respective plurality of documents stored in the second computer under the control of the respective third-tier program module; and

the third-tier program modules each receiving the document vector for the new document and comparing the document vector to a respective composite vector to determine similarity of the new document to the plurality of documents stored in the second computer under the control of the third-tier program module;

the third-tier program module each returning a similarity value resulting from comparison of the new document vector to a respective composite vector.

3. The computer method of claim 2, wherein

said first-tier program module transmits said new document to the second-tier program module for further transmission to a selected one of said third-tier program modules;

and

wherein upon receiving said new document, said third-tier program module transmits the new document vector to at least one fourth-tier program module;

wherein the fourth-tier program modules compares the new document vector to a respective composite vector to determine similarity of the new document to the plurality of documents stored in the second computer under the control of the fourth-tier program module;

wherein the fourth-tier program modules transmits a similarity value to the third-tier program module; and

based on evaluation of the similarity value returned by said fourth-tier program module, said third-tier program module determines whether the new document is to be transmitted to the fourth-tier program module.

4. The method of claim 2, wherein

based on the evaluation of the similarity value returned by the second-tier program module, said first-tier program module transmits the new document to the second-tier program module for storage under control of an existing third-tier and an existing fourth-tier program module, which are in communication with the second-tier program module.

5. The method of claim 2, wherein

based on the evaluation of the similarity value returned by the second-tier program module, said first-tier program module transmits the new document to the second-tier program module for storage under control of a third-tier program module and a fourth-tier program module, which are to be created by the second-tier program module.

6. The method of claim 1, wherein

each new document has been converted to a data file using a tag-identifier language before determining the document vector.

7. The method of claim 1, 2, 3, 4, 5 or 6, further comprising

searching for stored documents according to a search query having at least one term and identifying the documents found in the search;

and

displaying the documents so as to indicate similarity of the documents to each other.

8. The method of claim 7, wherein the documents are displayed as nodes of a tree structure having links and nodes in which similarity of documents is indicated by proximity of nodes to each other and by a length of links connecting the nodes to a common vertex.

9. The method of claim 8, wherein the documents are displayed as nodes of a tree structure that represents documents stored under the control of at least one third-tier program module.

10. The method of claim 1, 2, 3, 4, 5 or 6, wherein the information, including the new document, is collected from a plurality of Internet web sites.

11. The method of claim 1, wherein the first-tier program module and the second-tier program module are agent program modules that are originated in the JAVA computer language.

12. The method of claim 1, further comprising:

the first-tier program module transmitting the new document vector to a second, second-tier program module operating on a third computer in the computer system;

the second, second-tier program module transmitting a similarity value to said first-tier program module which represents a comparison of the new document vector to at least one composite vector characterizing a similarity of a plurality of documents stored in the third computer; and

based on said similarity value received from each of said second-tier program modules, said first-tier program module determining whether said new document should be transmitted to a selected one of said second-tier program modules for storage in the computer system.

13. A computer system for storing, retrieving and displaying information, the computer system being operable on at least one computer having a software operating system, the computer system comprising:

a multiplexing program module running on a first computer for receiving a new document and for calculating a new document vector for the new document, and for transmitting said new document vector to at least one next-tier program module; and

a next-tier program module running on a second computer for receiving said document vector from said multiplexing program module and for comparing said document vector to at least one composite vector representing the documents being stored for access through a third-tier program running on the second computer to determine if said document should be added to the documents being stored for access through the third-tier program module.

14. The computer system of claim 13, wherein the next-tier program module transmits the document vector to a plurality of third-tier program modules operating on the second computer in the computer system;

the third-tier program modules each storing a composite vector representing the similarity of a respective plurality of documents stored in the second computer under the control of the respective third-tier program module; and

the third-tier program modules each receiving the document vector for the new document and comparing the document vector to a respective composite vector to determine similarity of the new document to the plurality of documents stored in the second computer under the control of the third-tier program module;

the third-tier program module each returning a similarity value resulting from comparison of the new document vector to a respective composite vector.

15. The computer system of claim 14, wherein said multiplexing program module transmits said new document to the next-tier program module for further transmission to a selected one of said third-tier program modules;

and

wherein upon receiving said new document, said third-tier program module transmits the new document vector to at least one fourth-tier program module;

wherein the fourth-tier program module compares the new document vector to a respective composite vector to determine similarity of the new document to the plurality of documents stored in the second computer under the control of the fourth-tier program module;

wherein the fourth-tier program module transmits a similarity value to the third-tier program module; and

based on evaluation of the similarity value returned by said fourth-tier program module, said third-tier program module determines whether the new document is to be transmitted to the fourth-tier program module.

16. The computer system of claim 14, wherein

based on the evaluation of the similarity value returned by the next-tier program module, said multiplexing program module transmits the new document to the next-tier program module for storage under control of an existing third-tier and an existing fourth-tier program module, which are in communication with the next-tier program module.

17. The computer system of claim 14, wherein

based on the evaluation of the similarity value returned by the next-tier program module, said multiplexing program module transmits the new document to the next-tier program module for storage under control of a third-tier program module and a fourth-tier program module, which are to be created by the next-tier program module.

18. The computer system of claim 13, further comprising:

a second next-tier program module running on a third computer for receiving the new document received by the first next-tier program module; and

wherein said next-tier program module running on said second computer and said next-tier program module running on said third computer each compare the new document vector to a respective composite vector to determine a similarity value representing the similarity of the new document to the documents already stored in the second computer and the third computer, respectively, and wherein each of said next-tier program modules communicate said respective similarity value to said multiplexing module for a determination in which of the second computer and the third computer the document should be stored for access.

19. The computer system of claim 13, 14, 15, 16 or 17 further comprising:

a computer display and a display program module for displaying the documents so as to indicate similarity of the documents to each other.

20. The computer system of claim 19, wherein the documents are displayed by the display program module as nodes of a tree structure having links and nodes in which similarity of documents is indicated by proximity of nodes to each other and by a length of links connecting the nodes to a common vertex.

21. The computer system of claim 20, wherein the documents are displayed by the display program module as nodes of a tree structure that represents documents stored under the control of at least one third-tier program module.

22. The method of claim 13, 14, 15, 16 or 17, wherein the information, including the new document, is collected from a plurality of Internet web sites.

23. The method of claim 13, wherein the multiplexing program module and the next-tier program modules are agent program modules originated in the JAVA computer language.