Information resource taxonomy

• US 20050080781A1
• Filed: 12/18/2002
• Published: 04/14/2005
• Est. Priority Date: 12/18/2001
• Status: Active Grant

• Current Assignee: Telstra Corporation Limited (Telstra Group Ltd.)
• Original Assignee: Telstra New Wave Pty Limited
• Inventors: Ryan, Simon David, Raskutti, Bhavani, Sember, Peter Paul, Phiet, Do Quang
• Granted Patent: US 8,166,030 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing

Abstract

An information resource taxonomy system, including a data collector for collecting information resources from a communications network; and a taxonomy generator for generating a taxonomy represented by a hierarchy of resource clusters, using cluster criteria generated from the collected resources. The system includes an editor for editing the criteria, and a renderer for generating linked document data for displaying the hierarchy. A parallel cluster search system is used to evaluate clusters in parallel. The system also includes a parallel classifier for classifying further collected resources.

Claims

1. A process for generating a taxonomy for a plurality of information resources in a communications network, including:

collecting said resources from said network;

generating clusters of said resources on the basis of content of said resources; and

iteratively generating sub-clusters of said clusters based on content of each resource to provide said taxonomy as a hierarchy of resource clusters, wherein the number of said resource clusters is determined by content of said resources.

2. A process as claimed in claim 1, wherein said step of iteratively generating includes selecting one or more of said clusters if one or more intra-cluster similarity values for respective resources of said clusters exceeds a threshold similarity value, and iteratively generating sub-clusters of the selected clusters.

3. A process as claimed in claim 1, wherein said step of generating clusters includes:

selecting one or more portions of said resources on the basis of respective metrics of the relevance of said one or more portions; and

generating clusters of said resources, whereby the number of said clusters is determined on the basis of the selected one or more portions of said resources.

4. A process as claimed in claim 3, wherein clusters are generated on the basis of similarity values generated from word frequency data generated from one or more selected portions of said resources.

5. A process as claimed in claim 1, including generating linked document data for displaying said taxonomy.

6. A process as claimed in claim 5, wherein said linked document data includes markup language data.

7. A process as claimed in claim 5, wherein said linked document data includes metadata of said resources.

8. A process as claimed in claim 1, including generating descriptive text for said resources and descriptive text for each node of said hierarchy.

9. A process as claimed in claim 1, wherein said resources include dynamically generated content of said network.

10. A process as claimed in claim 1, wherein components of said hierarchy are sorted based on access frequencies of said resources.

11. A process as claimed in claim 1, wherein a resource is added to an existing cluster if the similarity of said resource to said cluster meets a similarity requirement.

12. A process as claimed in claim 1, wherein a new cluster is generated if the similarity of said resource to each existing cluster does not meet a similarity requirement.

13. A process as claimed in claim 1, wherein said step of generating clusters includes selecting a similarity value for clustering on the basis of goodness values for respective groupings of said resources generated for respective similarity values.

14. A process as claimed in claim 13, wherein the goodness value for each grouping is generated on the basis of similarity values for resources within the clusters of the grouping and differences between cluster centroids for the clusters of the grouping and a global centroid for said resources.

15. A process as claimed in claim 1, wherein said steps of generating clusters and iteratively generating sub-clusters are scalable with the number of said resources.

16. A process as claimed in claim 1, wherein said clusters are generated from a subset of said resources.

17. A process as claimed in claim 16, including selecting said subset by random sampling of said resources.

18. A process as claimed in claim 16, including classifying resources into said resource clusters.

19. A process as claimed in claim 16, including using said clusters as a training set for a classifier.

20. A process as claimed in claim 19, wherein said classifier includes a support vector machine.

21. A process as claimed in claim 16, including clustering using a similarity value determined whilst clustering the subset of said resources.

22. A process as claimed in claim 1, including generating one or more new clusters for resources that are not substantially similar to existing clusters.

23. A process as claimed in claim 21, including maintaining the coherence of clusters as additional resources are collected by reducing said similarity value with increasing number of resources.

24. A process as claimed in claim 16, including maintaining the coherence of clusters as additional resources are collected by selecting a random sample of clustered resources, generating a new hierarchy of resource clusters from the selected resources, and classifying the unselected resources into said new hierarchy of resource clusters.

25. A process as claimed in claim 16, including selecting a subset of clustered resources as the number of clustered resources increases to generate a metric for evaluating the quality of existing clusters.

26. A process as claimed in claim 23, including determining a new similarity value for reclustering said existing clusters on the basis of the quality of existing clusters.

27. A process as claimed in claim 1, wherein said step of iteratively generating includes selecting one or more of said clusters on the basis of respective measures of coherence of said clusters, and iteratively generating sub-clusters of the selected clusters.

28. An information resource taxonomy system having components for executing the steps of claim 1.

29. A computer-readable storage medium, having stored thereon program code for executing the steps of claim 1.

30. An information resource taxonomy system, including

a data collector for collecting information resources from a communications network;

and

a taxonomy generator for generating clusters of said resources, and for iteratively generating sub-clusters of said clusters based on content of each resource to provide a taxonomy for said resources as a hierarchy of resource clusters, wherein the number of said resource clusters is determined by content of said resources.

31. A system as claimed in claim 30, including an editor for editing said taxonomy, and a renderer for generating linked document data for displaying said hierarchy.

32. A system as claimed in claim 30, including a classifier for classifying further resources collected by said system.

33. A system as claimed in claim 30, wherein said system is scalable with respect to the number of said resources.

34. A system as claimed in claim 30, including a parallel cluster search system for evaluating clusters in parallel.

35. A system as claimed in claim 30, including a parallel classifier for classifying further resources in parallel.