Methods and apparatus for steering the analyses of collections of documents

• US 20060179051A1
• Filed: 11/03/2005
• Published: 08/10/2006
• Est. Priority Date: 02/09/2005
• Status: Abandoned Application

• Current Assignee: Battelle Memorial Institute
• Original Assignee: Battelle Memorial Institute
• Inventors: McGee, David R., Whitney, Paul D., Havre, Susan L.
• Application Number:
  US11/268,283
• Publication Number:
  US 20060179051A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3347 using vector based model

Abstract

A method for steering the analysis of a collection of documents includes receiving query terms for use in querying a database including a collection of documents; representing at least some of the query terms in a matrix; rotating document vectors associated with the documents to match the matrix to produce a matrix of rotated document vectors, each document vector representing a numeric vector created in association with individual documents; grouping the rotated document vectors into clusters, each cluster having one or more documents; and projecting the clusters to display visual information of the documents, the visual information including a summary view of the collection of documents. Program code and a system are also provided.

Claims

1. A method of steering the analysis of a collection of documents, comprising:

receiving query terms for use in querying a database including a collection of documents;

representing at least some of the query terms in a matrix;

rotating document vectors associated with the documents to match the matrix to produce a matrix of rotated document vectors, each document vector representing a numeric vector created in association with individual documents;

grouping the rotated document vectors into clusters, each cluster having one or more documents; and

projecting the clusters to display visual information of the documents, the visual information including a summary view of the collection of documents.

2. The method of claim 1, further comprising labeling the clusters using labels representing contents of the query.

3. The method of claim 1, wherein representing contents of the query comprises:

separating the query into atomic terms, the atomic terms including query terms for retrieving the collection of documents; and

constructing the matrix with the atomic terms.

4. The method of claim 3, wherein the matrix comprises an incidence matrix.

5. The method of claim 3, further comprising classifying the atomic terms as topic words by increasing a topicality value associated with the respective atomic terms.

6. The method of claim 1, wherein rotating the document vectors comprises changing the document vectors to reflect contents of the query.

7. The method of claim 1, wherein rotating the document vectors comprises rotating the document vectors using canonical correlations.

8. The method of claim 1, wherein grouping the rotated document vectors comprises grouping the rotated document vectors based on contents of the query.

9. The method of claim 1, wherein grouping the rotated document vectors is not based solely on contents of documents retrieved by the query.

10. The method of claim 1, wherein grouping the rotated document vectors comprises grouping documents associated with the rotated document vectors using statistical determination.

11. The method of claim 1, wherein grouping the rotated document vectors comprises grouping documents associated with the rotated document vectors into an unsupervised classification using a statistical technique.

12. The method of claim 1, wherein displaying visual information comprises displaying a summary view of the documents.

13. The method of claim 12, wherein the summary view is created based on contents of the documents projected into the clusters as well as on the contents of the query used to produce the clusters.

14. The method of claim 1, wherein the clusters comprise a collection of documents, the clusters being consistent with the contents of the query used to produce the clusters.

15. A method of steering the analysis of a collection of documents, comprising:

receiving a query against a database;

obtaining a query result set having a collection of documents;

grouping the collection of documents into a classification to produce a plurality of clusters, each cluster having a set of documents from the collection of documents, the grouping of the collection of documents into the clusters being based on contents of the query; and

displaying the clusters to display visual information of the collection of documents.

16. The method of claim 15, further comprising labeling the clusters using labels representing contents of the query.

17. The method of claim 15, the grouping comprises:

representing contents of the query as an incidence matrix, the incidence matrix comprising keywords of the query for retrieving the collection of documents;

rotating document vectors associated with the documents to match the incidence matrix, each document vector representing a numeric vector created in association with individual documents;

producing a matrix of rotated document vectors; and

classifying the rotated document vectors to produce the plurality of clusters.

18. The method of claim 17, wherein the rotating comprises changing the document vectors to reflect contents of the query.

19. The method of claim 17, wherein the rotating comprises rotating the document vectors using canonical correlations.

20. The method of claim 17, wherein the grouping further comprises:

separating the query into atomic terms;

constructing the incidence matrix using the atomic terms; and

classifying the atomic terms as topic words by increasing a topicality value associated with the respective atomic terms.

21. The method of claim 17, wherein the grouping further comprises grouping documents associated with the rotated document vectors using statistical determination.

22. The method of claim 17, wherein the grouping is not based solely on contents of documents retrieved by the query.

23. The method of claim 15, wherein display of visual information comprises displaying a summary view of the collection of documents.

24. The method of claim 23, wherein the summary view is created based on contents of the collection of documents projected into the clusters as well as on the contents of the query used to produce the clusters.

25. The method of claim 15, wherein the clusters comprising the collection of documents is consistent with contents of the query used to produce the clusters.

26. A computer-readable medium comprising computer program code which, when loaded in a computer, causes the computer, in operation, to:

receive a query against a database;

obtain a query result set having a collection of documents;

group the collection of documents into a classification to produce a plurality of clusters, each cluster having a set of documents from the collection of documents, the grouping of the collection of documents into the clusters being based on contents of the query; and

display the clusters to display visual information of the collection of documents.

27. The computer readable medium of claim 26, wherein the computer program code is further configured to label the clusters using labels representing contents of the query.

28. The computer readable medium of claim 26, wherein grouping the collection documents comprises:

representing contents of the query as an incidence matrix, the incidence matrix comprising keywords of the query for retrieving the collection of documents;

rotating document vectors associated with the documents to match the incidence matrix, each document vector representing a numeric vector created in association with individual documents;

producing a matrix of rotated document vectors; and

classifying the rotated document vectors to produce the plurality of clusters.

29. The computer readable medium of claim 27, wherein rotating the document vectors comprises changing the document vectors to reflect contents of the query.

30. The computer readable medium of claim 27, wherein rotating the document vectors comprises rotating the document vectors using canonical correlations.

31. The computer readable medium of claim 27, wherein grouping the collection of documents further comprises:

separating the query into atomic terms;

constructing the incidence matrix using the atomic terms; and

classifying the atomic terms as topic words by increasing a topicality value associated with the respective atomic terms.

32. The computer readable medium of claim 27, wherein grouping the collection of documents further comprises grouping documents associated with the rotated document vectors using statistical determination.

33. The computer readable medium of claim 27, wherein grouping the collection of documents is not based solely on contents of documents retrieved by the query.

34. The computer readable medium of claim 26, wherein display of visual information comprises displaying a summary view of the collection of documents.

35. The computer readable medium of claim 34, wherein the summary view is created based on contents of the collection of documents projected into the clusters as well as on the contents of the query used to produce the clusters.

36. The computer readable medium of claim 26, wherein the clusters comprising the collection of documents is consistent with contents of the query used to produce the clusters.

37. An information analysis and steering method, comprising:

receiving an information collection including information objects, each information object having a descriptive vector;

associating the information object with an indicator vector, the indicator vector having a plurality of vector coordinates;

labeling each of the plurality of vector coordinates with contents of a query that is used to produce the information collection; and

projecting the information collection as clusters, the clusters including the descriptive vectors and contents of the indicator vectors.

38. The method of claim 37, wherein the associating comprises representing the contents of the query as an incidence matrix.

39. The method claim 38, wherein the incidence matrix is generated by separating the query into atomic terms, and arranging the atomic terms in the form of a matrix.

40. The method of claim 39, wherein the associating further comprises:

rotating descriptive vectors associated with the information objects to match the incidence matrix to produce a matrix of rotated descriptive vectors; and

grouping the rotated descriptive vectors into the clusters.

41. The method of claim 40, wherein the labeling comprises labeling the clusters based on contents of the query.

42. The method of claim 40, wherein the rotating comprises rotating the descriptive vectors using canonical correlations.

43. The method of claim 40, wherein the grouping comprises grouping the clusters into an unsupervised classification using statistical determination.

44. The method of claim 37, wherein projecting the information collection as clusters comprises displaying a summary view of the information collection, the summary view being created based on the information collection projected as the clusters as well as on the contents of a query used to produce the information collection.

45. An information analysis and steering system comprising a computer server configured to:

receive an information collection including information objects, each information object having a descriptive vector;

associate the information object with an indicator vector, the indicator vector having a plurality of vector coordinates;

label each of the plurality of vector coordinates with contents of a query that is used to produce the information collection; and

project the information collection as clusters, the clusters including the descriptive vectors and contents of the indicator vectors.

46. The system of claim 45, wherein associating the information object comprises representing the contents of the query as an incidence matrix.

47. The system of claim 46, wherein the incidence matrix is generated by separating the query into atomic terms, and arranging the atomic terms in the form of a matrix.

48. The system of claim 47, wherein associating the information object further comprises:

rotating descriptive vectors associated with the information objects to match the incidence matrix to produce a matrix of rotated descriptive vectors; and

grouping the rotated descriptive vectors into the clusters.

49. The system of claim 48, wherein labeling each of the plurality of vector coordinates comprises labeling the clusters based on contents of the query.

50. The system of claim 48, wherein rotating the descriptive vectors comprises rotating the descriptive vectors using canonical correlations.

51. The system of claim 48, wherein grouping the rotated descriptive vectors comprises grouping the clusters into an unsupervised classification using statistical determination.

52. The system of claim 45, wherein projecting the information collection as clusters comprises displaying a summary view of the information collection, the summary view being created based on the information collection projected as the clusters as well as on the contents of a query used to produce the information collection.

53. A method of steering the analysis of a collection of documents, comprising:

receiving a collection of documents, the collection being produced by a query against a database;

creating a numeric vector for each document of the collection;

encoding the query to create an incidence matrix;

rotating the numeric vectors to match the incidence matrix;

grouping the rotated numeric vectors into clusters; and

projecting the clusters to create a summary view of the documents.

54. The method of claim 53, wherein vector coordinates of the numeric vector reflect differences in contents of the documents of the collection.

55. The method of claim 53, wherein the rotating comprises rotating the numeric vectors using a canonical correlations technique.

56. The method of claim 53, wherein the grouping comprises grouping the clusters into an unsupervised classification using statistical determination.

57. The method of claim 53, wherein the summary view of the documents is created based on the collection of documents projected as the clusters as well as on the contents of a query is used to produce the collection of documents.

58. A computer readable medium embodying computer program code which, when loaded in a computer, causes the computer, in operation, to:

represent contents of a query, used to retrieve a collection of documents, as a matrix;

rotate document vectors associated with the documents to match the matrix to produce a matrix of rotated document vectors;

group the rotated document vectors into clusters; and

project the clusters to display visual information of the documents.

59. A computer readable medium in accordance with claim 58, wherein the computer program code is further configured to cause the computer to label the clusters, the labels representing contents of the query.

60. A computer readable medium in accordance with claim 58, wherein representing contents of a query comprises separating the query into atomic terms, and constructing the matrix with the atomic terms.

61. A computer readable medium in accordance with claim 60, wherein the computer program code is further configured to cause the computer to classify the atomic terms as topic words by increasing a topicality value associated with the respective atomic terms.

62. A computer readable medium in accordance with claim 60, wherein rotating document vectors comprises changing the document vectors to reflect contents of the query.

63. A computer readable medium in accordance with claim 58, wherein rotating document vectors comprises rotating the document vectors using canonical correlations.

64. A computer readable medium in accordance with claim 58, wherein grouping rotated document vectors comprises grouping the rotated document vectors based on contents of the query.

65. A computer readable medium in accordance with claim 58, wherein grouping rotated document vectors comprises grouping documents associated with the rotated document vectors into an unsupervised classification using a statistical technique.

66. A computer readable medium in accordance with claim 58, wherein displaying visual information comprises displaying a summary view of the documents, the summary view being created based on contents of the documents projected into the clusters as well as on the contents of a query used to produce the clusters.

67. A method of representing information objects in a concept-space, comprising:

receiving a query against a database;

obtaining a query result set having a collection of information objects from the database, the collection of information objects related to one or more concepts;

grouping the collection of information objects into an unsupervised classification to produce a plurality of clusters, each cluster having a set of information objects from the collection, the grouping being performed based on the one or more concepts; and

projecting the clusters to display visual information of the collection of information objects, each of the clusters identifying a concept, each cluster includes information objects related to the concept identified by the cluster.

68. A method of claim 67, wherein the grouping comprises grouping information objects comprising a plurality of concepts across a plurality of clusters depending on concepts indicated by the information objects.

69. A method of claim 68, wherein computational complexity of the grouping comprises the cost of evaluating each of the information objects against a concept-space of interest.

70. A method of claim 67, wherein grouping the collection into clusters comprises spatially arranging the clusters based on similarity of information objects.

71. A method of claim 67, wherein the projecting comprises projecting each concept at a single location.

72. A method of claim 67, further comprising combining concepts for each of the information objects to produce a summary view of the concepts.

73. A method of steering the analysis of a collection of information objects, comprising:

receiving a collection of information objects, the information objects representing one or more concepts;

grouping the collection of information objects into a plurality of clusters, each cluster representing a single concept and having a set of information objects from the collection; and

projecting the clusters to display visual information of the collection of information objects.

74. A method of claim 73, wherein the grouping comprises grouping information objects comprising a plurality of concepts across a plurality of clusters depending on concepts indicated by the information objects.

75. A method of claim 73, wherein grouping the collection into clusters comprises spatially arranging the clusters based on similarity of information objects.

76. A method of claim 73, wherein the projecting comprises projecting each concept at a single location.

77. A method of claim 73, further comprising combining concepts for each of the information objects to produce a summary view of the concepts.

78. A computer-readable medium comprising computer usable-code, when loaded in a computer, causes the computer, in operation to:

receive a collection of information objects, the information objects representing one or more concepts;

group the collection of information objects into a plurality of clusters, each cluster representing a single concept and having a set of information objects from the collection; and

project the clusters to display visual information of the collection of information objects.

79. A computer-readable medium of claim 78, wherein grouping the collection of information objects comprises grouping information objects comprising a plurality of concepts across a plurality of clusters depending on concepts indicated by the information objects.

80. A computer-readable medium of claim 78, wherein grouping the collection comprises spatially arranging the clusters based on similarity of information objects.

81. A method comprising:

semantically filtering a set of documents in a database to extract a set of semantic concepts, to improve an efficiency of a predictive relationship to its content, based on at least one of word frequency, overlap and topicality;

defining a topic set, said topic set being characterized as the set of semantic concepts which best discriminate the content of the documents containing them, said topic set being defined based on at least one of word frequency, overlap and topicality;

forming a matrix with the semantic concepts contained within the topic set defining one dimension of said matrix and the semantic concepts contained within the filtered set of documents comprising another dimension of said matrix;

calculating matrix entries as the conditional probability that a document in the database will contain each semantic concept in the topic set given that it contains each semantic concept in the filtered set of documents;

providing the matrix entries as document vectors to interpret the document contents of the database;

inputting query terms;

augmenting the topic set by the query terms;

making an incidence matrix of query terms for the documents;

rotating the document vectors to match the incidence matrix; and

clustering and projecting the rotated document vectors.