SYSTEM AND METHOD FOR MEASURING THE QUALITY OF DOCUMENT SETS

• US 20090006438A1
• Filed: 06/25/2008
• Published: 01/01/2009
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Oracle OTC Subsidiary LLC (Oracle Corporation)
• Original Assignee: Oracle OTC Subsidiary LLC (Oracle Corporation)
• Inventors: Kapell, Joshua William, Tunkelang, Daniel, Zelevinsky, Vladimir, Wang, Joyce Jeanpin
• Granted Patent: US 8,219,593 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/3331 Query processing

Abstract

Systems and methods are described that calculate the interestingness of a set of one or more records in a database, either absolutely (i.e., compared to an overall collection of records) or relative to some other set of records. In one embodiment, the measure is a relative entropy value that has been normalized. Various applications of the measure are described in the context of an information retrieval system. These applications include, for example, guiding query interpretation, guiding view selection and summarization, intelligent ranges, event detection, concept triggers and interpreting user actions, hierarchy discovery, and adaptive data mining.

Claims

1. A method for organizing a database, the method comprising:

analyzing the database for a statistical distribution of at least one identifying characteristic;

generating a measurement of distinctiveness based on the statistical distribution of the at least one identifying characteristic;

identifying at least one similar group of elements within the database based on the measurement of distinctiveness;

generating a descriptor associated with the identified at least one similar group of elements; and

organizing the database based on the descriptor.

2. The method according to claim 1, further comprising an act of normalizing the measurement of distinctiveness.

3. The method according to claim 1, wherein the act of analyzing the database further comprises an act of generating at least one subset from the database.

4. The method according to claim 3, further comprising an act of manipulating a size of the generated subset within the database.

5. The method according to claim 1, further comprising an act of comparing the statistical distribution of the at least one identifying characteristic against a baseline statistical distribution.

6. The method according to claim 1, further comprising an act of generating a hierarchy of organization based on the measurement of distinctiveness.

7. The method according to claim 6, further comprising an act of identifying a relationship between elements of the database based, at least in part, on the statistical distribution of at least one identifying characteristic.

8. The method according to claim 7, further comprising an act of determining a relationship based on at least one identifying characteristic in common, and a measurement of distinctiveness.

9. The method according to claim 7, wherein the relationship identifies at least one of a parent, child, and sibling element within the database.

10. The method according to claim 6, further comprising an act of updating the hierarchy of organization based on review of the database.

11. The method according to claim 10, wherein review of the database comprises at least one of access to the database, navigation of the database, at least one query run against the database, at least one search in the database, a filtering operation on the database, a ranking operation on the database, and a data mining operation on the database.

12. The method according to claim 1, further comprising an act of caching information associated with the measurement of distinctiveness.

13. The method according to claim 12, wherein the caching information comprises at least one of age of the measurement of distinctiveness, age of any of the underlying calculations used to generate the measurement of distinctiveness, intermediate computation values, partial computation values, computational expense associated with generation of the measurement of distinctiveness, resource usage to maintain the measurement of distinctiveness, and resource usage to maintain the group.

14. The method according to claim 1, further comprising an act of modifying the at least one descriptor in response to review of the database.

15. The method according to claim 1, further comprising an act of indexing the database based, at least in part, the at least one descriptor.

16. The method according to claim 1, wherein the at least one identifying characteristic comprises at least one element of review of the database.

17. The method according to claim 16, wherein the acts of generating and grouping are repeated for subsequent review.

18. The method according to claim 16, further comprising an act of generating an additional identifying characteristic based on review of the database.

19. The method according to claim 1, further comprising an act of generating at least one additional descriptor in response to review of the database.

20. The method according to claim 19, wherein the act of grouping is further based on the at least one additional descriptor.

21. The method according to claim 1, further comprising an act of determining the at least one identifying characteristic based on review of the database.

22. The method according to claim 21, wherein review of the database comprises at least one of access to the database, navigation of the database, at least one query run against the database, at least one search in the database, a filtering operation on the database, a ranking operation on the database, and a data mining operation on the database.

23. A computer-readable medium having computer-readable instructions stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to perform a method for organizing a database, the method comprising the acts of:

analyzing the database for a statistical distribution of at least one identifying characteristic;

generating a measurement of distinctiveness based on the statistical distribution of the at least one identifying characteristic;

identifying at least one similar group of elements within the database based on the measurement of distinctiveness;

generating a descriptor associated with the identified at least one similar group of elements; and

organizing the database based on the descriptor.

24. A system for organizing a database, the system comprising:

an analysis component adapted to determine a measurement of distinctiveness based on a statistical distribution of at least one identifying characteristic;

a generation component adapted to generate a descriptor for at least one element of the database based on the measurement of distinctiveness; and

an organization component adapted to group a plurality of elements within the database based on the at least one description.

25. The system according to claim 24, further comprising a normalization component adapted to normalize the measurement of distinctiveness.

26. The system according to claim 24, wherein the analysis component is further adapted to generate at least one subset from the database.

27. The system according to claim 26, wherein the analysis component is further adapted to manipulate a size of the generated subset.

28. The system according to claim 24, wherein the analysis component is further adapted to compare the statistical distribution of the at least one identifying characteristic against a baseline statistical distribution.

29. The system according to claim 24, wherein the organization component is further adapted to generate a hierarchy of organization based on the measurement of distinctiveness.

30. The system according to claim 29, wherein the organization component is further adapted to identify a relationship between elements of the database based, at least in part, on the statistical distribution of at least one identifying characteristic.

31. The system according to claim 30, wherein the organization component is further adapted to determine a relationship based on at least one identifying characteristic in common, and a measurement of distinctiveness.

32. The system according to claim 30, wherein the organization component is further adapted to identify at least one of a parent, child, and sibling element within the database.

33. The system according to claim 29, wherein the organization component is further adapted to update the hierarchy of organization based on review of the database.

34. The system according to claim 33, wherein review of the database comprises at least one of access to the database, navigation of the database, at least one query run against the database, at least one search in the database, a filtering operation on the database, a ranking operation on the database, and a data mining operation on the database.

35. The system according to claim 24, further comprising a storage component adapted to cache information associated with the measurement of distinctiveness.

36. The system according to claim 35, wherein the caching information comprises at least one of age of the measurement of distinctiveness, age of any of the underlying calculations used to generate the measurement of distinctiveness, intermediate computation values, partial computation values, computational expense associated with generation of the measurement of distinctiveness, resource usage to maintain the measurement of distinctiveness, and resource usage to maintain the group.

37. The system according to claim 24, wherein the generation component is further adapted to modify the at least one descriptor in response to review of the database.

38. The system according to claim 24, wherein the organization component is further adapted to index the database based, at least in part, the at least one descriptor.

39. The system according to claim 24, wherein the at least one identifying characteristic comprises at least one element of review of the database.

40. The system according to claim 39, wherein the analysis component is further adapted to generate an additional identifying characteristic based on review of the database.

41. The system according to claim 24, wherein the generation component is further adapted to generate at least one additional descriptor in response to review of the database.

42. The system according to claim 41, wherein the organization component is further adapted to group based on the at least one additional descriptor.

43. The system according to claim 24, wherein the analysis component is further adapted to determine the at least one identifying characteristic based on review of the database.