System and method for measuring the quality of document sets

• US 8,832,140 B2
• Filed: 06/25/2008
• Issued: 09/09/2014
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Oracle OTC Subsidiary LLC (Oracle Corporation)
• Original Assignee: Oracle OTC Subsidiary LLC (Oracle Corporation)
• Inventors: Tunkelang, Daniel, Wang, Joyce Jeanpin, Zelevinsky, Vladimir, Sheu, Herng Albert
• Primary Examiner(s): Alam, Hosain
• Assistant Examiner(s): JOHNSON, JOHNESE T
• Application Number:
  US12/146,280
• Publication Number:
  US 20090006387A1
• Time in Patent Office: 2,267 Days
• Field of Search: 707/722, 707/737, 707/723, 707/736, 707/748
• US Class Current: 707/769
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/3331 Query processing

Abstract

Systems and methods are described that calculate the interestingness of a set of one or more records in a database, either absolutely (i.e., compared to an overall collection of records) or relative to some other set of records. In one embodiment, the measure is a relative entropy value that has been normalized. Various applications of the measure are described in the context of an information retrieval system. These applications include, for example, guiding query interpretation, guiding view selection and summarization, intelligent ranges, event detection, concept triggers and interpreting user actions, hierarchy discovery, and adaptive data mining.

Claims

1. A computer implemented method for optimizing results returned from interaction with a collection of information, the method comprising:

establishing criteria associated with at least one operation on a collection of information, wherein the criteria is based, at least in part, on a comparison of a measurement of the distinctiveness of a set of results and a distinctiveness score threshold;

establishing a rule that comprises the criteria and the at least one operation;

determining the set of results from interaction with a collection of information, wherein the set of results comprises a plurality of documents retrieved from the collection of information, and wherein each of the plurality of documents further comprise a unit of storage of digital data;

determining, by a computer system, a measurement of distinctiveness for the set of results based on a statistical distribution of at least one identifying characteristic, wherein the distinctiveness of the set of results is measured in relation to the collection of information, and wherein the determining the measurement of distinctiveness comprises;

identifying the at least one identifying characteristic within an evaluated set, and

determining a measure of distinctiveness of the evaluated set within the collection of information;

modifying, by the computer system, the set of results according to the at least one operation by applying the rule to the set of results in response to determining that the set of results matches the criteria based, at least in part, on the comparison of the measurement of distinctiveness for the set of results and the distinctiveness score threshold; and

outputting a modified result.

2. The method according to claim 1, further comprising establishing additional criteria.

3. The method according to claim 1, further comprising establishing additional operations.

4. The method according to claim 1, wherein the interaction with the collection of information comprises at least one of a query against the collection of information, a search applied to the collection of information, a search entered in a text box, a filtering operation applied to the collection of information, navigation within the collection of information, access to the collection of information, access to the collection of information through a browser, or a data mining operation.

5. The method according to claim 1, further comprising determining a measurement of distinctiveness based on a statistical distribution of at least one identifying characteristic within the set of results and a statistical distribution of at least one identifying characteristic within another set.

6. The method according to claim 5, further comprising generating the another set from the collection of information.

7. The method according to claim 1, further comprising generating a linear combination of the statistical distributions for the set of results and the another set, and wherein the measurement of distinctiveness is further based on the linear combination.

8. The method according to claim 1, wherein the establishing criteria comprises establishing at least one of a minimum value for the measurement of distinctiveness of the set results, a maximum value for the measurement of distinctiveness, or a range of values for the measurement of distinctiveness.

9. The method according to claim 1, wherein the modifying the set of results further comprises incorporating at least one additional element from the collection of information.

10. The method according to claim 1, wherein the modifying the set of results further comprises performing at least one additional operation.

11. The method according to claim 1, wherein the modifying the set of results comprises at least one of grouping content within the modified set, sorting content within the modified set, or filtering content within the modified set.

12. The method according to claim 1, wherein the modifying the set of results comprises generating a suggested query.

13. The method according to claim 12, wherein the outputting the modified result comprises outputting the suggested query.

14. The method according to claim 1, further comprising defining the at least one operation to identify desired content within the collection of information.

15. The method according to claim 1, further comprising defining the at least one operation to replace at least one element of the interaction with the collection of information with at least one predefined element.

16. The method according to claim 1, further comprising normalizing a measurement of distinctiveness.

17. The method according to claim 1, further comprising:

modifying a size of at least one set; and

determining a measurement of distinctiveness from the at least one modified set.

18. The method according to claim 1, further comprising determining a measurement of distinctiveness for at least one set, wherein the measurement of distinctiveness is determined relative to a baseline measure.

19. The method according to claim 1, further comprising determining a measurement of distinctiveness for at least one set, wherein the measurement of distinctiveness is determined relative to the collection of information.

20. The method according to claim 1, further comprising tracking a state associated with the interaction with the collection of information.

21. The method according to claim 20, wherein the modification of the set of results comprises a modification of the state associated with the interaction with the collection of information.

22. The method according to claim 1, further establishing criteria associated with a state associated with the interaction with the collection of information.

49. The method according to claim 1, further comprising defining the at least one operation to incorporate at least one predefined element into the set of results.

23. A method for information processing, comprising:

generating a salience measure that measures a quality of a set of documents, wherein the set of documents comprises a plurality of documents retrieved from an information retrieval system, based on their ambiguity relative to similarly sized sets of documents retrieved from the information retrieval system; and

using a distinctiveness value generated from the salience measure to take a given action in the information retrieval system, wherein the distinctiveness value is measured in relation to the collection of information, and wherein the using the distinctiveness value comprises establishing criteria associated with the given action on a collection of information, wherein the criteria is based, at least in part, on a comparison of the distinctiveness value and a distinctiveness score threshold, establishing a rule that comprises the criteria and the at least one operation, and applying the rule to the set of documents to take the given action.

24. The method as described in claim 23, wherein the salience measure is a normalized clarity value, and wherein the normalized clarity value is generated by accounting for a size of a measured set.

25. The method as described in claim 24 wherein, for the set of documents S, the normalized clarity value is the number of standard deviations by which the clarity of S exceeds to expected value of the clarity of a set of documents of the same size as S.

26. A non-transitory computer-readable medium having computer-readable instructions stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to optimize results returned from interaction with a collection of information, the optimizing comprising:

establishing criteria associated with at least one operation on a collection of information, wherein the criteria is based, at least in part, on a comparison of a measurement of the distinctiveness of a set of results and a distinctiveness score threshold;

establishing a rule that comprises the criteria and the at least one operation;

determining the set of results from interaction with a collection of information, wherein the set of results comprises a plurality of documents retrieved from the collection of information, and wherein each of the plurality of documents further comprise a unit of storage of digital data;

determining a measurement of distinctiveness for the set of results based on a statistical distribution of at least one identifying characteristic, wherein the distinctiveness of the set of results is measured in relation to the collection of information, and wherein the determining the measurement of distinctiveness comprises;

identifying the at least one identifying characteristic within an evaluated set, and

determining a measure of distinctiveness of the evaluated set within the collection of information;

modifying the set of results according to the at least one operation by applying the rule to the set of results in response to determining that the set of results matches the criteria based, at least in part, on the comparison of the measurement of distinctiveness for the set of results and the distinctiveness score threshold; and

outputting a modified result.

50. The non-transitory computer-readable medium of claim 26, the optimizing further comprising determining a measurement of distinctiveness based on a statistical distribution of at least one identifying characteristic within the set of results and a statistical distribution of at least one identifying characteristic within another set.

51. The non-transitory computer-readable medium of claim 50, the optimizing further comprising generating the another set from the collection of information.

27. A system for optimizing results returned from interaction with a collection of information, the system comprising:

at least one processor operatively connected to a memory, wherein the at least one processor when executing provides;

a rules engine adapted to establish criteria associated with at least one operation on a collection of information, wherein execution of the operation is based on a comparison of a measurement of a distinctiveness of the set of results and a distinctiveness score threshold, and wherein the rules engine is further configured to establish a rule that comprises the criteria and the at least one operation;

a measurement engine adapted to measure the distinctiveness of a set of results, wherein the distinctiveness of the set of results is measured in relation to the collection of information, and wherein the measurement engine is further adapted to;

determine a measurement of distinctiveness for the set of results based on a statistical distribution of at least one identifying characteristic

identify the at least one identifying characteristic within an evaluated set, and

determine a measure of distinctiveness of the evaluated set within the collection of information;

a retrieval engine adapted to return a set of results from a collection of information in response to interaction with the collection of information, wherein the set of results comprises a plurality of documents retrieved from the collection of information, wherein each of the plurality of documents further comprise a unit of storage of digital data;

a modification engine adapted to modify the set of results according to the at least one operation by applying the rule to the set of results in response to a determination that the set of results matches the established criteria, wherein the determination is based, at least in part, on the comparison of the measurement of distinctiveness for the set of results and the distinctiveness score threshold; and

an output engine adapted to output the modified result.

28. The system according to claim 27, wherein the rules engine is further adapted to establish additional criteria.

29. The system according to claim 27, wherein the rules engine is further adapted to establish additional operations.

30. The system according to claim 27, further comprising an input engine adapted to manage interaction with the collection of information, wherein interaction comprises at least one of a query against the collection of information, a search applied to the collection of information, a search entered in a text box, a filtering operation applied to the collection of information, navigation within the collection of information, access to the collection of information, access to the collection of information through a browser, and a data mining operation.

31. The system according to claim 27, wherein the measurement engine is further adapted to determine a measurement of distinctiveness based on a statistical distribution of at least one identifying characteristic within the set of results and a statistical distribution of at least one identifying characteristic within another set.

32. The system according to claim 31, further comprising a generation engine adapted to generate the another set from the collection of information.

33. The system according to claim 27, wherein the measurement engine is further adapted to generate a linear combination of the statistical distributions for the set of results and the another set, and wherein the measurement of distinctiveness is further based on the linear combination.

34. The system according to claim 27, wherein the rules engine is further adapted to establish at least one of a minimum value for the measurement of distinctiveness of the set results, a maximum value for the measurement of distinctiveness, and a range of values for the measurement of distinctiveness.

35. The system according to claim 27, wherein the modification engine is further adapted to incorporate at least one additional element from the collection of information.

36. The system according to claim 27, wherein the modification engine is further adapted to perform at least one additional operation.

37. The system according to claim 27, wherein the modification engine is further adapted to include at least one of grouping content within the modified set, sorting content within the modified set, and filtering content within the modified set.

38. The system according to claim 27, wherein the modification engine is further adapted to generate a suggested query.

39. The system according to claim 38, wherein the output engine is further adapted to output the suggested query.

40. The system according to claim 27, wherein the rules engine is further adapted to define the at least one operation to identify desired content within the collection of information.

41. The system according to claim 27, wherein the rules engine is further adapted to define the at least one operation to replace at least one element of the interaction with the collection of information with at least one predefined element.

42. The system according to claim 27, further comprising a normalization engine adapted to normalize a measurement of distinctiveness.

43. The system according to claim 27, further comprising a sizing engine adapted to modifying a size of at least one set, and wherein the measurement engine is further adapted to determine a measurement of distinctiveness from the at least one modified set.

44. The system according to claim 27, wherein the measurement engine is further adapted to determine a measurement of distinctiveness for at least one set relative to a baseline measure.

45. The system according to claim 27, wherein the measurement engine is further adapted to determine the measurement of distinctiveness relative to the collection of information.

46. The system according to claim 27, further comprising a tracking engine adapted to track a state associated with the interaction with the collection of information.

47. The system according to claim 46, wherein the modification engine is further adapted to modify the state associated with the interaction with the collection of information.

48. The system according to claim 27, wherein the rules engine is further adapted to establish criteria associated with a state associated with the interaction with the collection of information.