Grouping interdependent fields

• US 9,135,280 B2
• Filed: 02/11/2013
• Issued: 09/15/2015
• Est. Priority Date: 02/11/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Milenova, Boriana Lubomirova, Campos, Marcos M.
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US13/764,658
• Publication Number:
  US 20140229482A1
• Time in Patent Office: 946 Days
• Field of Search: 707/737, 707/802
• US Class Current: 1/1
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases

Abstract

Processes, machines, and stored machine instructions are provided for grouping interdependent fields. Field grouping logic may include specially configured machines and/or stored instructions that identify group(s) of interdependent fields of a data set. The field grouping logic may receive, from a client on a customizable interface, a request for interdependent fields in a data set and, in response, cause generation of an output object that identifies the similar fields in the data set. The field grouping logic may exclude field(s) of the data set that are not interdependent, are not frequently accessed, or do not consume much space in storage, even though the request may not identify which fields are interdependent. The output object identifies the similar fields in set(s) or list(s) of fields, or in a hierarchy or hierarchies of groups and sub-groups.

Claims

1. A method

for determining which fields, of a plurality of fields in a data set, are interdependent, comprising:

determining different levels of interdependence between different pairs of fields of the plurality of fields at least in part by, for each pair of the different pairs of fields;

for a respective sample of the data set in said each pair;

computing respective measured frequencies of co-occurrences of values in said each pair,

computing respective predicted frequencies of co-occurrences of values in said each pair based at least in part on measured frequencies of values in each separate field of said each pair;

computing a respective divergence score that measures how divergent values are in the separate fields of said each pair based at least in part on the respective measured frequencies of co-occurrences and the respective predicted frequencies of co-occurrences;

determining a maximum divergence score among respective divergence scores computed by said computing a respective divergence score for each pair of said different pairs;

scaling the respective divergence score based at least in part on the maximum divergence score to a respective scaled divergence score that is different from the respective divergence score; and

identifying a group of fields as interdependent based on said levels of interdependence between different pairs of fields.

2. The method of claim 1, wherein the group comprises a hierarchy that includes two or more sub-groups of interdependent fields, wherein the two or more sub-groups have a different level of interdependence than the fields within each of the two or more sub-groups.

3. The method of claim 1, wherein the plurality of fields comprises a first field, a second field, and a third field;

wherein the group comprises the first field and the second field and excludes at least the third field.

4. The method of claim 1, wherein the maximum divergence score for a particular pair, of said different pairs of fields, comprising a first field and a second field, is based at least in part on a first number of unique values in the first field or a second number of unique values in the second field.

5. The method of claim 1, wherein the respective divergence score for a particular pair, of said different pairs of fields, comprising a first field and a second field, is based at least in part on a sum of:

for each combination of values in the first field and values in the second field, a measured frequency of co-occurrence of said each combination of values multiplied by a logarithm of;

the respective measured frequency of co-occurrence denominated by

the respective predicted frequency of co-occurrence of said each combination of values.

6. The method of claim 1, wherein, for a particular pair of said different pairs, before scaling the respective divergence score, the respective divergence score is proportional to the maximum divergence score.

7. The method of claim 1, wherein the plurality of fields comprises a first field, a second field, and a third field;

wherein the group comprises the first field and the second field and excludes at least the third field;

wherein determining the levels of interdependence comprises;

determining a first level of interdependence between the first field and the second field;

determining a second level of interdependence between the first field and the third field;

determining that the first level of interdependence satisfies a threshold level of interdependence and the second level of interdependence does not satisfy the threshold level of interdependence.

8. The method of claim 1, wherein the group is a first group, further comprising identifying a second group of fields as interdependent based at least in part on the levels of interdependence, wherein the second group includes different fields of the plurality of fields than the first group.

9. The method of claim 1, wherein, for each pair of the different pairs, the respective divergence score for said each pair accounts for measured frequencies of co-occurrences and predicted frequencies of co-occurrences of values in said each pair.

10. The method of claim 1, wherein the group of interdependent fields is identified based at least in part on how frequently at least one field in the group is accessed.

11. The method of claim 1, further comprising automatically selecting the plurality of fields from the data set based at least in part on how frequently fields in the data set are accessed.

12. The method of claim 1, wherein the group of interdependent fields is identified based at least in part on a size of at least one field in the group.

13. The method of claim 1, further comprising automatically selecting the plurality of fields from the data set based at least in part on sizes of fields in the data set.

14. The method of claim 1, further comprising, in response to identifying the group, generating one or more density models that approximate values of fields in the group.

15. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause

determining which fields, of a plurality of fields in a data set, are interdependent and comprise instructions which, when executed by the one or more computing devices further cause performance of steps comprising:

determining different levels of interdependence between different pairs of fields of the plurality of fields at least in part by, for each pair of the different pairs of fields;

for a respective sample of the data set in said each pair;

computing respective measured frequencies of co-occurrences of values in said each pair,

computing respective predicted frequencies of co-occurrences of values in said each pair based at least in part on measured frequencies of values in each separate field of said each pair;

computing a respective divergence score that measures how divergent values are in the separate fields of said each pair based at least in part on the respective measured frequencies of co-occurrences and the respective predicted frequencies of co-occurrences;

determining a maximum divergence score among respective divergence scores computed by said computing a respective divergence score for each pair of said different pairs;

scaling the respective divergence score based at least in part on the maximum divergence score to a respective scaled divergence score that is different from the respective divergence score; and

identifying a group of fields as interdependent based on said levels of interdependence between different pairs of fields.

16. The one or more non-transitory storage media of claim 15, wherein the group comprises a hierarchy that includes two or more sub-groups of interdependent fields, wherein the two or more sub-groups have a different level of interdependence than the fields within each of the two or more sub-groups.

17. The one or more non-transitory storage media of claim 15,

wherein the plurality of fields comprises a first field, a second field, and a third field;

wherein the group comprises the first field and the second field and excludes at least the third field.

18. The one or more non-transitory storage media of claim 15,

wherein the maximum divergence score for a particular pair, of said different pairs of fields, comprising a first field and a second field, is based at least in part on a first number of unique values in the first field or a second number of unique values in the second field.

19. The one or more non-transitory storage media of claim 15, wherein the respective divergence score for a particular pair, of said different pairs of fields, comprising a first field and a second field, is based at least in part on a sum of:

for each combination of values in the first field and values in the second field, a measured frequency of co-occurrence of said each combination of values multiplied by a logarithm of;

the respective measured frequency of co-occurrence denominated by

the respective predicted frequency of co-occurrence of said each combination of values.

20. The one or more non-transitory storage media of claim 15, wherein, for a particular pair of said different pairs, before scaling the respective divergence score, the respective divergence score is proportional to the maximum divergence score.

21. The one or more non-transitory storage media of claim 15, wherein the plurality of fields comprises a first field, a second field, and a third field;

wherein the group comprises the first field and the second field and excludes at least the third field;

wherein determining the levels of interdependence comprises;

determining a first level of interdependence between the first field and the second field;

determining a second level of interdependence between the first field and the third field;

determining that the first level of interdependence satisfies a threshold level of interdependence and the second level of interdependence does not satisfy the threshold level of interdependence.

22. The one or more non-transitory storage media of claim 15, wherein the group is a first group, wherein the instructions, when executed, further cause identifying a second group of fields as interdependent based at least in part on the levels of interdependence, wherein the second group includes different fields of the plurality of fields than the first group.

23. The one or more non-transitory storage media of claim 15, wherein, for each pair of the different pairs, the respective divergence score for said each pair accounts for measured frequencies of co-occurrences and predicted frequencies of co-occurrences of values in said each pair.

24. The one or more non-transitory storage media of claim 15, wherein the group of interdependent fields is identified based at least in part on how frequently at least one field in the group is accessed.

25. The one or more non-transitory storage media of claim 15, wherein the instructions, when executed, further cause automatically selecting the plurality of fields from the data set based at least in part on how frequently fields in the data set are accessed.

26. The one or more non-transitory storage media of claim 15, wherein the group of interdependent fields is identified based at least in part on a size of at least one field in the group.

27. The one or more non-transitory storage media of claim 15, wherein the instructions, when executed, further cause automatically selecting the plurality of fields from the data set based at least in part on sizes of fields in the data set.

28. The one or more non-transitory storage media of claim 15, wherein the instructions further cause, in response to identifying the group, generating one or more density models that approximate values of fields in the group.