Generating partitioned hierarchical groups based on data sets for business intelligence data models

• US 10,698,924 B2
• Filed: 05/22/2014
• Issued: 06/30/2020
• Est. Priority Date: 05/22/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rais-Ghasem, Mohsen M., Vierich, Ralf
• Primary Examiner(s): Ortiz Ditren, Belix M
• Assistant Examiner(s): Mahmood, Rezwanul
• Application Number:
  US14/285,269
• Publication Number:
  US 20150339369A1
• Time in Patent Office: 2,231 Days
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Techniques are described for generating a hierarchical group based on a set of data. In one example, a method includes classifying two or more data items from a set of data with respect to a library of ontological concepts. The method further includes classifying the two or more data items with respect to lexical correlations between the two or more data items. The method further includes generating a hierarchical group in which the two or more data items are partitioned into one or more hierarchical partitions based at least in part on the classifying with respect to the library of ontological concepts and the classifying with respect to the lexical correlations, wherein each of the one or more hierarchical partitions comprises the two or more data items.

Claims

1. A method for generating a hierarchical group based on a set of data, the method comprising:

classifying two or more data items from a set of data with respect to a library of ontological concepts based at least in part on properties of the two or more data items, including detecting correlations between the properties of the two or more data items and one or more ontological concepts from the library of ontological concepts, wherein the properties of the two or more data items include data types defined for the two or more data items and ranges of data values in data fields of the two or more data items;

classifying the two or more data items with respect to lexical correlations between the two or more data items, including determining correlations between one or more elements of headers of the two or more data items;

analyzing the two or more data items based on one or more factors to determine whether the one or more factors contribute to defining a hierarchical relationship, wherein the analysis utilizes the one or more factors that comprise a set of heuristic rules and relative cardinality, wherein the set of heuristic rules discounts or disqualifies quantifiers or metrics associated with the two or more data items, and wherein the relative cardinality minimizes quantifiers or metrics through merging;

generating a hierarchical group in which the two or more data items are partitioned into one or more hierarchical partitions based at least in part on the classifying with respect to the library of ontological concepts, the classifying with respect to the lexical correlations, and the analysis of the two or more data items based on the one or more factors, wherein each of the one or more hierarchical partitions comprises the two or more data items; and

verifying a sampling of data in the one or more hierarchical partitions, including measuring correlations between data in the two or more data items in a particular hierarchical partition from the one or more hierarchical partitions to determine whether the particular hierarchical partition has a first data item at a leaf level of the particular hierarchical partition in a one-to-many relationship with a second data item at a base level of the particular hierarchical partition.

2. The method of claim 1, wherein the properties of the two or more data items further include one or more of:

numbers of distinct counts of data entries in the two or more data items or density of data in the two or more data items.

3. The method of claim 1, wherein generating the hierarchical group comprises selecting one of the two or more data items as the base level of the particular hierarchical partition from the hierarchical partitions, and selecting at least one other of the two or more data items as at least one leaf level of the particular hierarchical partition.

4. The method of claim 1, wherein verifying the sampling of the data further includes modifying an arrangement of the two or more data items in the particular hierarchical partition.

5. The method of claim 1, wherein verifying the sampling of the data further includes modifying the arrangement of the two or more data items in the particular hierarchical partition such that the particular hierarchical partition does not have a data item at the leaf level of the particular hierarchical partition in a one-to-many relationship with a data item at the base level of the particular hierarchical partition.

6. The method of claim 1, further comprising providing the hierarchical group in a user interface and enabling user inputs to modify the hierarchical group.

7. The method of claim 1, further comprising:

receiving an input selecting the set of data at the data store,

wherein the generating of the hierarchical group is in response to the receiving of the input selecting the set of data.

8. The method of claim 1, wherein the set of data comprises a table from a relational database, and wherein the two or more data items comprise two or more columns from the table.

9. The method of claim 1, wherein the library of ontological concepts comprises one or more hierarchical classifications of categories, and wherein classifying the two or more data items with respect to the library of ontological concepts comprises identifying one or more of the data items with one or more hierarchical levels in one of the hierarchical classifications of categories.

10. The method of claim 9, wherein the one or more hierarchical classifications of categories comprise one or more of a time category, a geography category, a product category, a business organization category, or an accounting category.

11. The method of claim 10, wherein the time category comprises a year hierarchical level, a quarter hierarchical level, and a month hierarchical level.

12. The method of claim 10, wherein the geography category comprises a country hierarchical level, a state/province hierarchical level, and a city hierarchical level.

13. The method of claim 10, wherein the product category comprises a department hierarchical level, a product line hierarchical level, and a product hierarchical level.

14. The method of claim 10, wherein the business organization category comprises a department hierarchical level, a manager hierarchical level, and an employee hierarchical level.

15. The method of claim 10, wherein the accounting category comprises a revenue hierarchical level and a profit hierarchical level.

16. The method of claim 9, wherein each of the one or more hierarchical classifications of categories comprises two or more hierarchical levels, and one or more of the hierarchical levels comprises one or more attributes.

17. A computer program product for generating a hierarchical group based on a set of data, the computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by a computing device to:

classify two or more data items from a set of data with respect to a library of ontological concepts based at least in part on properties of the two or more data items, including detecting correlations between the properties of the two or more data items and one or more ontological concepts from the library of ontological concepts, wherein the properties of the two or more data items include data types defined for the two or more data items and ranges of data values in data fields of the two or more data items;

classify the two or more data items with respect to lexical correlations between the two or more data items, including determining correlations between one or more elements of headers of the two or more data items;

analyze the two or more data items based on one or more factors to determine whether the one or more factors contribute to defining a hierarchical relationship, wherein the analysis utilizes the one or more factors that comprise a set of heuristic rules and relative cardinality, wherein the set of heuristic rules discounts or disqualifies quantifiers or metrics associated with the two or more data items, and wherein the relative cardinality minimizes quantifiers or metrics through merging;

generate a hierarchical group in which the two or more data items are partitioned into one or more hierarchical partitions based at least in part on the classifying with respect to the library of ontological concepts, the classifying with respect to the lexical correlations, and the analysis of the two or more data items based on the one or more factors, wherein each of the one or more hierarchical partitions comprises the two or more data items; and

verify a sampling of data in the one or more hierarchical partitions, including measuring correlations between data in the two or more data items in a particular hierarchical partition from the one or more hierarchical partitions to determine whether the particular hierarchical partition has a first data item at a leaf level of the particular hierarchical partition in a one-to-many relationship with a second data item at a base level of the particular hierarchical partition.

18. The computer program product of claim 17,

wherein the library of ontological concepts comprises one or more hierarchical classifications of categories,

wherein classifying the two or more data items with respect to the library of ontological concepts comprises identifying one or more of the data items with one or more hierarchical levels in one of the hierarchical classifications of categories, and

wherein the one or more hierarchical classifications of categories comprise one or more of a time category, a geography category, a product category, a business organization category, or an accounting category.

19. A computer system for generating a hierarchical group based on a set of data, the computer system comprising:

one or more processors, one or more computer-readable memories, and one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to classify two or more data items from a set of data with respect to a library of ontological concepts based at least in part on properties of the two or more data items, including determining correlations between the properties of the two or more data items and one or more ontological concepts from the library of ontological concepts, wherein the properties of the two or more data items include data types defined for the two or more data items and ranges of data values in data fields of the two or more data items;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to classify the two or more data items with respect to lexical correlations between the two or more data items;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to analyze the two or more data items based on one or more factors to determine whether the one or more factors contribute to defining a hierarchical relationship, wherein the analysis utilizes the one or more factors that comprise a set of heuristic rules and relative cardinality, wherein the set of heuristic rules discounts or disqualifies quantifiers or metrics associated with the two or more data items, and wherein the relative cardinality minimizes quantifiers or metrics through merging;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to generate a hierarchical group in which the two or more data items are partitioned into one or more hierarchical partitions based at least in part on the classifying with respect to the library of ontological concepts, the classifying with respect to the lexical correlations, and the analysis of the two or more data items based on the one or more factors, wherein each of the one or more hierarchical partitions comprises the two or more data items; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to verify a sampling of data in the one or more hierarchical partitions, including measuring correlations between data in the two or more data items in a particular hierarchical partition from the one or more hierarchical partitions to determine whether the particular hierarchical partition has a first data item at a leaf level of the particular hierarchical partition in a one-to-many relationship with a second data item at a base level of the particular hierarchical partition.

20. The computer system of claim 19,

wherein the library of ontological concepts comprises one or more hierarchical classifications of categories,

wherein classifying the two or more data items with respect to the library of ontological concepts comprises identifying one or more of the data items with one or more hierarchical levels in one of the hierarchical classifications of categories, and

wherein the one or more hierarchical classifications of categories comprise one or more of a time category, a geography category, a product category, a business organization category, or an accounting category.