Operational-related data computation engine

• US 8,768,976 B2
• Filed: 05/15/2009
• Issued: 07/01/2014
• Est. Priority Date: 05/15/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Apptio Inc (International Business Machines Corporation)
• Inventors: McLachlan, Paul, Elliott, Tavis
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Hoffler, Raheem
• Application Number:
  US12/467,120
• Publication Number:
  US 20100293163A1
• Time in Patent Office: 1,873 Days
• Field of Search: None
• US Class Current: 707/805
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A computation system includes a receiver, a schema generator, and a table generator. The receiver receives operational related data relating to an IT environment of an organization from remote systems. The schema generator organizes the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data to generate a schema. The operational related data includes data tables corresponding to IT assets of the organization. The table generator processes an input table based on the schema to generate an output table. The input table includes a starting table having one or more columns. The output table identifies a subset of interrelated data included in the starting table.

Claims

1. A computation system to process data, comprising:

a network interface to receive operational related data relating to an Information Technology (IT) environment of an organization over a network;

a memory device to store at least instructions and a data processing engine that is coupled to the network interface; and

a processor device that executes the instructions and the data processing engine to perform actions, including;

employing a schema generator to generate a schema by organizing at least part of the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

2. The computation system of claim 1 wherein the operational related data comprises a plurality of data tables corresponding to IT assets of the organization.

3. The computation system of claim 2 wherein the receiver is configured to receive the plurality of data tables with at least one communication port from a plurality of remote systems, each remote system including at least one data table.

4. The computation system of claim 2 further comprising:

a storage device coupled to the schema generator, the storage device configured to store user-defined specifications on the relationship between at least two data tables, the schema generator configured to determine the relationship based on the user-defined specifications.

5. The computation system of claim 2 wherein the schema generator is configured to iteratively process each value of every column of every data table.

6. The computation system of claim 5 wherein the schema generator is configured to record a new type of column when a processed value changes the column type, to record a new cardinality when the processed value changes the column cardinality, to associate the column and table of the processed value with a dictionary entry when the processed value is in the dictionary, to add the processed value to the dictionary when the processed value is not in the dictionary.

7. The computation system of claim 2 wherein the schema generator is configured to compare two or more schemas to detect when the relationship between the data tables has changed, to detect the introduction of new values in the data tables.

8. The computation system of claim 2 further comprising:

a table generator coupled to the schema generator, the table generator configured to process an input table based on the schema to generate an output table, the input table comprising a starting table having one or more columns, the output table identifying a subset of interrelated data included in the starting table.

9. The computation system of claim 8 wherein the table generator is further configured to process the input table and the output table based on the schema to map relationships between the input table and the output table.

10. The computation system of claim 9 wherein the table generator is further configured to estimate the relationship between unrelated data included in the input table and the output table.

11. The computation system of claim 2, further comprising when there are a plurality of unique values for a given cell in the destination table or the result table, determining a count of a number of the plurality of unique values and employing the count in the given cell.

12. A computer-implemented method to process data with a computer system machine, comprising:

receiving operational related data related to an Information Technology (IT) environment of an organization at a data reception system;

employing the computer system machine to generate a schema by organizing at least part of the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

13. The computer-implemented method of claim 12 wherein the operational related data comprises a plurality of data tables corresponding to IT assets of the organization.

14. The computer-implemented method of claim 13 further comprising:

receiving the plurality of data tables with at least one communication port from a plurality of remote systems, each remote system including at least one data table.

15. The computer-implemented method of claim 13 further comprising:

storing user-defined specifications on the relationship between at least two data tables; and

determining the relationship based on the user-defined specifications.

16. The computer-implemented method of claim 13 further comprising:

iteratively processing each value of every column of every data table.

17. The computer-implemented method of claim 16 further comprising:

recording a new type of column when a processed value changes the column type;

recording a new cardinality when the processed value changes the column cardinality;

associating the column and table of the processed value with a dictionary entry when the processed value is in the dictionary; and

adding the processed value to the dictionary when the processed value is not in the dictionary.

18. The computer-implemented method of claim 13 further comprising:

comparing two or more schemas to detect when the relationship between the data tables has changed, to detect the introduction of new values in the data tables.

19. The computer-implemented method of claim 13 further comprising:

processing an input table based on the schema to generate an output table, the input table comprising a starting table having one or more columns, the output table identifying a subset of interrelated data included in the starting table.

20. The computer-implemented method of claim 19 further comprising:

processing the input table and the output table based on the schema to map relationships between the input table and the output table.

21. The computer-implemented method of claim 20 further comprising:

estimating the relationship between unrelated data included in the input table and the output table.

25. The computer-readable non-volatile storage medium of claim 12 wherein the method further comprises:

storing user-defined specifications on the relationship between at least two data tables; and

determining the relationship based on the user-defined specifications.

22. A computer-readable non-volatile storage medium, having instructions stored therein, which when executed by a processor device, cause a computer system machine to perform a method to process data, comprising:

receiving operational related data related to an Information Technology (IT) environment of an organization at a data reception system;

employing the computer system machine to generate a schema by organizing the data into at least one class of IT metric-oriented data based on at least intrinsic characteristic of the operational related data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

23. The computer-readable non-volatile storage medium of claim 22 wherein the operational related data comprises a plurality of data tables corresponding to IT assets of the organization.

24. The computer-readable non-volatile storage medium of claim 23 wherein the method further comprises:

receiving the plurality of data tables with at least one communication port from a plurality of remote systems, each remote system including at least one data table.

26. The computer-readable non-volatile storage medium of claim 23 wherein the method further comprises:

iteratively processing each value of every column of every data table.

27. The computer-readable non-volatile storage medium of claim 23 wherein the method further comprises:

recording a new type of column when a processed value changes the column type;

recording a new cardinality when the processed value changes the column cardinality;

associating the column and table of the processed value with a dictionary entry when the processed value is in the dictionary; and

adding the processed value to the dictionary when the processed value is not in the dictionary.

28. The computer-readable non-volatile storage medium of claim 23 wherein the method further comprises:

comparing two or more schemas to detect when the relationship between the data tables has changed, to detect the introduction of new values in the data tables.

29. The computer-readable non-volatile storage medium of claim 23 wherein the method further comprises:

processing an input table based on the schema to generate an output table, the input table comprising a starting table having one or more columns, the output table identifying a subset of interrelated data included in the starting table.

30. The computer-readable non-volatile storage medium of claim 29 wherein the method further comprises:

processing the input table and the output table based on the schema to map relationships between the input table and the output table.

31. The computer-readable non-volatile storage medium of claim 30 wherein the method further comprises:

estimating the relationship between unrelated data included in the input table and the output table.

32. A computation system to process data, comprising:

a network interface to receive operational related data relating to an Information Technology (IT) environment of an organization over a network;

a memory device to store instructions, data, and a table generator; and

a processor device that executes the instructions and the table generator, that is coupled to the network interface, and performs actions, including;

employing the table generator to process an input table based on a schema to generate an output table, wherein the schema is generated based on organizing at least a portion of the operational related data into at least one class of IT metric-oriented data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

33. The computation system of claim 32 wherein the input table includes a mapping table, or a starting table comprising at least one column.

34. The computation system of claim 33 wherein the table generator is configured to apply the schema to the starting table to extrapolate the output table, the output table comprising a linked table including at least two different classes of information based on the schema.

35. The computation system of claim 32 wherein the table generator is configured to process input data organized in a plurality of input tables to identify a subset of interrelated data included in the input data corresponding to at least two classes of information based on the schema.

36. The computation system of claim 32 further comprising:

a schema generator coupled to the receiver, the schema generator configured to organize the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data, the operational related data comprising a plurality of data tables corresponding to IT assets of the organization.

37. The computation system of claim 32 wherein the table generator is further configured to identify a plurality of intermediate tables consisting of at least two input tables that include logically-connected input data, the intermediate tables including rows and columns, to extract at least a portion of the input data included in each intermediate table, and to construct at least one output table based on the extracted input data while maintaining conceptual integrity among at least a subset of the extracted input data by collapsing at least two individual values included in the extracted input data to produce at least one corresponding aggregated value summary.

38. The computation system of claim 37 wherein the at least one aggregate value summary is the total of corresponding individual values, the average of corresponding individual values, or a measure of a statistical distribution of corresponding individual values.

39. The computation system of claim 36 further comprising:

a storage device coupled to the schema generator, the storage device configured to store user-defined specifications on the relationship between at least two data tables, the schema generator configured to determine the relationship based on the user-defined specifications.

40. The computation system of claim 39 wherein the schema generator is configured to iteratively process each value of every column of every data table.

41. The computation system of claim 40 wherein the schema generator is configured to record a new type of column when a processed value changes the column type, to record a new cardinality when the processed value changes the column cardinality, to associate the column and table of the processed value with a dictionary entry when the processed value is in the dictionary, to add the processed value to the dictionary when the processed value is not in the dictionary.

42. A computer-implemented method for processing data through the execution of instructions by a computer system machine that enables actions, comprising:

receiving operational related data relating to an Information Technology (IT) environment of an organization at a data reception system with the computer system machine;

processing an input table based on a schema to generate an output table at a table generator with the computer system machine, wherein the schema is generated based on organizing at least a portion of the operational related data into at least one class of IT metric-oriented data at a schema generator;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

43. The computer-implemented method of claim 42 wherein the input table includes a mapping table, or a starting table comprising at least one column.

44. The computer-implemented method of claim 43 further comprising:

applying the schema to the starting table to extrapolate the output table, the output table comprising a linked table including at least two different classes of information based on the schema.

45. The computer-implemented method of claim 42 further comprising:

processing input data organized in a plurality of input tables to identify a subset of interrelated data included in the input data corresponding to at least two classes of information based on the schema.

46. The computer-implemented method of claim 42 further comprising:

organizing the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data, the operational related data comprising a plurality of data tables corresponding to IT assets of the organization.

47. The computer-implemented method of claim 42 further comprising:

identifying a plurality of intermediate tables consisting of at least two input tables that include logically-connected input data, the intermediate tables including rows and columns with the table generator;

extracting at least a portion of the input data included in each intermediate table with the table generator; and

constructing at least one output table based on the extracted input data while maintaining conceptual integrity among at least a subset of the extracted input data by collapsing at least two individual values included in the extracted input data to produce at least one corresponding aggregated value summary with the table generator.

48. The computer-implemented method of claim 47 wherein the at least one aggregate value summary is the total of corresponding individual values, the average of corresponding individual values, or a measure of a statistical distribution of corresponding individual values.

49. The computer-implemented method of claim 46 further comprising:

storing user-defined specifications on the relationship between at least two data tables in a storage device; and

determining the relationship based on the user-defined specifications with the schema generator.

50. The computer-implemented method of claim 49 further comprising:

iteratively processing each value of every column of every data table with the schema generator.

51. The computer-implemented method of claim 50 further comprising:

recording a new type of column when a processed value changes the column type with the schema generator;

recording a new cardinality when the processed value changes the column cardinality with the schema generator;

associating the column and table of the processed value with a dictionary entry when the processed value is in the dictionary with the schema generator; and

adding the processed value to the dictionary when the processed value is not in the dictionary with the schema generator.

52. A computer-readable non-volatile storage medium, having instructions to process operational data stored therein, which when executed by a processor device, cause a computer system to perform a method comprising:

receiving operational related data relating to an Information Technology (IT) environment of an organization;

processing an input table based on a schema to generate an output table, wherein the schema is generated based organizing at least a portion of the operational related data into at least one class of IT metric-oriented data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.

53. The computer-readable non-volatile storage medium of claim 52 wherein the input table includes a mapping table, or a starting table comprising at least one column.

54. The computer-readable non-volatile storage medium of claim 53 further comprising:

applying the schema to the starting table to extrapolate the output table, the output table comprising a linked table including at least two different classes of information based on the schema.

55. The computer-readable non-volatile storage medium of claim 52 further comprising:

processing input data organized in a plurality of input tables to identify a subset of interrelated data included in the input data corresponding to at least two classes of information based on the schema.

56. The computer-readable non-volatile storage medium of claim 52 further comprising:

organizing the operational related data into at least one class of IT metric-oriented data based on at least one intrinsic characteristic of the operational related data, operational related data comprising a plurality of data tables corresponding to IT assets of the organization.

57. The computer-readable non-volatile storage medium of claim 52 further comprising:

identifying a plurality of intermediate tables consisting of at least two input tables that include logically-connected input data, the intermediate tables including rows and columns;

extracting at least a portion of the input data included in each intermediate table; and

constructing at least one output table based on the extracted input data while maintaining conceptual integrity among at least a subset of the extracted input data by collapsing at least two individual values included in the extracted input data to produce at least one corresponding aggregated value summary.

58. The computer-readable non-volatile storage medium implemented method of claim 57 wherein the at least one aggregate value summary is the total of corresponding individual values, the average of corresponding individual values, or a measure of a statistical distribution of corresponding individual values.

59. The computer-readable non-volatile storage medium of claim 56 further comprising:

storing user-defined specifications on the relationship between at least two data tables; and

determining the relationship based on the user-defined specifications.

60. The computer-readable non-volatile storage medium of claim 53 further comprising:

recording a new type of column when a processed value changes the column type;

recording a new cardinality when the processed value changes the column cardinality;

associating the column and table of the processed value with a dictionary entry when the processed value is in the dictionary; and

adding the processed value to the dictionary when the processed value is not in the dictionary.

61. A computer system to process operational data, comprising:

a network interface to receive operational related data relating to an Information Technology (IT) environment of an organization over a network;

a memory device to store instructions, data, and a data processing engine and a processor device that executes the instructions and the data processing engine to perform actions, including;

organizing at least a portion of the operational related data into at least one class of IT metric-oriented data;

generating at least one weak mapping table for determining a relationship between a plurality of source tables that lack an explicit mapping between respective source tables based on customizable linking rules, a target probability, and a target cardinality;

employing the organized data and the at least one weak mapping table to generate a destination table and a result table based on each row and column of a starting table and the columns of each other table that is related to the starting table, wherein any determined path between related tables is based on a dictionary concept that lists at least one or more columns;

comparing at least a portion of the organized data of the schema in the destination table to at least a portion of the organized data of at least another schema in another destination table that is generated with other operational related data relating to the IT environment; and

comparing at least a portion of the organized data of the schema in the result table to at least a portion of the organized data of at least the other schema in another result table that is generated with other operational related data relating to the IT environment, wherein at least one row and at least one cell of the result table is re-ordered and filtered in accordance with a key of the starting table and further re-ordered and collapsed in accordance with a cardinality, and wherein a presentation of the result table comparison enables identifying a change in the schema over time.