Data processing system

• US 20060004746A1
• Filed: 08/04/2005
• Published: 01/05/2006
• Est. Priority Date: 09/04/1998
• Status: Active Grant

• Current Assignee: Kalido Ltd. (Insightsoftware UK Trading Ltd.)
• Original Assignee: Kalido Ltd. (Insightsoftware UK Trading Ltd.)
• Inventors: Angus, Christopher, Ottmann, Douglas John Bruce
• Granted Patent: US 7,774,371 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A data processing system is provided for storing and managing multiple data entries. The data processing system employs a data structure which allows the storage and management of a vast number of interrelated data entries the interrelations of which change over time. The data structure reflects such changing interrelations over time and allows the querying and extracting of data entries on the basis of their interrelations as they were or are defined at any desired point in time.

Claims

1-30. -30. (canceled)

31. A data processing system, comprising:

processing means for generating a data model in accordance with a data structure, the data model being adaptable to represent a change in the data structure; and

storage means for storing the data in accordance with the generated data model.

32. The data processing system of claim 31, wherein the stored data comprises information representative of the time of change in the data structure.

33. The data processing system of claim 31, wherein the stored data comprises:

transaction data representative of one or more measures which are determined relative to one or more references;

reference data representative of said one or more references; and

metadata descriptive of the transaction data and the reference data.

34. The data processing system of claim 33, wherein the metadata defines hierarchical associations between classes of the reference data.

35. The data processing system of claim 33, wherein the stored data comprises a number of elements of reference data, each element of reference data comprising information which defines an association with one or more other elements of reference data.

36. The data processing system of claim 35, wherein each element of reference data further comprises information representative of a first period of validity of a defined association.

37. The data processing system of claim 36, wherein the information representative of the first period of validity comprises a start date of validity and an end date of validity.

38. The data processing system of claim 33, wherein the one or more measures each are associated with one or more units.

39. The data processing system of claim 38, wherein the associations between the one or more measures and the one or more units are associated with a second period of validity.

40. The data processing system of claim 39, wherein the second period of validity comprises a start date of validity and an end date of validity.

41. The data processing system of claim 33, wherein the stored data comprises a number of items of transaction data, each item of transaction data being associated with a date of transaction.

42. The data processing system of claim 33, wherein the metadata defines associations between classes of reference data and the one or more measures, the associations between the classes of reference data and the one or more measures being representative of classes of transaction data.

43. The data processing system claim 33, further comprising:

first interface means for receiving data of any structure from a data source for storage in the data processing system.

44. The data processing system claim 43, further comprising:

second interface means for outputting data from the storage means in a required format.

45. A data processing system, comprising:

processing means for generating a data model representative of data of a first structure, and for adapting the data model to represent also data of a second structure; and

storage means for storing data in accordance with the data model.

46. The data processing system of claim 45, wherein the stored data includes information representative of the time of adaptation of the data model.

47-50. -50. (canceled)

51. A method for managing data in a data structure, comprising the steps of:

storing a data model in said data structure, said data model identifying a plurality of relationships among classes of entities;

storing a plurality of datasets in said data structure, each of said datasets including an identification of at least one of said classes of entities and at least one measure; and

relating each of said datasets to a respective subset of relationships in said data model, at least first and second ones of said datasets being related to respective first and second different ones of said subsets.

52. A method according to claim 51, wherein at least one of the relationships in said first subset is inconsistent with one of the relationships in said second subset.

53. A method according to claim 51, wherein the relationships identified in at least one of said subsets are hierarchical.

54. A method according to claim 51, wherein each of said entity class relationships further has associated therewith a temporal period of validity, and wherein said step of relating each of said datasets to a respective subset or relationships in said data model comprises the step of associating with each of said datasets a respective temporal period of coverage.

55. A method according to claim 54, wherein the subset of relationships related to each given one of said datasets is defined to exclude each relationship in said data model whose temporal period of validity does not overlap with the temporal period of coverage of the given dataset.

56. A method according to claim 51, further comprising the step of returning data from said data structure in response to a query.

57. A method according to claim 51, further comprising the steps of:

determining, in response to a query requiring data values for a specified measure distributed across entities in a specified class of entities, that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities; and

satisfying said query from datasets in said data structure at a level of granularity of hierarchically higher class of entities which includes said specified class of entities.

58. A method according to claim 51, further comprising the steps of:

identifying, in response to a query, a superset of said datasets which together are more than sufficient to satisfy said query;

selecting, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, the datasets in said subset being sufficient to satisfy said query; and

satisfying said query from datasets in said subset.

59. A method for managing data in a data structure, comprising the steps of storing a data model in said data structure, said data model identifying a plurality of relationships among classes of entities each of said entity class relationships having associated therewith a respective temporal period of validity;

and

storing a plurality of datasets in said data structure, each of said datasets including an identification of at least one of said classes of entities and at least one measure, and further having associated therewith a respective temporal period of coverage.

60. A method according to claim 59, wherein the subset of the relationships in said data model whose temporal periods of validity include a first temporal position are hierarchical.

61. A method according to claim 59, wherein the subset of the relationships in said data model whose temporal periods of validity include a first temporal position differs from the subset of the relationships in said data model whose temporal periods of validity include a second temporal position different from said first temporal position.

62. A method according to claim 61, wherein the subset of the relationships in said data model whose temporal periods of validity include said first temporal position is consistent with the subset of the relationships in said data model whose temporal periods of validity include said second temporal position.

63. A method according to claim 59, further comprising the step of returning data from said data structure in response to a query.

64. A method for extracting data from a data structure, said data structure including a data model identifying a plurality of relationships among classes of entities, a first subset of said relationships having a hierarchical organisation, said data structure further including a plurality of datasets each including an identification of at least one entity and a data value for at least one measure, comprising the steps of:

receiving a query requiring data values for a specified measure distributed across entities in a specified class of entities;

determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities;

determining a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

satisfying said query from datasets in said data structure at the level of granularity of said hierarchically class of entities.

65. A method according to claim 64, wherein said step of receiving a query requiring data values for a specified measure comprises the step of receiving a query requesting information derivable from said data values for said specified measure.

66. A method according to claim 64, wherein said data model includes a plurality of subsets of said relationships among classes of entities including said first set of relationships, each of said subset having a respective hierarchical organisation, each of said datasets being related to a respective one of said subsets,

and wherein said step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities comprises the step of determining that a first one of the datasets in said data structure which are required to satisfy said query is related to said first subset of relationships whereas a second one of the datasets in said data structure which are required to satisfy said query is related to a second one of said subsets of relationships different from said first subset of relationships, and first subset of relationships including a relationship involving said specified class of entities but said second subset of relationships not including any relationship involving said specified class of entities.

67. A method according to claim 66, wherein said first and second subsets of relationships share a sub-hierarchy which reaches down to the level of granularity of said hierarchically higher class of entities.

68. A method according to claim 66, wherein each of the relationships in said data model further has associated therewith a temporal period of validity, and each of said datasets has associated therewith a respective temporal period of coverage.

69. A method according to claim 68, wherein each of said datasets is related to a respective one of said relationship subsets at least in part by the temporal coverage of the dataset relative to the temporal period of validity of the entity class relationships in the respective subset.

70. A method according to claim 68, wherein said step of determining that a first one of the datasets in said data structure which are required to satisfy said query is related to said first subset of relationships whereas a second one of the datasets in said data structure which are required to satisfy said query is related to a second one of said subsets of relationships different from said first subset of relationships, said first subset of relationships including a relationship involving said specified class of entities but said second subset of relationships not including any relationship involving said specified class of entities, includes the step of determining that a relationship exists in said data structure between said specified class of entities and a higher class of entities, said relationship having a period of validity which overlaps with the temporal period of coverage of said first dataset but excludes the temporal period of coverage of said second dataset.

71. A method according to claim 64, wherein said step of satisfying said query from datasets in said data structure at the level of granularity of said hierarchically higher class of entities, comprises the step of selecting datasets from said data structure which identify said hierarchically higher class of entities.

72. A method according to claim 64, wherein said step of satisfying said query from datasets in said data structure at the level of granularity of said hierarchically higher class of entities, comprises the steps of:

selecting the group of datasets from said data structure which identify classes of entities which are hierarchically below said hierarchically higher class of entities; and

aggregating data values from the datasets in said group up to said hierarchically higher class of entities.

73. A method according to claim 64, wherein said step of satisfying said query comprises the steps of:

identifying, in response to said query, a superset of said datasets which together are more than sufficient to satisfy said query at the granularity of said hierarchically higher class of entities;

selecting, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, the datasets in said proper subset being both necessary and sufficient to satisfy said query; and

satisfying said query from datasets in said proper subset.

74. A method for extracting data from a data structure, said data structure including a plurality of datasets each including an identification of at least one entity in a hierarchically defined class of entities, each of said datasets further identifying a data value for at least one measure, comprising the steps of:

receiving a query requiring data values for a specified measure distributed across entities in a specified class of entities;

determining that not all datasets in said data structures which are required to satisfy said query are broken down according to entities in said specified class of entities;

determining a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

satisfying said query from datasets in said data structures at the level of granularity of said hierarchically higher class of entities.

75. A method according to claim 74, wherein said step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities, comprises the step of determining that none of the datasets in said data structures which are required to satisfy said query are broken down according to entities in said specified class of entities.

76. A method for extracting data from a data structure, said data structure including a plurality of datasets, comprising the steps of:

identifying, in response to a query, a superset of said datasets which together are more than sufficient to satisfy said query;

selecting, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, the datasets in said subset being sufficient to satisfy said query; and

satisfying said query from datasets in said subset.

77. A method according to claim 76, wherein said query requires data values from a query set of at least one measure, and wherein said step of selecting a proper subset of the datasets in said superset, the datasets in said subset of being sufficient to satisfy said query, comprises the step of selecting a proper subset of the datasets in said superset, the datasets in which subset are sufficient to allow all of the required measures in said query set to be sourced.

78. A method according to claim 76, wherein said query cannot be satisified from fewer than all of the datasets in said subset.

79. A method according to claim 76, wherein datasets in said data structure each include an identification of at least one entity in a hierarchically defined class of entities, and wherein said query requires data values for a specified measure distributed across entities in a specified class of entities, and wherein said step of identifying comprises the steps of:

determining a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

identifying a superset of said datasets which together are more than sufficient to satisfy said query at the granularity of said hierarchically higher class of entities.

80. A method according to claim 79, further comprising the step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities.

81. A method according to claim 76, wherein said query requests data broken down with a particular level of granularity, and wherein said step of satisfying said query comprises the step of satisfying said query with data broken down with a level of granularity that is coarser than said particular level of granularity.

82. A method according to claim 76, wherein said step of selecting, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, comprises the step of selecting, from all proper subsets of datasets in said superset, the datasets in which subsets are both necessary and sufficient to satisfy said query, a subset which has the lowest aggregate cost to satisfy said query.