Data processing system

• US 7,774,371 B2
• Filed: 08/04/2005
• Issued: 08/10/2010
• Est. Priority Date: 09/04/1998
• Status: Expired due to Fees

• Current Assignee: Kalido Ltd. (Insightsoftware UK Trading Ltd.)
• Original Assignee: Kalido Ltd. (Insightsoftware UK Trading Ltd.)
• Inventors: Angus, Christopher, Ottmann, Douglas John Bruce
• Primary Examiner(s): Fleurantin; Jean B
• Application Number:
  US11/196,918
• Publication Number:
  US 20060004746A1
• Time in Patent Office: 1,832 Days
• Field of Search: 707 1- 10, 707100-104, 707200-206, 711/1, 711/100
• US Class Current: 707/795
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A data processing system is provided for storing and managing multiple data entries. The data processing system employs a data structure which allows the storage and management of a vast number of interrelated data entries the interrelations of which change over time. The data structure reflects such changing interrelations over time and allows the querying and extracting of data entries on the basis of their interrelations as they were or are defined at any desired point in time.

Claims

1. A data processing system, comprising:

a processor for generating a data model in accordance with a data structure, the data model being adaptable to represent a change in the data structure; and

computer memory for storing the data in accordance with the generated data model, said stored data including

transaction data representative of one or more measures associated with one or more units, said measure being determined relative to one or more references;

reference data representative of said one or more references, each element of reference data comprising information which defines an association with one or more other elements of reference data said elements of reference data further including information representative of the time of change in the data structure, the first period of validity, represented by a start date of validity and an end date of validity of a defined association

metadata descriptive of the transaction data and the reference data.

2. A data processing system, comprising:

a processor for generating a data model in accordance with a data structure, the data model being adaptable to represent a change in the data structure; and

computer memory for storing the data in accordance with the generated data model, said stored data includes information representative of the time of change in the data structure and said stored data comprises;

transaction data representative of one or more measures associated with one or more units, said measures being determined relative to one or more references;

reference data representative of said one or more references; and

metadata descriptive of the transaction data and the reference data

wherein the associations between the one or more measures and the one or more units are associated with a second period of validity where said second period of validity comprises a start date of validity and an end date of validity.

3. A method for managing data in a data structure, comprising the steps of:

storing in computer memory a data model in said data structure, said data model identifying a plurality of relationships among classes of entities, each of said entity class relationships having associated therewith a temporal period of validity;

storing in computer memory a plurality of datasets in said data structure, each of said datasets including an identification of at least one of said classes of entities and at least one measure; and

using a processor to relate each of said datasets to a respective subset of relationships in said data model, at least first and second ones of said datasets being related to respective first and second different ones of said subsets and to associate with each of said datasets a respective temporal period of coverage;

wherein the subset of relationships related to each given one of said datasets is defined to exclude each relationship in said data model whose temporal period of validity does not overlap with the temporal period of coverage of the given dataset.

4. A method for managing data in a data structure, comprising the steps of:

storing in computer memory a data model in said data structure, said data model identifying a plurality of relationships among classes of entities, each of said entity class relationships having associated therewith a temporal period of validity;

storing in computer memory a plurality of datasets in said data structure, each of said datasets including an identification of at least one of said classes of entities and at least one measure; and

using a processor to relate each of said datasets to a respective subset of relationships in said data model, at least first and second ones of said datasets being related to respective first and second different ones of said subsets

determining, in response to a query requiring data values for a specified measure distributed across entities in a specified class of entities, that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities; and

satisfying said query from datasets in said data structure at a level of granularity of hierarchically higher class of entities which includes said specified class of entities.

5. A method for managing data in a data structure, comprising the steps of:

storing in computer memory a data model in said data structure, said data model identifying a plurality of relationships among classes of entities, each of said entity class relationships having associated therewith a temporal period of validity;

storing in computer memory a plurality of datasets in said data structure, each of said datasets including an identification of at least one of said classes of entities and at least one measure; and

using a processor to relate each of said datasets to a respective subset of relationships in said data model, at least first and second ones of said datasets being related to respective first and second different ones of said subsets;

identifying, in response to a query, a superset of said datasets which together are more than sufficient to satisfy said query;

selecting, in accordance with a predetermined optimization function, a proper subset of the datasets in said superset, the datasets in said subset being sufficient to satisfy said query; and

satisfying said query from datasets in said subset.

6. A method for managing data in a data structure stored in computer memory, comprising the steps of storing a data model in said data structure in the computer memory, said data model identifying a plurality of relationships among classes of entities each of said entity class relationships having associated therewith a respective temporal period of validity;

and

storing a plurality of datasets in said data structure in the computer memory, each of said datasets including an identification of at least one of said classes of entities and at least one measure, and further having associated therewith a respective temporal period of coverage

wherein the subset of the relationships in said data model whose temporal periods of validity include a first temporal position differs from the subset of the relationships in said data model whose temporal periods of validity include a second temporal position different from said first temporal position.

7. A method according to claim 6, wherein the subset of the relationships in said data model whose temporal periods of validity include said first temporal position is consistent with the subset of the relationships in said data model whose temporal periods of validity include said second temporal position.

8. A method for extracting data from a data structure stored in a computer memory, said data structure including a data model identifying a plurality of relationships among classes of entities, a first subset of said relationships having a hierarchical organisation, said data structure further including a plurality of datasets each including an identification of at least one entity and a data value for at least one measure, comprising the steps of:

receiving a query requiring data values for a specified measure distributed across entities in a specified class of entities;

determining with a processor that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities;

determining with a processor a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

satisfying said query from datasets in said data structure at the level of granularity of said hierarchically class of entities.

9. A method according to claim 8, wherein said step of receiving a query requiring data values for a specified measure comprises the step of receiving a query requesting information derivable from said data values for said specified measure.

10. A method according to claim 8, wherein said data model includes a plurality of subsets of said relationships among classes of entities including said first set of relationships, each of said subset having a respective hierarchical organisation, each of said datasets being related to a respective one of said subsets,

and wherein said step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities comprises the step of determining that a first one of the datasets in said data structure which are required to satisfy said query is related to said first subset of 7 a second one of the datasets in said data structure which are required to satisfy said query is related to a second one of said subsets of relationships different from said first subset of relationships, and first subset of relationships including a relationship involving said specified class of entities but said second subset of relationships not including any relationship involving said specified class of entities.

11. A method according to claim 10, wherein said first and second subsets of relationships share a sub-hierarchy which reaches down to the level of granularity of said hierarchically higher class of entities.

12. A method according to claim 10, wherein each of the relationships in said data model further has associated therewith a temporal period of validity, and each of said datasets has associated therewith a respective temporal period of coverage.

13. A method according to claim 12, wherein each of said datasets is related to a respective one of said relationship subsets at least in part by the temporal coverage of the dataset relative to the temporal period of validity of the entity class relationships in the respective subset.

14. A method according to claim 12, wherein said step of determining that a first one of the datasets in said data structure which are required to satisfy said query is related to said first subset of relationships whereas a second one of the datasets in said data structure which are required to satisfy said query is related to a second one of said subsets of relationships different from said first subset of relationships, said first subset of relationships including a relationship involving said specified class of entities but said second subset of relationships not including any relationship involving said specified class of entities, includes the step of determining that a relationship exists in said data structure between said specified class of entities and a higher class of entities, said relationship having a period of validity which overlaps with the temporal period of coverage of said first dataset but excludes the temporal period of coverage of said second dataset.

15. A method according to claim 8, wherein said step of satisfying said query from datasets in said data structure at the level of granularity of said hierarchically higher class of entities, comprises the step of selecting datasets from said data structure which identify said hierarchically higher class of entities.

16. A method according to claim 8, wherein said step of satisfying said query from datasets in said data structure at the level of granularity of said hierarchically higher class of entities, comprises the steps of:

selecting the group of datasets from said data structure which identify classes of entities which are hierarchically below said hierarchically higher class of entities; and

aggregating data values from the datasets in said group up to said hierarchically higher class of entities.

17. A method according to claim 8, wherein said step of satisfying said query comprises the steps of:

identifying, in response to said query, a superset of said datasets which together are more than sufficient to satisfy said query at the granularity of said hierarchically higher class of entities;

selecting, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, the datasets in said proper subset being both necessary and sufficient to satisfy said query; and

satisfying said query from datasets in said proper subset.

18. A method for extracting data from a data structure stored in a computer memory, said data structure including a plurality of datasets each including an identification of at least one entity in a hierarchically defined class of entities, each of said datasets further identifying a data value for at least one measure, comprising the steps of:

receiving a query requiring data values for a specified measure distributed across entities in a specified class of entities;

determining with a processor that not all datasets in said data structures which are required to satisfy said query are broken down according to entities in said specified class of entities;

determining with a processor a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

satisfying said query from datasets in said data structures at the level of granularity of said hierarchically higher class of entities.

19. A method according to claim 18, wherein said step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities, comprises the step of determining that none of the datasets in said data structures which are required to satisfy said query are broken down according to entities in said specified class of entities.

20. A method for extracting data from a data structure stored in computer memory, said data structure including a plurality of datasets, comprising the steps of:

identifying with a processor, in response to a query that requires data values of at least one measure, a superset of said datasets which together are more than sufficient to satisfy said query;

selecting with a processor, in accordance with a predetermined optimisation function, a proper subset of the datasets in said superset, the datasets in said subset being sufficient to satisfy said query; and

satisfying said query from datasets in said subset wherein said query requires data values from a query set of at least one measure, and wherein said step of selecting a proper subset of the datasets in said superset, the datasets in said subset of being sufficient to satisfy said query, comprises the step of selecting a proper subset of the datasets in said superset, the datasets in which subset are sufficient to allow all of the required measures in said query set to be sourced.

21. A method according to claim 20, wherein said query cannot be satisfied from fewer than all of the datasets in said subset.

22. A method for extracting data from a data structure stored in computer memory, said data structure including a plurality of datasets, comprising the steps of:

identifying with a processor, in response to a query that requires data values of at least one measure, a superset of said datasets which together are more than sufficient to satisfy said query, wherein said datasets in said data structure each including an identification of at least one entity in a hierarchically defined class of entities, said step of identifying further comprising the steps of;

determining a hierarchically higher class of entities which includes said specified class of entities, said query being satisfiable from datasets in said data structure at the level of granularity of said hierarchically higher class of entities; and

identifying a superset of said datasets which together are more than sufficient to satisfy said query at the granularity of said hierarchically higher class of entities;

satisfying said query from datasets in said subset, said query requiring data values for a specified measure distributed across entities in a specified class of entities.

23. A method according to claim 22, further comprising the step of determining that not all datasets in said data structure which are required to satisfy said query are broken down according to entities in said specified class of entities.

24. A method for extracting data from a data structure stored in computer memory, said data structure including a plurality of datasets, comprising the steps of:

identifying with a processor, in response to a query that requires data values of at least one measure, a superset of said datasets which together are more than sufficient to satisfy said query;

selecting with a processor from all proper subsets of datasets in said superset in accordance with a predetermined optimization function, a proper subset of the datasets in said superset, the datasets in said selected subset being necessary and sufficient to satisfy said query, said proper subset of datasets being selected based on the datasets in the selected subset being sufficient to allow all of the required measures in said query set to be sourced, said selected subset having the lowest aggregate cost to satisfy said query;

satisfying said query from datasets in said subset.