SYSTEM AND METHOD OF INTEGRATING TIME-AWARE DATA FROM MULTIPLE SOURCES

• US 20150169667A1
• Filed: 12/16/2013
• Published: 06/18/2015
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Tan, Wang-Chiew, Roth, Mary A
• Granted Patent: US 9,418,124 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/83 Querying

Abstract

A time-aware union operator is disclosed for consistent integration of time-aware data, wherein the time-aware union produces a time-aware consistent integrated view of underlying sources according to specified key constraints and policies. The implementation of time-aware union is idempotent, commutative, and associative, thus making it suitable for data integration, and it produces the same integrated outcome, modulo representation of time, regardless of the order in which sources are integrated.

Claims

1. A computer-implemented method for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the computer-implemented method comprising:

defining a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;

comparing all data records in a cumulative, pair-wise fashion;

identifying time-based conflicts between pairs of records and identifying time-conflicted pairs;

determining which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;

adjusting the time interval of every said determined record in accordance with said defined policy; and

outputting a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

2. The computer-implemented method of claim 1, wherein said candidate policies depend on at least one of the following:

the source of the data records, the n-dimensional time intervals, and values in the data records.

3. The computer-implemented method of claim 1, wherein said method is implemented as part of a parallel database system.

4. The computer-implemented method of claim 3, wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

5. The computer-implemented method of claim 1, wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

6. The computer-implemented method of claim 5, wherein said standard XML manipulation language is XPath.

7. A system for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the system comprising:

one or more processors; and

a memory comprising instructions which, when executed by the one or more processors, cause the one or more processors to;

define a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;

compare all data records in a cumulative, pair-wise fashion;

identify time-based conflicts between pairs of records and identify time-conflicted pairs;

determine which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;

adjust the time interval of every said determined record to be adjusted in accordance with said defined policy; and

output a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

8. The system of claim 7, wherein said candidate policies depend on at least one of the following:

the source of the data records, the n-dimensional time intervals, and values in the data records.

9. The system of claim 7, wherein said method is implemented as part of a parallel database system.

10. The system of claim 9, wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

11. The system of claim 7, wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

12. The system of claim 11, wherein said standard XML manipulation language is XPath.

13. A non-transitory computer-readable medium for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the computer-readable medium comprising computer readable program code which, when executed by a computer, cause the computer to computer-implemented method, the medium comprising:

computer readable program code receiving inputs defining a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;

computer readable program code comparing all data records in a cumulative, pair-wise fashion;

computer readable program code identifying time-based conflicts between pairs of records and identifying time-conflicted pairs;

computer readable program code determining which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;

computer readable program code adjusting the time interval of every said determined record in accordance with said defined policy; and

computer readable program code outputting a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

14. The non-transitory computer-readable medium of claim 13, wherein said candidate policies depend on at least one of the following:

the source of the data records, the n-dimensional time intervals, and values in the data records.

15. The non-transitory computer-readable medium of claim 13, wherein said method is implemented as part of a parallel database system.

16. The non-transitory computer-readable medium of claim 15, wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

17. The non-transitory computer-readable medium of claim 13, wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

18. The non-transitory computer-readable medium of claim 17, wherein said standard XML manipulation language is XPath.

19. A computer based method comprising:

receiving an input schema S;

receiving two instances, T₁ and T₂, each of which conforms to said input schema, S;

computing a merged instance of T₁ and T₂, T₁T₂, as an instance that also conforms to S such that every path in T₁ or T₂ can be found in T₁T₂, every path in T₁T₂ can be found in T₁ or T₂, and a temporal context associated with an element in any path in T₁T₂ is a result of adjusting temporal contexts of corresponding elements in T₁ and/or T₂ under the same path according to a given adjustment policy, said given adjustment policy designed to resolve time conflicts between data records having time conflicts, where said merged instance T₁T₂ provides an integrated view of instances T₁ and T₂ by fusing identical elements in instances T₁ and/or T₂ together and providing an interpretation of their corresponding temporal contexts according to said given adjust policy; and

outputting said merged instance T₁T₂.

20. The computer-based method of claim 19, wherein said computer-based method is implemented as part of a parallel database system.

21. The computer-based method of claim 19, wherein said outputted merged instance T₁T₂ is one among many intermediate results and said computer-based method further comprises the step of merging and outputting all said intermediate results.

22. The computer-based method of claim 19, wherein said outputted merged instance T₁T₂ comprise a concise and complete temporal history of all data records such that a history of an entity associated with the data records is retrievable via a standard XML manipulation language.

23. The computer-based method of claim 22, wherein said standard XML manipulation language is XPath.