Managing data integrity using a filter condition

• US 20040034699A1
• Filed: 08/16/2002
• Published: 02/19/2004
• Est. Priority Date: 08/16/2002
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Schmitt, Michael, Lutz, Steffen, Gotz, Martina
• Granted Patent: US 7,464,097 B2
• Time in Patent Office: Days
• US Class Current: 709/223
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 41/024 using relational databases for representation of network management data, e.g. managing via structured query language [SQL]
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Techniques are provided to manage the integrity of data stored in two or more data management systems by detecting inconsistencies between the data management systems. The techniques identify missing records in one or more data management systems by comparing the records in the data management systems. A filter condition is used to identify the records to be compared. For records that exist in two or more data management systems, the techniques identify records that are not identical in the data management systems. A user checkpoint is provided between the identification of missing records and the identification of records that are not identical. The detected inconsistencies also may be corrected.

Claims

1. A computer-implemented method for managing data integrity in a first data management system and a second data management system, each system having stored therein a plurality of data collections, each data collection being associated with a set of attributes and having at least one attribute that uniquely identifies a single data collection from the other data collections, the method comprising:

detecting the presence, in the first and second data management systems, of data collections that meet a filter condition, the filter condition comprising at least one value of an attribute, the attribute occurring in one or more data collections in the first data management system and the attribute occurring in one or more data collections in the second data management system; and

managing the integrity of the data collections identified in the presence detection such that each data collection is included both in the first data management system and in the second data management system and each data collection includes a set of values that is the same for the set of attributes associated with the data collection in the first data management system and for the set of attributes associated with the data collection in the second data management system.

2. The method of claim 1 wherein managing the integrity of the data collections identified in the presence detection comprises when a data collection that meets the filter condition occurs only in the first data management system, sending the data collection from the first data management system to the second data management system.

3. The method of claim 1 wherein managing the integrity of the data collections identified in the presence detection comprises when a data collection that meets the filter condition occurs in both the first data management system and in the second data management system, determining whether a first set of attribute values associated with the data collection in the first data management system is equal to a second set of attribute values associated with the data collection in the second data management system.

4. The method of claim 3 wherein managing the integrity of the data collections comprises when the first set of attribute values is not equal to the second set of attribute values, sending the data collection from the first data management system to the second data management system.

5. The method of claim 3 wherein managing the integrity of the data collections comprises when the first set of attribute values is not equal to the second set of attribute values, deleting the data collection from the second data management and sending the data collection from the first data management system to the second data management system.

6. The method of claim 1 wherein the filter condition comprises a user-definable filter condition.

7. The method of claim 1 wherein the filter condition comprises one or more values for one or more attributes associated with a data collection.

8. The method of claim 1 wherein the filter condition comprises one or more values for an attribute that uniquely identifies a data collection.

9. The method of claim 1 wherein detecting the presence of data collections that meet the filter condition comprises:

extracting from the first data management system a first group of values of a uniquely identifying attribute for one or more data collections, each data collection having an attribute value that meets the filter condition, each value of the uniquely identifying attribute being associated with a single data collection, and

extracting from the second data management system a second group of values of the uniquely identifying attribute for one or more data collections, each data collection having an attribute value that meets the filter condition, each value of the uniquely identifying attribute being associated with a single data collection.

10. The method of claim 1 further comprising:

providing a user with an indication of the results of the presence detection; and

permitting the user to determine whether to proceed with managing the integrity of the data collections identified in the presence detection; and

managing the integrity of the data collections identified in the presence detection proceeds only after receiving an indication from the user to proceed.

11. The method of claim 1 wherein a data collection comprises an object instance, and an attribute that uniquely identifies a single data collection from the other data collections is an object identifier.

12. The method of claim 1 wherein a data collection comprises a data record, and an attribute that uniquely identifies a single data collection from the other data collections is a record identifier.

13. The method of claim 1 wherein a data collection comprises a row in a database table, and an attribute that uniquely identifies a single data collection from the other data collections is a row identifier.

14. The method of claim 1 wherein a data collection comprises a data segment, and an attribute that uniquely identifies a single data collection from the other data collections is a data segment identifier.

15. The method of claim 1 wherein detecting the presence, in the first and second data management systems, of data collections that meet a filter condition further comprises:

determining the number of data collections that occur in both the first data management system and the second data management system and meet the filter condition;

determining the number of data collections that occur in only the first data management system and meet the filter condition; and

determining the number of data collections that occur in only the second data management system and meet the filter condition.

16. The method of claim 15 wherein detecting the presence, in the first and second data management systems, of data collections that meet a filter condition further comprises:

presenting the number of data collections that occur in both the first data management system and the second data management system and meet the filter condition;

presenting the number of data collections that occur in only the first data management system and meet the filter condition; and

presenting the number of data collections that occur in only the second data management system and meet the filter condition.

17. The method of claim 1 wherein managing integrity of the data collections identified in the presence detection comprises:

extracting from the first data management system a first group of data collections that meet the filter condition; and

extracting from the second data management system a second group of data collections that meet the filter condition.

18. The method of claim 17 wherein managing integrity of the data collections identified in the presence detection comprises:

receiving from the user an indication of one or more data collections that are associated both with the first group of data collections and with the second group of data collections, each single data collection having a first set of attribute values associated with the first group of data collections and a second set of attribute values associated with the second group of data collections; and

providing the user with an indication of whether the first set of attribute values is equal to the second set of attribute values.

19. The method of claim 1 wherein managing the integrity of data collections identified in the presence detection further comprises transforming one or more values for a set of attributes associated with a data collection such that the set of attributes associated with the data collection in the first data management system may be compared with the set of attributes associated with the data collection in the second data management system.

20. A computer-readable medium or propagated signal having embodied thereon a computer program configured to manage data integrity in a first data management system and a second data management system, each system having stored therein a plurality of data collections, each data collection being associated with a set of attributes and having at least one attribute that uniquely identifies a single data collection from the other data collections, the medium or signal comprising one or more code segments configured to:

detect the presence, in the first and second data management systems, of data collections that meet a filter condition, the filter condition comprising at least one value of an attribute, the attribute occurring in one or more data collections in the first data management system and the attribute occurring in one or more data collections in the second data management system; and

manage the data integrity of the data collections identified in the presence detection such that each data collection is included both in the first data management system and in the second data management system and each data collection includes a set of values that is the same for the set of attributes associated with the data collection in the first data management system and for the set of attributes associated with the data collection in the second data management system.

21. The medium or signal of claim 20 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to send the data collection from the first data management system to the second data management system when a data collection that meets the filter condition occurs only in the first data management system.

22. The medium or signal of claim 20 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to determine whether a first set of attribute values associated with the data collection in the first data management system is equal to a second set of attribute values associated with the data collection in the second data management system.

23. The medium or signal of claim 22 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to send the data collection from the first data management system to the second data management system when the first set of attribute values is not equal to the second set of attribute values.

24. The medium or signal of claim 22 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to delete the data collection from the second data management and send the data collection from the first data management system to the second data management system when the first set of attribute values is not equal to the second set of attribute values.

25. The medium or signal of claim 20 wherein the filter condition is a user-definable filter condition.

26. The medium or signal of claim 20 wherein the filter condition comprises one or more values for one or more attributes associated with a data collection.

27. The medium or signal of claim 20 wherein the filter condition comprises one or more values for an attribute that uniquely identifies a data collection.

28. The medium or signal of claim 20 wherein the one or more code segments configured to detect the presence of data collections that meet the filter condition comprise one or more code segments configured to:

extract from the first data management system a first group of values of a uniquely identifying attribute for one or more data collections, each data collection having an attribute value that meets the filter condition, each value of the uniquely identifying attribute being associated with a single data collection; and

extract from the second data management system a second group of values of the uniquely identifying attribute for one or more data collections, each data collection having an attribute value that meets the filter condition, each value of the uniquely identifying attribute being associated with a single data collection.

29. The medium or signal of claim 20 further comprising one or more code segments configured to:

provide a user with an indication of the results of the presence detection; and

permit the user to determine whether to proceed with managing the integrity of the data collections identified in the presence detection; and

manage the data integrity of the data collections identified in the presence detection comprise proceeds only after receiving an indication from the user to proceed.

30. The medium or signal of claim 20 wherein a data collection comprises an object instance, and an attribute that uniquely identifies a single data collection from the other data collections is an object identifier.

31. The medium or signal of claim 20 wherein a data collection comprises a data record, and an attribute that uniquely identifies a single data collection from the other data collections is a record identifier.

32. The medium or signal of claim 20 wherein a data collection comprises a row in a database table, and an attribute that uniquely identifies a single data collection from the other data collections is a row identifier.

33. The medium or signal of claim 20 wherein a data collection comprises a data segment, and an attribute that uniquely identifies a single data collection from the other data collections is a data segment identifier.

34. The medium or signal of claim 20 wherein the one or more code segments configured to detect the presence, in the first and second data management systems, of data collections that meet a filter condition comprise one or more code segments configured to:

determine the number of data collections that occur in both the first data management system and the second data management system and meet the filter condition;

determine the number of data collections that occur in only the first data management system and meet the filter condition; and

determine the number of data collection entities that occur in only the second data management system and meet the filter condition.

35. The medium or signal of claim 34 wherein the one or more code segments configured to detect the presence, in the first and second data management systems, of data collections that meet a filter condition comprise one or more code segments configured to:

present the number of data collections that occur in both the first data management system and the second data management system and meet the filter condition;

present the number of data collections that occur in only the first data management system and meet the filter condition; and

present the number of data collections that occur in only the second data management system and meet the filter condition.

36. The medium or signal of claim 20 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to:

extract from the first data management system a first group of data collections that meet the filter condition; and

extract from the second data management system a second group of data collections that that meet the filter condition.

37. The medium or signal of claim 20 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection comprise one or more code segments configured to:

receive from the user an indication of one or more data collections that are associated both with the first group of data collections and with the second group of data collections, each single data collection having a first set of attribute values associated with the first group of data collections and a second set of attribute values associated with the second group of data collections; and

provide the user with an indication of whether the first set of attribute values is equal to the second set of attribute values.

38. The medium or signal of claim 20 wherein the one or more code segments configured to manage the integrity of the data collections identified in the presence detection further comprise one or more code segments configured to transform one or more values for a set of attributes associated with a data collection such that the set of attributes associated with the data-collection in the first data management system may be compared with the set of attributes associated with the data collection in the second data management system.

39. A computer-readable medium or propagated signal having embodied thereon a computer program configured to manage data integrity in a leading data management system and a contrast data management system, the medium or signal comprising one or more code segments configured to:

receive a filter condition from a user, the filter condition comprising at least one value of an attribute, the attribute occurring in one or more data collections in the first data management system and the attribute occurring in one or more data collections in the second data management system;

extract from the leading data management system a first group of values of at least one attribute that uniquely identifies a single data collection from the other data collections, each value being associated with a data collection that meets the filter condition;

extract from the contrast data management system a second group of values of at least one attribute that uniquely identifies a single data collection from the other data collections, each value being associated with a data collection that meets the filter condition;

compare the first group of values of the uniquely identifying attribute with the second group of values of the uniquely identifying attribute to determine whether each value of the uniquely identifying attribute in the first group represent the same data collection as a value of the uniquely identifying attribute in the second group;

present a list of the values of the uniquely identifying attribute that are associated with both the leading data management system and the contrast data management system and meet the filter condition, the values of the uniquely identifying attribute that are associated with only the leading data management system and meet the filter condition, and the values of uniquely identifying attribute that are associated with only the contrast data management system and meet the filter condition;

permit the user to determine whether to proceed with managing data integrity; and

after receiving an indication from the user to proceed,

extract from the leading data management system a first group of attribute value sets, each attribute value set being associated with a single data collection that is included in both the leading data management system and the contrast data management system and that meets the filter condition,

extract from the contrast data management system a second group of attribute value sets, each attribute value set being associated with a particular data collection that is included in both the leading data management system and the contrast data management system and that meets the filter condition, and

for each attribute value set extracted from the leading data management system, identify the attribute value set from the contrast data management system that represents the same data collection and compare the attribute value set from the leading data management system with the attribute value set from the contrast data management system to determine whether the attribute value sets match, and present an indication whether the attribute value set from the leading data management system matches the attribute value set from the contrast data management system.

40. The medium or signal of claim 39 wherein the one or more code segments configured to manage data integrity comprise one or more code segments configured to receive an indication from a user to compare data collections in the leading data management system with data collections in the contrast data management system.

41. The medium or signal of claim 39 wherein the one or more code segments configured to manage data integrity comprise one or more code segments configured to:

determine which values of the at least one uniquely identifying attribute are associated with both the leading data management system and the contrast data management system and meet the filter condition;

determine which values of the at least one uniquely identifying attribute are associated with only the leading data management system and meet the filter condition; and

determine which values of the at least one uniquely identifying attribute are associated with only the contrast data management system and meet the filter condition.

42. The medium or signal of claim 39 wherein the one or more code segments configured to manage data integrity comprise one or more code segments configured to:

present the number of the values of the at least one uniquely identifying attribute are associated with both the leading data management system and the contrast data management system and meet the filter condition;

present the number of the values of the at least one uniquely identifying attribute are associated with only the leading data management system and meet the filter condition; and

present the number of the values of the at least one uniquely identifying attribute are associated with only the contrast data management system and meet the filter condition.