DATA COMPARISON SYSTEM

• US 20110035371A1
• Filed: 07/20/2010
• Published: 02/10/2011
• Est. Priority Date: 08/06/2009
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services GmbH (Accenture PLC)
• Inventors: Pong, Geoffrey, Hui, Anita
• Granted Patent: US 9,122,732 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems

Abstract

A data comparison system is described. The system may include a memory, an interface, and a processor. The memory may store a compare data structure containing multiple data item pairs, each pair including a legacy data item of a legacy dataset and a corresponding new data item of a new dataset, and a tolerance associated with each data item pair. The processor may receive the compare data structure and the associated tolerances. The processor may call a compare data subroutine to compare each data item pair in accordance with the associated tolerance if the data items are fields. Otherwise, the processor may recursively call the compare data subroutine for each record the data items until the data item are fields. The processor may then compare data items in accordance with the associated tolerance.

Claims

1. A computer-implemented method for performing a tolerance based comparison between a legacy data store and a new data store, the method comprising:

(a) receiving, by a processor from a device of a user, a compare data structure comprising of a plurality of data item pairs, each data item pair identifying a legacy data item of a legacy dataset and a new data item of a new dataset, wherein each data item pair comprises a data type of a table data type, a flat structure data type, a deep structure data type, or a field data type, further wherein the table data type, the flat structure data type and the deep structure data type each comprise a plurality of records;

(b) receiving, by the processor from the device of the user, a plurality of tolerances, each tolerance being associated with one of the data item pairs;

(c) calling, by the processor, a compare data subroutine to compare each data item pair of the plurality of data item pairs;

(d) determining, by the processor, whether the data type of each data item pair comprises the table data type, the flat structure data type, or the field data type;

(e) comparing, by the processor, each legacy data item and each new data item of each data item pair in accordance with the associated tolerance if the data type of each data item pair comprises the field data type; and

(f) otherwise, repeating, by the processor, steps (c)-(e) for each record of each data item pair until the data type of each data item pair comprises the field data type, and then comparing each legacy data item and each new data item of each data item pair in accordance with the associated tolerance.

2. The computer-implemented method of claim 1 further comprising:

(g) transforming, by the processor, a result of the comparing into a report, wherein the report describes a percentage of the legacy and new data items which were an exact match, a percentage of the legacy and new data items which were within the associated tolerance, and a percentage of the legacy and new data items which were outside the associated tolerance; and

(h) providing, by the processor to the device of the user, the report.

3. The computer-implemented method of claim 2 wherein the report comprises a plurality of field-level statistics and a plurality of record level statistics.

4. The computer-implemented method of claim 3 wherein the report groups the plurality of field-level statistics based on a field in the plurality of fields.

5. The computer-implemented method of claim 1 wherein receiving, by the processor from the device of the user, the plurality of tolerances, each tolerance being associated with one of the data item pairs further comprises:

providing, by the processor to the device of the user, a user interface for identifying the plurality of tolerances and the data item pair associated with each tolerance; and

receiving, from the device of the user, the plurality of tolerances, each tolerance being associated with one of the data item pairs.

6. The computer-implemented method of claim 1 wherein comparing, by the processor, each legacy data item and each new data item of each data item pair in accordance with the associated tolerance if the data type of each data item pair comprises the field data type further comprises:

comparing, by the processor, each legacy data item and each new data item of each data item pair to determine a difference between each legacy data item and each new data item if the data type of each data item pair comprises the field data type; and

assigning, by the processor, a category for each data item pair, the category based on whether the difference of each data item pair is within the tolerance associated with each data item pair.

7. The computer-implemented method of claim 6 wherein each category comprises one of an exact match category, a within tolerance category or an outside of tolerance category.

8. The computer-implemented method of claim 7 further comprising:

(g) transforming, by the processor, each category into the report, wherein the report describes a percentage of the data item pairs which were assigned the exact match category, a percentage of the data item pairs which were assigned the within tolerance category, and a percentage of the data item pairs which were assigned the outside of tolerance category; and

(h) providing, by the processor to the device of the user, the report.

9. The computer-implemented method of claim 1 wherein at least one data item comprises a data type of a character data type and the associated tolerance comprises at least one of a case sensitive tolerance and a space sensitive tolerance.

10. The computer-implemented method of claim 1 wherein at least one data item comprises a data type of a date data type and the associated tolerance comprises a number of days difference tolerance.

11. The computer-implemented method of claim 1 wherein the legacy dataset and the new dataset comprise a plurality of invoices.

12. The computer-implemented method of claim 1 further comprising:

(g) providing, to the device of the user, a user interface for identifying which of the data item pairs to compare; and

(h) receiving, by the processor from the device of the user, a selection of data item pairs to compare.

13. The computer-implemented method of claim 12 wherein comparing, by the processor, each legacy data item and each new data item of each data item pair in accordance with the associated tolerance if the data type of each data item pair comprises the field data type further comprises comparing, by the processor, each legacy data item and each new data item of each data item pair in accordance with the associated tolerance if the data type of each data item pair comprises the field data type and if the data item pair was included in the selection provided by the device of the user in step (g).

14. A computer-implemented method for performing a tolerance based comparison between a legacy data store and a new data store, the method comprising:

receiving, by a processor from a device of a user, a compare data structure, wherein the compare data structure describes a plurality of mappings between a first plurality of data items of a first dataset and a second plurality of data items of a second dataset, wherein the data items comprise at least one of a table, a field, a deep structure, or a flat structure;

populating, by the processor, the compare data structure with a first plurality of data items from the first dataset and the second plurality of data items from the second dataset;

receiving, by the processor, a plurality of tolerances, each tolerance being associated with one of the mappings between the first plurality of data items and the second plurality of data items wherein each tolerance describes an acceptable variance for each mapping of the plurality of mappings;

comparing, by the processor, the first plurality of data items in the compare data structure with the second plurality of data items in the compare structure in accordance with the plurality of mappings and the plurality of tolerances, wherein the first plurality of data items and the second plurality of data items are compared recursively if the first plurality of data items and the second plurality of data items comprise the table, the flat structure, or the deep structure;

transforming, by the processor, a result of the comparing into a report, wherein the report describes a percentage of the first and second data items which were an exact match, a percentage of the first and second data items which were within the associated tolerances, and a percentage of the first and second data items which were outside the associated tolerances; and

providing, by the processor to the device of the user, the report.

15. The computer-implemented method of claim 14 wherein the report comprises a plurality of field-level statistics and a plurality of record level statistics.

16. The computer-implemented method of claim 15 wherein the report groups the plurality of field-level statistics based on a field in the plurality of fields.

17. The computer-implemented method of claim 14 wherein at least one data item comprises a data type of a character data type and the associated tolerance comprises at least one of a case sensitive tolerance and a space sensitive tolerance.

18. The computer-implemented method of claim 14 wherein at least one data item comprises a data type of a date data type and the associated tolerance comprises a number of days difference tolerance.

19. The computer-implemented method of claim 14 wherein the first dataset and the second dataset comprise a plurality of invoices.

20. The computer-implemented method of claim 14 further comprising:

providing, to the device of the user, a user interface for identifying which of the mappings to compare; and

receiving, by the processor from the device of the user, a selection of mappings to compare.

21. The computer-implemented method of claim 20 wherein comparing, by the processor, the first plurality of data items in the compare data structure with the second plurality of data items in the compare structure in accordance with the plurality of mappings and the plurality of tolerances if each mapping was selected by the user.

22. A data comparison system, the system comprising:

a memory to store a compare data structure comprising of a plurality of data item pairs, each data item pair identifying a legacy data item of a legacy dataset and a new data item of a new dataset, wherein each data item pair comprises a data type of a field data type, a table data type, a flat structure data type, or a deep structure data type, further wherein the table data type, the flat structure data type and the deep structure data type each comprise a plurality of records, and a plurality of tolerances, each tolerance being associated with one of the data item pairs;

an interface operatively connected to the memory, the interface operative to communicate with a device of a user; and

a processor operatively connected to the memory and the interface, the processor operative to receive, from the device of the user via the interface, the compare data structure and the plurality of tolerances, call a compare data subroutine to compare each legacy data item and each new data item identified by each data item pair, determine whether the data type of each data item pair comprises field data type, the table data type, the flat structure data type or the deep structure data type, compare each legacy data item and each new data item in accordance with the associated tolerance if the data type of each data item pair comprises the field data type, and otherwise recursively call the compare data subroutine for each record of each data item pair until the data type of each data item pair comprises the field data type, and then compare the legacy data item and the new data item of each data item pair in accordance with the associated tolerance.

23. The system of claim 22 wherein the processor is further operative to transform a result of the compare into a report, wherein the report describes a percentage of the first and second data items which were an exact match, a percentage of the first and second data items which were within the associated tolerance, and a percentage of the first and second data items which were outside the associated tolerance, and provide, to the device of the user via the interface, the report.

24. The system of claim 23 wherein the report comprises a plurality of field-level statistics and a plurality of record level statistics.

25. The system of claim 24 wherein the report groups the plurality of field-level statistics based on a field in the plurality of fields.

26. The system of claim 22 wherein the processor is further operative to provide, to the device of the user via the interface, a user interface for identifying the plurality of tolerances and the data item pair associated with each tolerance, and receive, from the device of the user via the interface, the plurality of tolerances, each tolerance being associated with one of the data item pairs.

27. The system of claim 22 wherein the processor is further operative to compare each legacy data item and each new data item to determine a difference between each legacy data item and each new data item if each legacy data item and each new data item comprise the field, and assign a category for each data item pair, the category based on whether the difference of each data item pair is within the associated tolerance.

28. The system of claim 27 wherein each category comprises one of an exact match category, a within tolerance category and an outside of tolerance category.

29. The system of claim 28 wherein the processor is further operative to transform each category into the report, wherein the report describes a percentage of the data item pairs which assigned the exact match category, a percentage of the data item pairs which were assigned the within tolerance category, and a percentage of the data item pairs which were assigned the outside of tolerance category, and provide, to the device of the user via the interface, the report.

30. The system of claim 22 wherein at least one data item comprises a data type of a character data type and the associated tolerance comprises at least one of a case sensitive tolerance and a space sensitive tolerance.

31. The system of claim 22 wherein at least one data item comprises a data type of a date data type and the associated tolerance comprises a number of days difference tolerance.

32. The system of claim 22 wherein the legacy dataset and the new dataset comprise a plurality of invoices.

33. The system of claim 22 wherein the processor is further operative to provide, to the device of the user via the interface, a user interface for identifying which of the data item pairs to compare, and receive, from the device of the user via the interface, a selection of data item pairs to compare.

34. The system of claim 33 wherein the processor is further operative to compare each legacy data item and each new data item in accordance with the associated tolerance if each legacy data item and each new data item comprise the field and if the data item pair was selected by the user.