Systems and methods for capturing data refinement actions based on visualized search of information

• US 8,645,332 B1
• Filed: 08/20/2012
• Issued: 02/04/2014
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Cohen, Ronen, Vishnia-Shabtai, Nimrod
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US13/589,414
• Publication Number:
  US 20140052695A1
• Time in Patent Office: 533 Days
• Field of Search: 707/1, 707/610, 707/691, 709204-206, 713/1
• US Class Current: 707/691
• CPC Class Codes: G06F 16/1767 Concurrency control, e.g. optimistic or pessimistic approaches
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2315 Optimistic concurrency control

Abstract

A method for refining data discrepancies in data loads from multiple data stores includes receiving a data set containing data records from one or more data sources, comparing the received data set with a prior data set to identify delta records among corresponding records of the received and the prior data sets, recalculating a best record for data located in each of the corresponding delta records, and if a discrepancy exists in a data value for corresponding delta records, executing a refinement action rule. The refinement action rule includes at least one replace statement and search criteria. The method also includes capturing a user'"'"'s interaction with a user interface presenting the data set, and storing the interaction in a data store as a refinement action rule. A system for implementing the method and a computer readable medium are also described.

Claims

1. A computer-implemented method comprising:

receiving at a processor-controlled data store more than one data set, each data set containing respective data records from one or more respective data sources;

merging data of the respective data records into respective single records, each of the respective single records containing one or more data items from corresponding data records;

comparing data within the respective single records by a processor-implemented matching component to identify delta data within the respective single records;

calculating respective single best records for data located in each of the respective single records containing delta data by a processor-implemented best record recalculator component;

if a discrepancy exists in a data value among the respective single records, executing a refinement action rule by a processor-implemented refinement execution component; and

the refinement execution component implementing a heuristic algorithm associated with at least one filter configured to perform a fuzzy search of data records, the heuristic algorithm being trained by data steward definitions monitored by an interactive dialog box.

2. The method of claim 1, wherein the refinement action rule includes at least one replace statement and a search criteria.

3. The method of claim 1, further including the refinement execution component configured to change the data value of the discrepancy.

4. The method of claim 1, further including:

capturing an interaction by a user with the prior data set; and

storing the interaction in a data store as a refinement action rule.

5. The method of claim 1, further including:

performing a search of at least one data set to detect data discrepancies;

presenting in a user interface a graphical representation of the data discrepancies;

capturing an interaction by a user with the at least one data set; and

storing the interaction in a data store as a refinement action rule.

6. The method of claim 1, further including:

applying string pattern analysis to the data records to identify data field format inconsistencies;

displaying on a user interface a graphical representation of the aggregate of occurrences of the data field format inconsistencies;

capturing an interaction by a user with the graphical representation of the aggregate of occurrences; and

storing the interaction in a data store as a refinement action rule.

7. The method of claim 1, further including:

displaying on a user interface a graphical representation of the hierarchal perspective of at least one data set;

capturing an interaction by a user with the graphical representation of the hierarchal perspective; and

storing the interaction in a data store as a refinement action rule.

8. A non-transitory computer readable medium having stored thereon instructions which when executed by a processor cause the processor to perform the method of:

receiving more than one data set, each data set containing respective data records from one or more respective data sources;

merging data of the respective data records into respective single records, each of the respective single records containing one or more data items from corresponding data records;

comparing data within the respective single records to identify delta data within the respective single records;

calculating respective single best records for data located in each of the respective single records containing delta data;

if a discrepancy exists in a data value among the respective single records, executing a refinement action rule; and

the refinement action rule based on data steward definitions monitored by an interactive dialog box to train a heuristic algorithm associated with at least one filter configured to perform a fuzzy search of data records.

9. The computer readable medium of claim 8, the step of executing a refinement action rule includes causing the processor to perform at least one replace statement based on a search criteria included in the refinement action rule.

10. The computer readable medium of claim 8, further including executable instructions to cause a processor to perform the steps of:

capturing an interaction by a user with the prior data set; and

storing the interaction in a data store as a refinement action rule.

11. The computer readable medium of claim 8, further including executable instructions to cause a processor to perform the steps of:

performing a search of at least one data set to detect data discrepancies;

presenting in a user interface a graphical representation of the data discrepancies;

capturing an interaction by a user with the at least one data set; and

storing the interaction in a data store as a refinement action rule.

12. The computer readable medium of claim 8, further including executable instructions to cause a processor to perform the steps of:

applying string pattern analysis to the data records to identify data field format inconsistencies;

displaying on a user interface a graphical representation of the aggregate of occurrences of the data field format inconsistencies;

capturing an interaction by a user with the graphical representation of the aggregate of occurrences; and

storing the interaction in a data store as a refinement action rule.

13. The computer readable medium of claim 8, further including executable instructions to cause a processor to perform the steps of:

displaying on a user interface a graphical representation of the hierarchal perspective of at least one data set;

capturing an interaction by a user with the graphical representation of the hierarchal perspective; and

storing the interaction in a data store as a refinement action rule.

14. A system comprising:

a server under processor control configured to provide user interfaces to a user;

the processor configured to;

implement user interactions with the user interface to change data values in data records stored in a master database coupled to the server; and

store the user interactions as refinement action rules in a database coupled to the server;

the processor including;

a matching component configured to compare data within respective single records to identify delta data within the respective single records;

best record recalculator component configured to calculate respective single best records for data located in each of the respective single records containing delta data;

a refinement execution component configured to execute a refinement action rule if a discrepancy exists in a data value among the respective single records; and

the refinement execution component implementing a heuristic algorithm associated with at least one filter configured to perform a fuzzy search of data records, the heuristic algorithm being trained by data steward definitions monitored by an interactive dialog box.

15. The system of claim 14, wherein the refinement action rule includes at least one replace statement and a search criteria.

16. The system of claim 15, wherein the heuristic algorithms access the refinement action rule replace statement and the search criteria.

17. The system of claim 14, the server further configured to:

present a refinement dialog box on the user interface;

receive user entries in the refinement dialog box; and

store the user entries in the database as refinement action rules.