Systems and methods for master data management using record and field based rules

• US 8,341,131 B2
• Filed: 09/16/2010
• Issued: 12/25/2012
• Est. Priority Date: 09/16/2010
• Status: Active Grant

• Current Assignee: Regional Resources Ltd (Imperial Metals Corp.)
• Original Assignee: SAP AG (SAP SE)
• Inventors: Cohen, Ronen
• Primary Examiner(s): Ly, Anh
• Application Number:
  US12/883,562
• Publication Number:
  US 20120072464A1
• Time in Patent Office: 831 Days
• Field of Search: 707/687, 707/690, 707/691, 707/694, 707/705, 707/759, 707/802, 707/953, 707/E17.005, 707/E17.044, 707/803, 707/E17.045, 714/48, 714/49, 714/100, 714/E11.023
• US Class Current: 707/694
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors

Abstract

According to some embodiments, a plurality of input records may be received from a plurality of sources, and each input record may include a plurality of fields. It may then be detected that a set of input records from different sources are related to each other (e.g., are duplicates). One of the set of input records may be automatically selected as a master record in accordance with a record level rule. At least one field in the master record may, according to some embodiments, be automatically modified based on a corresponding field in another input record in accordance with a field level rule. The modified master record could then be stored for subsequent use by other applications.

Claims

1. A computer implemented method, comprising:

receiving a plurality of input records from a plurality of sources, each input record including a plurality of relational database fields;

detecting that a set of input records from different sources are related to each other;

automatically selecting one of the set of input records as a master record in accordance with a record level rule;

automatically modifying at least one relational database field in the master record based on a corresponding relational database field in another input record in accordance with a field level rule;

appending a relational database field from another input record to the master record without deleting the value of the corresponding relational database field that is related to the appended relational database field in the master record;

storing the modified master record;

receiving an undo indication from a user; and

responsive to the undo indication, restoring the master record to the unmodified state based on the appended relational database field, wherein each source is associated with a reliability score and the record level rule comprises automatically selecting the input record from the source having the best reliability score.

2. The method of claim 1, wherein the record level rule is associated with at least one of:

(i) a completeness criteria, (ii) an occurrence frequency associated with combinations of values across relational database fields, or (iii) when the record was created or modified.

3. The method of claim 1, wherein said detecting that the set of input records are related to each other comprises finding similar values in corresponding relational database fields of the input records.

4. The method of claim 1, wherein the field level rule is associated with at least one of:

(i) a value length, (ii) a value frequency of corresponding fields in different input records, (iii) a record or field reliability score, or (iv) when the relational database field was created or modified.

5. The method of claim 1, wherein a field in the master record is automatically modified based on values in a plurality of corresponding fields in other input records in accordance with:

(i) a minimum function, (ii) a maximum function, or (iii) an average function.

6. The method of claim 1, further comprising:

storing an indication of the other input record along with the modification to the relational database field of the master record.

7. The method of claim 6, further comprising:

generating a master record lineage report based at least in part on the stored indication.

8. The method of claim 1, wherein the input records comprise simulation records.

9. The method of claim 1, wherein at least one of a record level rule or field level rule is automatically adjusted based on historical information.

10. The method of claim 1, further comprising:

transmitting an exception associated with at least one of a record or a relational database field indicating that manual review is recommended.

11. The method of claim 1, further comprising:

placing a value into a relational database field of the master record based on at least one of;

(i) a value in a non-corresponding relational database field, or (ii) data received from a third party service.

12. A non-transitory, computer-readable storage medium storing program code executable by a computer to:

receive a plurality of input records from a plurality of sources, each source being associated with a reliability score and each input record including a plurality of relational database fields;

detect that a set of input records from different sources are related to each other by finding similar values in corresponding relational database fields of the input records;

automatically select the input record from the source having the best reliability score as a master record in accordance with a record level rule;

automatically modify at least one relational database field in the master record based on a corresponding relational database field in another input record in accordance with a field level rule;

appending a relational database field from another input record to the master record without deleting the value of the corresponding relational database field that is related to the appended relational database field in the master record;

store the modified master record;

receive an undo indication from a user; and

responsive to the undo indication, restore the master record to the unmodified state based on the appended relational database field, wherein each source is associated with a reliability score and the record level rule comprises automatically selecting the input record from the source having the best reliability score.

13. The medium of claim 12, wherein a plurality of field level rules are applied to modify the master record.

14. The medium of claim 12, wherein the field level rule is associated with at least one of:

(i) the source reliability scores or a field level reliability score, (ii) a completeness criteria, (iii) an occurrence frequency, or (iv) when information was created or modified.

15. A system, comprising:

an input data store to provide a plurality of input records associated with different sources, each input record including a plurality of relational database fields;

a master record data store; and

a master data server to (i) receive the input records from the input data store and (ii) store a modified master record into the master record store, wherein the master data server;

detects that a set of input records from different sources are related to each other,

selects one of the set of input records as a master record in accordance with a record level rule,

creates the modified master record by modifying at least one relational database field in the master record based on a corresponding relational database field in another input record in accordance with a field level rule and appending a relational database field from another input record to the master record without deleting the value of the corresponding relational database field that is related to the appended relational database field in the master record;

receives an undo indication from a user; and

responsive to the undo indication, restores the master record to the unmodified state based on the appended relational database field, wherein each source is associated with a reliability score and the record level rule comprises automatically selecting the input record from the source having the best reliability score.

16. The system of claim 15, wherein the master data server is further to store an indication of the other input record along with the modification to the relational database field of the master record.

17. The system of claim 15, wherein at least one of the record or field level rule are associated with at least one of:

(i) a reliability score, (ii) a completeness criteria, (iii) an occurrence frequency, or (iv) when information was created or modified.