Data mapper

• US 10,223,433 B2
• Filed: 01/25/2017
• Issued: 03/05/2019
• Est. Priority Date: 01/25/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Coleman, Andrew J., Reeve, John A.
• Primary Examiner(s): Uddin, Md I
• Application Number:
  US15/414,853
• Publication Number:
  US 20180210905A1
• Time in Patent Office: 769 Days
• Field of Search: 707754, 707755, 707763
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/90344 by using string matching techniques

Abstract

Systems, methods and tools for mapping data elements between a source data structure and a target data structure without the needing exactly matching values or a dictionary of synonyms. The embodiments of the present disclosure auto map data elements having similar phonetic sounds, even in the presence of one or more misspellings or alternative spellings. Embodiments of the present disclosure utilize Soundex algorithms to generate a key corresponding to each source and target data elements. Each Soundex key may be generated based on the phonetical sounds of the source and target data elements. By using phonetical sounds to generate the key, a Soundex algorithm allows for similarly sounding words, alternatively spelled words or misspelled words to generate the same Soundex key, creating equivalence between the naming of source and target data elements once the auto mapping function is implemented.

Claims

1. A method for mapping data elements between a source data structure and a target data structure, said method comprising the steps of:

generating, by a processor, a Soundex key as a function of a Soundex algorithm applied to a source data element;

pairing, by the processor, the Soundex key with a value equal to the source data element, forming a source key and value pair;

storing, by the processor, the source key and value pair in the source data structure in a first hash table;

further generating, by the processor, a second Soundex key as a function of the Soundex algorithm applied to a target data element;

further pairing, by the processor, the second Soundex key with a second value equal to the target data element, forming a target key and value pair;

further storing, by the processor, the target key and value pair in the target data structure in a second hash table;

matching, by the processor, the source key and value pair to the target key and value pair;

automatically mapping, by the processor, the source data element to the target data element as a function of matching the source key and value pair stored in the source data structure to the target key and value pair of the target data structure, wherein the source key and value pairs and the target key and value pairs are encoded to allow for alternative spellings or misspellings of the data elements stored by the source and target data structures, and wherein the step of matching results in a plurality of target data elements, each target data element comprising Soundex keys matching the Soundex key of the source data element;

creating, by the processor, a parent source Soundex key for each parent element of the source data element;

creating, by the processor, a parent target Soundex key for each parent element of the plurality of target data elements matching the Soundex key of the source data element;

comparing, by the processor, the parent source Soundex key for each parent element of the source data element with each parent target Soundex key; and

mapping, by the processor the source data element to the target data element having the parent target Soundex key that matches the source parent Soundex key.

2. The method of claim 1, wherein the source data structure and the target data structure are selected from the group consisting of an array, file, record and table.

3. The method of claim 2, wherein the first hash table and the second hash table are different hash tables.

4. The method of claim 1, wherein a match identified during the matching step comprises a target key and value pair having the second Soundex key with a starting letter equal to a starting letter of the Soundex key and a three-digit number that is within a pre-configured margin, but not equal to, the Soundex key.

5. The method of claim 1, wherein the step of matching includes identifying the target key and value pair closest, but unequal to, the source key and value pair.

9. The method of claim 1, wherein the step of matching includes identifying the target key and value pair closest, but unequal to, the source key and value pair.

6. A computer system, comprising:

a processor;

a memory device coupled to the processor; and

a computer readable storage device coupled to the processor, wherein the storage device contains program code executable by the processor via the memory device to implement a method for mapping data elements between a source data structure and a target data structure, said method comprising the steps of;

generating, by the processor, a Soundex key as a function of a Soundex algorithm applied to a source data element;

pairing, by the processor, the Soundex key with a value equal to the source data element, forming a source key and value pair;

storing, by the processor, the source key and value pair in the source data structure in a first hash table;

further generating, by the processor, a second Soundex key as a function of the Soundex algorithm applied to a target data element;

further pairing, by the processor, the second Soundex key with a second value equal to the target data element, forming a target key and value pair;

further storing, by the processor, the target key and value pair in the target data structure in a second hash table;

matching, by the processor, the source key and value pair to the target key and value pair;

automatically mapping, by the processor, the source data element to the target data element as a function of matching the source key and value pair stored in the source data structure to the target key and value pair of the target data structure, wherein the source key and value pairs and the target key and value pairs are encoded to allow for alternative spellings or misspellings of the data elements stored by the source and target data structures, and wherein the step of matching results in a plurality of target data elements, each target data element comprising Soundex keys matching the Soundex key of the source data element;

creating, by the processor, a parent source Soundex key for each parent element of the source data element;

creating, by the processor, a parent target Soundex key for each parent element of the plurality of target data elements matching the Soundex key of the source data element;

comparing, by the processor, the parent source Soundex key for each parent element of the source data element with each parent target Soundex key; and

mapping, by the processor the source data element to the target data element having the parent target Soundex key that matches the source parent Soundex key.

7. The computer system of claim 6, wherein the source data structure and the target data structure are selected from the group consisting of an array, file, record and table.

8. The computer system of claim 6, wherein a match identified during the matching step comprises a target key and value pair having the second Soundex key with a starting letter equal to a starting letter of the Soundex key and a three-digit number that is within a pre-configured margin, but not equal to, the Soundex key.

10. A computer program product comprising:

one or more computer readable hardware storage devices having computer readable program code stored therein, said program code containing instructions executable by one or more processors to implement a method for mapping data elements between a source data structure and a target data structure, said method comprising the steps of;

generating, by a processor, a Soundex key as a function of a Soundex algorithm applied to a source data element;

pairing, by the processor, the Soundex key with a value equal to the source data element, forming a source key and value pair;

storing, by the processor, the source key and value pair in the source data structure in a first hash table;

further generating, by the processor, a second Soundex key as a function of the Soundex algorithm applied to a target data element;

further pairing, by the processor, the second Soundex key with a second value equal to the target data element, forming a target key and value pair;

further storing, by the processor, the target key and value pair in the target data structure in a second hash table;

matching, by the processor, the source key and value pair to the target key and value pair;

automatically mapping, by the processor, the source data element to the target data element as a function of matching the source key and value pair stored in the source data structure to the target key and value pair of the target data structure, wherein the source key and value pairs and the target key and value pairs are encoded to allow for alternative spellings or misspellings of the data elements stored by the source and target data structures, and wherein the step of matching results in a plurality of target data elements, each target data element comprising Soundex keys matching the Soundex key of the source data element;

creating, by the processor, a parent source Soundex key for each parent element of the source data element;

creating, by the processor, a parent target Soundex key for each parent element of the plurality of target data elements matching the Soundex key of the source data element;

comparing, by the processor, the parent source Soundex key for each parent element of the source data element with each parent target Soundex key; and

mapping, by the processor the source data element to the target data element having the parent target Soundex key that matches the source parent Soundex key.

11. The computer program product of claim 10, wherein the source data structure and the target data structure are selected from the group consisting of an array, file, record and table.

12. The computer program product of claim 10, wherein a match identified during the matching step comprises a target key and value pair having the second Soundex key with a starting letter equal to a starting letter of the Soundex key and a three-digit number that is within a pre-configured margin, but not equal to, the Soundex key.

13. The computer program product of claim 10, wherein the step of matching includes identifying the target key and value pair closest, but unequal to, the source key and value pair.