Supplier Deduplication Engine

• US 20080275874A1
• Filed: 02/12/2008
• Published: 11/06/2008
• Est. Priority Date: 05/03/2007
• Status: Active Grant

• Current Assignee: Ketera Technologies Inc. (Jack Taylor Family Voting Trust U A D 4 14 99)
• Original Assignee: Ketera Technologies Inc. (Jack Taylor Family Voting Trust U A D 4 14 99)
• Inventors: Goyal, Ram Dayal
• Granted Patent: US 8,234,107 B2
• Time in Patent Office: Days
• US Class Current: 707/6
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation

Abstract

Disclosed herein is a method of grouping similar supplier names together in a database. The syntactical errors in the supplier names are corrected. The supplier names are grouped after correcting the syntactical errors. The abbreviations in the supplier names are captured. The ordering, pronunciation and stemming errors in the supplier names are corrected. A matching algorithm that matches and compares two supplier names is applied that comprises the steps of grouping supplier names based on first set of characters in the supplier names and calculating a matching score between the two supplier using Levenshtein distance between the two supplier names, along with the supplier names'"'"' sound codes obtained from a modified metaphone algorithm, length of each word, position of matching and mismatching characters, and stem of words in the supplier names. The matching scores are compared with set thresholds in order to further group the supplier names into clusters.

Claims

1. A method of grouping similar supplier names from a plurality of supplier names together, comprising the steps of:

correcting syntactical errors in said supplier names;

grouping the supplier names after said step of correcting syntactical errors;

capturing abbreviations of the supplier names;

correcting ordering, pronunciation and stemming errors;

a matching algorithm that matches and compares two of said supplier names further comprising the steps of;

grouping supplier names based on the first set of characters in the supplier names;

calculating a matching score using Levenshtein distance between said two supplier names, along with the supplier names'"'"' sound codes obtained from a modified metaphone algorithm, length of each word, position of matching and mismatching characters, and stem of words in the two supplier names.

2. The method of claim 1, further comprising the step of applying manual intervention to make a final decision;

3. The method of claim 1, wherein said grouping is agglomerative and the grouping result is normalized in the form of a table.

4. The method of claim 3, wherein said agglomerative grouping is performed in a plurality of levels.

5. The method of claim 4, wherein the first level out of said plurality of levels comprises correcting syntactical errors and capturing abbreviations.

6. The method of claim 4, wherein the second level out of said plurality of levels comprises handling order of words, pronunciation errors and stemming errors.

7. The method of claim 4, wherein the third level out of said plurality of levels comprises grouping of clusters created in said second level using approximate string matching criteria.

8. The method of claim 7, wherein said string matching criteria comprises Levenshtien distance scores, position of matching and non-matching characters, number of words, length of words, etc.

9. The method of claim 4, wherein the fourth level out of said plurality of levels comprises manual verification in order to verify the accuracy in grouping and identify probable matches in supplier names.

10. The method of claim 4, further comprising the step of setting threshold values for all said levels of grouping of supplier names.

11. The method of claim 1, wherein a matching score is calculated for matching two supplier names from different groups.

12. The method of claim 11, wherein grouping of supplier names further comprises the steps of comparing said matching scores with said set thresholds, and if the matching score is greater than the set threshold, said supplier names are merged in third level.

13. The method of claim 1, wherein said modified metaphone algorithm determines the sound code for each word in the supplier name.

14. The method of claim 1, wherein the number of words in said supplier names and the number of words matching between said two supplier names is considered for determining matching scores between the two supplier names in said matching algorithm.

15. The method of claim 1, wherein the position of mismatching characters is considered for determining matching scores between said two supplier names in said matching algorithm.

16. The method of claim 1, wherein the length of each word present in said supplier names is considered in said matching algorithm.

17. The method of claim 1, wherein small sub groups of supplier names are created based on a similar first set of characters, wherein said step of creating small sub groups eliminates unnecessary matches.

18. The method of claim 1, wherein said supplier names are grouped by assigning same record identifiers to matched supplier names at different levels of grouping, whereby each group comprises a unique identifier.

19. The method of claim 1, wherein common words and word forms in said supplier names are ignored before grouping the supplier names.

20. A computer program product comprising computer executable instructions embodied in a computer-readable medium, said computer program product including:

a first computer parsable program code for correcting syntactical errors in supplier names;

a second computer parsable program code for grouping the supplier names after said step of correcting syntactical errors;

a third computer parsable program code for capturing abbreviations of the supplier names;

a fourth computer parsable program code for correcting ordering, pronunciation and stemming errors;

a fifth computer parsable program code of a matching algorithm that matches and compares two of said supplier names further comprising;

a sixth computer parsable program code for grouping supplier names based on the first set of characters in the supplier names to avoid unnecessary squared n matching;

a seventh computer parsable program code for calculating a matching score using levenshtein distance between said two supplier names, along with the supplier names'"'"' sound codes obtained from a modified metaphone algorithm, length of each word, position of matching and mismatching characters, and stem of words in the two supplier names