Generating rules for matching new customer records to existing customer records in a large database

• US 5,960,430 A
• Filed: 07/10/1998
• Issued: 09/28/1999
• Est. Priority Date: 08/23/1996
• Status: Expired due to Fees

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Phillips, Mary Clarkeson, Murren, Brian Terence, Lander, Henry, Pierce, Barbara Ann, Haimowitz, Ira Joseph
• Primary Examiner(s): Amsbury, Wayne
• Assistant Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US09/113,406
• Time in Patent Office: 445 Days
• Field of Search: 707/1, 707/2, 707/3, 707/6, 707/8, 707/7, 707/10, 707/100, 707/101, 707/104, 707/200, 707/201, 707/203, 707/511, 707/532, 707/533, 707/534, 707/536, 707/541, 707/102, 707/103, 707/204 , 707/205, 704/7, 704/9, 706/45, 706/47, 706/50, 706/53, 706/52, 706/101, 364/274.7
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A method and system for matching new customer records to existing customer records in a database. The new customer records are validated for quality and normalized into a standard form. A hash key is selected to generate a candidate set of records from the existing records in the database that likely matches the new customer records. The new customer records are then matched to each of the records in the candidate set. Once the matching has been performed, a decision is made on whether to create a new customer record, update an existing record, or save the new record in a pending file for resolution at a later time. In another embodiment, there is a methodology for learning matching rules for matching records in a database. The matching rules are then used for matching a new customer record to existing records in a database.

Claims

1. A method for generating rules for matching data in a database containing a plurality of records each having a collection of fields, comprising the steps of:

obtaining a sample of training data from the database;

identifying pairs of records from the sample of training data that are similar;

applying field matching functions to each of the corresponding fields in the similar pairs of records, each field matching function generating a score indicating a strength of a match between items in the field;

generating an intermediate file of vectors containing matching scores for all of the fields from each of the similar pair of records; and

converting the intermediate file of vectors into a plurality of matching rules for matching data in the database.

2. The method according to claim 1, wherein the field matching functions comprise exact matches, phonetic-based matches, and numeric string matches.

3. The method according to claim 1, wherein each vector comprises an overall matching score indicating whether each of the similar pair of records is a match or a non-match.

4. The method according to claim 1, wherein the plurality of matching rules indicate instances of matches, possible matches, and no matches.

5. The method according to claim 4, further comprising the step of using the plurality of matching rules for matching a new data set containing a record and a collection of fields to an existing data set in a database containing a plurality of records each having a collection fields.

6. A system for generating rules for matching data in a database containing a plurality of records each having a collection of fields, comprising:

means for obtaining a sample of training data from the database;

means for identifying pairs of records from the sample of training data that are similar;

means for applying field matching functions to each of the corresponding fields in the similar pairs of records, each field matching function generating a score indicating a strength of a match between items in the field;

means for generating an intermediate file of vectors containing matching scores for all of the fields from each of the similar pair of records; and

means for converting the intermediate file of vectors into a plurality of matching rules for matching data in the database.

7. The system according to claim 6, wherein the field matching functions comprise exact matches, phonetic-based matches, and numeric string matches.

8. The system according to claim 6, wherein each vector comprises an overall matching score indicating whether each of the similar pair of records is a match or a non-match.

9. The system according to claim 6, wherein the plurality of matching rules indicate instances of matches, possible matches, and no matches.

10. The system according to claim 9, further comprising means for using the plurality of matching rules for matching a new data set containing a record and a collection of fields to an existing data set in a database containing a plurality of records each having a collection fields.