Method and system for processing data records

• US 7,403,942 B1
• Filed: 02/04/2003
• Issued: 07/22/2008
• Est. Priority Date: 02/04/2003
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Bayliss, David
• Primary Examiner(s): Truong; Cam Y T
• Application Number:
  US10/357,447
• Time in Patent Office: 1,995 Days
• Field of Search: 707 3- 6
• US Class Current: 707/748
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Disclosed herein are various exemplary systems and methods for linking entity references to entities and identifying associations between entities. In particular, a method for identifying an entity from a plurality of entity references, each entity reference being linked with a separate ghost entity, is provided. The method comprises the steps of comparing an entity reference of a first ghost entity with an entity reference of a second ghost entity to determine a match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity, linking the entity reference of the first ghost entity additionally with the second ghost entity and the entity reference of the second ghost entity additionally with the first ghost entity when the match probability is greater than or equal to a match threshold and repeating the steps of comparing and linking for one or more ghost entity pairings possible from the ghost entities. The method further comprises determining, for one or more entity references linked to a ghost entity, a score for the entity reference based at least in part on a match probability between the entity reference and a value representing the one or more entity references linked to the ghost entity and identifying the ghost entity as an actual entity based at least in part on one or more scores for the one or more entity references linked to the ghost entity.

Claims

1. A method for identifying an entity from a plurality of entity references, each entity reference being linked with a separate ghost entity, the method comprising the steps of:

comparing an entity reference of a first ghost entity with an entity reference of a second ghost entity to determine a match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity, wherein the match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity comprises a content weight W_c,i for a particular data field i, wherein the content weight W_c,i for a particular data field i is inversely related to a sum of a total number of occurrences of a data field value and a cautiousness value;

assigning a definitive identifier to an entity reference based on the match probability;

linking the entity reference of the first ghost entity additionally with the second ghost entity and the entity reference of the second ghost entity additionally with the first ghost entity when the match probability is greater than or equal to a match threshold;

repeating steps of the comparing and the linking for one or more ghost entity pairings possible from the ghost entities;

determining, for one or more entity references linked to a ghost entity, a score for an entity reference based at least in part on a match probability between an entity reference and the entity and a match probability between the entity reference and the ghost entity; and

identifying the ghost entity as an actual entity based at least in part on one or more scores for the one or more entity references linked to the ghost entity, wherein the identifying the ghost entity as an actual entity comprises associating a same definitive identifier with the one or more entity references linked to the ghost entity.

2. The method as in claim 1, wherein the ghost entity is identified as an actual entity based in part on a comparison of a sum of the scores with a defined threshold.

3. The method as in claim 1, further comprising the step of adjusting the score for the entity reference based at least in part on one or more grading criteria.

4. The method as in claim 3, wherein the step of adjusting the score comprises generating an adjusted score based on a cross-product of the one or more grading criteria and the score.

5. The method as in claim 1, wherein the score for the entity reference is further based upon a parameter selected from a group consisting of:

an average field value for one or more data fields of the one or more entity references linked to the entity;

a weighted average field value for one or more data fields of the one or more entity references linked to the entity;

a median field value for one or more data fields of the one or more entity references linked to the entity; and

a field value selected at random from the one or more entity references linked to the entity.

6. A computer readable storage medium for identifying an entity from a plurality of entity references, each entity reference being linked with a separate ghost entity, the computer readable storage medium comprising a set of executable instructions manipulated by a processor to:

compare an entity reference of a first ghost entity with an entity reference of a second ghost entity to determine a match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity, wherein the match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity comprises a content weight W_c,i for a particular data field i, wherein the content weight W_c,i for a particular data field i is inversely related to a sum of a total number of occurrences of a data field value and a cautiousness value;

assign a definitive identifier to an entity reference based on the match probability;

link the entity reference of the first ghost entity additionally with the second ghost entity and the entity reference of the second ghost entity additionally with the first ghost entity when the match probability is greater than or equal to a match threshold;

repeat steps of the comparing and the linking for one or more ghost entity pairings possible from the ghost entities;

determine, for one or more entity references linked to a ghost entity, a score for an entity reference based at least in part on a match probability between an entity reference and the entity and a match probability between the entity reference and the ghost entity; and

identify the ghost entity as an actual entity based at least in part on one or more scores for the one or more entity references linked to the ghost entity, wherein the identifying the ghost entity as an actual entity comprises associating a same definitive identifier with the one or more entity references linked to the ghost entity.

7. The computer readable storage medium as in claim 6, wherein the ghost entity is identified as an actual entity based in part on a comparison of a sum of the scores with a defined threshold.

8. The computer readable storage medium as in claim 6, further comprising instructions manipulated by the processor to adjust the score for the entity reference based at least in part on one or more grading criteria.

9. The computer readable storage medium as in claim 6, wherein the score for the entity reference is further based upon a parameter selected from a group consisting of:

an average field value for one or more data fields of the one or more entity references linked to the entity;

a weighted average field value for one or more data fields of the one or more entity references linked to the entity;

a median field value for one or more data fields of the one or more entity references linked to the entity; and

a field value selected at random from the one or more entity references linked to the entity.

10. A system for identifying an entity from a plurality of entity references, each entity reference being linked with a separate ghost entity, the system comprising:

memory;

a processor operably connected to the memory; and

a set of executable instructions stored in the memory and manipulated by the processor to;

compare an entity reference of a first ghost entity with an entity reference of a second ghost entity to determine a match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity, wherein the match probability between the entity reference of the first ghost entity and the entity reference of the second ghost entity comprises a content weight W_c,i for a particular data field i, wherein the content weight W_c,i for a particular data field i is inversely related to a sum of a total number of occurrences of a data field value and a cautiousness value;

assign a definitive identifier to an entity reference based on the match probability;

link the entity reference of the first ghost entity additionally with the second ghost entity and the entity reference of the second ghost entity additionally with the first ghost entity when the match probability is greater than or equal to a match threshold;

repeat steps of the comparing and the linking for one or more ghost entity pairings possible from the ghost entities;

determine, for one or more entity references linked to a ghost entity, a score for an entity reference based at least in part on a match probability between an entity reference and an entity and a match probability between the entity reference and the ghost entity; and

identify the ghost entity as an actual entity based at least in part on one or more scores for the one or more entity references linked to the ghost entity, wherein the identifying the ghost entity as an actual entity comprises associating a same definitive identifier with the one or more entity references linked to the ghost entity.

11. The system as in claim 10, wherein the ghost entity is identified as an actual entity based in part on a comparison of a sum of the scores with a defined threshold.

12. The system as in claim 10, further comprising instructions manipulated by the processor to adjust the score for the entity reference based at least in part on one or more grading criteria.

13. The system as in claim 10, wherein the score for the entity reference is further based upon a parameter selected from a group consisting of:

an average field value for one or more data fields of the one or more entity references linked to the entity;

a weighted average field value for one or more data fields of the one or more entity references linked to the entity;

a median field value for one or more data fields of the one or more entity references linked to the entity; and

a field value selected at random from the one or more entity references linked to the entity.