Method and system for filtering false positives

• US 8,589,415 B2
• Filed: 01/14/2010
• Issued: 11/19/2013
• Est. Priority Date: 09/15/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Adams, Norm, Ellard, Scott, Schumacher, Scott
• Primary Examiner(s): LIAO, JASON G
• Application Number:
  US12/687,304
• Publication Number:
  US 20100114877A1
• Time in Patent Office: 1,405 Days
• Field of Search: None
• US Class Current: 707/749
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

Embodiments of systems and methods for reducing false positives during the linking of data records are disclosed herein. Broadly speaking, embodiments of the present invention may be used in the generation of an overall weight from the comparison of various attributes of data records, where the linking of the data records is dependent on the overall weight. More specifically, embodiments of the present invention may calculate a false positive penalty based on a set of results, each of the set of results based on a comparison of an attribute. The false positive penalty may be subtracted from the overall weight generated from the comparison of the attributes of data records to adjust the overall weight. By configuring which attributes of the data records are used as the set of attributes for generating the false positive penalty, and the penalties associated with a particular combination of results for the comparisons of these attributes, the incidence of false positives in the linking of data records may be significantly reduced.

Claims

1. A method for association of data records, comprising:

providing a system comprising an identity hub running an identity hub engine, the identity hub coupled to one or more external data sources through one or more networks, each external data source at a corresponding database;

receiving a first data record and a second data record from the one or more external data sources at the identity hub;

obtaining a first set of results, wherein each of the first set of results is a value generated based on a comparison between one of a first set of attributes from the first data record and the second data record, wherein the comparison between each of the first set of attributes is performed by the identity hub engine;

determining a first overall weight for a comparison between the first data record and the second data record using the first set of results;

generating a first false positive penalty based on the first set of results, wherein the first false positive penalty is associated with the comparison of the first data record and the second data record, and wherein each possible permutation of the first set of results corresponds to a different false positive penalty;

adjusting the first overall weight to reduce a likelihood of incorrect linking of the first data record and second data record; and

determining whether the first data record and the second data record should be linked based on the adjusted first overall weight.

2. The method of claim 1, wherein the first overall weight is determined using a second set of results wherein each of the second set of results is a value generated based on a comparison between one of a second set of attributes from the first data record and the second data record.

3. The method of claim 1, wherein the first false positive penalty is zero.

4. The method of claim 1, wherein the first set of attributes to use in determining the first false positive penalty was configured by a user.

5. The method of claim 1, further comprising:

obtaining a second set of results, wherein each of the second set of results is based on a comparison between one of the first set of attributes from a third data record and a fourth data record, the second set of results differing from the first set of results, wherein obtaining the second set of results is performed by the identity hub engine;

determining a second overall weight for a comparison between the third data record and the fourth data record using the second set of results;

generating a second false positive penalty based on the second set of results wherein the second false positive penalty is associated with the comparison of the third data record and the fourth data record;

adjusting the second overall weight to reduce the likelihood of the incorrect linking of the third data record and fourth data record; and

determining whether the third data record and the fourth data record should be linked based on the adjusted second overall weight.

6. A computer readable storage media, comprising instructions translatable for implementing an identity hub engine on an identity hub the identity hub coupled to one or more external data sources through one or more networks, each external data source at a corresponding database the identity hub engine operable for:

receiving a first data record and a second data record from the one or more external data sources at the identity hub;

obtaining a first set of results, wherein each of the first set of results is a value generated based on a comparison between one of a first set of attributes from the first data record and the second data record, wherein the comparison between each of the first set of attributes is performed by the identity hub engine;

determining a first overall weight for a comparison between the first data record and the second data record using the first set of results;

generating a first false positive penalty based on the first set of results, wherein the first false positive penalty is associated with the comparison of the first data record and the second data record, and wherein each possible permutation of the first set of results corresponds to a different false positive penalty;

adjusting the first overall weight to reduce a likelihood of incorrect linking of the first data record and second data record; and

determining whether the first data record and the second data record should be linked based on the adjusted first overall weight.

7. The computer readable storage media of claim 6, wherein the first overall weight is determined using a second set of results wherein each of the second set of results is a value generated based on a comparison between one of a second set of attributes from the first data record and the second data record.

8. The computer readable storage media of claim 6, wherein the first false positive penalty is zero.

9. The computer readable storage media of claim 6, wherein the first set of attributes to use in determining the first false positive penalty was configured by a user.

10. The computer readable storage media of claim 6, wherein the identity hub engine is operable for:

obtaining a second set of results, wherein each of the second set of results is based on a comparison between one of the first set of attributes from a third data record and a fourth data record, the second set of results differing from the first set of results, wherein obtaining the second set of results is performed by the identity hub engine;

determining a second overall weight for a comparison between the third data record and the fourth data record using the second set of results;

generating a second false positive penalty based on the second set of results wherein the second false positive penalty is associated with the comparison of the third data record and the fourth data record;

adjusting the second overall weight to reduce the likelihood of the incorrect linking of the third data record and fourth data record; and

determining whether the third data record and the fourth data record should be linked based on the adjusted second overall weight.

11. A system for the linking of data records, comprising:

one or more data sources, each data source at a corresponding database; and

an identify hub linked to the one or more data sources through one or more networks, wherein the identity hub comprising a computer readable medium including instructions operable for implementing an identity hub engine for;

receiving a first data record and a second data record from the one or more external data sources at the identity hub;

obtaining a first set of results, wherein each of the first set of results is a value generated based on a comparison between one of a first set of attributes from the first data record and the second data record, wherein the comparison between each of the first set of attributes is performed by the identity hub engine;

determining a first overall weight for a comparison between the first data record and the second data record using the first set of results;

generating a first false positive penalty based on the first set of results, wherein the first false positive penalty is associated with the comparison of the first data record and the second data record, and wherein each possible permutation of the first set of results corresponds to a different false positive penalty;

adjusting the first overall weight to reduce a likelihood of incorrect linking of the first data record and second data record; and

determining whether the first data record and the second data record should be linked based on the adjusted first overall weight.

12. The system of claim 11, wherein the first overall weight is determined using a second set of results wherein each of the second set of results is a value generated based on a comparison between one of a second set of attributes from the first data record and the second data record.

13. The system of claim 11, wherein the first false positive penalty is zero.

14. The system of claim 11, wherein the first set of attributes to use in determining the first false positive penalty was configured by a user.

15. The system of claim 11, wherein the identity hub engine is operable for:

obtaining a second set of results, wherein each of the second set of results is based on a comparison between one of the first set of attributes from a third data record and a fourth data record, the second set of results differing from the first set of results, wherein obtaining the second set of results is performed by the identity hub engine;

determining a second overall weight for a comparison between the third data record and the fourth data record using the second set of results;

generating a second false positive penalty based on the second set of results wherein the second false positive penalty is associated with the comparison of the third data record and the fourth data record;

adjusting the second overall weight to reduce the likelihood of the incorrect linking of the third data record and fourth data record; and

determining whether the third data record and the fourth data record should be linked based on the adjusted second overall weight.