Best-value determination rules for an entity resolution system

• US 9,910,875 B2
• Filed: 12/22/2008
• Issued: 03/06/2018
• Est. Priority Date: 12/22/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Caceres, Barry M.
• Primary Examiner(s): Uddin, Mohammed R
• Application Number:
  US12/341,526
• Publication Number:
  US 20100161634A1
• Time in Patent Office: 3,361 Days
• Field of Search: 707692, 707 90, 707694, 707701, 707999101
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409

Abstract

Primary value determination rules may be used by entity resolution system to select a “best” or “primary” value of an attribute from a plurality of attribute values. For example, the “best” name, address, phone number, etc. to use in presenting a summary of information about that entity may be determined. Further, the primary value determination rules may each be configured to assign a confidence score to the “best” values selected for of a given entity. Doing so allows a selection of a “best” value for a given attribute made by one rule to be overridden by a selection of another “best” value made by another rule for that same attribute.

Claims

1. A computer-implemented method to reconcile distinct candidate values in an entity resolution system storing distinct identity records that resolve to a plurality of entities, the computer-implemented method comprising:

receiving a selection of an entity from the plurality of entities stored in the entity resolution system, wherein each entity is associated with a respective plurality of distinct identity records that previously resolved to the respective entity based on one or more entity resolution rules, wherein each identity record includes two or more attribute types and attribute values, wherein the attribute values in each identity record includes information corresponding to one individual, wherein a relationship is identified between at least two of the plurality of entities;

subsequent to resolving the plurality of distinct identity records to the selected entity based on the one or more entity resolution rules, evaluating the plurality of distinct identity records of the selected entity against a plurality of representative value determination rules distinct from the one or more entity resolution rules, wherein each representative value determination rule has a respective rule name and specifies to evaluate a respective attribute type using a distinct criterion for selecting a representative value, wherein the evaluation for each representative value determination rule comprises;

identifying two or more attribute types associated with the respective representative value determination rule, the representative value determination rule specifying to qualify a first attribute type by a specified value of a second attribute type different from the first attribute type, wherein at least two of the representative value determination rules are associated with at least the first attribute types;

identifying a set of attribute values, stored in the plurality of distinct identity records of the selected entity, that correspond to the first attribute type qualified by the specified value of the second attribute type;

determining whether the representative value determination rule applies to the entity, based on the identified set of attribute values;

upon determining that the representative value determination value rule applies to the entity, selecting, from the identified set of attribute values, a respective, distinct, candidate value to represent the first attribute types qualified by the specified value of the second attribute type, wherein each of the at least two representative value determination rules assigns a respective, distinct confidence level to the candidate value selected by the respective representative value determination rule;

determining, by operation of one or more computer processors, the candidate value having the highest confidence level among the selected candidate values;

designating only the candidate value as a sole value representing the first attribute type qualified by the specified value of the second attribute type, including storing an indication of the sole value as representing the first attribute type; and

retaining each of the identified set of attribute values, represented by the sole value, as part of the plurality of distinct identity records of the selected entity without merging or replacing the identified set of attribute values in the plurality of distinct identity records in the selected entity; and

generating an entity display summary including the sole value representing the first attribute type qualified by the specified value of the second attribute type, whereafter the entity display summary is output.

2. The computer-implemented method of claim 1, wherein two or more of the plurality of representative value determination rules are applied in a specified order determined from a ranking value assigned to two or more representative value determination rules.

3. The computer-implemented method of claim 1, further comprising:

receiving a first identity record;

resolving the first identity record to one of the plurality of entities;

associating the first identity record; and

upon determining that the identity record satisfies a relevance detection rule, generating an alert, wherein the alert includes the representative value selected for the first attribute type qualified by the specified value of the second attribute type.

4. The computer-implemented method of claim 1, wherein the relationship is selected from an express relationship and an implied relationship, wherein a respective, qualifying criterion is includable in each representative value determination rule, wherein each of the respective plurality of identity records for each of the plurality of entities is evaluated against the at least two representative value determination rules, wherein the computer-implemented method further comprises providing:

(i) a first representative value determination rule that applies to the first attribute type of the respective identity record of the respective entity using a first confidence level; and

(ii) a second representative value determination rule that applies, to the first attribute type of the respective identity record of the respective entity using a second confidence level.

5. The computer-implemented method of claim 4, wherein the respective plurality of identity records for each of the plurality of entities together constitute an available set of identity records, wherein the plurality of representative value determination rules includes:

(iii) a third representative value determination rule that applies, to the second attribute type of only a first subset of the available set of identity records, a first qualifying criterion and using a third confidence level, wherein the first subset is selected based on the second attribute type having a first value specified by the first qualifying criterion.

6. The computer-implemented method of claim 5, wherein the plurality of representative value determination rules includes further includes:

(iv) a fourth representative value determination rule that applies, to the second attribute type of only a second subset of the available set of identity records, a second qualifying criterion and using a fourth confidence level, wherein the first subset is selected based on the second attribute type having a second value specified by the second qualifying criterion;

wherein the first and second values are distinct, wherein the first and second criteria are distinct, wherein the first, second, third, and fourth confidence levels are distinct, wherein the first and second subsets are mutually exclusive and are each less than all of the available set of identity records.

7. The computer-implemented method of claim 6, wherein each representative value determination rule specifies a respective plurality of distinct properties including:

(i) the rule name;

(ii) a processing rank;

(iii) the attribute type;

(iv) the criterion, which comprises a detection technique;

(v) the confidence level; and

(v) an indication of what, if any, qualifying criterion to apply to identity records.

8. The computer-implemented method of claim 7, wherein the first representative value determination rule specifies:

(i) a rule name indicating to determine a most suitable phone number;

(ii) a processing rank that is greater than that of the second representative value determination rule but that is less than that of the fourth representative value determination rule;

(iii) an attribute type of phone number;

(iv) a detection technique indicating to determine a most common value within a time period measured until two years prior;

(v) a confidence level lowest among the first, second, third, and fourth representative value determination rules; and

(vi) no qualifying criterion to apply.

9. The computer-implemented method of claim 8, wherein the second representative value determination rule specifies:

(i) a rule name indicating to determine a latest phone number;

(ii) a processing rank lowest among the first, second, third, and fourth representative value determination rules;

(iii) an attribute type of phone number;

(iv) a detection technique indicating to determine the phone number provided in a most recent identity record of the each entity;

(v) a confidence level highest among the first, second, third, and fourth representative value determination rules; and

(vi) no qualifying criterion to apply.

10. The computer-implemented method of claim 9, wherein the third representative value determination rule specifies:

(i) a rule name indicating to determine a representative full name;

(ii) a processing rank that is greater than that of the second representative value determination rule but that is less than that of the fourth representative value determination rule, wherein the processing rank is identical to that of the first representative value determination rule; and

(iii) an attribute type of full name.

11. The computer-implemented method of claim 10, wherein the third representative value determination rule further specifies:

(iv) a detection technique indicating to determine a most common value by exact match;

(v) a confidence level second highest among the first, second, third, and fourth representative value determination rules; and

(vi) the first qualifying criterion, which limits applicability of the third representative value determination rule only to identity records having a gender attribute type that matches a male gender value.

12. The computer-implemented method of claim 11, wherein the fourth representative value determination rule specifies:

(i) a rule name indicating to determine one or more representative names;

(ii) a processing rank highest among the first, second, third, and fourth representative value determination rules; and

(iii) an attribute type of names including full names, given names, and surnames.

13. The computer-implemented method of claim 12, wherein the fourth representative value determination rule further specifies:

(iv) a detection technique indicating to determine a most common value by exact match;

(v) a confidence level second lowest among the first, second, third, and fourth representative value determination rules; and

(vi) the second qualifying criterion, which limits applicability of the fourth representative value determination rule only to identity records having a gender attribute type that matches a female gender value.

14. The computer-implemented method of claim 13, wherein the first, second, third, and fourth representative value determination rules specify at least three distinct detection techniques, wherein each representative value determination rule is applied in a specified order relative to other representative value determination rules, wherein the specified order is determined based on the processing rank of the respective representative value determination rule.

15. The computer-implemented method of claim 14, wherein the relationship is, in respective instances, the express relationship and the implied relationship, wherein each representative value determination rule is applied by an entity resolution application executing on a computing system, wherein the computing system includes the one or more computer processors, wherein the computer-implemented method further comprises:

receiving a first identity record;

resolving the first identity record to one of the plurality of entities by the entity resolution application;

associating the first identity record; and

upon determining that the identity record satisfies a relevance detection rule distinct from each representative value determination rule, generating an alert by an alert analysis application executing on the computing system, wherein the alert analysis application is distinct from the entity resolution application, wherein the alert includes the candidate value selected for the first attribute type qualified by the specified value of the second attribute type, whereafter the alert is output.

16. A computer program product to reconcile distinct candidate values in an entity resolution system storing distinct identity records that resolve to a plurality of entities, the computer program product comprising:

a non-transitory computer-readable medium having computer-usable program code embodied therewith, the computer-usable program code executable by one or more computer processors to;

receive a selection of an entity from the plurality of entities stored in the entity resolution system, wherein each entity is associated with a respective plurality of distinct identity records that previously resolved to the respective entity based on one or more entity resolution rules, wherein each identity record includes two or more attribute types and attribute values, wherein the attribute values in each identity record includes information corresponding to one individual, wherein a relationship is identified between at least two of the plurality of entities;

subsequent to resolving the plurality of distinct identity records to the selected entity based on the one or more entity resolution rules, evaluate the plurality of distinct identity records of the selected entity against a plurality of representative value determination rules distinct from the one or more entity resolution rules, wherein each representative value determination rule has a respective rule name and specifies to evaluate a respective attribute type using a distinct criterion for selecting a representative value, wherein the evaluation for each representative value determination rule comprises;

identifying two or more attribute types associated with the respective representative value determination rule, the representative value determination rule specifying to qualify a first attribute type by a specified value of a second attribute type different from the first attribute type, wherein at least two of the representative value determination rules are associated with at least the first attribute types;

identifying a set of attribute values, stored in the plurality of distinct identity records of the selected entity, that correspond to the first attribute types;

qualified by the specified value of the second attribute type;

determining whether the representative value determination rule applies to the entity, based on the identified set of attribute values;

upon determining that the representative value determination value rule applies to the entity, selecting, from the identified set of attribute values, a respective, distinct, candidate value to represent the first attribute type qualified by the specified value of the second attribute type, wherein each of the at least two representative value determination rules assigns a respective, distinct confidence level to the candidate value selected by the respective representative value determination rule;

determining the candidate value having the highest confidence level among the selected candidate values;

designating only the candidate value as a sole value representing the first attribute types qualified by the specified value of the second attribute type, including storing an indication of the sole value as representing the first attribute type; and

retaining each of the identified set of attribute values, represented by the sole value, as part of the plurality of distinct identity records of the selected entity without merging or replacing the identified set of attribute values in the plurality of distinct identity records in the selected entity; and

generate an entity display summary including the sole value representing the first attribute types qualified by the specified value of the second attribute type, whereafter the entity display summary is output.

17. The computer program product of claim 16, wherein two or more of the plurality of representative value determination rules are applied in a specified order determined from a ranking value assigned to two or more representative value determination rules.

18. The computer program product of claim 16, wherein the computer-useable program code is further executable to:

receive a first identity record;

resolve the first identity record to one of the plurality of entities;

associate the first identity record; and

upon determining that the identity record satisfies a relevance detection rule, generate an alert, wherein the alert includes the candidate value selected for the first attribute type qualified by the specified value of the second attribute type.

19. A system to reconcile distinct candidate values in an entity resolution system storing distinct identity records that resolve to a plurality of entities, the system comprising:

one or more computer processors; and

a memory containing a program which, when executed by the one or more computer processors, is configured to perform an operation comprising;

receiving a selection of an entity from the plurality of entities stored in the entity resolution system, wherein each entity is associated with a respective plurality of distinct identity records that previously resolved to the respective entity based on one or more entity resolution rules, wherein each identity record includes two or more attribute types and attribute values, wherein the attribute values in each identity record includes information corresponding to one individual, wherein a relationship is identified between at least two of the plurality of entities;

subsequent to resolving the plurality of distinct identity records to the selected entity based on the one or more entity resolution rules, evaluating the plurality of distinct identity records of the selected entity against a plurality of representative value determination rules distinct from the one or more entity resolution rules, wherein each representative value determination rule has a respective rule name and specifies to evaluate a respective attribute type using a distinct criterion for selecting a representative value, wherein the evaluation for each representative value determination rule comprises;

identifying two or more attribute types associated with the representative value determination rule, the representative value determination rule specifying to qualify a first attribute type by a specified value of a second attribute type different from the first attribute type, wherein at least two of the representative value determination rules are associated with at least the first attribute types;

identifying a set of attribute values, stored in the plurality of distinct identity records of the selected entity, that correspond to the first attribute type qualified by the specified value of the second attribute type;

determining whether the representative value determination rule applies to the entity, based on the identified set of attribute values;

upon determining that the representative value determination value rule applies to the entity, selecting, from the identified set of attribute values, a respective, distinct, candidate representative value to represent the first attribute type qualified by the specified value of the second attribute type, wherein each of the at least two representative value determination rules assigns a respective, distinct confidence level to the candidate value selected by the respective representative value determination rule;

determining the candidate value having the highest confidence level among the selected candidate values;

designating only the candidate value as a sole value representing the first attribute types qualified by the specified value of the second attribute type, including storing an indication of the sole value as representing the first attribute types; and

retaining each of the identified set of attribute values, represented by the sole value, as part of the plurality of distinct identity records of the selected entity without merging or replacing the identified set of attribute values in the plurality of distinct identity records in the selected entity; and

generating an entity display summary including the sole value representing the first attribute type qualified by the specified value of the second attribute type, whereafter the entity display summary is output.

20. The system of claim 19, wherein two or more of the plurality of representative value determination rules are applied in a specified order determined from a ranking value assigned to two or more representative value determination rules.

21. The system of claim 19, wherein the operation further comprises:

receiving a first identity record;

resolving the first identity record to one of the plurality of entities;

associating the first identity record; and

upon determining that the identity record satisfies a relevance detection rule, generating an alert, wherein the alert includes the candidate value selected for the first attribute type qualified by the specified value of the second attribute type.