Statistical measure and calibration of search criteria where one or both of the search criteria and database is incomplete

• US 8,495,076 B2
• Filed: 11/18/2011
• Issued: 07/23/2013
• Est. Priority Date: 07/02/2008
• Status: Active Grant

• Current Assignee: LexisNexis Risk Solutions, Inc. (RELX PLC)
• Original Assignee: LexisNexis Risk Solutions, Inc. (RELX PLC)
• Inventors: Bayliss, David Alan
• Primary Examiner(s): Kim, Charles
• Assistant Examiner(s): HUANG, MIRANDA M
• Application Number:
  US13/299,968
• Publication Number:
  US 20120072417A1
• Time in Patent Office: 613 Days
• Field of Search: None
• US Class Current: 707/758
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24578 using ranking

Abstract

Disclosed is a system for, and method of, identifying an entity representation. In some embodiments, search criteria are used to identify an entity representation in a universal database, and this identification is then used to identify a corresponding entity representation in a foreign database. Certain embodiments provide assurance, with a know probability of error, that the entity representation identified in the universal database is correct.

Claims

1. A method of identifying an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight, the method comprising:

receiving a plurality of search criteria field values identifying an entity representation in the foreign database;

ranking entity representations in the universal database according to summed field value weights, wherein each summed field value weight comprises weights corresponding to field values, from records within a same entity representation from the universal database, that match a search criteria field value, and wherein, for each summed field value weight, each weight corresponding to field values is counted at most once for each search criteria field value;

determining a highest ranked entity representation;

calculating a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the entity representation identified by the search criteria field values;

wherein the calculating a confidence level comprises a sum of terms, each term comprising an exponent, each exponent comprising a difference between a summed weight for the highest ranked entity reference and a summed weight for another entity representation; and

outputting, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

2. The method of claim 1, wherein each field value weight for the universal database comprises a logarithm of a probability that an arbitrary entity representation in the universal database comprises a corresponding field value in a field of a record in the arbitrary entity representation.

3. The method of claim 1, wherein the calculating a confidence level comprises calculating a difference between a summed weight for the highest ranked entity reference and a summed weight for a next highest ranked entity reference, and wherein the threshold comprises a logarithm of a term comprising a confidence level.

4. The method of claim 1, wherein the sum is over an initial portion of highest ranked entity representations.

5. A method of identifying an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight, the method comprising:

receiving a plurality of search criteria field values identifying an entity representation in the foreign database;

ranking entity representations in the universal database according to summed field value weights, wherein each summed field value weight comprises weights corresponding to field values, from records within a same entity representation from the universal database, that match a search criteria field value, and wherein, for each summed field value weight, each weight corresponding to field values is counted at most once for each search criteria field value;

determining a highest ranked entity representation;

calculating a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the entity representation identified by the search criteria field values;

wherein the calculating a confidence level comprises calculating a difference between a summed weight for the highest ranked entity reference and a term comprising a difference between a number of entities reflected in the universal database and a number of entities reflected in the foreign database, and wherein the threshold comprises a logarithm of a term comprising a confidence level; and

outputting, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

6. A system for identifying an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight, the system comprising:

an electronic universal database comprising a plurality of electronically stored entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight;

an electronic memory storing a plurality of search criteria field values identifying an entity representation in the foreign database;

a processor programmed to rank entity representations in the universal database according to summed field value weights, wherein each summed field value weight comprises weights corresponding to field values, from records within a same universal entity representation, that match a search criteria field value, and wherein, for each summed field value weight, each weight corresponding to field values is counted at most once for each search criteria field value;

a processor programmed to determine a highest ranked universal entity representation;

a processor programmed to calculate a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the entity representation identified by the search criteria field values;

wherein the calculating a confidence level comprises a sum of terms, each term comprising an exponent, each exponent comprising a difference between a summed weight for the highest ranked entity reference and a summed weight for another entity representation; and

a processor programmed to output, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

7. The system of claim 6, wherein each field value weight for the universal database comprises a logarithm of a probability that an arbitrary entity representation in the universal database comprises a corresponding field value in a field of a record in the arbitrary entity representation.

9. The system of claim 6, wherein the sum is over an initial portion of highest ranked entity representations.

11. The system of claim 6, wherein the identifier comprises a social security number.

8. A system for identifying an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight, the system comprising:

an electronic universal database comprising a plurality of electronically stored entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight;

an electronic memory storing a plurality of search criteria field values identifying an entity representation in the foreign database;

a processor programmed to rank entity representations in the universal database according to summed field value weights, wherein each summed field value weight comprises weights corresponding to field values, from records within a same universal entity representation, that match a search criteria field value, and wherein, for each summed field value weight, each weight corresponding to field values is counted at most once for each search criteria field value;

a processor programmed to determine a highest ranked universal entity representation;

a processor programmed to calculate a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the entity representation identified by the search criteria field values, wherein the processor configured to calculate a confidence level is further configured to calculate a difference between a summed weight for the highest ranked entity reference and a summed weight for a next highest ranked entity reference, and wherein the threshold comprises a logarithm of a term comprising a confidence level and;

a processor programmed to output, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

10. A system for identifying an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight, the system comprising:

an electronic universal database comprising a plurality of electronically stored entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight;

an electronic memory storing a plurality of search criteria field values identifying an entity representation in the foreign database;

a processor programmed to rank entity representations in the universal database according to summed field value weights, wherein each summed field value weight comprises weights corresponding to field values, from records within a same universal entity representation, that match a search criteria field value, and wherein, for each summed field value weight, each weight corresponding to field values is counted at most once for each search criteria field value;

a processor programmed to determine a highest ranked universal entity representation;

a processor programmed to calculate a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the entity representation identified by the search criteria field values, wherein the processor configured to calculate a confidence level is further configured to calculate a difference between a summed weight for the highest ranked entity reference and a term comprising a difference between a number of entities reflected in the universal database and a number of entities reflected in the foreign database, and wherein the threshold comprises a logarithm of a term comprising a confidence level, and;

a processor programmed to output, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.