Statistical measure and calibration of reflexive, symmetric and transitive fuzzy search criteria where one or both of the search criteria and database is incomplete

• US 8,190,616 B2
• Filed: 07/02/2009
• Issued: 05/29/2012
• Est. Priority Date: 07/02/2008
• Status: Active Grant

• Current Assignee: LexisNexis Risk Solutions, Inc. (RELX PLC)
• Original Assignee: LexisNexis Risk & Information Analytics Group, Inc. (RELX PLC)
• Inventors: Bayliss, David Alan
• Primary Examiner(s): Betit, Jacob F
• Assistant Examiner(s): HUANG, MIRANDA M
• Application Number:
  US12/496,929
• Publication Number:
  US 20100005091A1
• Time in Patent Office: 1,062 Days
• Field of Search: None
• US Class Current: 707/748
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24578 using ranking

Abstract

Disclosed is a system for, and method of, searching for and identifying an entity representation. Some embodiments utilize a reflexive, symmetric and transitive function to allow for non-identical matches between field values. The function may be used to generate field value codes, which are associated with a portion of a field value weight for the original field value. In such embodiments, the field value weight for the original field values may be distributed among the original field value and the associated field value code.

Claims

1. A method of identifying, using a search criteria, an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight which is calculated based on the data value stored in the field of the record, wherein the search criteria comprises at least one field value that is not identical to a field value in a record in an entity representation that is identified by the method, the method comprising:

selecting a field;

applying a symmetric, reflexive and transitive function to each field value in the selected field of each of a plurality of records, whereby a plurality of field value codes are generated, and whereby applying the symmetric, reflexive and transitive function to each field value in the selected field of each of a plurality of records in the database defines a partition of the plurality of records;

populating a field of each of the plurality of records with a field value code;

computing a field value weight for each field value code;

distributing, for each record, a field value weight associated with a field value in the selected field, among the field value in the selected field and a field value code, wherein the distributing comprises, for each record of the plurality of records, calculating a difference between a field value weight associated with a field value in the selected field and a field value weight for a field value code;

receiving a plurality of search criteria field values;

determining a highest ranked entity representation according to summed field value weights for field values matching the plurality of search criteria field values;

calculating a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the plurality of search criteria field values; and

outputting, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

2. The method of claim 1, wherein the distributing further comprises, for each record in the plurality of records, replacing a field value weight associated with a field value in the selected field by the difference.

3. The method of claim 1, wherein each field value weight comprises a logarithm of a probability that an arbitrary entity representation in the database comprises a corresponding field value in a field of a record in the arbitrary entity representation.

4. The method of claim 1, wherein the populating comprises adding a field to each record of the plurality of records.

5. The method of claim 4, wherein the populating further comprises, for each record of the plurality of records, replacing a field value in the selected field by a field value code, and wherein the populating further comprises, for each record in the plurality of records, moving a field value from the selected field to the added field.

6. The method of claim 1 further comprising, after the step of populating, storing field value weights for each field value code in each record.

7. The method of claim 1 further comprising computing a match template partition.

8. The method of claim 1, wherein the outputting an identifier comprises outputting the identifier to a user in a human readable form.

9. The method of claim 1, wherein the outputting an identifier comprises outputting the identifier to a computer configured to perform a search for an entity representation in the database based on the search criteria.

10. The method of claim 1, wherein the identifier comprises a social security number.

11. A system of identifying, using a search criteria, an entity representation in an electronic universal database that corresponds to an entity representation in an electronic foreign database, each database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight which is calculated based on the data value stored in the field of the record, wherein the search criteria comprises at least one field value that is not identical to a field value in a record in an entity representation that is identified, the system comprising:

an electronic database comprising a plurality of entity representations, each entity representation comprising a plurality of linked records, each record comprising a plurality of fields, each field capable of containing a field value, each field value associated with a field value weight;

a processor programmed to apply a symmetric, reflexive and transitive function to each field value in a selected field of each of a plurality of records, whereby a plurality of field value codes are generated and electronically stored, and whereby applying the symmetric, reflexive and transitive function to each field value in the selected field of each of a plurality of records in the database defines a partition of the plurality of records;

a processor programmed to store, in a field of each of the plurality of records, a field value code;

a processor programmed to compute a field value weight for each field value code;

a processor programmed to distribute electronic storage of, for each record, a field value weight associated with a field value in the selected field, among an electronic storage of a field value weight for the field value in the selected field and an electronic storage of the field value weight of the field value, wherein the distributing comprises, for each record of the plurality of records, calculating a difference between a field value weight associated with a field value in the selected field and a field value weight for a field value code;

an electronic memory storing a plurality of search criteria field values;

a processor programmed to determine a highest ranked entity representation according to summed field value weights for field values matching the plurality of search criteria field values;

a processor programmed to calculate a confidence level reflecting a likelihood that the highest ranked entity representation corresponds to the plurality of search criteria field values; and

an output configured to output, if the confidence level exceeds a predetermined threshold, an identifier for the highest ranked entity representation.

12. The system of claim 11, wherein the processor programmed to distribute electronic storage is further programmed to, for each record in the plurality of records, replace a field value weight associated with a field value in the selected field by the difference.

13. The system of claim 12, wherein each field value weight comprises a logarithm of a probability that an arbitrary entity representation in the database comprises a corresponding field value in a field of a record in the arbitrary entity representation.

14. The system of claim 11, further comprising a processor programmed to add a field to each record of the plurality of records.

15. The system of claim 14, further comprising a processor programmed to, for each record of the plurality of records, replace a field value in the selected field by a field value code and move a field value from a storage of the selected field to a storage of the added field.

16. The system of claim 11 further comprising an electronic storage of field value weights for each field value code in each record.

17. The system of claim 11 further comprising an electronic storage of a match template partition.

18. The system of claim 11, wherein the output comprises a human readable display.

19. The system of claim 11, wherein the output comprises an electronic interface coupled to a computer configured to perform a search for an entity representation in the database based on the search criteria.

20. The system of claim 11, wherein the identifier comprises a social security number.