SYSTEM AND METHOD FOR ASSESSING DATA ACCURACY

• US 20160364427A1
• Filed: 06/09/2015
• Published: 12/15/2016
• Est. Priority Date: 06/09/2015
• Status: Active Grant

• Current Assignee: Early Warning Services, LLC
• Original Assignee: Early Warning Services, LLC
• Inventors: Wedgeworth, Frederick L. III
• Granted Patent: US 9,910,905 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Data from a plurality of data sources is provided to a multi-source data management system, which stores the data and provides it to a data accuracy system for purposes of assessing the accuracy of data records and the individual fields within data records. Data accuracy scores may be stored at the data management system with the data records to which they pertain. Accuracy scores may be periodically recalculated and monitored, and alerts provided if an accuracy score changes a predetermined amount over a given period of time. Also, data records may be provided by a data user for accuracy assessment, using other data records stored at the multi-sourced data management system.

Claims

1. A method for evaluating the accuracy of a data record, comprising:

storing, at a data storage system, a plurality of data records received from a plurality of different data sources, wherein the data records have varying degrees of trustworthiness, and wherein each of the stored data record are associated with an entity;

receiving a data record based on a specified entity, the received data record to be evaluated for accuracy;

retrieving a group of data records from the stored plurality for data records, each retrieved data record associated with the specified entity, each retrieved data record having a data field corresponding to a data field of the data record to be evaluated, and each retrieved data record associated with one of the data sources; and

calculating, at a data accuracy system, an accuracy score for the data field of the data record to be evaluated, comprising;

determining a trustworthiness weight for at least each one of the retrieved group of data records based on the trustworthiness of the data source providing each one of the group of data records;

determining a matching score for at least the corresponding data field in each of the retrieved group of data records, based on the degree of similarity between (1) data in the corresponding data field in each of the retrieved group of data records and (2) data in the data field of the data record to be evaluated; and

combining, for the corresponding data field in each one of the retrieved group of data records, (1) the trustworthiness weight for that one of the retrieved group of records, with (2) the matching score for the corresponding data field in that one of retrieved group of data records; and

summing together the combined trustworthiness weight and matching score for every corresponding data field.

2. The method of claim 1, further comprising:

determining a trustworthiness weight for the data record to be evaluated;

determining a matching score for the data field of the data record to be evaluated;

combining the trustworthiness weight for the data record to be evaluated and the matching score for the data field of the data record to be evaluated; and

summing, with the previously summed trustworthiness weight and matching score for every corresponding data field, the combined trustworthiness weight for the data record to be evaluated and the matching score for the data field of the data record to be evaluated.

3. The method of claim 2, wherein the matching score for the data field of the data record to be evaluated is assigned a predetermined value.

4. The method of claim 2, wherein the assigned predetermined value is 1.

5. The method of claim 1, further comprising:

storing, in the data management system, the accuracy score in association with the corresponding data field of the data record for which the accuracy score was calculated.

6. The method of claim 1, further comprising:

calculating an accuracy score for each of other data fields in the data record to be evaluated.

7. The method of claim 6, further comprising:

calculating an overall accuracy score for the data record to be evaluated, by combining accuracy scores for individual data fields of the data record.

8. The method of claim 7, further comprising:

calculating at accuracy score for an identified entity, by combining accuracy scores for a plurality of data records associated with the identified entity.

9. The method of claim 7, further comprising:

calculating an accuracy score for one of the plurality of data sources, by combining accuracy scores for a plurality of data records that are provided by that one of the plurality of data sources.

10. The method of claim 1, further comprising:

monitoring, at the data accuracy system, the accuracy score, including;

waiting a predetermined period of time after calculating the accuracy score,

re-calculating the accuracy score; and

comparing the re-calculated accuracy score to a previously calculated accuracy score.

11. The method of claim 1, wherein the received data record associated with the specified entity is provided with a query from a data user, and wherein the method further comprises:

providing the calculated accuracy score to the data user as a response to the query.

12. The method of claim 1, wherein the plurality of different data sources are selected from a group comprising:

Published Research;

News;

Wikipedia;

Social Media;

Financial Transactions;

Financial Account Ownership Data;

Government Benefits/Licensing Records;

Court Records;

Voter Registration; and

Aggregated Public Data.

13. The method of claim 1, wherein the received data record associated with the specified entity is received with other data records from a financial institution, wherein the calculated accuracy score is provided to the financial institution, and wherein the calculated accuracy score is used by the financial institution to assess the accuracy of data maintained at the financial institution or assess the accuracy of data received at the financial institution from customers.

14. A system for evaluating the accuracy of a data record, comprising:

a data storage system for storing a plurality of data records received from a plurality of different data sources, wherein the data records have varying degrees of trustworthiness, and wherein each of the stored data records are associated with an entity;

a data accuracy system including a processor and a memory, the memory storing instructions that are executable by the processor and configure the data accuracy system to;

receive a data record associated with a specified entity, the received data record to be evaluated for accuracy;

retrieve a group of data records from the stored plurality for data records stored at the data storage system, each retrieved data record associated with the specified entity, each retrieved data record having a data field corresponding to a data field of the data record to be evaluated, and each retrieved data record associated with one of the data sources; and

calculate an accuracy score for the data field of the data record to be evaluated, comprising;

determine a trustworthiness weight for at least each one of the retrieved group of data records based on the trustworthiness of the data source providing each one of the group of data records;

determine a matching score for at least the corresponding data field in each of the retrieved group of data records, based on the degree of similarity between (1) data in the corresponding data field in each of the retrieved group of data records and (2) data in the data field of the data record to be evaluated; and

combine, for the corresponding data field in each one of the retrieved group of data records, (1) the trustworthiness weight for that one of the retrieved group of records, with (2) the matching score for the corresponding data field in that one of retrieved group of data records; and

sum together the combined trustworthiness weight and matching score for every corresponding data field.

15. The system of claim 14, wherein the instructions executable by the processor further configure the data accuracy system to:

determine a trustworthiness weight for the data record to be evaluated;

determine a matching score for the data field of the data record to be evaluated;

combine the trustworthiness weight for the data record to be evaluated and the matching score for the data field of the data record to be evaluated; and

sum, with the previously summed trustworthiness weight and matching score for every corresponding data field, the combined trustworthiness weight for the data record to be evaluated and the matching score for the data field of the data record to be evaluated.

16. The system of claim 15, wherein the matching score for the data field of the data record to be evaluated is assigned a predetermined value.

17. The system of claim 15, wherein the assigned predetermined value is 1.

18. The system of claim 14, wherein the instructions executable by the processor further configure the data accuracy system to:

store, in the data management system, the accuracy score in association with the corresponding data field of the data record for which the accuracy score was calculated.

19. The system of claim 14, wherein the instructions executable by the processor further configure the data accuracy system to:

calculate an accuracy score for each of other data fields in the data record to be evaluated.

20. The system of claim 19, wherein the instructions executable by the processor further configure the data accuracy system to:

calculate an overall accuracy score for the data record to be evaluated, by combining accuracy scores for individual data fields of the data record.

21. The system of claim 19, wherein the instructions executable by the processor further configure the data accuracy system to:

calculate an accuracy score for an identified entity, by combining accuracy scores for a plurality of data records associated with the identified entity.

22. The system of claim 19, wherein the instructions executable by the processor further configure the data accuracy system to:

calculate an accuracy score for one of the plurality of data sources, by combining accuracy scores for a plurality of data records that are provided by that one of the plurality of data sources.

23. The system of claim 14, wherein the instructions executable by the processor further configure the data accuracy system to:

monitor, at the data accuracy system, the accuracy score, including;

waiting a predetermined period of time after calculating the accuracy score,

re-calculating the accuracy score; and

comparing the re-calculated accuracy score to a previously calculated accuracy score.

24. The system of claim 14, wherein the received data record associated with the specified entity is provided with a query from a data user, and wherein the instructions executable by the processor further configure the data accuracy system to:

provide the calculated accuracy score to the data user as a response to the query.

25. The system of claim 14, wherein the plurality of different data sources are selected from a group comprising:

Published Research;

News;

Wikipedia;

Social Media;

Financial Transactions;

Financial Account Ownership Data;

Government Benefits/Licensing Records;

Court Records;

Voter Registration; and

Aggregated Public Data.

26. The system of claim 14, wherein the received data record associated with the specified entity is received with other data records from a financial institution, wherein the calculated accuracy score is provided to the financial institution, and wherein the calculated accuracy score is used by the financial institution to assess the accuracy of data maintained at the financial institution or assess the accuracy of data received at the financial institution from customers.

27. A method for assessing the accuracy of a data record maintained at a data source, comprising:

receiving, from one data source, a data record to be assessed, the data record to be assessed associated with an entity and comprising a plurality of data fields;

identifying other data records from one or more other data sources that are associated with the same entity as the entity associated data record to be assessed, each of the other data records having data fields corresponding to data fields in the data record to be assessed;

assigning an accuracy score to at least one data field of the data record to be assessed, comprising;

assigning a trustworthiness weight to each of the data sources;

assigning a matching score for the degree of similarity between (1) the data in the at least one data field of the data record to be assessed and (2) the data in a corresponding data field in each of the data records;

calculating the accuracy score (DAS) for the at least one data field of the data record to be assessed, based on;