Server architecture for electronic data quality processing

• US 9,529,851 B1
• Filed: 11/26/2014
• Issued: 12/27/2016
• Est. Priority Date: 12/02/2013
• Status: Active Grant

• Current Assignee: Experian Information Solutions Incorporated (Experian plc)
• Original Assignee: Experian Information Solutions Incorporated (Experian plc)
• Inventors: Smith, Donna Meryl
• Primary Examiner(s): Siddiqi, Mohammad A
• Application Number:
  US14/555,444
• Time in Patent Office: 762 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/256 in federated or virtual databases
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 63/04 for providing a confidential data exchange among entities communicating through data packet networks
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload

Abstract

In one embodiment, a server architecture is disclosed that provides for processing and analyzing data received from data furnishers to evaluate quality of the provided data. The system may format the data received from the data furnishers into standardized form. Based on configuration information and rules for the data furnishers and the provided data, the system may analyze the data set to calculate one or more data quality indicators.

Claims

1. A data quality review architecture platform for conducting an analysis of data quality within data sets provided by data furnishers for addition to a large-scale credit data store, the data quality review architecture comprising:

a primary system configured to electronically communicate with a set of remote data furnisher systems, to access encrypted data sets of a data furnisher which include account data for a plurality of the data furnisher'"'"'s consumers, and to communicate with a large-scale credit data store storing billions of records, wherein certain of the encrypted data sets include errors or inaccuracies and are potentially to be added to the large-scale credit data store;

a data format manager module configured to electronically communicate with the primary system to access the encrypted data sets, to decrypt the encrypted data sets, and to format the data sets to conform with or determine the data set already conforms with a first predetermined format, and to generate decrypted, processed data sets;

a data loader module configured to electronically communicate with the data format manager module to access the decrypted processed data sets and external data and make them available for analysis;

a configuration and control module configured to;

access data furnisher-specific instructions from a data furnisher information database,

use the data furnisher-specific instructions to select a set of services for searching for errors or inaccuracies within the corresponding data furnisher'"'"'s decrypted, processed data, to select a set of metrics to run on the corresponding data furnisher'"'"'s decrypted, processed data, and

instruct the data loader module to make the corresponding data furnishers'"'"' decrypted, processed data available for analysis,

wherein the set of services for searching for errors or inaccuracies in the data set include at least one of;

determining whether status code values are logical for specific date field values;

determining whether balances are logical compared to one or more credit limits;

determining whether one or more fields are complete and logically valid;

determining whether certain criteria have been met indicating fatal errors associated with the data set;

one or more application servers remote from the set of remote data furnisher systems and remote from the large-scale credit data store and configured to;

access the data furnisher'"'"'s decrypted, processed data set,

execute the data furnisher-specific selected set of services and metrics on the decrypted, processed data set to automatically generate data quality indicators which represent a quality assessment of the data in the decrypted processed data set indicating a quantity of errors or number of inaccuracies within the decrypted, processed data set,

generate an analytics result data package based on the performed services and metrics and generated data quality indicators, and to store the analytics result data package in an analytics database, and

perform a determination on whether to allow the data set to be added to the large-scale credit data store based upon a comparison between the data quality indicators and one or more pre-determined data quality metrics related to the data furnisher; and

a reporting application configured to electronically communicate with the analytics database to;

provide electronic access to a remote data furnisher user system of the data furnisher to review information about the metrics and generated data quality indicators associated with the decrypted, processed data set of the data furnisher;

electronically create report displays using the analytics result data package,

electronically create benchmarking displays comparing the quantity of errors or the quantity of inaccuracies with those of one or more additional data furnishers associated with a peer group of the data furnisher for access by the remote data furnisher user system using the analytics result data package,

electronically create metric displays using the analytics result data package, and

enable access of the report displays, the benchmarking displays,

and the metric displays to the remote data furnisher user system.

2. The server data quality review architecture platform of claim 1 wherein the reporting application is further configured to electronically communicate with a system of an analyst and to enable access of the report displays, the benchmarking displays, and the metric displays to the system of the analyst.

3. The data quality review architecture platform of claim 1, wherein the benchmarking displays do not indicate an identity of the one or more additional data furnishers associated with the peer group of the data furnisher.

4. The data quality review architecture platform claim 1, wherein the one or more application servers is further configured to access historical metric data related to the data furnisher and to use the historical metric data to perform the selected set of services and metrics on the data to automatically generate the data quality indicators.

5. The data quality review architecture platform claim 1, wherein the one or more application servers is further configured to automatically generate a suggestion to the data furnisher for enhancing the data furnisher'"'"'s data quality based on the generated data quality indicators.

6. The data quality review architecture platform claim 5, wherein the one or more application servers is further configured to simulate a revised set of data quality indicators based on the automatically generated suggestion.

7. A computer-implemented method for conducting an analysis of data quality within a data set provided by a remote data furnisher for addition to a large-scale credit database, the computer-implemented method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

accessing a data set of a remote data furnisher for updating a large-scale credit database, wherein certain records of the data set includes errors or inaccuracies and are potentially to be added to the large-scale credit database;

formatting the data set to conform to or determining that the data set already conforms with a predetermined format;

obtaining data furnisher-specific configuration information specific to the data furnisher from a data furnisher information database, the obtained data furnisher-specific configuration information used to select a set of services for searching for errors or inaccuracies and metrics to be run on the data set, wherein the set of services for searching for errors or inaccuracies in the data set include at least one of;

determining whether status code values are logical for specific date field values;

determining whether balances are logical compared to one or more credit limits;

determining whether one or more fields are complete and logically valid;

determining whether certain criteria have been met indicating fatal errors associated with the data set;

obtaining historical records of the data furnisher that are related to the data set;

at an application server system comprising one or more application servers and remote from the remote data furnisher and the large-scale credit database, analyzing the data set and the obtained historical records in accordance with the obtained data furnisher-specific configuration information to;

perform the selected set of services and metrics on the data set to automatically calculate one or more data quality indices that represent quality of the data set indicating a quantity of errors or quantity of inaccuracies within the data set,

generate an analytics result data package based on the performed services and metrics and generated data quality indicators, and

store the analytics result data package in an analytics database;

generating a data quality report, the data quality report including at least one of the calculated one or more data quality indices; and

generating an instruction to allow the data set to be added to the large-scale credit database if the calculated one or more data quality indices meet a predetermined criterion.

8. The computer-implemented method of claim 7, wherein the data quality report includes a comparison between the one or more data quality indices of the data furnisher and corresponding data quality indices of at least one or more peer data furnishers.

9. The computer-implemented method of claim 8, wherein the data quality report does not reveal an actual identity of the at least one or more peer data furnishers.

10. The computer-implemented method of claim 7, wherein the configuration information includes at least one configuration that is specific to the data furnisher and wherein the data quality report includes at least one result of a data furnisher-specific analysis conducted based on the at least one data furnisher-specific configuration.

11. The computer-implemented method of claim 7, wherein the data furnisher-specific configuration information includes at least one configuration that is specific for at least one data type within the data set and wherein the data quality report includes at least one result of a data type-specific analysis conducted based on the at least one data furnisher-specific configuration.

12. The computer-implemented method of claim 7 further comprises automatically generating a suggestion to the data furnisher for enhancing data quality based on the calculated data quality indices.

13. The computer-implemented method of claim 12 further comprises simulating data quality of the data furnisher by implementing the generated suggestion.

14. A non-transitory computer storage medium storing computer-executable instructions that direct a computing system to perform operations for conducting an analysis of data quality within a data set provided by a remote data furnisher for addition to a large-scale credit database, the operations comprising:

accessing a data set of a remote data furnisher for updating a large-scale credit database, wherein certain records of the data set includes errors or inaccuracies and are potentially to be added to the large-scale credit database;

formatting the data set to conform to or determining that the data set already conforms with a predetermined format;

obtaining data furnisher-specific configuration information specific to the data furnisher from a data furnisher information database, the obtained data furnisher-specific configuration information used to select a set of services for searching for errors or inaccuracies and metrics to be run on the data set, wherein the set of services for searching for errors or inaccuracies in the data set include at least one of;

determining whether status code values are logical for specific date field values;

determining whether balances are logical compared to one or more credit limits;

determining whether one or more fields are complete and logically valid;

determining whether certain criteria have been met indicating fatal errors associated with the data set;

obtaining historical records of the data furnisher that are related to the data set;

at an application server system comprising one or more application servers and remote from the remote data furnisher and the large-scale credit database, analyzing the data set and the obtained historical records in accordance with the obtained data furnisher-specific configuration information to;

perform the selected set of services and metrics on the data set to automatically calculate one or more data quality indices that represent quality of the data set indicating a quantity of errors or quantity of inaccuracies within the data set,

generate an analytics result data package based on the performed services and metrics and generated data quality indicators, and

store the analytics result data package in an analytics database;

generating a data quality report, the data quality report including at least one of the calculated one or more data quality indices; and

generating an instruction to allow the data set to be added to the large-scale credit database if the calculated one or more data quality indices meet a predetermined criterion.

15. The non-transitory computer storage medium of claim 14, wherein the data quality report includes a comparison between the one or more data quality indices of the data furnisher and corresponding data quality indices of at least one or more peer data furnishers.

16. The non-transitory computer storage medium of claim 14, wherein the data quality report does not reveal an actual identity of the at least one or more peer data furnishers.

17. The non-transitory computer storage medium of claim 14, wherein the data furnisher-specific configuration information includes at least one configuration that is specific to the data furnisher and wherein the data quality report includes at least one result of a data furnisher-specific analysis conducted based on the at least one data furnisher-specific configuration.

18. The non-transitory computer storage medium of claim 14, wherein the configuration information includes at least one data furnisher-specific configuration that is specific for at least one data type within the data set and wherein the data quality report includes at least one result of a data type-specific analysis conducted based on the at least one data furnisher-specific configuration.

19. The non-transitory computer storage medium of claim 14, further comprises automatically generating a suggestion to the data furnisher for enhancing data quality based on the calculated data quality indices index.

20. The non-transitory computer storage medium of claim 19, further comprises simulating data quality of the data furnisher by implementing the generated suggestion.