Systems and methods for data exchange and conversion

• US 10,445,343 B2
• Filed: 07/14/2016
• Issued: 10/15/2019
• Est. Priority Date: 07/16/2015
• Status: Active Grant

• Current Assignee: ICE Data Pricing & Reference Data LLC (Intercontinental Exchange, Inc.)
• Original Assignee: Interactive Data Pricing and Reference Data Incorporated (Intercontinental Exchange, Inc.)
• Inventors: Gartland, William J., Monahan, Timothy
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US15/210,036
• Publication Number:
  US 20170017712A1
• Time in Patent Office: 1,188 Days
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange

Abstract

Systems and methods for data conversion and distribution. A data collection system receives data from plural data sources; extracts data values for each data source from the data; classifies each extracted value among categories; and stores each extracted value in a database according to a data mapping based on a category index, such that the extracted value is stored according to the associated category index. A data performance system retrieves, for each category, data values associated with the category and a predetermined period, by querying the database based on the category index; and runs, for each category, the retrieved data through statistical algorithms, to generate data performance metrics representative of the data from all of the data sources. A data distribution system generates a predictive performance metric for further data based on a comparison between the further data and the data performance metrics for at least one category.

Claims

1. A data conversion and distribution system comprising:

a data collection system configured to;

receive electronic data from each of a plurality of electronic data sources via electronic communication over a network, said electronic data comprising real-time market data,

extract, for each electronic data source, one or more data values from the corresponding received electronic data,

classify, for each extracted data value from each electronic data source, the extracted data value among one or more predetermined categories, and

store each extracted data value from each electronic data source in a first database according to a data mapping based on a category index corresponding to each predetermined category, such that the respective extracted data value is stored according to the category index associated with the corresponding classified category;

a data performance system communicatively coupled to the data collection system, the data performance system configured to;

retrieve, from the first database, for each predetermined category, stored data values associated with the respective predetermined category and a predetermined time period, by querying the first database based on the respective category index, and

run, for each predetermined category, the respective retrieved data through one or more statistical algorithms, to generate data performance metrics for the respective predetermined category representative of the electronic data from all of the plurality of electronic data sources, wherein the data performance metrics include a statistical profile representative of data execution quality for all retrieved data values in the corresponding predetermined category; and

a data distribution system communicatively coupled to the data collection system and the data performance system, the data distribution system configured to;

generate a predictive performance metric for further electronic data based on a comparison of the further electronic data to the data performance metrics for at least one of the one or more predetermined categories,

wherein the data collection system, the data performance system and the data distribution system comprise one or more computers.

2. The system of claim 1, wherein the electronic data received from each electronic data source includes at least one of one or more electronic data feeds, one or more electronic data files, one or more emails, and one or more text messages.

3. The system of claim 1, wherein the data collection system is configured to assign a classification indication to each classified data value corresponding to the classified category, and map the corresponding classified data value to one or more locations of the first database based on the respective classification indication.

4. The system of claim 1, wherein the data collection system is configured to further classify, for at least one of the classified values, the classified value among one or more predetermined attributes associated with the classified predetermined category.

5. The system of claim 1, wherein the data collection system is configured to at least one of authenticate each data source and validate at least a portion of the received electronic data.

6. The system of claim 1, wherein the data collection system is configured to receive, from among the plurality of electronic data sources, one or more benchmark data values and store the received one or more benchmark data values in a second database.

7. The system of claim 6, wherein the second database is further configured to store historical data.

8. The system of claim 7, wherein the data distribution system is configured to generate the predictive performance metric responsive to a request for performance information received from a receiver device and is configured to transmit, to the receiver device, at least one of the predictive performance metric and other data associated with the further electronic data corresponding to one or more values stored in the second database.

9. The system of claim 6, wherein:

the data classification system is further configured to determine, for each classified data value, a difference value between the classified data value and a benchmark data value among the one or more benchmark data values stored in the second database, and store the associated difference value in the first database with the corresponding extracted data value, and

the data performance system is configured to retrieve, for each predetermined category, each corresponding stored difference value from the first database associated with the respective predetermined category and a predetermined time period and generate the data performance metrics for the respective predetermined category based on each retrieved difference value.

10. The system of claim 1, wherein the data performance metrics represent a transaction price distribution for price differences for a range of transactions in the respective predetermined category.

11. The system of claim 6, wherein the electronic data includes transaction data associated with one or more assets, and the one or more benchmark data values includes reference price data.

12. A method for data conversion and distribution, the method comprising:

receiving, by a data collection system, electronic data from each of a plurality of electronic data sources via electronic communication over a network, said electronic data comprising real-time market data;

extracting by the data collection system, for each electronic data source, one or more data values from the corresponding received electronic data;

classifying, by the data collection system, for each extracted data value from each electronic data source, the extracted data value among one or more predetermined categories;

storing, by the data collection system, each extracted data value from each electronic data source in a first database according to a data mapping based on a category index corresponding to each predetermined category, such that the respective extracted data value is stored according to the category index associated with the corresponding classified category;

retrieving, by a data performance system, from the first database, for each predetermined category, stored data values associated with the respective predetermined category and a predetermined time period, by querying the first database based on the respective category index;

running, by the data performance system, for each predetermined category, the respective retrieved data through one or more statistical algorithms, to generate data performance metrics for the respective predetermined category representative of the electronic data from all of the plurality of electronic data sources, wherein the data performance metrics include a statistical profile representative of data execution quality for all retrieved data values in the corresponding predetermined category; and

generating, by a data distribution system, a predictive performance metric for further electronic data based on a comparison of the further electronic data to the data performance metrics for at least one of the one or more predetermined categories,

wherein the data collection system, the data performance system and the data distribution system comprise one or more computers.

13. The method of claim 12, wherein the classifying of the extracted data further includes assigning a classification indication to each classified data value corresponding to the classified category, and map the corresponding classified data value to one or more locations of the first database based on the respective classification indication.

14. The method of claim 12, wherein the classifying of the extracted data further includes classifying, for at least one of the classified values, the classified value among one or more predetermined attributes associated with the classified predetermined category.

15. The method of claim 12, the method further comprising:

retrieving, by the data classification system, for each classified data value, a benchmark data value from among one or more benchmark data values stored in a second database corresponding to the classified data value;

generating, for each classified data value, a difference value between the classified data value and the respective retrieved benchmark data value; and

store the associated difference value in the first database with the corresponding extracted data value.

16. The method of claim 15, wherein:

the retrieving of the stored data values includes retrieving, for each predetermined category, each corresponding stored difference value from the first database associated with the respective predetermined category and a predetermined time period, and

the running of the respective retrieved data includes running for each predetermined category, each corresponding retrieved difference value through the one or more statistical algorithms to generate the respective data performance metrics.

17. The method of claim 16, wherein the generating of the predictive performance metric includes determining a further difference value between the further electronic data and a further benchmark value corresponding to the further electronic data and comparing the further difference value to the data performance metrics for the at least one of the one or more predetermined categories to identify the predictive performance metric for the further electronic data.

18. The method of claim 12, wherein the data performance metrics represent a transaction price distribution for price differences for a range of transactions in the respective predetermined category.

19. A non-transitory computer-readable storage medium programmed to include instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform functions including:

causing a data collection system to;

receive electronic data from each of a plurality of electronic data sources via electronic communication over a network, said electronic data comprising real-time market data;

extract, for each electronic data source, one or more data values from the corresponding received electronic data;

classify, for each extracted data value from each electronic data source, the extracted data value among one or more predetermined categories, and

store each extracted data value from each electronic data source in a first database according to a data mapping based on a category index corresponding to each predetermined category, such that the respective extracted data value is stored according to the category index associated with the corresponding classified category;

causing a data performance system to;

retrieve, from the first database, for each predetermined category, stored data values associated with the respective predetermined category and a predetermined time period, by querying the first database based on the respective category index, and

run, for each predetermined category, the respective retrieved data through one or more statistical algorithms, to generate data performance metrics for the respective predetermined category representative of the electronic data from all of the plurality of electronic data sources, wherein the data performance metrics include a statistical profile representative of data execution quality for all retrieved data values in the corresponding predetermined category; and

cause a data distribution system to;

generate a predictive performance metric for further electronic data based on a comparison of the further electronic data to the data performance metrics for at least one of the one or more predetermined categories.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions include further instructions that cause the data collection system to:

retrieve, for each classified data value, a benchmark data value from among one or more benchmark data values stored in a second database corresponding to the classified data value;

generate, for each classified data value, a difference value between the classified data value and the respective retrieved benchmark data value; and

store the associated difference value in the first database with the corresponding extracted data value.

21. The non-transitory computer-readable storage medium of claim 20, wherein the instructions include further instructions that cause the data performance system to:

retrieve, for each predetermined category, each corresponding stored difference value from the first database associated with the respective predetermined category and a predetermined time period, and

run, for each predetermined category, each corresponding retrieved difference value through the one or more statistical algorithms to generate the respective data performance metrics.