Asynchronous sensors

• US 10,353,761 B2
• Filed: 02/25/2016
• Issued: 07/16/2019
• Est. Priority Date: 04/29/2011
• Status: Active Grant

• Current Assignee: Dark Matter Technologies LLC
• Original Assignee: Black Knight Ip Holding Company, Llc (Intercontinental Exchange, Inc.)
• Inventors: Sogorka, Daniel Brian, Johnson, Mark Richard, Holbrook, John David, Sanderson, Jeffrey Albert, Frazier, Ronald Lynn, Smith, Donald P.
• Primary Examiner(s): Park, Grace
• Application Number:
  US15/053,700
• Publication Number:
  US 20160299803A1
• Time in Patent Office: 1,237 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]

Abstract

Described herein are apparatuses, methods, and computer readable media for asynchronous sensing and/or validating of data using a first sensor and a second sensor for a purpose of transforming input data into validated output data when at least one attribute of external data or the input data, as determined by a sensor, is missing. The external data is associated with the input data and is used for determining a set of attributes for the first sensor. The asynchronous sensing and/or validating of the input data enables faster transforming of the input data into the validated output data.

Claims

1. A system for parallel sensing and validating of data using a first sensor and a second sensor for a purpose of transforming input data into validated output data when at least one attribute of external data or the input data is missing, the external data being associated with the input data and being used for determining a set of attributes for the first sensor, wherein the parallel sensing and validating of the input data enables faster transforming of the input data into the validated output data, the system comprising:

a data communication interface for;

establishing a first connection to an input data system;

receiving, on the first connection, input data from the input data system;

establishing a second connection to an external data system; and

receiving, on the second connection, external data from the external data system, the external data being associated with the input data;

a pre-validation sensor for

sensing a first pre-validation attribute associated with the input data;

a missing-attribute validation sensor for;

sensing a first missing attribute associated with the external data or the input data;

a first validation sensor for;

sensing a first validation attribute associated with pre-validated input data;

a second validation sensor for;

sensing, in parallel with the sensing of the first validation attribute by the first validation sensor, a second validation attribute associated with the pre-validated input data;

a chipset and interfaced with a processor, the chipset having hardware for supporting the first connection to the input data system, and the second connection to the external data system; and

the processor for;

determining, using the pre-validation sensor, the first pre-validation attribute associated with the input data;

pre-validating, based on the first pre-validation attribute, the input data;

determining, using the missing-attribute validation sensor, the first missing attribute associated with the external data or the input data;

determining, based on the first missing attribute, an insensible attribute associated with the first validation sensor;

determining, using the first validation sensor, a first set of validation attributes associated with the pre-validated input data, the first set of validation attributes not comprising the insensible attribute;

determining, using the second validation sensor, a second set of validation attributes associated with the pre-validated input data;

validating, based on the first set of validation attributes, the pre-validated input data; and

validating, based on the second set of validation attributes and in parallel with the validating based on the first set of validation attributes, the pre-validated input data.

2. The system of claim 1, wherein the data communication interface is further for:

establishing a third connection to an output data system; and

transmitting, on the third connection, output data to the output data system, wherein the hardware further supports a third connection to the output data system.

3. The system of claim 2, wherein the first connection and the third connection comprise the same connection.

4. The system of claim 1, wherein the first connection and the second connection comprise the same connection.

5. The system of claim 1, wherein the processor is further for:

validating, based on the first missing attribute, the pre-validated input data.

6. The system of claim 1, wherein the processor is further for:

determining, based on the first missing attribute, a quality of the validated data.

7. The system of claim 1, wherein the processor is further for:

indicating, using a graphical indicator, a degree of validity of the validated data.

8. The system of claim 7, wherein the graphical indicator comprises a red flag.

9. The system of claim 1, wherein the input data comprises at least some external data.

10. The system of claim 1, wherein no dependency exists between the first sensor and the second sensor.

11. The system of claim 1, wherein the processor is further for:

integrating the validated data based on the first set of validation attributes and the validated data based on the second set of validation attributes.

12. The system of claim 1, wherein the processor is further for:

requesting and receiving the external data based on the input data.

13. A method for parallel sensing and validating of data using a first sensor and a second sensor for transforming input data into validated output data when at least one attribute of external data or the input data is missing, the external data being associated with the input data and being used for determining a set of attributes for the first sensor, the method comprising:

establishing a first connection to an input data system;

receiving, on the first connection, input data from the input data system;

establishing a second connection to an external data system; and

receiving, on the second connection, external data from the external data system, the external data being associated with the input data;

sensing, using a pre-validation sensor, a first pre-validation attribute associated with the input data;

sensing, using a missing-attribute validation sensor, a first missing attribute associated with the external data or the input data;

sensing, using a first validation sensor, a first validation attribute associated with pre-validated input data;

sensing, using a second validation sensor, in parallel with the sensing of the first validation attribute by the first validation sensor, a second validation attribute associated with the pre-validated input data;

determining, using the pre-validation sensor, the first pre-validation attribute associated with the input data;

pre-validating, based on the first pre-validation attribute, the input data;

determining, using the missing-attribute validation sensor, the first missing attribute associated with the external data or the input data;

determining, based on the first missing attribute, an insensible attribute associated with the first validation sensor;

determining, using the first validation sensor, a first set of validation attributes associated with the pre-validated input data, the first set of validation attributes not comprising the insensible attribute;

determining, using the second validation sensor, a second set of validation attributes associated with the pre-validated input data;

validating, based on the first set of validation attributes, the pre-validated input data; and

validating, based on the second set of validation attributes and in parallel with the validating based on the first set of validation attributes, the pre-validated input data.

14. The method of claim 13, wherein the input data comprises at least some external data.

15. The method of claim 13, wherein no dependency exists between the first sensor and the second sensor.

16. The method of claim 13, wherein the first connection and the second connection comprise the same connection.

17. A non-transitory computer readable medium comprising code configured for parallel sensing and validating of data using a first sensor and a second sensor for transforming input data into validated output data when at least one attribute of external data or the input data is missing, the external data being associated with the input data and being used for determining a set of attributes for the first sensor, the code configured for:

establishing a first connection to an input data system;

receiving, on the first connection, input data from the input data system;

establishing a second connection to an external data system; and

receiving, on the second connection, external data from the external data system, the external data being associated with the input data;

sensing, using a pre-validation sensor, a first pre-validation attribute associated with the input data;

sensing, using a missing-attribute validation sensor, a first missing attribute associated with the external data or the input data;

sensing, using a first validation sensor, a first validation attribute associated with pre-validated input data;

sensing, using a second validation sensor, in parallel with the sensing of the first validation attribute by the first validation sensor, a second validation attribute associated with the pre-validated input data;

determining, using the pre-validation sensor, the first pre-validation attribute associated with the input data;

pre-validating, based on the first pre-validation attribute, the input data;

determining, using the missing-attribute validation sensor, the first missing attribute associated with the external data or the input data;

determining, based on the first missing attribute, an insensible attribute associated with the first validation sensor;

determining, using the first validation sensor, a first set of validation attributes associated with the pre-validated input data, the first set of validation attributes not comprising the insensible attribute;

determining, using the second validation sensor, a second set of validation attributes associated with the pre-validated input data;

validating, based on the first set of validation attributes, the pre-validated input data; and

validating, based on the second set of validation attributes and in parallel with the validating based on the first set of validation attributes, the pre-validated input data.

18. The non-transitory computer readable medium of claim 17, wherein the input data comprises at least some external data.

19. The non-transitory computer readable medium of claim 17, wherein no dependency exists between the first sensor and the second sensor.

20. The non-transitory computer readable medium of claim 17, wherein the first connection and the second connection comprise the same connection.