CLINICAL DATA AGGREGATION ARCHITECTURE AND PLATFORM

• US 20180101584A1
• Filed: 10/09/2017
• Published: 04/12/2018
• Est. Priority Date: 10/07/2016
• Status: Active Grant

• Current Assignee: Pankaj Manon, Sudeep Pattnaik
• Original Assignee: Pankaj Manon, Sudeep Pattnaik
• Inventors: Pattnaik, Sudeep, Manon, Pankaj
• Granted Patent: US 10,642,854 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/86 Mapping to a database
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A clinical data aggregation system and method, comprising ingesting, transforming and storing data in a clinical data lake. The present invention uses an Extract Load and Transform (ELT) rather than traditional Extract Transform and Load (ETL) design principle. The data hub platform leverages modern noSQL databases which makes the platform highly flexible to configure studies with any design complexity with relative ease.

Claims

1. A system for loading and mapping source data into a predetermined standard format, the system comprising:

a database for storing the source data;

a graphical user interface comprised of multiples screens for displaying on a display screen, the graphical user interface having;

i) an area having a user selectable list of data formats of the source data;

ii) an area configured to allow a user to input a plurality of key identifiers for the source data, wherein each key identifier identifies a predetermined data item or items of the source data;

iii) an area for setting mapping rules by mapping a plurality of data items to a plurality of target fields or variables of the predetermined standard format;

iv) an area for inputting a plurality of expressions, wherein each of the expressions are used to transform a predetermined data item or items of the source data into a target value;

a hardware processing system, the hardware processing system programmed with instructions for executing on the hardware processing system for;

     1) loading data items of the source data into the database as pairs of key identifier and value for each data item or items;

     2) transforming the loaded source data using the mapping rules into a predetermined standard format.

2. The system of claim 1, wherein the key is a unique identifier and the value is a value identified by the key.

3. The system of claim 1, wherein the graphical user interface has an area for defining the characteristics of the incoming file.

4. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to automatically configure key identifiers for source data that are in a standard data format.

5. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to:

1) compare loaded source data with stored ontology data; and

     2) map data items from the source data to variables of the predetermined standard format.

6. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the source data with a model of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

7. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the source data with a model of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

8. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the metadata of the source data with the variables of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

9. The system of claim 1, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to use configurable matching algorithms to compare metadata of the source data with a stored mapping library to automatically map the source data to the predetermined standard format.

10. The system of claim 9, wherein the stored mapping library is a library of mapping between domains of the predetermined standard format with source data metadata.

11. A system for loading and mapping source data into a predetermined standard format, the system comprising:

a database for storing the source data;

a graphical user interface comprised of multiple screens for displaying on a display screen, the graphical user interface having;

i) a first area having a user selectable list of types of source data that can be loaded by the system;

ii) a second area having a user selectable list of data formats of the source data;

iii) a third area configured to allow a user to input a plurality of key identifiers for the source data, wherein each key identifier identifies a predetermined data item or items of the source data;

iv) a fourth area for setting mapping rules by mapping a plurality of data items to a plurality of target fields or variables of the predetermined standard format;

v) a fifth area for inputting a plurality of expressions, wherein each of the expressions are used to transform a predetermined data item or items of the source data into a target value;

a hardware processing system, the hardware processing system programmed with instructions for executing on the hardware processing system for;

     1) loading data items of the source data into the database as pairs of key identifier and value for each data item or items;

     2) transforming the loaded source data using the mapping rules into a predetermined standard format;

     3) comparing loaded source data with stored ontology data; and

     4) mapping data items from the source data to variables of the predetermined standard format.

12. The system of claim 11, wherein the key is a unique identifier and the value is a value identified by the key.

13. The system of claim 11, wherein the graphical user interface has an area for defining the characteristics of the incoming file.

14. The system of claim 11, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to automatically configure key identifiers for source data that are in a standard data format.

15. The system of claim 11, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the source data with a model of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

16. The system of claim 11, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the source data with a model of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

17. The system of claim 11, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the metadata of the source data with the variables of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

18. The system of claim 11, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to use configurable matching algorithms to compare metadata of the source data with a stored mapping library to automatically map the source data to the predetermined standard format.

19. The system of claim 18, wherein the stored mapping library is a library of mapping between domains of the predetermined standard format with source data metadata.

20. A system for loading and mapping source data into a predetermined standard format, the system comprising:

a database for storing the source data;

a graphical user interface comprised of multiple screens for displaying on a display screen, the graphical user interface having;

i) a first area having a user selectable list of types of source data that can be loaded by the system;

ii) a second area having a user selectable list of data formats of the source data;

iii) a third area configured to allow a user to input a plurality of key identifiers for the source data, wherein each key identifier identifies a predetermined data item or items of the source data;

iv) a fourth area for setting mapping rules by mapping a plurality of data items to a plurality of target fields or variables of the predetermined standard format;

v) a fifth area for inputting a plurality of expressions, wherein each of the expressions are used to transform a predetermined data item or items of the source data into a target value;

a hardware processing system, the hardware processing system programmed with instructions for executing on the hardware processing system for;

     1) loading data items of the source data into the database as pairs of key identifier and value for each data item or items;

     2) transforming the loaded source data using the mapping rules and expressions into a predetermined standard format;

     3) comparing loaded source data with stored ontology data and mapping data items from the source data to variables of the predetermined standard format;

     5) comparing the source data with a model of the predetermined standard format and using pattern matching algorithms to automatically map the source data to the predetermined standard format.

21. The system of claim 20, wherein the key is a unique identifier and the value is a valued identified by the key.

22. The system of claim 20, wherein the graphical user interface has an area for defining the characteristics of the incoming file.

23. The system of claim 20, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to automatically configure key identifiers for source data that are in a standard data format.

24. The system of claim 20, wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to compare the source data with a model of the predetermined standard format using pattern matching algorithms to automatically map the source data to the predetermined standard format.

25. The system of claim 20 wherein the hardware processing system is programmed with instructions for executing on the hardware processing system to use configurable matching algorithms to compare metadata of the source data with a stored mapping library to automatically map the source data to the predetermined standard format.

26. The system of claim 25, wherein the stored mapping library is a library of mapping between domains of the predetermined standard format with source data metadata.