Method and apparatus for remotely managed data extraction

• US 10,120,913 B1
• Filed: 08/30/2011
• Issued: 11/06/2018
• Est. Priority Date: 08/30/2011
• Status: Expired due to Fees

• Current Assignee: Intalere, Inc.
• Original Assignee: Intalere, Inc.
• Inventors: Mullins, Scott David, Morgan, Anthony John, Bakthavatchalu, Suganya
• Primary Examiner(s): Nguyen, Merilyn P
• Application Number:
  US13/221,407
• Time in Patent Office: 2,625 Days
• Field of Search: 707722
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/90344 by using string matching techniques

Abstract

Techniques for remotely managed data extraction are disclosed whereby a client data extractor (CDE) is installed on a customer computer system to extract data from the customer computer system for transmission to a destination. Soft configuration techniques can be employed where the configuration data for governing the data extraction are provided to the customer computer system from a service provider computer system on an as-needed basis. Moreover, during extraction, data translation from a source format to a destination format can be performed at the query level to improve efficiency.

Claims

1. A computer-implemented data extraction method comprising:

providing a graphical user interface (GUI) for access by a user, wherein the providing step is performed by a client data extractor (CDE) module executed by a processor resident within a first computer system;

the GUI receiving scheduling information from the user;

the GUI interfacing the scheduling information from the user with a separate scheduling application;

the separate scheduling application defining a scheduled basis for the CDE module to initiate communication with a second computer system in response to the scheduling information;

the CDE module initiating a communication with the second computer system on the scheduled basis, wherein the second computer system is remote from the first computer system;

after initiation, the CDE module receiving a query string and a connection string from the second computer system;

the CDE module connecting to a data source within the first computer system based on the received connection string;

the CDE module querying the data source based on the received query string, the query string defining the data sought to be extracted and a translation of the data sought to be extracted from a format of the data source to a format of a destination;

the CDE module receiving data from the data source in response to the query;

the CDE module translating the received data to the format of the destination based on the received query string;

the CDE module assembling the translated data into a data structure; and

the CDE module sending the data structure to the destination; and

repeating the the initiating step, the query string and connection string receiving step, the connecting step, the querying step, the data receiving step, the translating step, the assembling step, and the sending step each time that a new data extraction from the data source is to be performed such that the first computer system is not required to locally manage the query strings and the connection strings.

2. The method of claim 1 wherein the initiating step further comprises the processor communicating an identifier associated with the first computer system to the second computer system, and wherein the query string and connection string receiving step comprises the processor receiving a query string and a connection string from the second computer system that are associated with the communicated identifier.

3. The method of claim 1 wherein the data source comprises a database configured to store relational data, and wherein the data structure comprises an XML data structure.

4. The method of claim 1 wherein the initiating step and the sending step are performed using web service requests.

5. The method of claim 1 wherein the translating step comprises translating a field identifier for data sought to be retrieved from the format of the data source to the format of the destination based on the received query.

6. The method of claim 5 wherein the destination is resident within the second computer system.

7. The method of claim 5 wherein the destination comprises a software application that provides a user with access to extracted data for analysis thereof.

8. The method of claim 1 wherein the destination comprises the second computer system.

9. The method of claim 1 further comprising:

the processor receiving input from a user that defines the destination.

10. The method of claim 1 wherein the first computer system comprises a plurality of databases from which data is to be extracted, wherein the initiating step comprises the processor initiating a communication with the second computer system to receive a plurality of query strings and a plurality of connection strings, each connection string associated with a query string and defining a connection to one of the plurality of databases such that the plurality of connection strings define a plurality of connections to the plurality of databases, wherein the receiving step comprises the processor receiving the plurality of query strings and associated connection strings from the second computer system, the processor performing the connecting step, the querying step, the translating step, and the data receiving step for each of the plurality of received query strings and associated connection strings.

11. The method of claim 1 wherein the query string defines the data sought to be extracted and a plurality of translations, the plurality of translations comprising a first translation from the format of the data source to a format of a first destination and a second translation from the format of the data source to a format of a second destination;

wherein the translating step comprises the processor translating the received data to the formats of the first and second destinations based on the received query string;

wherein the assembling step comprises the processor assembling the translated data into a plurality of data structures, the plurality of data structures comprising a first data structure into which the translated data for the first destination has been assembled and a second data structure into which the translated data for the second destination has been assembled; and

wherein the sending step comprises the processor sending the first data structure to the first destination and the processor sending the second data structure to the second destination.

12. The method of claim 1 wherein the query string includes a portion that is configured to control the querying step to only extract data that has been added to or modified within the data source since a previous extraction operation from the data source.

13. The method of claim 1 wherein the first computer system comprises a healthcare provider computer system.

14. A computer program product for data extraction comprising:

a plurality of instructions that are part of a client data extractor (CDE) module for execution by a first computer system and are executable by a processor to (1) provide a graphical user interface (GUI) for access by a user, wherein the GUI is configured to (i) receive scheduling information from the user, and (ii) interface the scheduling information from the user with a separate scheduling application that is configured to define a scheduled basis for the CDE module to initiate a communication with a second computer system in response to the scheduling information, (2) initiate, by the first computer system, a communication with the second computer system on the scheduled basis, wherein the second computer system is remote from the first computer system, (3) after initiation, receive, at the first computer system, a query string and a connection string from the second computer system, (4) connect to a data source within the first computer system based on the received connection string, (5) query the data source based on the received query string, the query string configured to define the data sought to be extracted and a translation of the data sought to be extracted from a format of the data source to a format of a destination, (6) receive data from the data source in response to the query, (7) translate the received data to the format of the destination based on the received query string, (8) assemble the translated data into a data structure, (9) send the data structure to the destination, and (10) repeat the initiate, string receive, connect, query, data receive, translate, assemble, and send operations for a plurality of new data extractions from the data source such that the first computer system is not required to locally manage the query strings and the connection strings; and

wherein the plurality of instructions are resident on a non-transitory computer-readable storage medium.

15. The computer program product of claim 14 wherein the instructions are further configured to (1) as part of the initiate operation, communicate an identifier associated with the first computer system to the second computer system, and (2) as part of the string receive operation, receive a query string and a connection string from the second computer system that are associated with the communicated identifier.

16. The computer program product of claim 14 wherein the data source comprises a database configured to store relational data, and wherein the data structure comprises an XML data structure.

17. The computer program product of claim 14 wherein instructions are further configured to perform the initiate and send operations via web service requests.

18. The computer program product of claim 14 wherein the instructions are further configured to, as part of the translate operation, translate a field identifier for data sought to be retrieved from the format of the data source to the format of the destination based on the received query.

19. The computer program product of claim 14 wherein the first computer system comprises a plurality of databases from which data is to be extracted;

wherein the instructions are further configured to (1) as part of the initiate operation, initiate a communication with the second computer system to receive a plurality of query strings and a plurality of connection strings, each connection string associated with a query string and defining a connection to one of the plurality of databases such that the plurality of connection strings define a plurality of connections to the plurality of databases, (2) as part of the string receive operation, receive the plurality of query strings and associated connection strings from the second computer system, and (3) perform the connect, query, translate, and data receive operations for each of the plurality of received query strings and associated connection strings.

20. The computer program product of claim 14 wherein the query string defines the data sought to be extracted and a plurality of translations, the plurality of translations comprising a first translation from the format of the data source to a format of a first destination and a second translation from the format of the data source to a format of a second destination;

and

wherein the instructions are further configured to (1) as part of the translate operation, translate the received data to the formats of the first and second destinations based on the received query string, (2) as part of the assemble operation, assemble the translated data into a plurality of data structures, the plurality of data structures comprising a first data structure into which the translated data for the first destination has been assembled and a second data structure into which the translated data for the second destination has been assembled, and (3) as part of the send operation, send the first data structure to the first destination and the second data structure to the second destination.

21. The computer program product of claim 14 wherein the query string includes a portion that is configured to control the query operation to only extract data that has been added to or modified within the data source since a previous extraction operation from the data source.

22. An apparatus for data extraction comprising:

a processor resident on a first computer system, the processor configured to execute a client data extractor (CDE) module, wherein execution of the CDE module causes the processor to (1) provide a graphical user interface (GUI) for access by a user, wherein the GUI is configured to (i) receive scheduling information from the user, and (ii) interface the scheduling information from the user with a separate scheduling application that is configured to define a scheduled basis for the CDE module to initiate a communication with a second computer system in response to the scheduling information, (2) initiate a communication with the second computer system on the scheduled basis, wherein the second computer system is remote from the first computer system, (3) after initiation, receive a query string and a connection string from the second computer system, (4) connect to a data source within the first computer system based on the received connection string, (5) query the data source based on the received query string, the query string configured to define the data sought to be extracted and a translation of the data sought to be extracted from a format of the data source to a format of a destination, (6) receive data from the data source in response to the query, (7) translate the received data to the format of the destination based on the received query string, (8) assemble the translated data into a data structure, (9) send the data structure to the destination, and (10) repeat the initiate, string receive, connect, query, data receive, translate, assemble, and send operations for a plurality of new data extractions from the data source such that the first computer system is not required to locally manage the query strings and the connection strings.

23. The apparatus of claim 22 wherein the data source comprises a database configured to store relational data;

wherein the data structure comprises an XML data structure;

wherein the query string includes a portion that is configured to control the query operation to only extract data that has been added to or modified within the database since a previous extraction operation from the database and

wherein the processor is further configured to (1) perform the initiate and send operations via web service requests, (2) as part of the initiate operation, communicate an identifier associated with the first computer system to the second computer system, and (3) as part of the string receive operation, receive a query string and a connection string from the second computer system that are associated with the communicated identifier.

24. The apparatus of claim 22 wherein the processor is further configured to, as part of the translate operation, translate a field identifier for data sought to be retrieved from the format of the data source to the format of the destination based on the received query.

25. The apparatus of claim 22 wherein the query string defines the data sought to be extracted and a plurality of translations, the plurality of translations comprising a first translation from the format of the data source to a format of a first destination and a second translation from the format of the data source to a format of a second destination;

and

wherein the processor is further configured to (1) as part of the translate operation, translate the received data to the formats of the first and second destinations based on the received query string, (2) as part of the assemble operation, assemble the translated data into a plurality of data structures, the plurality of data structures comprising a first data structure into which the translated data for the first destination has been assembled and a second data structure into which the translated data for the second destination has been assembled, and (3) as part of the send operation, send the first data structure to the first destination and the second data structure to the second destination.

26. A computer-implemented data extraction method comprising:

providing a graphical user interface (GUI) for access by a user, wherein the providing step is performed by a client data extractor (CDE) module executed by a processor resident within a first computer system;

the GUI receiving scheduling information from the user;

the GUI interfacing the scheduling information from the user with a separate scheduling application; and

the separate scheduling application defining a scheduled basis for the CDE module to initiate communication with a second computer system in response to the scheduling information;

the client data extractor (CDE) module executing on the first computer system to perform a data extraction from a database of the first computer system, wherein the CDE module executing step comprises;

the CDE module determining whether a data extraction is to be performed according to the scheduled basis; and

in response to determining that a data extraction is to be performed on the scheduled basis the CDE module (1) sending a request to a second computer system, (2) obtaining configuration data from the second computer system in response to the sent request, wherein the configuration data comprises a plurality of query strings and a plurality of connection strings for use in the data extraction, each connection string associated with a query string and defining a connection to a database of the first computer system, (3) for each query string and its associated connection string within the configuration data, (i) using the connection string associated with that query string, connecting to the database corresponding to that associated connection string, and (ii) extracting data from that connected database using that query string, wherein the extracting step includes translating the extracted data from a format of that connected database to a format of a destination during extraction at a query string level based on data within that query string, (4) assembling the extracted data into a data structure, and (5) sending the assembled data structure to a destination.

27. A system for data extraction, the system comprising:

a first computer system; and

a second computer system for communication with the first computer system via a network;

wherein the first computer system comprises a data source and a processor, the processor configured to execute a client data extractor (CDE) module to perform a data extraction from the data source;

wherein the second computer system comprises at least one server and a memory, wherein memory is configured to store configuration data in association with a plurality of identifiers, the configuration data comprising a plurality of query strings and a plurality of connection strings, a plurality of the query strings being configured to define (1) the data sought to be extracted and (2) a translation of the data sought to be extracted from a format of a data source to a format of a destination;

wherein the CDE module is configured to provide a graphical user interface (GUI) for access by a user, wherein the GUI is configured to (i) receive scheduling information from the user, and (ii) interface the scheduling information from the user with a separate scheduling application that is configured to define a scheduled basis for the CDE module to initiate a communication with a second computer system in response to the scheduling information

wherein the CDE module is further configured to (1) determine whether a data extraction is to be performed according to the scheduled basis, and (2) in response to a determination that a data extraction is to be performed on the scheduled basis (1) send a request to the second computer system, the request comprising a request for configuration data and an identifier;

wherein the at least one server is configured to (1) receive the request for configuration data, and (2) in response to the received request, automatically (i) access the memory to identify the configuration data associated with the identifier within the received request, and (ii) communicate the identified configuration data to the first computer system;

wherein the CDE module is configured to (1) obtain the communicated configuration data from the second computer system, the communicated configuration data comprising a query string and a connection string for use in the data extraction, (2) connect to the data source based on the connection string, (3) extract data from the connected data source based on the query string, wherein the extracting operation is configured to translate the extracted data from a format of the data source to a format of a destination during extraction at a query level based on data within the query string, (4) assemble the extracted data into a data structure, and (5) send the assembled data structure to the second computer system; and

wherein the at least one server is further configured to (1) receive the sent data structure, and (2) in response to the received data structure, automatically store the extracted data within the received data structure in the memory.