Method for service oriented data extraction transformation and load

• US 8,307,025 B2
• Filed: 05/29/2008
• Issued: 11/06/2012
• Est. Priority Date: 01/19/2007
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Alba, Alfredo
• Primary Examiner(s): Moore, Ian N
• Assistant Examiner(s): ALEXANDER, JENEE LAUREN
• Application Number:
  US12/129,211
• Publication Number:
  US 20080294790A1
• Time in Patent Office: 1,622 Days
• Field of Search: 707/1, 707/100, 707/602, 709217-219, 709223-226, 709/231, 709/203, 709/201
• US Class Current: 709/200
• CPC Class Codes: H04L 65/103 in the network
• CPC Class Codes: H04L 65/765 intermediate

Abstract

The present invention relates to a method for the configurable real time transformation of dissimilar data sources, the method further consisting of the steps of acquiring real time information pertaining to at least one data source, wherein the information comprises reference information that is associated with the data source, data transformation specification information that is associated with the data source, and scheduled event specification information that is associated with the data source, and maintaining the data source information. The method further comprises the steps of acquiring data from the data source in accordance with a specified scheduled event, converting the acquired data into a predetermined standardized format, performing at least one data transformation function from the real time stream upon the converted data in accordance with the acquired data transformation specification information that was associated with the data source; and transmitting the transformed data to a destination data source.

Claims

1. A computer program product that includes a non-transitory computer readable medium useable by a machine, the medium having stored thereon a sequence of instructions which, when executed by the machine, causes the machine to transform a data source into a predetermined format, wherein the machine executes:

maintaining a comprehensive listing of data sources in a data source repository;

acquiring real time information pertaining to at least one data source, wherein the real time information includes;

reference information that is associated with the data source,

data transformation specification information that is associated with the data source, the data transformation specification information that is associated with the data source being stored in a transformation specification repository, and

scheduled event specification information that is associated with the data source,

maintaining the real time information;

acquiring data from the data source in accordance with at least one specified scheduled event identified in the scheduled event specification information, the acquiring from the data source being performed at a rate that complies with performance limitations of the data source;

converting the real time stream acquired data into a predetermined standardized and normalized format, wherein the extracted data is normalized in a per fact basis into a XML document;

performing at least one data transformation function from the real time stream upon the converted data in accordance with the acquired data transformation specification information stored in a transformation specification repository that was associated with the data source; and

transmitting the transformed data to a destination data source;

wherein changes to the data sources, transformation specification information, and scheduled event specification information are provided to a maintenance module to update a registry including;

detecting a change;

determining what type of configuration modification is to be made;

if the change is to be made to the data transformation specification, then a transformation specification repository is accessed and the modifications are performed to the data transformation specification;

if the change is to be made to the data source repository, then accessing the data source repository and modifying the configuration for the relevant data;

if the change is to be made to the scheduled event specification information, then accessing the scheduled event specification information and modifying the configuration for the relevant data.