×

PARALLEL PROCESSING FOR ETL PROCESSES

  • US 20080222634A1
  • Filed: 03/06/2007
  • Published: 09/11/2008
  • Est. Priority Date: 03/06/2007
  • Status: Abandoned Application
First Claim
Patent Images

1. A computer-implemented method for parallel processing of data from a plurality of data sources in conjunction with an Extract-Transform-Load (ETL) process, the data being part of a related data set, comprising:

  • staging a unit of extracted data from each of the plurality of data sources, thereby generating a plurality of units of staged data;

    identifying a plurality of tasks for transforming the staged data;

    assigning a subset of the tasks to each of a plurality of child processes being managed by a master process, such that dependent tasks are assigned to a same child process;

    concurrently executing the subsets of tasks assigned to the child processes, thereby generating a plurality of units of transformed data from the plurality of units of staged data; and

    publishing the transformed data to at least one data store after all of the plurality of tasks are completed, thereby ensuring that the published data represent the related data set.

View all claims
  • 3 Assignments
Timeline View
Assignment View
    ×
    ×