×

Enhancing performance of extract, transform, and load (ETL) jobs

  • US 9,529,873 B2
  • Filed: 05/16/2013
  • Issued: 12/27/2016
  • Est. Priority Date: 09/21/2012
  • Status: Expired due to Fees
First Claim
Patent Images

1. A computer implemented method for accelerating a read stage and a write stage of an extract, transform, and load job comprising:

  • receiving a file having configuration information that defines a plurality of stages of at least one extract, transform, and load (ETL) job;

    creating a configuration file, wherein the configuration file is associated with the file, and wherein the configuration file includes one or more compression parameters, and wherein each compression parameter of the one or more compression parameters is selected from the group consisting of a compressor stage or a decompressor stage;

    identifying a read stage that is preceded by a write stage within the file, wherein the read stage reads data from a source location, and wherein the data that is read or a modified version of the data that is read is being written by the write stage that writes data to the source location, and wherein the read stage and write stage each have a first parameter that represents a name of a stage, and wherein the first parameter is associated with a second parameter that represents a type of stage;

    adding the identified read stage to the configuration file;

    in response to identify the read stage that is preceded by the write stage within the file, replacing the read stage with a decompressor stage that reads data from a hard disk, decompresses the data from the hard disk and passes the data that is decompressed to another stage or to a computer program for further processing, wherein the decompressor stage is configured to be executed on a field programmable gate array (FPGA), and wherein parameters for the read stage are a decompressor stage in the configuration file;

    in response to identifying the read stage that is preceded by the write stage within the file, replacing the write stage with a compressor stage that compresses the data, and writes the data to a hard disk, wherein the compressor stage is configured to be executed on the FPGA, and wherein the parameters for the write stage are a compressor stage in the configuration file;

    receiving one or more first indications from a user, wherein each first indication of the one or more first indications includes adding, modifying, or deleting a first stage of the plurality of stages;

    receiving one or more second indications from the user, wherein each second indication of the one or more second indications include modifying the one or more parameters of a second stage of the plurality of stages in the configuration file; and

    executing the decompressor stage and the compressor stage on the FPGA based on the configuration file.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×