Systems and methods for building an extract, transform, load pipeline
First Claim
1. A computer-implemented method for building an extract, transform, load (ETL) pipeline, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
- identifying a plurality of ETL resources employable in the ETL pipeline that are available to a user;
categorizing each of the plurality of ETL resources based on at least one characteristic of each of the plurality of ETL resources that affects a capability of the ETL pipeline when employed in the ETL pipeline;
provisioning the plurality of ETL resources for use with containers based on the at least one characteristic of each of the plurality of ETL resources;
presenting a user environment to the user, the user environment including a plurality of container images corresponding to available ETL resources of the plurality of ETL resources;
receiving, from the user, a selection of at least one container image of the plurality of container images, wherein the at least one container image includes at least one application that performs at least one action to be carried out in the ETL pipeline;
determining two or more ETL resources from the plurality of ETL resources that are best fit to carry out the at least one action based on the at least one characteristic of each of the plurality of ETL resources; and
running each of at least one container from the at least one container image that includes files and parameters for running the at least one container, the at least one container utilizing the two or more ETL resources of the plurality of ETL resources, wherein at least one of the two or more ETL resources comprises at least one server that hosts the at least one container, the at least one container isolating user space of the at least one container from other processes while sharing kernel space with the other processes on the at least one server.
5 Assignments
0 Petitions
Accused Products
Abstract
The disclosed computer-implemented method for building an extract, transform, load pipeline may include (i) identifying a plurality of ETL resources available to a user, (ii) categorizing each of the plurality of ETL resources based on at least one characteristic, (iii) provisioning the plurality of ETL resources for use with containers, (iv) presenting a user environment to the user, the user environment including a plurality of container images corresponding to available ETL resources, (v) receiving, from the user, a selection of at least one container image of the plurality of container images, and (vi) running at least one container from the at least one container image, the at least one container utilizing two or more ETL resources of the plurality of ETL resources. Various other methods, systems, and computer-readable media are also disclosed.
-
Citations
20 Claims
-
1. A computer-implemented method for building an extract, transform, load (ETL) pipeline, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
-
identifying a plurality of ETL resources employable in the ETL pipeline that are available to a user; categorizing each of the plurality of ETL resources based on at least one characteristic of each of the plurality of ETL resources that affects a capability of the ETL pipeline when employed in the ETL pipeline; provisioning the plurality of ETL resources for use with containers based on the at least one characteristic of each of the plurality of ETL resources; presenting a user environment to the user, the user environment including a plurality of container images corresponding to available ETL resources of the plurality of ETL resources; receiving, from the user, a selection of at least one container image of the plurality of container images, wherein the at least one container image includes at least one application that performs at least one action to be carried out in the ETL pipeline; determining two or more ETL resources from the plurality of ETL resources that are best fit to carry out the at least one action based on the at least one characteristic of each of the plurality of ETL resources; and running each of at least one container from the at least one container image that includes files and parameters for running the at least one container, the at least one container utilizing the two or more ETL resources of the plurality of ETL resources, wherein at least one of the two or more ETL resources comprises at least one server that hosts the at least one container, the at least one container isolating user space of the at least one container from other processes while sharing kernel space with the other processes on the at least one server. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for building an ETL pipeline, the system comprising:
-
an identifying module, stored in memory, that identifies a plurality of ETL resources employable in the ETL pipeline that are available to a user; a categorizing module, stored in memory, that categorizes each of the plurality of ETL resources based on at least one characteristic of each of the plurality of ETL resources that affects a capability of the ETL pipeline when employed in the ETL pipeline; a provisioning module, stored in memory, that provisions the plurality of ETL resources for use with containers based on the at least one characteristic of each of the plurality of ETL resources; an interface module, stored in memory, that presents a user environment to the user, the user environment including a plurality of container images corresponding to available ETL resources of the plurality of ETL resources; a receiving module, stored in memory, that receives, from the user, a selection of at least one container image of the plurality of container images, wherein the at least one container image includes at least one application that performs at least one action to be carried out in the ETL pipeline; a determining module, stored in memory, that determines two or more ETL resources from the plurality of ETL resources that are best fit to carry out the at least one action based on the at least one characteristic of each of the plurality of ETL resources; a running module, stored in memory, that runs each of at least one container from the at least one container image that includes files and parameters for running the at least one container, the at least one container utilizing two or more ETL resources of the plurality of ETL resources, wherein at least one of the two or more ETL resources comprises at least one server that hosts the at least one container, the at least one container isolating user space of the at least one container from other processes while sharing kernel space with the other processes on the at least one server; and at least one processor that executes the identifying module, the categorizing module, the provisioning module, the interface module, the receiving module, the determining module, and the running module.
-
-
20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
-
identify a plurality of ETL resources employable in an ETL pipeline that are available to a user; categorize each of the plurality of ETL resources based on at least one characteristic of each of the plurality of ETL resources that affects a capability of the ETL pipeline when employed in the ETL pipeline; provision the plurality of ETL resources for use with containers based on the at least one characteristic of each of the plurality of ETL resources; present a user environment to the user, the user environment including a plurality of container images corresponding to available ETL resources of the plurality of ETL resources; receive, from the user, a selection of at least one container image of the plurality of container images, wherein the at least one container image includes at least one application that performs at least one action to be carried out in the ETL pipeline; determine two or more ETL resources from the plurality of ETL resources that are best fit to carry out the at least one action based on the at least one characteristic of each of the plurality of ETL resources; and run each of at least one container from the at least one container image that includes files and parameters for running the at least one container, the at least one container utilizing two or more ETL resources of the plurality of ETL resources, wherein at least one of the two or more ETL resources comprises at least one server that hosts the at least one container, the at least one container isolating user space of the at least one container from other processes while sharing kernel space with the other processes on the at least one server.
-
Specification