Extensible data transformation authoring and validation system
First Claim
1. A method performed by one or more processors, the method comprising:
- receiving user code comprising at least a portion of a computer program via data retrieval from a computer data storage device or input from a computer input device;
digitally storing the user code at a first code repository;
building the user code into executable code comprising one or more machine executable computer program files, wherein building the user code comprises;
generating the executable code, based on the user code and a first criteria, and digitally storing the executable code in a second repository which is different from the first code repository, wherein the first criteria comprises a criteria for obfuscating sensitive data and the second repository is a blobstore;
generating a job specification, based on the user code and a second criteria, and digitally storing the job specification in a computer memory;
launching computer execution of an application server, using the executable code and the job specification;
receiving data transformation instructions that specify a process for transforming a first dataset to a second dataset;
in response to receiving data transformation instructions, executing, using the executable code, the data transformation instructions in the application server.
8 Assignments
0 Petitions
Accused Products
Abstract
Data transformation in a distributed system of applications and data repositories is described. The subsystems for the overall framework are distributed, thereby allowing for customization to require only isolated changes to one or more subsystems. In one embodiment, a source code repository is used to receive and store source code. A build subsystem can retrieve source code from the source code repository and build it, using one or more criteria. By building the source code, the build subsystem can generate an artifact, which is executable code, such as a JAR or SQL file. Likewise, by building the source code, the build subsystem can generate one or more job specifications for executing the executable code. In one embodiment, the artifact and job specification may be used to launch an application server in a cluster. The application server can then receive data transformation instructions and execute the data transformation instructions.
186 Citations
12 Claims
-
1. A method performed by one or more processors, the method comprising:
-
receiving user code comprising at least a portion of a computer program via data retrieval from a computer data storage device or input from a computer input device; digitally storing the user code at a first code repository; building the user code into executable code comprising one or more machine executable computer program files, wherein building the user code comprises; generating the executable code, based on the user code and a first criteria, and digitally storing the executable code in a second repository which is different from the first code repository, wherein the first criteria comprises a criteria for obfuscating sensitive data and the second repository is a blobstore; generating a job specification, based on the user code and a second criteria, and digitally storing the job specification in a computer memory; launching computer execution of an application server, using the executable code and the job specification; receiving data transformation instructions that specify a process for transforming a first dataset to a second dataset; in response to receiving data transformation instructions, executing, using the executable code, the data transformation instructions in the application server. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. One or more non-transitory computer-readable storage medium storing one or more sequences of instructions which, when executed by one or more processors, cause:
-
receiving user code comprising at least a portion of a computer program via data retrieval from a computer data storage device or input from a computer input device; digitally storing the user code at a first code repository; building the user code into executable code comprising one or more machine executable computer program files, wherein building the user code comprises; generating the executable code, based on the user code and a first criteria, and digitally storing the executable code in a second repository which is different from the first code repository, wherein the first criteria comprises a criteria for obfuscating sensitive data and the second repository is a blobstore; generating a job specification, based on the user code and a second criteria, and digitally storing the job specification in a computer memory; launching computer execution of an application server, using the executable code and the job specification; receiving data transformation instructions that specify a process for transforming a first dataset to a second dataset; and in response to receiving data transformation instructions, executing, using the executable code, the data transformation instructions in the application server. - View Dependent Claims (8, 9, 10, 11, 12)
-
Specification