Program development in a distributed server environment

• US 9,280,407 B2
• Filed: 04/16/2014
• Issued: 03/08/2016
• Est. Priority Date: 04/16/2014
• Status: Expired due to Fees

• Current Assignee: Snowflake, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Brodsky, Stephen A., Dawra, Anshul, Friedrich, Thomas, Nguyen, Benjamin, Noble-Thomas, Robin M., Thirumurthy, Rohit R., Thitte, Sudarshan R.
• Primary Examiner(s): VU, TUAN A
• Application Number:
  US14/254,418
• Publication Number:
  US 20150301928A1
• Time in Patent Office: 692 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/32 with visual or acoustical indication of the functioning of the machine
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route

Abstract

According to one embodiment of the present invention, a system for processing a computer program for a distributed server environment executes the computer program in the distributed server environment in accordance with a configuration for the computer program. The computer program is generated within a development environment. The configuration indicates a type of server environment and is associated with one or more data sets for processing by the computer program. The system monitors execution of the computer program within the distributed server environment and provides a program status to the development environment. The system displays results produced by the computer program within the distributed server environment via the development environment. Embodiments of the present invention further include a method and computer program product for processing a computer program in substantially the same manners described above.

Claims

1. A system for processing a computer program in a distributed server environment comprising:

a development environment comprising at least one processor and a memory configured to;

generate a computer program and configuration information associated with one or more data sets for processing by the computer program, wherein the configuration information includes;

a specification of artifacts of the computer program, and

a definition of a distributed server environment to execute the computer program comprising credentials to access the distributed server environment and storage location information defined within the configuration information, wherein the storage location information includes storage locations for one or more input data sets and at least one storage location at which the results produced by the computer program are stored;

submit the computer program and the configuration information to a distributed server system to execute the computer program in the distributed server environment in accordance with the configuration information;

receive a program status from the distributed server system comprising information about a state of execution of the computer program and the results produced by the computer program within the distributed server environment; and

identify errors of the computer program by comparing results produced by one or more program runs within the distributed server environment.

2. The system of claim 1, wherein the distributed server environment provides the program status to the development environment in response to the development environment polling the distributed server environment.

3. The system of claim 1, wherein the development environment imports the results produced by the computer program to a client system.

4. The system of claim 1, wherein displaying the results further comprises:

displaying the results based on a type of data within the results.

5. The system of claim 1, wherein the at least one first processor and the first memory are further configured to:

retrieve one or more logs of information from the distributed server environment pertaining to execution of the computer program for display in the development environment.

6. The system of claim 1, wherein the configuration information is associated with a plurality of computer program units, and wherein:

executing the computer program further comprises;

executing the plurality of computer program units in the distributed server environment in accordance with the configuration information;

monitoring execution of the computer program further comprises;

monitoring execution of the plurality of computer program units within the distributed server environment and providing a program status for each computer program unit to the development environment; and

displaying results further comprises;

displaying results produced by the plurality of computer program units within the distributed server environment via the development environment.

7. The system of claim 1, wherein the distributed server system is a cluster using a distributed processing framework,

wherein the cluster comprises a manager system and at least two processing nodes, and

wherein the manager system:

distributes the computer program to the at least two processing nodes, and

combines results from the at least two processing nodes.

8. A computer program product for processing a computer program in a distributed server environment comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith for execution on a processing system, the computer readable program code comprising computer readable program code configured to;

generate a computer program and configuration information associated with one or more data sets for processing by the computer program, wherein the configuration information includes;

a specification of artifacts of the computer program, and

a definition of a distributed server environment to execute the computer program comprising credentials to access the distributed server environment and storage location information defined within the configuration information, wherein the storage location information includes storage locations for one or more input data sets and at least one storage location at which the results produced by the computer program are stored;

submit the computer program and the configuration information to a distributed server system to execute the computer program in the distributed server environment in accordance with the configuration information;

receive a program status from the distributed server system comprising information about a state of execution of the computer program and the results produced by the computer program within the distributed server environment; and

identify errors of the computer program by comparing results produced by one or more program runs within the distributed server environment in the first memory.

9. The computer program product of claim 8, wherein the distributed server environment provides the program status to the development environment in response to the development environment polling the distributed server environment.

10. The computer program product of claim 8, wherein the development environment imports the results produced by the computer program to a client system.

11. The computer program product of claim 8, wherein displaying the results further comprises:

displaying the results based on a type of data within the results.

12. The computer program product of claim 8, wherein the computer readable program code is further configured to:

retrieve one or more logs of information from the distributed server environment pertaining to execution of the computer program for display in the development environment.

13. The computer program product of claim 8, wherein the configuration information is associated with a plurality of computer program units, and wherein:

executing the computer program further comprises;

executing the plurality of computer program units in the distributed server environment in accordance with the configuration information;

monitoring execution of the computer program further comprises;

monitoring execution of the plurality of computer program units within the distributed server environment and providing a program status for each computer program unit to the development environment; and

displaying results further comprises;

displaying results produced by the plurality of computer program units within the distributed server environment via the development environment.

14. The computer program product of claim 8, wherein the distributed server system is a cluster using a distributed processing framework,

wherein the cluster comprises a manager system and at least two processing nodes, and

wherein the manager system:

distributes the computer program to the at least two processing nodes, and combines results from the at least two processing nodes.