Resilient programming frameworks for handling failures in parallel programs

• US 10,296,424 B2
• Filed: 04/07/2017
• Issued: 05/21/2019
• Est. Priority Date: 03/13/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Iyengar, Arun, Milthorpe, Joshua J.
• Primary Examiner(s): Kudirka, Joseph R
• Application Number:
  US15/481,888
• Publication Number:
  US 20170212808A1
• Time in Patent Office: 774 Days
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

An information processing system, computer readable storage medium, and method for supporting resilient execution of computer programs. A method provides a resilient store wherein information in the resilient store can be accessed in the event of a failure. The method periodically checkpoints application state in the resilient store. A resilient executor comprises software which executes applications by catching failures. The method uses the resilient executor to execute at least one application. In response to the resilient executor detecting a failure, restoring application state information to the at least one application from a checkpoint stored in the resilient store, the resilient executor resuming execution of the at least one application with the restored application state information.

Claims

1. An information processing system capable of supporting resilient execution of applications written in a programming language with exception handling, the information processing system comprising:

memory;

persistent memory for storing data and computer instructions;

a resilient executor, communicatively coupled with the memory and the persistent memory, for executing computations of applications while detecting failures in the execution of the computations; and

at least one processor, communicatively coupled with the resilient executor, the memory, the persistent memory, and wherein the at least one processor, responsive to executing computer instructions, performs operations comprising;

providing an interface allowing programs to explicitly reference a place to communicate with or execute at least one computation on the place, wherein each place comprises an entity executing a computation;

providing a virtual place abstraction layer which defines a mapping between virtual places and physical places;

providing an interface allowing an application to communicate with or execute at least one computation on a physical place p1 by referencing a virtual place p2 which is mapped to physical place p1; and

in response to a physical place p3 failing, wherein virtual place p4 maps to physical place p3, updating the mapping so that virtual place p4 maps to physical place p5 wherein physical place p5 is live.

2. The information processing system of claim 1 further comprising:

a resilient store, communicatively coupled with the memory and the persistent memory, wherein application state information stored in the resilient store is accessible in response to detection of a failure of an application executing in the information processing system.

3. The information processing system of claim 2 wherein the at least one processor performs operations further comprising:

periodically checkpointing an application state in the resilient store.

4. The information processing system of claim 3 wherein the at least one processor performs operations further comprising:

executing, with the resilient executor, computations of the application while detecting failures in the execution of the computations, wherein the resilient executor includes computer code which is part of the application.

5. The information processing system of claim 4 wherein the at least one processor performs operations further comprising:

restoring, based on the resilient executor detecting a failure in the execution of a computation of the application by catching with said computer code at least one exception, application state information for the application from a checkpoint in the resilient store; and

resuming, with the resilient executor, execution of the computation of the application with the restored application state information.

6. The information processing system of claim 5, wherein the resilient executor comprises:

an interface allowing applications to use the resilient executor;

a resilient run method which the application invokes via the interface which executes the application while detecting and catching place failures as exceptions; and

a recovery method which is invoked when the resilient run method catches an exception resulting from a failed place wherein the recovery method recovers from the place failure, restores the application to a previous checkpoint in the resilient store, and resumes execution of the application from the restored checkpoint.

7. The information processing system of claim 1, wherein at least one of physical place p1, physical place p3, and physical place p5, comprises at least one of a process and a thread.

8. The information processing system of claim 1, wherein the resilient executor includes a method for resiliently executing an iterative computation.

9. The information processing system of claim 1, wherein the resilient executor handles an iterative computation by invoking at least one of an application-specific method to execute an iteration of a computation of the application and an application-specific method to determine if the computation has finished.

10. The information processing system of claim 1, wherein the resilient executor handles at least one exception by recursively catching and handling additional exceptions which occur.

11. A computer readable storage medium, comprising computer instructions which, responsive to being executed by a processor of an information processing system, cause the processor to perform operations for supporting resilient execution of applications written in a programming language with exception handling, the information processing system comprising memory for storing data and computer instructions, and a resilient executor for executing computations of the applications while detecting failures in the execution of the computations, the operations comprising:

providing an interface allowing programs to explicitly reference a place to communicate with or execute at least one computation on the place, wherein each place comprises an entity executing a computation;

providing a virtual place abstraction layer which defines a mapping between virtual places and physical places;

providing an interface allowing an application to communicate with or execute at least one computation on a physical place p1 by referencing a virtual place p2 which is mapped to physical place p1; and

in response to a physical place p3 failing, wherein virtual place p4 maps to physical place p3, updating the mapping so that virtual place p4 maps to physical place p5 wherein physical place p5 is live.

12. The computer readable storage medium of claim 11, wherein the information processing system further comprises a resilient store, communicatively coupled with the memory, wherein application state information stored in the resilient store is accessible in response to detection of a failure of an application executing in the information processing system, and wherein the operations further comprise:

in response to detection of a failure of an application executing in the information processing system, the resilient store accessing the application state information stored in the resilient store.

13. The computer readable storage medium of claim 12, wherein the operations further comprise:

periodically checkpointing an application state in the resilient store.

14. The computer readable storage medium of claim 13, wherein the operations further comprise:

executing, with the resilient executor, computations of the application while detecting failures in the execution of the computations, wherein the resilient executor includes computer code which is part of the application.

15. The computer readable storage medium of claim 14, wherein the operations further comprise:

restoring, based on the resilient executor detecting a failure in the execution of a computation of the application by catching with said computer code at least one exception, application state information for the application from a checkpoint in the resilient store; and

resuming, with the resilient executor, execution of the computation of the application with the restored application state information.

16. The computer readable storage medium of claim 15, wherein the resilient executor further includes:

an interface allowing applications to use the resilient executor;

a resilient run method which the application invokes via the interface which executes the application while detecting and catching place failures as exceptions; and

a recovery method which is invoked when the resilient run method catches an exception resulting from a failed place wherein the recovery method recovers from the place failure, restores the application to a previous checkpoint in the resilient store, and resumes execution of the application from the restored checkpoint, and wherein the operations further comprise;

invoking, via the interface, the resilient run method which executes the application while detecting and catching place failures as exceptions; and

invoking the recovery method based on the resilient run method catching an exception resulting from a particular failed place, the recovery method;

recovering from the particular failed place failure;

restoring the application to a previous checkpoint in the resilient store; and

resuming execution of the application from the restored checkpoint.

17. The computer readable storage medium of claim 11, wherein at least one of physical place p1, physical place p3, and physical place p5, comprises at least one of a process and a thread.

18. The computer readable storage medium of claim 11, wherein the resilient executor further includes a method for resiliently executing an iterative computation, and wherein the operations further comprise:

resiliently executing, with the method included in the resilient executor, an iterative computation on physical place p1 by referencing virtual place p2 which is mapped to physical place p1.

19. The computer readable storage medium of claim 11, wherein the resilient executor handles an iterative computation by invoking at least one of an application-specific method to execute an iteration of a computation of the application and an application-specific method to determine if the computation has finished, and wherein the operations further comprise:

handling, with the resilient executor, an iterative computation by invoking at least one of the application-specific method to execute an iteration of a computation of the application and the application-specific method to determine if the computation has finished.

20. The computer readable storage medium of claim 11, wherein the operations further comprise:

handling, with the resilient executor, at least one exception by recursively catching and handling additional exceptions which occur.