Method and system for providing coordinated checkpointing to a group of independent computer applications

• US 8,082,468 B1
• Filed: 12/15/2008
• Issued: 12/20/2011
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: Backensto, Keith Richard, Havemose, Allan
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Lottich, Joshua P
• Application Number:
  US12/334,640
• Time in Patent Office: 1,100 Days
• Field of Search: 709/213, 709/223, 714/4, 714/7, 714/15, 714/4.1, 714/6.3, 714/38.1, 717/128, 717/170, 717/171, 719/320
• US Class Current: 714/15
• CPC Class Codes: G06F 11/1407 Checkpointing the instruction stream
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035
• CPC Class Codes: G06F 9/30145 Instruction analysis, e.g. decoding, instruction word fields
• CPC Class Codes: G06F 9/461 Saving or restoring of program or task context
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence

Abstract

A system and method thereof for performing loss-less migration of an application group. In an exemplary embodiment, the system may include a high-availability services module structured for execution in conjunction with an operating system, and one or more computer nodes of a distributed system upon which at least one independent application can be executed upon. The high-availability services module may be structured to be executable on the one or more computer nodes for loss-less migration of the one or more independent applications, and is operable to perform checkpointing of all state in a transport connection.

Claims

1. A system for performing loss-less migration of an application group comprised of one or more independent applications, the system comprising:

a high-availability services module structured for execution in conjunction with an operating system;

interceptors configured to intercept fork( ) and exec( );

a checkpointer structured to checkpoint and restore applications that may use fork( ) and exec( ) and structured to incrementally checkpoint and restore applications that may write memory pages from kernel space; and

one or more computer nodes of a distributed system upon which at least one independent application can be executed upon, wherein

the high-availability services module is structured to be executable on the one or more computer nodes for loss-less migration of the one or more independent applications, and is operable to perform checkpointing and restoration of all state in a transport connection.

2. The system according to claim 1, wherein the high-availability services module is further operable to coordinate the checkpointing of the state of the transport connection across the application group.

3. The system according to claim 2, wherein the high-availability services module is further operable to restore all states in the transport connection to the state they were in at a last checkpoint.

4. The system according to claim 3, wherein the high-availability services module is further operable to coordinate recovery within a restore procedure that is coupled to the transport connection.

5. A method of performing loss-less migration of an application group comprised of one or more independent applications, the method comprising:

migrating independent applications within the application group, without loss, from their respective primary nodes to at least one backup node;

maintaining transparency to a client connected to the primary node over a transport connection; and

restoring the application group, including all of the independent applications, from checkpoints in response to initiating recovery of the application, wherein

said one or more independent applications can join or leave said application group at any time, and

said one or more independent applications can use both fork( ) and exec( ).

6. The method according to claim 5, further comprising flushing and halting the transport connection during a taking of the checkpoints.

7. The method according to claim 5, wherein the transparency is maintained by automatically coordinating transparent recovery of distributed applications of the application group.

8. A method for launching and running an independent application using a coordinator, the method comprising:

installing interceptors for fork( ) and exec( );

registering the application using the coordinator; and

running the application, wherein

said independent application is a member of an application group comprised of one or more independent applications;

upon the application issuing a fork call, control of the application passes to the interceptors, whereupon the interceptors call an operating system fork( ), which in turn creates a new application process from a parent process, and then passes control of the application back to the interceptors; and

upon the application issuing an exec call, control of the application passes to the interceptors, whereupon the interceptors call an operating system exec( ), which in turn overlays a new application image onto the current process and then passes control of the application back to the interceptors.

9. The method according to claim 8, further comprising registering the new application process and updating process information for the parent process.

10. The method according to claim 9, further comprising resuming execution of the parent process and executing a child process.

11. The method according to claim 10, further comprising terminating and unregistering the parent process and the child process.

12. The method according to claim 11, further comprising terminating the application.

13. A method for launching and running an independent application using a coordinator, the method comprising:

preloading a custom_init( ) function along with loading said independent application;

installing interceptors for fork( ) and exec( );

registering the application using the coordinator; and

calling the independent application'"'"'s entry point to run the independent application;

wherein

upon the application issuing an exec call, control of the application passes to the interceptors, whereupon the interceptors call an operating system exec( ), which in turn overlays a new application image onto an existing application process, and said interceptors for fork( ) and exec( ) run in user-space.

14. The method according to claim 13, further comprising initializing the new image by registering the new image, with the coordinator, restoring internal state from shared memory, and executing the new image.

17. The method according to claim 14, wherein the shared memory is configured to be outside the address space of the independent application.

19. The method according to claim 13, further comprising using shared memory to preserve key system states across the exec( ) and the fork( ), wherein said shared memory is configured to be outside the address space of the independent application.

15. A method for launching and running an independent application using a coordinator, the method comprising:

installing interceptors for fork( ) and exec( );

registering the application using the coordinator;

initializing the new image, including registration with the coordinator, restoration of all internal state from shared memory, and executing the new image; and

running the application, wherein

upon the application issuing an exec call, control of the application passes to the interceptors, whereupon the interceptors call an operating system exec( ), which in turn overlays a new image onto an existing application process, and wherein an environment variable is used to preserve across exec( ), a number of times a process has exec'"'"' ed.

16. The method according to claim 15, further comprising storing in shared memory at least one of a global application state including checkpoint barrier information having a barrier semaphore ID, a Virtual PID table, a Pipe table, a Semaphore ID table, and a SysV shared memory segment ID table.

18. The method according to claim 15, wherein the ID of said shared memory and CPENV_EXEC are both written to an environment and used for re-initialization.

20. A method for launching and running an independent application using a coordinator, the method comprising:

installing interceptors for fork( ) and exec( );

registering the application using the coordinator;

initializing the new image, including registration with the coordinator, restoration of all internal state from shared memory, and executing the new image; and

running the application, wherein

upon the application issuing an exec call, control of the application passes to the interceptors, whereupon the interceptors call an operating system exec( ), which in turn overlays a new image onto an existing application process, and wherein an environment variable CPENV_EXEC is used during restore to preserve, across exec, a number of times a process must exec prior to restoring its data.