Method and system for providing storage checkpointing to a group of independent computer applications

• US 8,826,070 B1
• Filed: 12/15/2008
• Issued: 09/02/2014
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: Havemose, Allan, Backensto, Keith Richard
• Primary Examiner(s): Chu, Gabriel
• Assistant Examiner(s): BRYAN, JASON B
• Application Number:
  US12/334,657
• Time in Patent Office: 2,087 Days
• US Class Current: 714/13
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method and system for checkpointing at least one application in an application group. At least one full checkpoint and at least one incremental checkpoint are created for the application in the application group. The at least one incremental application checkpoint is merged against the at least one full application checkpoint, and checkpointing across all applications in the application group is synchronized. A storage checkpoint is taken for at least one of the full checkpoint and the incremental checkpoint, and memory and storage checkpoints are synchronized and consistent.

Claims

1. A method of checkpointing one or more applications in an application group, the method comprising:

preloading interceptors for fork( ) and exec( ) for the one or more applications in the application group;

preloading interceptors for file operations for the one or more applications in the application group;

creating at least one full checkpoint for the one or more applications in the application group;

creating at least one incremental checkpoint for the one or more applications in the application group;

merging the at least one incremental application checkpoint against the at least one full application checkpoint;

synchronizing checkpointing across all applications in the application group; and

taking a storage checkpoint for at least one of the at least one full checkpoint and the at least one incremental checkpoint;

wherein the one or more applications are comprised of one or more processes, and each process comprised of one or more threads;

wherein the one or more applications in the application group uses fork( ) and exec( ) in any combination;

wherein intercepted file operations are added to a file operations data structure upon entering said file operations interceptors, intercepted file operations are removed from said file operations data structure upon completion of said file operations, and checkpointing is disabled between adding said intercepted file operations to said file operations data structure and a start of execution of said file operations; and

wherein checkpointing of user-space pages is comprised of write-protecting one or more user-space memory pages, detecting an attempt to write to said one or more user-space memory pages using an EFAULT error code, and upon receiving said EFAULT error code including said one or more user-space memory pages in a checkpoint.

2. The method as claimed in claim 1, wherein said memory and storage checkpoints are stored on locally attached storage.

3. The method as claimed in claim 1, wherein said memory and storage checkpoints of the one or more applications in an application group are stored on a Network Attached Storage.

4. The method as claimed claim 1, wherein said memory and storage checkpoints of the one or more applications in an application group are stored on Storage Area Network storage.

5. The method as claimed in claim 1, further comprising forcing cache consistency by performing one or more of writing of a token file, flushing caches, and reading the token file.

6. The method as claimed in claim 1, further comprising restoring consistent memory and storage checkpoints.

7. The method as claimed in claim 1, further comprising utilizing standard operating systems and system libraries and utilizing interception for all customization.

8. The method as claimed in claim 7, wherein the interception is built into the standard operating systems and the system libraries.

9. The method as claimed in claim 1, wherein the application group is a virtual machine program, and said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

10. A method as recited in claim 1, wherein said memory and storage checkpoints are written to storage and read from storage over a wireless network and said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

11. A non-transitory computer readable medium storing a plurality of computer executable instructions for execution by a processor to checkpoint one or more applications in an application group, the computer executable instructions for:

preloading interceptors for fork( ) and exec( ) for the one or more applications in the application group;

preloading interceptors for file operations for the one or more applications in the application group;

creating at least one full checkpoint for the one or more applications in an application group;

creating at least one incremental checkpoint for the one or more applications in the application group;

merging the at least one incremental application checkpoint against the at least one full application checkpoint;

synchronizing checkpointing across all applications in the application group; and

taking a storage checkpoint for at least one of the at least one full checkpoint and the at least one incremental checkpoint;

wherein the at one or more applications are comprised of one or more processes, and each process comprised of one or more threads;

wherein the one or more applications in the application group use fork( ) and exec( ) in any combination;

wherein intercepted file operations are added to a file operations data structure upon entering said file operations interceptors, intercepted file operations are removed from said file operations data structure upon completion of said file operations, and checkpointing is disabled between adding said intercepted file operations to said file operations data structure and a start of execution of said file operations; and

wherein checkpointing of user-space pages is comprised of write-protecting one or more user-space memory pages, detecting an attempt to write to said one or more user-space memory pages using an EFAULT error code, and upon receiving said EFAULT error code including said one or more user-space memory pages in a checkpoint.

12. The non-transitory computer readable medium as claimed in claim 11, wherein the instructions further comprise forcing cache consistency by performing one or more of writing of a token file, flushing caches, and reading the token file.

13. The non-transitory computer readable medium as claimed in claim 11, wherein the instructions further comprise restoring consistent memory and storage checkpoints.

14. The non-transitory computer readable medium as claimed in claim 11, wherein the instructions further comprise utilizing standard operating systems and system libraries and utilizing interception for all customization.

15. The non-transitory computer readable medium as claimed in claim 11, wherein the instructions further comprise accessing said memory and storage checkpoints over a wireless work, and said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

16. A system for checkpointing one or more applications in an application group, the system comprising:

a storage disk structured to store files;

a storage access interface structured to access the files stored on the storage disk; and

a processor structured to process instructions for;

preloading interceptors for fork( ) and exec( ) for the one or more applications in the application group;

preloading interceptors for file operations for the one or more applications in the application group;

creating at least one full checkpoint for the one or more applications in the application group;

creating at least one incremental checkpoint for the one or more applications in the application group;

merging the at least one incremental application checkpoint against the at least one full application checkpoint;

synchronizing checkpointing across all applications in the application group; and

taking a storage checkpoint for at least one of the at least one full checkpoint and the at least one incremental checkpoint;

wherein the one or more applications are comprised of one or more processes, and each process comprised of one or more threads;

wherein the one or more applications in the application group uses fork( ) and exec( ) in any combination;

wherein intercepted file operations are added to a file operations data structure upon entering said file operations interceptors, intercepted file operations are removed from said file operations data structure upon completion of said file operations, and checkpointing is disabled between adding said intercepted file operations to said file operations data structure and a start of execution of said file operations; and

wherein checkpointing of user-space pages is comprised of write-protecting one or more user-space memory pages, detecting an attempt to write to said one or more user-space memory pages using an EFAULT error code, and upon receiving said EFAULT error code including said one or more user-space memory pages in a checkpoint.

17. The system as claimed in claim 16, wherein the storage disk is structured as part of Network Attached Storage, and

wherein said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

18. The system as claimed in claim 16, wherein the storage disk is structured as part of a Storage Area Network, and

wherein said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

19. The system as claimed in claim 16, wherein the application group comprises a virtual machine program, and

wherein said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.

20. The system as claimed in claim 16, further comprising a wireless network structured to access the files stored on the storage disk,

wherein said memory and storage checkpoints are comprised of checkpoints of the one or more applications in the application group.