Method and apparatus for creating and using persistent images of distributed shared memory segments and in-memory checkpoints

• US 7,779,295 B1
• Filed: 09/30/2005
• Issued: 08/17/2010
• Est. Priority Date: 06/28/2005
• Status: Expired due to Fees

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Shah, Veeral, Borate, Milind
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US11/239,766
• Time in Patent Office: 1,782 Days
• Field of Search: 714/6, 714/7, 714/13, 714/12
• US Class Current: 714/6.32
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A method and apparatus that enable quick recovery from failure or restoration of an application state of one or more nodes, applications, and/or communication links in a distributed computing environment, such as a cluster. Recovery or restoration is facilitated by regularly saving persistent images of the in-memory checkpoint data and/or of distributed shared memory segments using snapshots of the committed checkpoint data. When one or more nodes fail, the snapshots can be read and used to restart the application in the most recently-saved state prior to the failure or rollback the application to an earlier state.

Claims

1. A method comprising:

updating a first checkpoint image, responsive to a first change made by a first instance of an application executing on a plurality of nodes, wherein the first instance of the application is executing on a first node of the plurality of nodes, wherein the first checkpoint image is maintained at the first node, and wherein the first checkpoint image comprises a first state of the first instance of the application;

updating, responsive to a second change committed by any of the first instance of the application and at least one other instance of the application executing on at least one other node of the plurality of nodes, a second checkpoint image maintained at a distributed shared memory shared by the plurality of nodes wherein the distributed shared memory propagates the second checkpoint image to the plurality of nodes, and wherein the second checkpoint image comprises a second state of any of the first and the at least one other instances of the application;

creating, from at least a portion of the distributed shared memory and in response to the second change, a checkpoint-image snapshot comprising data indicative of the second state of any of the first and the at least one other instances of the application; and

storing the checkpoint-image snapshot on at least one node of the plurality of nodes.

2. The method of claim 1, wherein storing the checkpoint-image snapshot comprises:

storing the checkpoint-image snapshot along with at least one other checkpoint-image snapshot stored on at least one node at an earlier time, and wherein the data indicative of the second state comprises;

at least one difference between the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

3. The method of claim 1, further comprising:

executing any of the first and the at least one other instances of the application in accordance with the checkpoint-image snapshot.

4. The method of claim 1, wherein storing the checkpoint-image snapshot comprises:

storing, at the at least one node, a transaction identifier number corresponding to the checkpoint-image snapshot.

5. The method of claim 2, further comprising:

executing any of the first and the at least one other instances of the application in accordance with any of the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

6. The method of claim 5, wherein respective transaction identifiers are associated with the checkpoint-image snapshot and the at least one other checkpoint-image snapshot, and wherein executing any of the first and the at least one other instances of the application comprises:

using at least one of the transaction identifiers to select any of the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

7. The method of claim 5, wherein executing any of the first and the at least one other instances of the application comprises:

performing any of a rollback or roll-forward using any of the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

8. The method of claim 1, wherein the data indicative of the second change comprises the second state of the application.

9. The method of claim 2, further comprising:

updating, responsive to the second change, a bitmap to reflect that at least one portion of the second checkpoint image differs from the at least one other checkpoint-image snapshot.

10. The method of claim 1, wherein:

updating the second checkpoint image comprises;

updating, at a first node of the plurality of nodes, in response to the second change being committed by the first instance of the application, a first instance of the second checkpoint image, wherein the first instance of the second checkpoint image comprises a first instance of the second state; and

updating, at the at least one other node, in response to the second change being committed by the at least one other instance of the application, a second instance of the second checkpoint image, wherein the second instance of the second checkpoint image comprises a second instance of the second state; and

creating the checkpoint-image snapshot comprises;

creating, from at least a portion of the distributed shared memory and in response to the second change, a first checkpoint-image snapshot comprising data indicative of the first instance of the second state; and

creating, from at least a portion of the distributed shared memory and in response to the second change, a second checkpoint-image snapshot comprising data indicative of the second instance of the second state; and

storing the checkpoint-image snapshot comprises;

storing the first and second checkpoint-image snapshots on the at least one node of the plurality of nodes.

11. An apparatus comprising:

means for updating a first checkpoint image, responsive to a first change made by a first instance of an application executing on a plurality of nodes, wherein the first instance of the application is executing on a first node of the plurality of nodes, wherein the first checkpoint image is maintained at the first node, and wherein the first checkpoint image comprises a first state of the first instance of the application;

means for updating, responsive to a second change committed by any of the first instance of the application and at least one other instance of the application executing on at least one other node of the plurality of nodes, a second checkpoint image maintained at a distributed shared memory shared by the plurality of nodes wherein the distributed shared memory propagates the second checkpoint image to the plurality of nodes, and wherein the second checkpoint image comprises a second state of any of the first and the at least one other instances of the application;

means for creating, from at least a portion of the distributed shared memory and in response to the second change, a checkpoint-image snapshot comprising data indicative of the second state of any of the first and the at least one other instances of the application; and

storage for storing the checkpoint-image snapshot.

12. The apparatus of claim 11, wherein said storage stores the checkpoint-image snapshot along with at least one other checkpoint-image snapshot stored at an earlier time, and wherein the data indicative of the second state comprises:

at least one difference between the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

13. The apparatus of claim 12, further comprising:

a means for executing any of the first and the at least one other instances of the application in accordance with any of the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

14. The apparatus of claim 11, further comprising:

means for executing any of the first and the at least one other instances of the application in accordance with the checkpoint-image snapshot.

15. The apparatus of claim 11, wherein the storage stores the checkpoint-image snapshot along with a transaction identifier number corresponding to the checkpoint-image snapshot.

16. The apparatus of claim 13, wherein respective transaction identifiers are associated with the checkpoint-image snapshot and the at least one other checkpoint-image snapshot, and wherein the means for executing any of the first and the at least one other instances of the application comprises:

a means for using at least one of the transaction identifiers to select any of the checkpoint-image snapshot and the at least one other checkpoint-image snapshot.

17. The apparatus of claim 11 wherein the storage comprises a disk drive.

18. The apparatus of claim 11 wherein the storage is managed by a file system or volume manager.

19. The apparatus of claim 11, wherein the data indicative of the second change comprises the second state of the application.

20. The apparatus of claim 12, further comprising:

a means for updating, responsive to the second change, a bitmap to reflect that at least one portion of the second checkpoint image differs from the at least one other checkpoint-image snapshot.