Methods and apparatus for shared storage journaling

• US 7,433,898 B1
• Filed: 04/26/2005
• Issued: 10/07/2008
• Est. Priority Date: 06/01/2004
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Sanbolic, Inc. (Cloud Software Group)
• Inventors: Georgiev, Ivan I.
• Primary Examiner(s): Alam; Shahid A.
• Assistant Examiner(s): NGUYEN, PHONG H
• Application Number:
  US11/115,041
• Time in Patent Office: 1,260 Days
• Field of Search: 707/9, 707/201, 707/202, 711/163, 726/27, 714/2, 714/15
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Multiple computers have access to a shared storage system. The computers negotiate with each other for access rights (e.g., READ or WRITE) to regions of the storage system. After negotiating for permission to modify a location (e.g., a range of addresses) of the shared storage system, a given computer copies data from the shared storage system into its own corresponding cache. The given computer granted the permission then modifies the data in its cache while maintaining a separate journal of corresponding changes to metadata. The given computer writes the journal to a dedicated region of the storage system. Thereafter, the given computer attempts to flush contents of the modified cache to the storage system. If the given computer crashes while attempting to flush the modified cache to the shared storage system, a healthy one of the multiple computers utilizes the journal information in the storage system to recover metadata.

Claims

1. A method for use in a shared storage system, the method comprising:

detecting a change to file system metadata stored in a first computer of multiple computers having access to the shared storage system;

creating a journal, the journal identifying executed operational steps used to change the file system metadata stored in the first computer;

storing the journal to the shared storage system, storage of the journal enabling a second computer to subsequently retrieve and execute at least portions of the journal to carry out the executed operational steps to modify the file system metadata of the shared storage system in accordance with a failed attempt by the first computer to change a copy of the file system metadata in the shared storage system;

maintaining access right schedule information, the access right schedule information identifying which of the multiple computers had access to different regions of the shared storage at different times; and

storing at least a portion of the access right schedule information to the journal.

2. A method as in claim 1 further comprising:

retrieving the file system metadata as well as corresponding application data from the shared storage system;

storing the retrieved file system metadata as well as the corresponding application data in the first computer; and

initiating the change to the retrieved file system metadata stored in the first computer in response to modifying the corresponding application data stored in the first computer.

3. A method as in claim 1, wherein storing the journal includes:

storing the journal to the shared storage system before storing of changed file system metadata to the shared storage.

4. A method as in claim 3 further comprising:

after storing the journal to the shared storage system, initiating storage of the file system metadata including the change to the shared storage system.

5. A method as in claim 1, wherein storing the journal includes:

storing the journal to a specific journal data region in the shared storage system allocated for use by the first computer to store the journal, other journal data regions of the shared storage system being allocated for use by other computers to store their corresponding journals.

6. A method as in claim 1, wherein creating a journal in the first computer includes:

generating a journal entry for the journal, the journal entry including a set of executable operational steps that are performed to recreate the change to the file system metadata in the first computer, wherein the operational steps are i) atomic and ii) idempotent.

7. A method as in claim 1 further comprising:

generating time stamps for corresponding journal entries stored in the journal, the time stamps indicating a relative time when the first computer makes a corresponding change to the file system metadata in the first computer.

8. A method as in claim 1, further comprising:

maintaining respective journals associated with the multiple computers having access to the shared storage system, the respective journals indicating operational steps associated with changes initiated by the multiple computers to different regions of the shared storage at different times;

detecting a failure in the first computer; and

utilizing the journal of the first computer stored in the shared storage system as well as all of the respective journals of other offline computers and the access schedule information of all online computers to generate a map identifying which non-overlapping regions in the shared storage system need to be modified by replaying portions of the journal associated with the first computer.

9. A method as in claim 1 further comprising:

detecting a failure in the first computer;

generating a request to access a journal in the shared storage system associated with the first computer in response to detecting the failure, the journal including journal entries of modification operations associated with the first computer prior to the failure; and

on behalf of the first computer, retrieving and replaying at least portions of the journal associated with the first computer in the shared storage system.

10. A method as in claim 1 further comprising:

via communications from the first computer, negotiating with the multiple computers to obtain access rights to a specific region of the shared storage system being accessed by the first computer.

23. A method as in claim 10, wherein storing at least a portion of the access right schedule information to the journal includes storing timing information indicating when the first computer initiated the change to the file system metadata stored in the first computer relative to a second computer attempting a modification of a copy of the file system metadata at the second computer.

11. A computer system for reducing data corruption in a shared storage system, the computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor;

a communication interface that supports communication with other nodes of the storage area network; and

an interconnect coupling the processor, the memory unit, and the communication interface, enabling the computer system to execute the application and perform operations of;

detecting a change to file system metadata initiated by a first computer of multiple computers having access to the shared storage system;

creating a journal, the journal identifying operational steps associated with the change to the file system metadata;

storing the journal to the shared storage system, storage of the journal enabling a second computer to subsequently retrieve and execute at least portions of the journal to carry out the executed operational steps to modify the file system metadata of the shared storage system in accordance with a failed attempt by the first computer to change a copy of the file system metadata in the shared storage system;

maintaining access right schedule information, the access right schedule information identifying which of the multiple computers had access to different regions of the shared storage at different times; and

storing at least a portion of the access right schedule information to the journal.

12. A computer system as in claim 11 that additionally performs operations of:

retrieving the file system metadata from the shared storage system;

storing the retrieved file system metadata in the first computer; and

initiating the change to the retrieved file system metadata stored in the first computer.

13. A computer system as in claim 11, wherein storing the journal includes:

storing the journal to the shared storage system is done before the storing of the changed file system metadata to the shared storage.

14. A computer system as in claim 13 that additionally performs operations of:

after storing the journal to the shared storage system, initiating storage of the file system metadata including the change to the shared storage system.

15. A computer system as in claim 11, wherein storing the journal includes:

storing the journal to a specific journal data region in the shared storage system allocated for use by the first computer to store the journal, other journal data regions of the shared storage system being allocated for use by other computers to store their corresponding journals.

16. A computer system as in claim 11, wherein creating a journal in the first computer includes:

generating a journal entry for the journal, the journal entry including a set of executable operational steps that may be performed by to recreate the change to the file system metadata in the first computer, wherein the operational steps are i) atomic and ii) idempotent.

17. A computer system as in claim 11 that additionally performs operations of:

generating time stamps for corresponding journal entries stored in the journal, the time stamps indicating a relative time when the first computer makes a corresponding change to the file system metadata in the first computer.

18. A computer system as in claim 11 that additionally performs operations of:

maintaining respective journals associated with the multiple computers having access to the shared storage system, the respective journals indicating operational steps associated with changes initiated by the multiple computers to different regions of the shared storage at different times;

detecting a failure in the first computer; and

utilizing the journal of the first computer stored in the shared storage system as well as all of the respective journals of other offline computers and the access schedule information of all online computers to generate a map identifying which non-overlapping regions in the shared storage system need to be modified by replaying portions of the journal associated with the first computer.

19. A computer system as in claim 11 that additionally performs operations of:

detecting a failure in the first computer;

generating a request to access a journal in the shared storage system associated with the first computer in response to detecting the failure, the journal including journal entries of modification operations associated with the first computer prior to the failure; and

on behalf of the first computer, retrieving and replaying at least portions of the journal associated with the first computer in the shared storage system.

20. A computer system as in claim 11 that additionally performs operations of:

negotiating with the multiple computers for access rights to a specific region of the shared storage system.

21. A computer program product including a computer-storage medium having instructions stored thereon for processing data information, such that the instructions, when carried out by a processing device, enable the processing device to perform the steps of:

detecting a change to file system metadata stored in a first computer of multiple computers having access to a shared storage system;

creating a journal, the journal identifying operational steps associated with the change to the file system metadata;

storing the journal to the shared storage system, storage of the journal enabling a second computer to subsequently retrieve and execute at least portions of the journal to carry out the executed operational steps to modify the file system metadata of the shared storage system in accordance with a failed attempt by the first computer to change a copy of the file system metadata in the shared storage system;

maintaining access right schedule information, the access right schedule information identifying which of the multiple computers had access to different regions of the shared storage at different times; and

storing at least a portion of the access right schedule information to the journal.

22. A computer system having access to a shared storage system, the computer system including a processor coupled to a memory, and the computer system including:

means for detecting a change to file system metadata stored in a first computer of multiple computers having access to the shared storage system;

means for creating a journal, the journal identifying operational steps associated with the change to the file system metadata; and

means for storing the journal to the shared storage system, storage of the journal enabling a second computer to subsequently retrieve and execute at least portions of the journal to carry out the executed operational steps to modify the file system metadata of the shared storage system in accordance with a failed attempt by the first computer to change a copy of the file system metadata in the shared storage system;

means for maintaining access right schedule information, the access right schedule information identifying which of the multiple computers had access to different regions of the shared storage at different times; and

means for storing at least a portion of the access right schedule information to the journal.