SYSTEMS AND METHODS FOR DATA MIGRATION IN A CLUSTERED FILE SYSTEM

• US 20100250508A1
• Filed: 03/11/2010
• Published: 09/30/2010
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Granted Patent: US 8,209,307 B2
• Time in Patent Office: Days
• US Class Current: 707/704
• CPC Class Codes: G06F 12/0864 using pseudo-associative means, e.g. set-associative or hashing
• CPC Class Codes: G06F 12/0891 using clearing, invalidating or resetting means
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 2212/60 Details of cache memory
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

Systems and methods for providing more efficient handling of I/O requests for clustered file system data subject to data migration or the like. For instance, exemplary systems can more quickly determine if certain files on primary storage represent actual file data or stub data for recalling file data from secondary storage. Certain embodiments utilize a driver cache on each cluster node to maintain a record of recently accessed files that represent regular files (as opposed to stubs). A dual-locking process, using both strict locking and relaxed locking, maintains consistency between driver caches on different nodes and the data of the underlying clustered file system, while providing improved access to the data by the different nodes. Moreover, a signaling process can be used, such as with zero-length files, for alerting drivers on different nodes that data migration is to be performed and/or that the driver caches should be flushed.

Claims

1. A method for coordinating access to data in a storage management system, the method comprising:

storing in a shared file system on a primary storage device a plurality of files comprising a first plurality of regular files and a plurality of stub files, the plurality of stub files being associated with a second plurality of regular files having been migrated to secondary storage;

maintaining in a first driver cache of a first computer a first indication of first inodes associated with at least one of the first plurality of regular files on the primary storage device;

maintaining in a second driver cache of a second computer a second indication of second inodes associated with at least one of the first plurality of regular files on the primary storage device;

requesting with each of the first and second computers a read-only lock on a signal file on the shared file system to determine whether or not a migration operation is about to occur with respect to at least one of the first plurality of regular files on the primary storage device;

when the request for the read-only lock on the signal file is denied of at least one of the first and second computers, clearing the respective first and/or second indications from the respective first and/or second driver caches of the at least one of the first and second computers; and

when the request for the read-only lock on the signal file is granted to at least one of the first and second computers, unlocking the signal file and re-requesting the read-only lock on the signal file after a predetermined period of time.

2. The method of claim 1, wherein said requesting is performed by file system drivers of the first and second computers.

3. The method of claim 1, additionally comprising, when said request for the read-only lock is denied, releasing a second read-only lock on a token file.

4. The method of claim 3, additionally comprising performing migration of the at least one of the first plurality of regular files only after no second read-only locks by the first and second computers remain on the token file.

5. The method of claim 4, additionally comprising obtaining with a migration service an exclusive lock on both the signal and token files prior to performing said migration.

6. The method of claim 3, wherein at least one of the signal file and the token file comprises a zero-length file.

7. The method of claim 1, additionally comprising populating the first driver cache with additional indications of first inodes associated with additional ones of the first plurality of regular files on the primary storage device after the request by the first computer for the read-only lock on the signal file is granted.

8. The method of claim 1, wherein the first and second computers are independent nodes of a clustered file system.

9. The method of claim 1, wherein the predetermined period of time is between approximately two seconds and approximately ten seconds.

10. A system for coordinating access to data in a shared storage environment, the system comprising:

a first computer comprising a first memory storing copies of first inodes in a shared file system that represent first data files that have not been migrated from primary storage to secondary storage;

a second computer comprising a second memory storing copies of second inodes in the shared file system that represent second data files that have not been migrated from primary storage to secondary storage, the first and second memories not storing copies of inodes that represent stub files;

first and second file system drivers executing on, respectively, said first and second computers, each of the first and second file system drivers being configured to request a shared lock on a signal file on the shared file system; and

when the request for the shared lock on the signal file is denied of at least one of the first and second file system drivers, clearing the respective copies of the first and/or second inodes from the respective first and/or second memories, and

when the request for the shared lock on the signal file is granted to at least one of the first and second file system drivers, unlocking the signal file and re-requesting the shared lock on the signal file after a predetermined period of time.

11. The system of claim 10, wherein the signal file comprises a zero-length file stored on primary storage.

12. The system of claim 10, wherein each of the first and second computers further comprises, respectively, first and second migration services for migrating data files from primary storage to secondary storage.

13. The system of claim 12, wherein the first and second migration services are each configured to obtain an exclusive lock on the signal file prior to performing said migration.

14. The system of claim 13, wherein, when the request for the shared lock on the signal file is denied of the first file system driver, the first file system driver is configured to release a shared lock on a token file.

15. The system of claim 14, wherein the first and second migration services are each configured to obtain an exclusive lock on the token file after all shared locks are released on the token file and prior to performing said migration.

16. The system of claim 10, wherein the first and second memories comprise, respectively, first and second driver caches.

17. The system of claim 10, wherein the first and second computers are independent nodes of a clustered file system.

18. The system of claim 10, wherein at least one of the first data files is the same as at least one of the second data files.

19. A system for coordinating access to data in a shared storage environment, the system comprising:

first means for storing on a first computing device a first indication of first inodes in a shared file system that represent first data files that have not been migrated from primary storage to secondary storage;

second means for storing on a second computing device a second indication of second inodes in the shared file system that represent second data files that have not been migrated from primary storage to secondary storage, the first and second means for storing not storing indications of inodes that represent stub files;

third means for requesting a non-exclusive lock on a signal file on the shared file system to determine whether or not a migration operation is about to occur on data in the shared file system; and

fourth means for requesting a non-exclusive lock on the signal file on the shared file system to determine whether or not a migration operation is about to occur at least on at least one of the first and second data files,

and when the request for the non-exclusive lock on the signal file is denied of at least one of the third and fourth means for requesting, clearing the first and/or second indications of the, respective, first and/or second means for storing,

and when the request for the non-exclusive lock on the signal file is granted to at least one of the third and fourth means for requesting, unlocking the signal file and re-requesting with the at least one of the third and fourth means for requesting the non-exclusive lock on the signal file after a predetermined period of time.

20. The system of claim 19, wherein, when the request for the non-exclusive lock on the signal file is denied of either of the third and fourth means for requesting, the third or fourth means for requesting is configured to release a shared lock on a token file.