Data storage system with on-demand recovery of file import metadata during file system migration

• US 10,545,913 B1
• Filed: 04/30/2017
• Issued: 01/28/2020
• Est. Priority Date: 04/30/2017
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Liao, Zexian, Bankier, John, Long, Fei, Qian, Yue, Wang, Xianfeng
• Primary Examiner(s): Kuddus, Daniel A
• Application Number:
  US15/582,713
• Time in Patent Office: 1,003 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency

Abstract

A user I/O operation is processed in a file system of a target data storage system (DSS) performing a file system migration. The target DSS assesses a migration state of a user file of the user I/O operation, including (1) determining whether a metadata file storing migration state data for the user file has an already recovered record indicating the migration state of the file, (2) if so, then using the already recovered record to assess the migration state of the user file, and (3) otherwise (a) initiating an on-demand recovery to obtain an on-demand record for the user file, and (b) using the on-demand recovered record to assess the migration state. If the migration state indicates that the file has not yet been migrated, then the user I/O operation is redirected to a separate source DSS, and otherwise the user I/O operation is completed locally at the target DSS.

Claims

1. A computer-implemented method of processing a user I/O operation in a file system of a data storage system, the data storage system operating as a target data storage system performing a migration operation to migrate the file system to the target data storage system from a separate source data storage system, the method comprising the steps, performed by the target data storage system during a restart of the target data storage system occurring during the migration operation, of:

assessing a migration state of a user file being the target of the user I/O operation, including (1) first determining whether a metadata file storing migration state data for the user file has an already recovered record for the user file accurately indicating the migration state of the file, the already recovered record having been recovered by a separate background recovery process performed during the restart, (2) when the metadata file has an already recovered record for the user file, then using the already recovered record to assess the migration state of the user file, and (3) when the metadata file does not have an already recovered record for the user file, then (a) initiating an on-demand recovery to obtain an on-demand recovered record for the user file, and (b) using the on-demand recovered record to assess the migration state of the user file; and

when the assessed migration state indicates that the file has not yet been migrated to the data storage system, then redirecting the user I/O operation to a separate source data storage system for processing the user I/O operation, and otherwise completing the user I/O operation by accessing the file as previously migrated to the data storage system from the source data storage system,

wherein the on-demand recovery includes use of an on-demand recovered map being a content-addressed listing of inode numbers for files whose metadata records have been on-demand recovered during a post-restart recovery period.

2. The method of claim 1, wherein the on-demand recovery is performed in addition to a separate background recovery to recover metadata records not involved in current user I/O operations, the background recovery using a background recovery scan pointer continually pointing to a next sequential record to be background recovered, and wherein all of the inode numbers in the on-demand recovered map are generally greater than the value of the background recovery scan pointer at a given time.

3. The method of claim 1, wherein the assessing includes use of an in-memory lock map to manage potential race conditions in accessing/updating records of the metadata file.

4. The method of claim 3, wherein the lock map is content-addressed and implemented using a hash function of inode number.

5. The method of claim 3, wherein reading or recovering a metadata record of the metadata file first acquires a read/write lock, the lock being added into the memory lock map when the lock is acquired for the first time, and the lock being removed from the memory lock map after a background recovery function has scanned the record.

6. The method of claim 1, wherein the determining includes two sub-determinations including (1) whether an inode number for the target file is less than a background recovery scan pointer, in which case a background recovery process has already recovered the record and thus the record is usable for assessing migration state, and (2) whether the inode number for the target file is in the on-demand recovered map, in which case the record has already been on-demand recovered and thus it is usable for assessing migration state.

7. The method of claim 1, wherein the metadata file is implemented as a sparse file having a logical size corresponding to a theoretical maximum number of inodes of the file system, whereas an actual number of inodes during operation corresponds to a number of actually existing files substantially less than the theoretical maximum, the sparse file being directly indexed by inode number for accessing individual entries, and wherein it is generally necessary to scan the entire metadata file for recovery, requiring a time period longer than a timeout period for processing the user I/O operation.

8. The method of claim 1, performed during a reboot after a non-clean shutdown in which the migration state as recorded in a metadata file does not accurately describe the actual migration state of the respective file.

9. The method of claim 1, further including use of a background recovery process implemented in a multi-threaded manner employing multiple independent execution threads to collectively scan an entirety of the metadata file in the background while user I/O operations are proceeding in the foreground.

10. A data storage system including secondary storage devices and processing and interface circuitry configured and operative to execute computer program instructions to realize a file system and to perform a method of processing a user I/O operation in the file system, and to operate the data storage system as a target data storage system performing a migration operation to migrate the file system to the target data storage system from a separate source data storage system during a restart of the target data storage system occurring during the migration operation, the method including:

assessing a migration state of a user file being the target of the user I/O operation, including (1) first determining whether a metadata file storing migration state data for the user file has an already recovered record for the user file accurately indicating the migration state of the file, the already recovered record having been recovered by a separate background recovery process performed during the restart, (2) when the metadata file has an already recovered record for the user file, then using the already recovered record to assess the migration state of the user file, and (3) when the metadata file does not have an already recovered record for the user file, then (a) initiating an on-demand recovery to obtain an on-demand recovered record for the user file, and (b) using the on-demand recovered record to assess the migration state of the user file; and

when the assessed migration state indicates that the file has not yet been migrated to the data storage system, then redirecting the user I/O operation to a separate source data storage system for processing the user I/O operation, and otherwise completing the user I/O operation by accessing the file as previously migrated to the data storage system from the source data storage system,

wherein the on-demand recovery includes use of an on-demand recovered map being a content-addressed listing of inode numbers for files whose metadata records have been on-demand recovered during a post-restart recovery period.

11. The data storage system of claim 10, wherein the on-demand recovery is performed in addition to a separate background recovery to recover metadata records not involved in current user I/O operations, the background recovery using a background recovery scan pointer continually pointing to a next sequential record to be background recovered, and wherein all of the inode numbers in the on-demand recovered map are generally greater than the value of the background recovery scan pointer at a given time.

12. The data storage system of claim 10, wherein the assessing includes use of an in-memory lock map to manage potential race conditions in accessing/updating records of the metadata file.

13. The data storage system of claim 12, wherein the lock map is content-addressed and implemented using a hash function of inode number.

14. The data storage system of claim 12, wherein reading or recovering a metadata record of the metadata file first acquires a read/write lock, the lock being added into the memory lock map when the lock is acquired for the first time, and the lock being removed from the memory lock map after a background recovery function has scanned the record.

15. The data storage system of claim 10, wherein the determining includes two sub-determinations including (1) whether an inode number for the target file is less than a background recovery scan pointer, in which case a background recovery process has already recovered the record and thus the record is usable for assessing migration state, and (2) whether the inode number for the target file is in the on-demand recovered map, in which case the record has already been on-demand recovered and thus it is usable for assessing migration state.

16. The data storage system of claim 10, wherein the metadata file is implemented as a sparse file having a logical size corresponding to a theoretical maximum number of inodes of the file system, whereas an actual number of inodes during operation corresponds to a number of actually existing files substantially less than the theoretical maximum, the sparse file being directly indexed by Mode number for accessing individual entries, and wherein it is generally necessary to scan the entire metadata file for recovery, requiring a time period longer than a timeout period for processing the user I/O operation.

17. The data storage system of claim 10, wherein the method is performed during a reboot after a non-clean shutdown in which the migration state as recorded in a metadata file does not accurately describe the actual migration state of the respective file.

18. The data storage system of claim 10, wherein the method further includes use of a background recovery process implemented in a multi-threaded manner employing multiple independent execution threads to collectively scan an entirety of the metadata file in the background while user I/O operations are proceeding in the foreground.