Invulnerable data movement for file system upgrade

• US 9,256,373 B1
• Filed: 05/19/2014
• Issued: 02/09/2016
• Est. Priority Date: 05/19/2014
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Kamat, Mahesh, Liang, Shuang
• Primary Examiner(s): ELMORE, STEPHEN C
• Application Number:
  US14/281,855
• Time in Patent Office: 631 Days
• Field of Search: 711/114, 711/154, 711/156, 711/203, 707/809, 707/810
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A single virtual storage device file system that abstracts multiple RAID groups of physical storage devices into one virtual device and one first blockset having a plurality of data blocks in a contiguous linear address space is converted into a multiple virtual device file system that abstracts the multiple RAID groups of physical storage devices as separate multiple virtual storage devices each having a separate second blockset and address space, by migrating data in allocated blocks at boundaries of the physical storage device groups to free blocks, partitioning the first blockset at the boundaries into the multiple second blocksets, updating the block metadata of each block, and rebuilding the file system using the block metadata to generate second blockset metadata.

Claims

1. A method of converting a first file system that abstracts a plurality of physical storage devices as a single first virtual device in a logical contiguous linear address space having one first blockset that includes a plurality of blocks, into a second file system that abstracts groups of said plurality of physical storage devices as multiple second virtual devices, comprising:

partitioning said first blockset at boundaries of said groups of said physical storage devices into a plurality of separate independent second blocksets, each second blockset having an independent logical address space and corresponding to one storage group of said physical storage devices;

migrating data from source blocks of said first blockset for each of said one storage groups of physical storage devices to target blocks of a corresponding second blockset of each of said one storage groups;

updating metadata in headers of target blocks of said second blocksets according to locations of said target blocks in said second blocksets; and

rebuilding said first file system as said second file system by abstracting said storage groups of physical storage devices and their corresponding second blocksets as separate virtual storage devices.

2. The method of claim 1, wherein said partitioning comprises splitting said first blockset at disk storage group boundaries into multiple second blocksets such the each second blockset and disk storage group have a one-to-one mapping, and creating new second blockset metadata for the second blockset of each storage group so that the second file system can separately access each second blockset.

3. The method of claim 2, wherein said partitioning comprises creating a separate independent second blockset having its own separate address space for each storage group.

4. The method of claim 1, wherein said migrating comprises moving data using invulnerable data movement to ensure the integrity of said data, and wherein said partitioning comprises identifying allocated blocks of said first blockset at said boundaries of said storage groups;

determining at each boundary a number of blocks necessary to provide a space required for second blockset metadata; and

moving a necessary number of said allocated blocks to unallocated blocks in said first blockset to free said space required for said second blockset metadata.

5. The method of claim 4, wherein said invulnerable data movement comprises writing data and metadata read from said source blocks into a buffer memory, determining that the data written to said buffer memory is correct, writing the data from the buffer memory to said target blocks, and verifying the data written to the target blocks by comparing it to the data in said buffer memory.

6. The method of claim 5, wherein said verifying comprises a read verify after write operation that compares the data written to the target blocks with the data in said buffer memory.

7. The method of claim 5, wherein said writing data from source blocks comprises calculating a checksum of the data written into the buffer memory from each source block, and said determining comprises comparing said checksum with another checksum in said each source block.

8. The method of claim 1, wherein said updating metadata comprises updating a location in said metadata of each target block according to an offset of said target block in a corresponding second blockset.

9. The method of claim 8, wherein said rebuilding said first file system comprises generating blockset header and trailer metadata for each second blockset using said updated metadata of target blocks of said each second blockset.

10. The method of claim 1, wherein said groups of physical storage devices each comprise a RAID array of disks.

11. A method of converting a first file system that abstracts a plurality of physical storage devices as a single virtual device in a logical contiguous linear address space comprising one first blockset that includes a plurality of blocks into a second file system that abstracts said plurality of physical storage devices as multiple separate virtual devices, each separate virtual device having its own separate second blockset and logical address space, comprising:

identifying in said first blockset a boundary between first and second groups of said plurality of physical storage devices, which groups respectively comprise first and second storage groups;

determining an amount of space required for second blockset metadata of a new second blockset associated with said first storage group;

migrating data in allocated blocks at said boundary to unallocated blocks in said first blockset to free a number of blocks at said boundary that is necessary to provide said space required for said second blockset metadata;

updating block metadata in headers of blocks of said first blockset that contain data of said first storage group according to locations of said blocks in said second blockset; and

rebuilding said first file system to convert the first file system to said second file system.

12. The method of claim 11 further comprising repeating said identifying, said determining, and said migrating multiple times to form multiple groups of physical storage devices corresponding to said multiple virtual devices, and wherein said second file system comprises a multiple virtual device file system.

13. The method of claim 11, wherein said migrating comprises moving data of said allocated blocks using invulnerable data movement.

14. The method of claim 13, wherein said invulnerable data movement comprises writing data and metadata read from said allocated blocks into a buffer memory, first verifying the data written to said buffer memory is correct, writing the data from buffer memory to an unallocated block in said first blockset, and second verifying the data written to the unallocated block by comparing it to the data in said buffer memory.

15. The method of claim 14, wherein said metadata read from said allocated blocks comprises a first checksum of the data in said allocated block, and said first verifying comprises computing a second checksum of the data written to said buffer from said allocated block, and comparing said first and second checksums.

16. The method of claim 11, wherein said updating comprises updating said block metadata according to offsets of said blocks within the second blockset.

17. The method of claim 11, wherein said rebuilding said first file system comprises generating blockset header and trailer metadata for said first and second blocksets from block metadata of the blocks in each blockset.