Data file migration from a mirrored RAID to a non-mirrored XOR-based RAID without rewriting the data

• US 20030188097A1
• Filed: 03/29/2002
• Published: 10/02/2003
• Est. Priority Date: 03/29/2002
• Status: Active Grant

• Current Assignee: Panasas
• Original Assignee: Panasas
• Inventors: Welch, Brent B., Holland, Mark C.
• Granted Patent: US 6,985,995 B2
• Time in Patent Office: Days
• US Class Current: 711/114
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A data storage methodology wherein a data file is initially stored in a format consistent with RAID-1 and RAID-X and then migrated to a format consistent with RAID-X and inconsistent with RAID-1 when the data file grows in size beyond a certain threshold. Here, RAID-X refers to any non-mirrored storage scheme employing XOR-based error correction coding (e.g., a RAID-5 configuration). Each component object (including the data objects and the parity object) for the data file is configured to be stored in a different stripe unit per object-based secure disk. Each stripe unit may store, for example, 64 KB of data. So long as the data file does not grow beyond the size threshold of a stripe unit (e.g., 64 KB), the parity stripe unit contains a mirrored copy of the data stored in one of the data stripe units because of the exclusive-ORing of the input data with “all zeros” assumed to be contained in empty or partially-filled stripe units. When the file grows beyond the size threshold, the parity stripe unit starts storing parity information instead of a mirrored copy of the file data. Thus, the data file can be automatically migrated from a format consistent with RAID-1 and RAID-X to a format consistent with RAID-X and inconsistent with RAID-1 without the necessity to duplicate or rewrite the stored data.

Claims

1. A method for storing a data file in a storage system having a plurality of storage disks, said method comprising:

striping said data file across said plurality of storage disks using a plurality of data stripe units and a parity stripe unit;

initially storing said data file in a first format consistent with RAID-1 and RAID-X using a first data stripe unit from said plurality of data stripe units and said parity stripe unit, wherein said parity stripe unit maintains a mirrored copy of a first data contained in said first data stripe unit, and RAID-X refers to a non-mirrored RAID scheme employing an XOR-based error correcting code;

continuing storing said data file in said first format consistent with RAID-1 and RAID-X until the size of said first data exceeds a storage space allocated to said first data stripe unit in one of said storage disks; and

migrating from storing said data file in said first format consistent with RAID-1 and RAID-X to storing said data file in a second format consistent with RAID-X and inconsistent with RAID-1 when the size of said first data exceeds said storage space allocated to said first data stripe unit, said migration being performed without rewriting said first data.

2. A method for storing a data file in a storage system having a plurality of storage disks, said method comprising:

striping said data file across said plurality of storage disks using a plurality of data stripe units and a parity stripe unit;

initially storing said data file in a first format consistent with RAID-1 and RAID-5 using a first data stripe unit from said plurality of data stripe units and said parity stripe unit, wherein said parity stripe unit maintains a mirrored copy of a first data contained in said first data stripe unit;

continuing storing said data file in said first format consistent with RAID-1 and RAID-5 until the size of said first data exceeds a storage space allocated to said first data stripe unit in one of said storage disks; and

migrating from storing said data file in said first format consistent with RAID-1 and RAID-5 to storing said data file in a second format consistent with RAID-5 and inconsistent with RAID-1 when the size of said first data exceeds said storage space allocated to said first data stripe unit, said migration being performed without rewriting said first data.

3. The method of claim 2, wherein striping said data file includes allocating each stripe unit in said plurality of data stripe units and said parity stripe unit a respective storage space in a different one of said plurality of storage disks.

4. The method of claim 2, wherein initially storing said data file in said first format consistent with RAID-1 and RAID-5 includes:

treating each data stripe unit except said first data stripe unit in said plurality of data stripe units as having all zeros stored therein;

combining said first data in said first data stripe unit with zeros assumed to be contained in each other data stripe unit in said plurality of data stripe units to generate a first parity data therefrom, wherein said first parity data being identical to said first data; and

storing said first parity data in said parity stripe unit.

5. The method of claim 4, wherein combining said first data includes exclusive-ORing said first data in said first data stripe unit with zeros assumed to be contained in each said other data stripe unit in said plurality of data stripe units.

6. The method of claim 5, wherein migrating to storing said data file in said second format consistent with RAID-5 and inconsistent with RAID-1 includes:

monitoring when the size of said first data exceeds said storage space allocated to said first data stripe unit;

storing a second data in a second data stripe unit from said plurality of data stripe units, wherein said second data constitutes a portion of said first data that fails to get stored in said storage space allocated to said first data stripe unit;

exclusive-ORing said first data in said first data stripe unit with said second data in said second data stripe unit and with zeros assumed to be contained in said second data stripe unit and in each said other data stripe unit in said plurality of data stripe units to generate a second parity data therefrom; and

storing said second parity data in said parity stripe unit, thereby effectively migrating storage of said data file from said first format consistent with RAID-1 and RAID-5 to said second format consistent with RAID-5 and inconsistent with RAID-1 without rewriting said first data.

7. A method for storing a data file object in a storage system having a plurality of storage disks, said method comprising:

striping said data file object across said plurality of storage disks using a plurality of data stripe units and one or more parity stripe units, wherein each stripe unit in said plurality of data stripe units and in said one or more parity stripe units is allocated a respective storage space in a different one of said plurality of storage disks;

initially storing said data file object in a first format consistent with RAID-1 and RAID-5 using a first data stripe unit from said plurality of data stripe units and a first parity stripe unit from said one or more parity stripe units, wherein said first parity stripe unit maintains a mirrored copy of a first data contained in said first data stripe unit;

continuing storing said data file object in said first format consistent with RAID-1 and RAID-5 until the size of said first data exceeds said respective storage space allocated to said first data stripe unit; and

migrating from storing said data file object in said first format consistent with RAID-1 and RAID-5 to storing said data file object in a second format consistent with RAID-5 and inconsistent with RAID-1 when the size of said first data exceeds said respective storage space allocated to said first data stripe unit, wherein said migration being performed without rewriting said first data.

8. The method of claim 7, wherein said plurality of storage disks includes a plurality of object based secure disks (OBDs).

9. The method of claim 7, wherein said plurality of data stripe units includes a variable number of data stripe units.

10. The method of claim 7, further comprising:

creating a plurality of component objects for said data file object prior to striping said data file object, wherein each said component object is configured to be stored on a different one of said plurality of storage disks, and wherein all component objects together constitute said data file object; and

associating each said component object with a different stripe unit selected from said plurality of data stripe units and said one or more parity stripe units.

11. The method of claim 10, further comprising:

creating said data file object prior to striping said data file object; and

associating a layout map with said data file object upon creation of said data file object, wherein said layout map describes how said data file object is to be stored in said storage system.

12. The method of claim 11, wherein said layout map contains at least one of the following:

a first information about a storage layout scheme for said data file object including an identification of said data file object and an indication of initial storage of said data file object in said first format consistent with RAID-1 and RAID-5;

a second information about which one or more of said plurality of storage disks are to be used to store said data file object;

a total number of stripe units in said plurality of data stripe units and said one or more parity stripe units;

said respective storage space allocated to each said stripe unit;

a set of object identifiers, wherein each object identifier is associated with a different one of said plurality of component objects; and

a set of initial entries created along with said plurality of component objects, wherein each initial entry marks a corresponding one of said plurality of component objects as a blank object so long as no data is written to said corresponding one of said plurality of component objects.

13. The method of claim 10, further comprising migrating one of said plurality of component objects along with data stored therein from a first sector in a storage disk from said plurality of storage disks to a second sector in said storage disk without disrupting striping for said data file object.

14. The method of claim 7, wherein initially storing said data file object in said first format consistent with RAID-1 and RAID-5 includes:

prior to storing said first data, initially treating each said stripe unit in said plurality of data stripe units as having all zeros stored therein;

commencing storing said first data in said first data stripe unit;

combining said first data in said first data stripe unit with zeros assumed to be contained in each other stripe unit in said plurality of data stripe units to generate a first parity data therefrom, wherein said first parity data being identical to said first data; and

storing said first parity data in said first parity stripe unit.

15. The method of claim 14, wherein combining said first data includes exclusive-ORing said first data in said first data stripe unit with zeros assumed to be contained in each said other stripe unit in said plurality of data stripe units.

16. The method of claim 14, wherein migrating to storing said data file object in said second format consistent with RAID-5 and inconsistent with RAID-5 includes:

monitoring when the size of said first data exceeds said respective storage space allocated to said first data stripe unit;

storing a second data in a second data stripe unit from said plurality of data stripe units, wherein said second data constitutes a portion of said first data that fails to get stored in said respective storage space allocated to said first data stripe unit;

exclusive-ORing said first data in said first data stripe unit with said second data in said second data stripe unit and with zeros assumed to be contained in said second data stripe unit and in each said other stripe unit in said plurality of data stripe units to generate a second parity data therefrom; and

storing said second parity data in said first parity stripe unit, thereby effectively migrating storage of said data file object from said first format consistent with RAID-1 and RAID-5 to said second format consistent with RAID-5 and inconsistent with RAID-1 without rewriting said first data.

17. The method of claim 7, wherein each said respective storage space stores a predetermined size of data.

18. The method of claim 7, further comprising dynamically allocating said respective storage space to each said stripe unit.

19. The method of claim 7, wherein striping said data file object includes:

creating a first stripe having a first plurality of stripe units, wherein said first plurality of stripe units includes a first set of data stripe units from said plurality of data stripe units and said first parity stripe unit, wherein said respective storage space for each stripe unit in said first plurality of stripe units is of a first size; and

creating a second stripe having a second plurality of stripe units, wherein said second plurality of stripe unit includes a second set of data stripe units from said plurality of data stripe units and a second parity stripe unit from said one or more parity stripe units, wherein said respective storage space for each stripe unit in said second plurality of stripe units is of a second size, and wherein said second size is different from said first size.

20. The method of claim 19, wherein said first size is smaller than said second size.

21. The method of claim 19, wherein creating said second stripe includes creating said second stripe when the combined storage space offered by said first set of data stripe units in said first plurality of stripe units is insufficient to store said data file object.

22. An object-based storage system for storing a data file object comprising:

a plurality of object based secure disks (OBDs);

means for treating said data file object as a unitary RAID-5 file object;

means for striping said RAID-5 file object across said plurality of OBDs using a plurality of data stripe units and a parity stripe unit;

means for allocating a respective storage space to each stripe unit in said plurality of data stripe units and said parity stripe unit in a different one of said plurality of OBDs;

means for initially storing said RAID-5 file object as a RAID-1 file object using a first data stripe unit from said plurality of data stripe units and said parity stripe unit, wherein said parity stripe unit maintains a mirrored copy of a first data contained in said first data stripe unit;

means for continuing storing said RAID-5 file object as said RAID-1 file object until the size of said first data exceeds said respective storage space allocated to said first data stripe unit; and

means for migrating from storing said data file object as said RAID-1 file object to said RAID-5 file object when the size of said first data exceeds said respective storage space allocated to said first data stripe unit, wherein said migration being performed without rewriting said first data.

23. The system of claim 22, wherein said means for initially storing said RAID-5 file object as said RAID-1 file object includes:

means for initially treating each said stripe unit in said plurality of data stripe units as having all zeros stored therein;

means for commencing storing said first data in said first data stripe unit;

means for exclusive-ORing said first data in said first data stripe unit with zeros assumed to be contained in each other stripe unit in said plurality of data stripc units to generate a first parity data therefrom, wherein said first parity data being identical to said first data; and

means for storing said first parity data in said parity stripe unit.

24. The system of claim 23, wherein said means for migrating from storing said data file object as said RAID-1 file object to said RAID-5 file object includes:

means for monitoring when the size of said first data exceeds said respective storage space allocated to said first data stripe unit;

means for storing a second data in a second data stripe unit from said plurality of data stripe units, wherein said second data constitutes a portion of said first data that fails to get stored in said respective storage space allocated to said first data stripe unit;

means for exclusive-ORing said first data in said first data stripe unit with said second data in said second data stripe unit and with zeros assumed to be contained in said second data stripe unit and in each said other stripe unit in said plurality of data stripe units to generate a second parity data therefrom; and

means for storing said second parity data in said parity stripe unit, thereby effectively migrating storage of said data file object as said RAID-1 file object to said RAID-5 file object without rewriting said first data.