Data file migration from a mirrored RAID to a non-mirrored XOR-based RAID without rewriting the data

• US 6,985,995 B2
• Filed: 03/29/2002
• Issued: 01/10/2006
• Est. Priority Date: 03/29/2002
• Status: Expired due to Term

• Current Assignee: Panasas
• Original Assignee: Panasas
• Inventors: Welch, Brent B., Holland, Mark C.
• Primary Examiner(s): Kim, Matthew
• Assistant Examiner(s): Li, Zhuo H.
• Application Number:
  US10/109,998
• Publication Number:
  US 20030188097A1
• Time in Patent Office: 1,383 Days
• Field of Search: 711/114, 711161-162, 711/100, 711/111, 711/112, 714 5- 6, 719/325, 709201-203, 707/204, 710/79, 716/6
• US Class Current: 711/114
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A data storage methodology wherein a data file is initially stored in a format consistent with RAID-1 and RAID-X and then migrated to a format consistent with RAID-X and inconsistent with RAID-1 when the data file grows in size beyond a certain threshold. Here, RAID-X refers to any non-mirrored storage scheme employing XOR-based error correction coding (e.g., a RAID-5 configuration). Each component object (including the data objects and the parity object) for the data file is configured to be stored in a different stripe unit per object-based secure disk. Each stripe unit may store, for example, 64 KB of data. So long as the data file does not grow beyond the size threshold of a stripe unit (e.g., 64 KB), the parity stripe unit contains a mirrored copy of the data stored in one of the data stripe units because of the exclusive-ORing of the input data with “all zeros” assumed to be contained in empty or partially-filled stripe units. When the file grows beyond the size threshold, the parity stripe unit starts storing parity information instead of a mirrored copy of the file data. Thus, the data file can be automatically migrated from a format consistent with RAID-1 and RAID-X to a format consistent with RAID-X and inconsistent with RAID-1 without the necessity to duplicate or rewrite the stored data.

Claims

1. A method for storing a data file in a storage system having a plurality of storage disks, said method comprising:

(a) providing a stripe across said plurality of storage disks, said stripe comprising a plurality of data stripe units and a parity stripe unit;

(b) storing said data file in a first format consistent with RAID-1 using only a first data stripe unit from said plurality of data stripe units and said parity stripe unit until a size of said data file exceeds an allocated storage space in said first data stripe unit, wherein during step (b) said parity stripe unit maintains a mirrored copy of data contained in said first data stripe unit; and

(c) when the size of the data file exceeds said allocated storage space, storing said data file in a second format consistent with RAID-X and inconsistent with RAID-1 using said first data stripe unit, at least one further data stripe unit and the parity stripe unit wherein no rewriting of the data contained in the first data stripe unit occurs when storage of the data file transitions from said first format to said second format, and wherein RAID-X refers to a non-mirrored RAID scheme employing an XOR-based error correcting code.

2. A method for storing a data file in a storage system having a plurality of storage disks, said method comprising:

(a) providing a stripe across said plurality of storage disks, said stripe comprising a plurality of data stripe units and a parity stripe unit;

(b) storing said data file in a first format consistent with RAID-1 using only a first data stripe unit from said plurality of data stripe units and said parity stripe unit until a size of said data file exceeds an allocated storage space in said first data stripe unit, wherein during step (b) said parity stripe unit maintains a mirrored copy of data contained in said first data stripe unit; and

(c) when the size of the data file exceeds said allocated storage space, storing said data file in a second format consistent with RAID-5 and inconsistent with RAID-1 using said first data stripe unit, at least one further data stripe unit and the parity stripe unit wherein no rewriting of the data contained in the first stripe unit occurs when storage of the data file transitions from said first format to said second format.

3. The method of claim 2, wherein step (a) includes allocating each stripe unit in said plurality of data stripe units and said parity stripe unit a respective storage space in a different one of said plurality of storage disks.

4. The method of claim 2, wherein the storing of said data file in said first format consistent with RAID-1 and RAID-5 in step (b) includes:

treating each data stripe unit except said first data stripe unit in said plurality of data stripe units as having all zeros stored therein;

combining data in said first data stripe unit with zeros assumed to be contained in each other data stripe unit in said plurality of data stripe units to generate a first parity data therefrom, wherein said first parity data is identical to said data in said first data stripe unit; and

storing said first parity data in said parity stripe unit.

5. The method of claim 4, wherein the combining of said data in said first data stripe unit includes exclusive-ORing said data in said first data stripe unit with zeros assumed to be contained in each said other data stripe unit in said plurality of data stripe units.

6. The method of claim 5, further comprising:

monitoring when the size of the data file exceeds said allocated storage space;

wherein step (c) comprises;

storing data in the at least one further data stripe unit, wherein said data stored in the at least one further data stripe unit constitutes a portion of said data file that fails to get stored in said allocated storage space in said first data stripe unit; and

exclusive-ORing said data in said first data stripe unit with said data in said at least one further data stripe unit.

7. A method for storing a data file object in a storage system having a plurality of storage disks, said method comprising:

(a) providing a stripe across said plurality of storage disks, said stripe comprising a plurality of data stripe units and one or more parity stripe units, wherein each stripe unit in said plurality of data stripe units and in said one or more parity stripe units is allocated a respective storage space in a different one of said plurality of storage disks;

(b) storing said data file object in a first format consistent with RAID-1 using only a first data stripe unit from said plurality of data stripe units and a first parity stripe unit from said one or more parity stripe units until a size of said data file object exceeds an allocated storage space in said first data stripe unit, wherein during step (b) said first parity stripe unit maintains a mirrored copy of data contained in said first data stripe unit; and

(c) when the size of the data file object exceeds said allocated storage space, storing said data file object in a second format consistent with RAID-5 and inconsistent with RAID-1 using said first data stripe unit, at least one further data stripe unit and one or more parity stripe units wherein no rewriting of the data contained in the first data stripe unit occurs when storage of the data file object transitions from said first format to said second format.

8. The method of claim 7, wherein said plurality of storage disks includes a plurality of object based secure disks (OBDs).

9. The method of claim 7, wherein said plurality of data stripe units includes a variable number of data stripe units.

10. The method of claim 7, further comprising:

creating a plurality of component objects for said data file object, wherein each said component object is configured to be stored on a different one of said plurality of storage disks, and wherein all component objects together constitute said data file object; and

associating each said component object with a different stripe unit selected from said plurality of data stripe units and said one or more parity stripe units.

11. The method of claim 10, further comprising:

creating said data file object prior to striping said data file object; and

associating a layout map with said data file object upon creation of said data file object, wherein said layout map describes how said data file object is to be stored in said storage system.

12. The method of claim 11, wherein said layout map contains at least one of the following:

a first information about a storage layout scheme for said data file object including an identification of said data file object and an indication of initial storage of said data file object in said first format consistent with RAID-1 and RAID-5;

a second information about which one or more of said plurality of storage disks are to be used to store said data file object;

a total number of stripe units in said plurality of data stripe units and said one or more parity stripe units;

said respective storage space allocated to each said stripe unit;

a set of object identifiers, wherein each object identifier is associated with a different one of said plurality of component objects; and

a set of initial entries created along with said plurality of component objects, wherein each initial entry marks a corresponding one of said plurality of component objects as a blank object so long as no data is written to said corresponding one of said plurality of component objects.

13. The method of claim 7, wherein the storing of said data file object in said first format consistent with RAID-1 and RAID-5 in step (b) includes:

initially treating each said stripe unit in said plurality of data stripe units as having all zeros stored therein;

commencing storing said data file object in said first data stripe unit;

combining data in said first data stripe unit with zeros assumed to be contained in each other stripe unit in said plurality of data stripe units to generate a first parity data therefrom, wherein said first parity data is identical to said data in said first data stripe unit; and

storing said first parity data in said first parity stripe unit.

14. The method of claim 13, wherein the combining of said data in said first data stripe unit includes exclusive-ORing said data in said first data stripe unit with zeros assumed to be contained in each said other stripe unit in said plurality of data stripe units.

15. The method of claim 13, further comprising:

monitoring when the size of the data file object exceeds said allocated storage space;

wherein step (c) comprises;

storing data in the at least one further data stripe unit, wherein said data stored in the at least one further data stripe unit constitutes a portion of said data file object that fails to get stored in said allocated storage space in said first data stripe unit; and

exclusive-ORing said data in said first data stripe unit with said data in said at least one further data stripe unit.

16. The method of claim 7, wherein said allocated storage space stores a predetermined size of data.

17. The method of claim 7, further comprising dynamically allocating storage space to each said stripe unit.

18. The method of claim 7, wherein step (a) further comprises:

creating a first stripe having a first plurality of stripe units, wherein said first plurality of stripe units includes a first set of data stripe units from said plurality of data stripe units and said first parity stripe unit, wherein respective storage space for each stripe unit in said first plurality of stripe units is of a first size; and

creating a second stripe having a second plurality of stripe units, wherein said second plurality of stripe unit includes a second set of data stripe units from said plurality of data stripe units and a second parity stripe unit from said one or more parity stripe units, wherein respective storage space for each stripe unit in said second plurality of stripe units is of a second size, and wherein said second size is different from said first size.

19. The method of claim 18, wherein said first size is smaller than said second size.

20. The method of claim 18, wherein creating said second stripe includes creating said second stripe when the combined storage space offered by said first set of data stripe units in said first plurality of stripe units is insufficient to store said data file object.

21. An object-based storage system for storing a data file object comprising:

a plurality of object based secure disks (OBDs);

means for striping said data file object across said plurality of OBDs using a plurality of data stripe units and a parity stripe unit;

means for allocating a respective storage space to each stripe unit in said plurality of data stripe units and said parity stripe unit in a different one of said plurality of OBDs;

first means for storing said data file object in a first format consistent with RAID-1 using only a first data stripe unit from said plurality of data stripe units and said parity stripe unit until a size of said data file object exceeds an allocated storage space in said first data stripe unit, wherein said first means for storing causes said parity stripe unit to maintain a mirrored copy of a data contained in said first data stripe unit; and

second means for storing said data file object in a second format consistent with RAID-5 and inconsistent with RAID-1 using said first data stripe unit, at least one further data stripe unit and said parity stripe unit when the size of said data file object exceeds said allocated storage space, wherein no rewriting of the data contained in the first data stripe unit occurs when storage of the data file object transitions from said first format to said second format.

22. The system of claim 21, wherein said first means for storing includes:

means for initially treating each said stripe unit in said plurality of data stripe units as having all zeros stored therein;

means for commencing storing said data file object in said first data stripe unit;

means for exclusive-ORing said data in said first data stripe unit with zeros assumed to be contained in each other stripe unit in said plurality of data stripe units to generate a first parity data therefrom, wherein said first parity data is identical to said data in said first data stripe unit; and

means for storing said first parity data in said parity stripe unit.

23. The system of claim 22, further comprising:

means for monitoring when the size of said data file object exceeds said allocated storage space in said first data stripe unit;

wherein said second means for storing comprises;

means for storing data in the at least one further data stripe unit, wherein said data stored in the at least one further data stripe unit constitutes a portion of said data file object that fails to get stored in said allocated storage space in said first data stripe unit;

means for exclusive-ORing said data in said first data stripe unit with said data in said at least one further data stripe unit.