Method for rebuilding meta-data in a data storage system and a data storage system

• US 6,738,863 B2
• Filed: 11/13/2001
• Issued: 05/18/2004
• Est. Priority Date: 11/18/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Butterworth, Henry Esmond, Nicholson, Robert Bruce
• Primary Examiner(s): Padmanabhan, Mano
• Assistant Examiner(s): ROJAS, MIDYS
• Application Number:
  US09/992,284
• Publication Number:
  US 20020062422A1
• Time in Patent Office: 917 Days
• Field of Search: 711/112, 711/114
• US Class Current: 711/114
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1038 LFS, i.e. Log Structured File System used in RAID systems with parity

Abstract

A method for rebuilding meta-data stored in a data storage system (104) having storage devices (106) in which segments of data are located; for example, a storage system in the form of a log structured array. Data is written in segments to the storage devices (106) from a plurality of flows (122) of data and each segment of data contains meta-data relating to that segment. The meta-data stored in the storage system (104) can be rebuilt in the event of a failure by scanning the meta-data in each segment. A first scan of the meta-data in each segment in the storage devices (106) identifies the last segment written from each flow (122) and these segments are excluded from the rebuilding process as they may contain incomplete or inconsistent information. A second scan of the meta-data in each segment in the storage devices (106) identifies any segments which do not contain any live data tracks and these segments are also excluded from the, rebuilding process. The meta-data in the storage system (104) can then be rebuilt using the meta-data in the segments, excluding those segments identified.

Claims

1. A method for rebuilding meta-data in a storage system having storage devices in which segments of data are located, wherein data is written in segments to the storage devices from a plurality of flows of data and each segment of data also contains meta-data relating to that segment;

the method comprising;

scanning the meta-data in each segment to identify the last segment written from each flow;

rebuilding the meta-data in the storage system using the meta-data in the segments excluding the meta-data for the segments identified as being the last segments written from each flow.

2. A method as claimed in claim 1, wherein the data storage system includes a processor and memory, and the data storage devices are an array of storage devices having a plurality of data blocks organized on the storage devices in segments distributed across the storage devices, wherein when a data block in a segment stored on the storage devices in a first location is updated, the updated data block is assigned to a different segment, written to a new storage location and designated as a current data block, and the data block in the first location is designated as an old data block, and having a main directory, stored in memory, containing the locations on the storage devices of the current data blocks.

3. A method as claimed in claim 2, wherein the data storage system is a log structured array and the storage devices are a plurality of direct access storage devices.

4. A method as claimed in claim 2, wherein method includes scanning the meta-data of each segment to identify any segments which do not contain any current data blocks and wherein the method also comprises excluding any such segments from the rebuilding process.

5. A method as claimed in claim 2, wherein the last segment written from each flow is excluded from a free space collection process.

6. A method as claimed in claim 1, wherein the meta-data in the segments includes a description of the data blocks in the segment and ordering information.

7. A method as claimed in claim 6, wherein the ordering information includes an identifier of the flow from which the segment was written and a sequence number relating to the order the segments are written from one flow.

8. A method as claimed in claim 6, wherein the description of the data blocks in the segment and the ordering information are written atomically in the segment.

9. A method as claimed in claim 8, wherein the description of the data blocks in the segment and the ordering information are written in a single sector in a segment.

10. A method as claimed in claim 8, wherein the description of the data blocks in the segment and the ordering information are written in more than one sector in a segment and wherein the writes of the sectors in a segment are atomicised by including the segment sequence number in each sector.

11. A method as claimed in claim 10, wherein a segment with a plurality of sectors containing meta-data is ignored in the rebuilding process if the sectors have different segment sequence numbers.

12. A method as claimed in claim 1, wherein writing a segment from a flow commits the previous segment written from that flow.

13. A method as claimed in claim 1, wherein a flow is flushed by writing an empty segment in order to commit the previous segment from that flow.

14. A data storage system as claimed in claim 13, wherein any segments which do not contain any current data blocks are excluded from a rebuilding process.

15. A data storage system having storage devices in which segments of data are located, including a plurality of flows provided in the data storage system from which data is written in segments to the storage devices, each segment of data also containing meta-data relating to that segment;

wherein meta-data in the data storage system can be rebuilt using the meta-data in the segments excluding the meta-data for the segments identified as being the last segments written from each flow.

16. A data storage system as claimed in claim 15, wherein the data storage system includes a processor and memory, and the data storage devices are an array of storage devices having a plurality of data blocks organized on the storage devices in segments distributed across the storage devices, wherein when a data block in a segment stored on the storage devices in a first location is updated, the updated data block is assigned to a different segment, written to a new storage location and designated as a current data block, and the data block in the first location is designated as an old data block, and having a main directory, stored in memory, containing the locations on the storage devices of the current data blocks.

17. A data storage system as claimed in claim 16, wherein the data storage system is a log structured array and the storage devices are a plurality of direct access storage devices.

18. A data storage system as claimed in claim 16, wherein the last segment written from each flow is excluded from a free space collection process.

19. A data storage system as claimed in claim 15, wherein the meta-data in the segments includes a description of the data blocks stored in the segment and ordering information.

20. A data storage system as claimed in claim 19, wherein the ordering information includes an identifier of the flow from which the segment was written and a sequence number relating to the order the segments are written from one flow.

21. A data storage system as claimed in claim 19, wherein the description of the data blocks and the ordering information are written atomically in the segment.

22. A data storage system as claimed in claim 21, wherein the description of the data blocks and the ordering information are written in a single sector in each segment.

23. A data storage system as claimed in claim 21, wherein the description of the data blocks in the segment and the ordering information are written in more than one sector in a segment and wherein the writes of the sectors in a segment are atomicised by including the segment sequence number in each sector.

24. A data storage system as claimed in claim 23, wherein a segment with a plurality of sectors containing meta-data is ignored in the rebuilding process if the sectors have different segment sequence numbers.

25. A data storage system as claimed in claim 15, wherein writing a segment from a flow commits the previous segment written from that flow.

26. A data storage system as claimed in claim 15, wherein a flow is flushed by writing an empty segment in order to commit the previous segment from that flow.

27. A computer program product stored on a computer readable storage medium, comprising computer readable program code means for performing the steps of:

scanning meta-data in each segment in storage devices to identify the last segment written from each of a plurality of flows of data;

rebuilding meta-data in the storage devices using the meta-data in the segments excluding the meta-data for the segments identified as being the last segments written from each flow.