Data integrity validation in storage systems

• US 7,873,878 B2
• Filed: 09/24/2007
• Issued: 01/18/2011
• Est. Priority Date: 09/24/2007
• Status: Expired due to Fees

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Weber, Bret S., Bish, John Edward, Day, Kenneth III, Hafner, James, Belluomini, Wendy
• Primary Examiner(s): Beausoliel; Robert
• Assistant Examiner(s): Lottich; Joshua P
• Application Number:
  US11/860,461
• Publication Number:
  US 20090083504A1
• Time in Patent Office: 1,212 Days
• Field of Search: 714/6, 714/15, 714/52
• US Class Current: 714/52
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/104 Metadata, i.e. metadata associated with RAID systems with parity
• CPC Class Codes: G06F 2211/109 Sector level checksum or ECC, i.e. sector or stripe level checksum or ECC in addition to the RAID parity calculation

Abstract

A data storage method comprises storing first data in at least a first data chunk, wherein the first data chunk is a logical representation of one or more sectors on at least a first disk drive in a storage system; storing first metadata, associated with the first data, in at least a first appendix, wherein the first appendix is a logical representation of a sector region on at least the first disk drive in the storage system, and wherein the first metadata comprises first atomicity metadata (AMD) and first validity metadata (VMD) associated with the first data; and storing a copy of the first VMD for the first data in at least one low latency non-volatile storage (LLNVS) device.

Claims

1. A data storage method comprising:

storing first data in at least a first data chunk in response to a write request to store the first data on a first disk drive, wherein the first data chunk is a logical representation of one or more sectors on at least the first disk drive in a storage system;

storing first metadata, associated with the first data, in at least a first appendix in response to the same write request, wherein the first appendix is a logical representation of a sector region on at least the first disk drive in the storage system, and wherein the first metadata comprises first atomicity metadata (AMD) and first validity metadata (VMD) associated with the first data; and

storing one or more copies of the first VMD for the first data in one or more low latency non-volatile storage (LLNVS) devices in response to the same write request, so that when an error associated with content in the first chunk or the first appendix is detected during a read request for the first data, the copy of first VMD stored in the LLNVS device can be accessed with a low level of latency for the purpose of comparison with the first VMD stored in the first appendix to determine whether the data written during the write request is the same as the data read during the read request,

wherein said storing of the first data, the first metadata and the one or more copies of the first VMD is performed prior to retrieval of the first metadata from the first appendix, wherein the retrieval is performed subsequent to said write request.

2. The data storage method of claim 1, wherein the first VMD provides information about the first data that can be used to determine whether the first data is corrupted.

3. The data storage method of claim 1, wherein the first AMD provides information on when the first data and the corresponding first VMD stored in the first appendix are both successfully written during an update phase.

4. The data storage method of claim 1 further comprising analyzing the first VMD stored in the appendix to determine whether the first data is corrupted, in response to determining an error associated with the first VMD.

5. The data storage method of claim 4 further comprising:

comparing the first VMD stored in the appendix with a copy of the first VMD stored in the LLNVS to determine whether the first VMD in the appendix is corrupted.

6. The data storage method of claim 5 further comprising rebuilding the first data in response to determining that the first VMD in the appendix is not corrupted.

7. The data storage method of claim 5 further comprising rebuilding the first VMD in response to determining that the first VMD in the appendix is corrupted.

8. The data storage method of claim 1 further comprising examining the first AMD to determine validity of the first VMD, in response to receiving a request from a host system to perform an operation on the first data.

9. The data storage method of claim 8 further comprising:

rebuilding the first data to produce second data, in response to determining that the first AMD is corrupted;

determining a second AMD associated with the second data;

comparing the first AMD with the second AMD to determine whether first AMD and the second AMD are consistent; and

replacing the first data stored in the first chunk with the second data, in response to determining that the first AMD is consistent with the second AMD.

10. The data storage method of claim 9 further comprising repairing the first AMD or the first VMD in response to determining that the first AMD and the second AMD are inconsistent.

11. A data storage system comprising:

at least a first disk drive for storing first data in at least a first data chunk in response to a write request to store the first data on the first disk drive, wherein the first data chunk is a logical representation of one or more sectors on at least the first disk drive in a storage system;

a first appendix for storing first metadata, associated with the first data, in response to the same write request, wherein the first appendix is a logical representation of a sector region on at least the first disk drive in the storage system, and wherein the first metadata comprises first atomicity metadata (AMD) and first validity metadata (VMD) associated with the first data; and

at least one low latency non-volatile storage (LLNVS) devices for storing a copy of the first VMD for the first data in response to the same write request, so that when an error associated with content in the first chunk or the first appendix is detected during a read request for the first data, the copy of first VMD stored in the LLNVS device can be accessed with a low level of latency for the purpose of comparison with the first VMD stored in the first appendix to determine whether the data written during the write request is the same as the data read during the read request,

wherein said storing of the first data, the first metadata and the one or more copies of the first VMD is performed prior to retrieval of the first metadata from the first appendix, wherein the retrieval is performed subsequent to said write request.

12. The data storage system of claim 11, wherein the first VMD provides information about the first data that can be used to determine whether the first data is corrupted.

13. The data storage system of claim 11, wherein the first AMD provides information on when the first data and the corresponding first VMD stored in the first appendix are both successfully written during an update phase.

14. The data storage system of claim 11 further wherein the first VMD stored in the appendix is analyzed to determine whether the first data is corrupted, in response to determining an error associated with the first VMD.

15. The data storage system of claim 14 wherein the first VMD stored in the appendix is compared with a copy of the first VMD stored in the LLNVS to determine whether the first VMD in the appendix is corrupted.

16. The data storage system of claim 15 further wherein the first data is rebuilt in response to determining that the first VMD in the appendix is not corrupted.

17. The data storage system of claim 15 wherein the first VMD is rebuilt in response to determining that the first VMD in the appendix is corrupted.

18. The data storage system of claim 11 further wherein the first AMD is examined to determine validity of the first VMD, in response to receiving a request from a host system to perform an operation on the first data.

19. The data storage system of claim 18 wherein:

the first data is rebuilt to produce second data, in response to determining that the first AMD is corrupted;

a second AMD associated with the second data is determined;

the first AMD is compared with the second AMD to determine whether first AMD and the second AMD are consistent; and

the first data stored in the first chunk is replaced with the second data, in response to determining that the first AMD is consistent with the second AMD.

20. A computer program product comprising a computer readable storage medium having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to:

store first data in at least a first data chunk in response to a write request to store the first data on a first disk drive, wherein the first data chunk is a logical representation of one or more sectors on at least the first disk drive in a storage system;

store first metadata, associated with the first data, in at least a first appendix in response to the same write request, wherein the first appendix is a logical representation of a sector region on at least the first disk drive in the storage system, and wherein the first metadata comprises first atomicity metadata (AMD) and first validity metadata (VMD) associated with the first data, wherein the AMD comprises one or more of cyclic redundancy check (CRC) metadata or longitudinal redundancy check (CRC) metadata; and

store a plurality of copies of the first VMD for the first data in a plurality of low latency non-volatile storage (LLNVS) device in response to the same write request, so that when an error associated with content in the first chunk or the first appendix is detected during a read request for the first data, the copy of first VMD stored in the LLNVS device can be accessed with a low level of latency for the purpose of comparison with the first VMD stored in the first appendix to determine whether the data written during the write request is the same as the data read during the read request,

wherein said storing of the first data, the first metadata and the one or more copies of the first VMD is performed prior to retrieval of the first metadata from the first appendix, wherein the retrieval is performed subsequent to said write request.