Long term data protection system and method

• US 7,177,995 B2
• Filed: 08/02/2006
• Issued: 02/13/2007
• Est. Priority Date: 03/15/2004
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Hitachi, Ltd.
• Inventors: Yagawa, Yuichi
• Primary Examiner(s): LANE, JOHN A
• Application Number:
  US11/498,449
• Publication Number:
  US 20070011501A1
• Time in Patent Office: 195 Days
• Field of Search: 711/162, 711/173, 714/6
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1612 where the redundant component is persistent storage
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures

Abstract

A file to be written can be partitioned into one or more partitions. Each such input partition is identified with a hash code and a group ID. Replica(s) of an input partition can be created to ensure a certain number of identical partitions. When a file is accessed, each partition that is read out can be checked for corruption. Corrupted readout partitions are then replaced with a validated replacement partition, whether the replacement partition is from a replica or from another file.

Claims

1. A method for accessing a storage system comprising:

accessing a first read-out partition of a data object, the first read-out partition comprising at least a portion of content comprising the data object;

if content in the first read-out partition is corrupted, then;

accessing the storage system to find a replacement partition from among one or more candidate partitions, including determining if a candidate partition is corrupted or not, the replacement partition being a candidate partition that is not corrupted; and

replacing content in the data object that constitutes the first read-out partition with content of the replacement partition; and

repeating the foregoing with a second read-out partition of the data object.

2. The method of claim 1 wherein each partition is associated with a partition ID, wherein if two partitions comprise identical content, then the two partitions have the same partition ID, wherein the read-out partition is associated with a first partition ID and the one or more candidate partitions each is associated with the first partition ID.

3. The method of claim 2 wherein the partition ID comprises a hash code and a group ID, wherein the hash code is based on content of a partition, wherein if the two partitions have the same hash code but their respective content is different from each other, then the two partitions each have a different group ID, wherein if the two partitions comprise identical content, then the two partitions have the same group ID.

4. The method of claim 1 wherein if a replacement partition cannot be found for the first read-out partition, then indicating an error condition.

5. The method of claim 1 further comprising receiving a data access request, the data access request including information indicative of the data object.

6. The method of claim 1 wherein the data object is a file.

7. A method for accessing data in a storage system comprising:

identifying a first data object;

obtaining a first partition of the first data object from the storage system;

performing a computation using data comprising the first data object to produce a first computed value;

obtaining partition identification information relating to the first partition, the partition identification information including a first previously computed value; and

if the first computed value does not match the first previously computed value, then;

obtaining a first candidate partition from the storage system;

performing a computation using data comprising the first candidate partition to produce a second computed value;

obtaining partition identification information relating to the first candidate partition, the partition identification information including a second previously computed value;

if the second computed value does not match the second previously computed value, then repeating with a second candidate partition; and

if the second computed value does match the second previously computed value, then replacing the data comprising the first partition of the first data object with the data comprising the first candidate partition.

8. The method of claim 7 wherein identifying a first data object is a step of receiving a read request for the first data object.

9. The method of claim 7 wherein if the first computed value does match the first previously computed value, then repeating with a second partition of the first data object.

10. The method of claim 7 wherein the first data object is a file.

11. The method of claim 7 wherein steps comprising the method are repeated for a second data object.

12. A data storage system comprising:

a storage subsystem; and

a data processing subsystem in data communication with the storage subsystem to store data to the storage subsystem and to access data stored on the storage subsystem,

the data processing subsystem configured to;

access a first file stored on the storage subsystem, wherein the first file comprises data, the data being logically grouped into one or more accessed partitions;

determine, for each accessed partition, whether the accessed partition is corrupt, referred to as a corrupt partition;

determine, for each corrupt partition, whether there is a replacement partition on the storage system, the replacement partition being identical to the accessed partition at a time when the accessed partition was not corrupt; and

modify the first file to replace each of its corrupt partitions with a replacement partition if it exists.

13. The system of claim 12 wherein the data processing subsystem is further configured to communicate with one or more users and to receive access requests for data stored on the storage subsystem, wherein one such access request is a request for the first file.

14. The system of claim 12 further comprising partition identity information, wherein each partition on the storage subsystem is associated with its corresponding partition identity information, the partition identity information including a partition ID, wherein the partition ID uniquely identifies content of a partition, wherein partitions that are identical have the same partition ID,

wherein the data processing subsystem is further configured to determine whether an accessed partition is corrupt based on its accessed content and on its partition ID.

15. The system of claim 12 wherein each partition on the storage subsystem is associated with a partition ID comprising a hash code component and a group ID component, wherein partitions that have the same hash code value also have the same group ID if they have identical content, wherein the data processing subsystem is further configured to:

compute a first hash value for an accessed partition; and

compare the first hash value with the hash code component of the partition ID associate with the accessed partition in order to determine whether the accessed partition is corrupt.

16. A data processing system comprising:

first means for accessing a partition comprising a file that is stored on a storage subsystem, a partition comprising data from the file, a partition having associated therewith a partition ID that uniquely identifies content of the partition, wherein partitions comprising identical content have the same partition ID;

second means for determining whether a partition is corrupt;

third means for identifying a replacement partition from among a plurality of partitions stored on the storage subsystem to replace a corrupt partition, based on a partition ID associated with the corrupt partition and on partition IDs of the plurality of partitions, the corrupt partition being a constituent partition of a target file; and

fourth means to modify the target file to replace content comprising the corrupt partition with content from a replacement partition.