DISTRIBUTED STORAGE NETWORK DATA REVISION CONTROL

• US 20110107113A1
• Filed: 07/16/2010
• Published: 05/05/2011
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Dhuse, Greg, Volvovski, Ilya, Resch, Jason K., Baptist, Andrew
• Granted Patent: US 10,142,115 B2
• Time in Patent Office: Days
• US Class Current: 713/193
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 12/00 Data switching networks interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units G06F13/00
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/3247 involving digital signatures

Abstract

Multiple revisions of an encoded data slice are generated, with each revision having the same slice name. Each of the data slices represents the same original data portion, but each is encoded so that no single data slice can be used to reconstruct the original data portion. Appropriate revision numbers are associated with each encoded data slice, and the encoded data slices and associated revision numbers are transmitted for storage in selected storage units of a distributed storage network. If write confirmations are received from at least a write threshold number of storage units, a commit command is transmitted so that the most recently written data slices will be available for access. After a commit command is issued, a current directory used to access the encoded data slices can be sliced, encoded, and stored in the same way as the data slices.

Claims

1. A method comprising:

generating a plurality of encoded data slices, each of the plurality of encoded data slices including different encoded versions of a data portion, the different encoded versions encoded to prevent reconstruction of the data portion from a single encoded data slice but allow reconstruction of the data portion using at least a threshold number of encoded data slices;

determining revision numbers associated with respective data slices of the plurality of encoded data slices;

associating revision numbers with each of the plurality of encoded data slices; and

transmitting at least a write threshold number of the plurality of encoded data slices and associated revision numbers to be stored in distributed storage units included in a distributed storage network.

2. The method of claim 1, further comprising:

determining selected distributed storage units of a plurality of distributed storage units to which each of the plurality of encoded data slices is to be transmitted.

3. The method of claim 2, further comprising:

transmitting write commands associated with the plurality of encoded data slices to the selected distributed storage units; and

determining if write confirmations are received from at least a write threshold number of the selected distributed storage units.

4. The method of claim 3, further comprising:

sending a write commit command to the selected distributed storage units in response to receiving the write confirmations from at least a write threshold number of distributed storage units.

5. The method of claim 4, further comprising:

determining a current directory mapping data objects to locations of encoded data slices generated from the data objects, wherein the current directory reflects write confirmations associated with storage of the plurality of encoded data slices;

caching an instance of the current directory;

generating a plurality of encoded cache slices from the cached instance of the current directory; and

transmitting at least a write threshold number of the plurality of the encoded cache slices to selected distributed storage units.

6. The method of claim 5, further comprising:

determining whether at least a threshold number of write confirmations associated with the encoded cache slices have been received;

in response to a favorable determination, sending a write commit command to the selected distributed storage units; and

in response to an unfavorable determination, retransmitting at least some of the encoded cache slices to different selected storage units.

7. The method of claim 1, further comprising:

determining the revision numbers based on at least one of a timestamp, a random number, a user vault identifier, a user identifier, a data object identifier, a hash of the data object, and a hash of the data object identifier.

8. The method of claim 1, further comprising:

transmitting the at least a write threshold number of the plurality of encoded data slices to be stored in a plurality of different pillars; and

associating a same revision number with each of the plurality of encoded data slices included in a same pillar.

9. The method of claim 1, further comprising:

associating different revision numbers with different encoded data slices associated with a same data object.

10. A method comprising:

determining target distributed storage units from which to retrieve encoded data slices including a revision number, each of the encoded data slices generated from a common data portion and encoded to prevent reconstruction of the common data portion from a single encoded data slice, but to allow reconstruction of the common data portion using at least a threshold number of encoded data slices;

receiving at least a read threshold number of the encoded data slices in response to sending a retrieve command to the target distributed storage units;

determining if the revision numbers included in the at least a threshold number of encoded data slices match each other; and

in response to a favorable match, decoding the at least a read threshold number of encoded data slices to recover the common data portion.

11. The method of claim 10, further comprising:

in response to an at least partially unfavorable match, determining at least one different distributed storage unit from which to request encoded data slices.

12. The method of claim 11, further comprising:

determining the at least one different distributed storage unit based, at least in part, on which of the target distributed storage units provided encoded data slices with matching revision numbers.

13. The method of claim 10, further comprising:

sending a retrieve command to a number of the target distributed storage units at least as great as a read threshold number, but less than a total number of encoded data slices generated from a common data portion.

14. A distributed storage processing unit comprising:

a grid module to generate a plurality of encoded data slices, each of the plurality of encoded data slices including different encoded versions of a data portion, the different encoded versions encoded to prevent reconstruction of the data portion from a single encoded data slice but allow reconstruction of the data portion using at least a threshold number of encoded data slices;

a processor to determine revision numbers associated with respective data slices of the plurality of encoded data slices, and associate the revision numbers with each of the plurality of encoded data slices; and

an interface to transmit at least a write threshold number of the plurality of encoded data slices to be stored in distributed storage units included in a distributed storage network.

15. The distributed storage processing unit of claim 14, further comprising:

the processor further to determine selected distributed storage units of a plurality of distributed storage units to which each of the plurality of encoded data slices is to be transmitted;

the interface further to transmit write commands command associated with the plurality of encoded data slices to the selected distributed storage units; and

16. The distributed storage processing unit of claim 15, further comprising:

the processor further to determine if write confirmations are received from at least a write threshold number of the selected distributed storage units; and

the interface further to send a write commit command to the selected distributed storage units in response to receiving the write confirmations from at least a write threshold number of distributed storage units.

17. The distributed storage processing unit of claim 16, further comprising:

determining a current directory mapping data objects to locations of encoded data slices generated from the data objects, wherein the current directory reflects write confirmations associated with storage of the plurality of encoded data slices;

caching an instance of the current directory;

generating a plurality of encoded cache slices from the cached instance of the current directory; and

transmitting at least a write threshold number of the plurality of the encoded cache slices to particular distributed storage units.

18. The distributed storage processing unit of claim 17, further comprising:

determining whether at least a threshold number of write confirmations associated with the encoded cache slices have been received;

in response to a favorable determination, sending a write commit command to the particular distributed storage units; and

in response to an unfavorable determination, retransmitting at least some of the encoded cache slices to different particular storage units.

19. A distributed storage processing unit comprising:

a processor to determine target distributed storage units from which to retrieve encoded data slices including a revision number, each of the encoded data slices generated from a common data portion and encoded to prevent reconstruction of the common data portion from a single encoded data slice, but to allow reconstruction of the common data portion using at least a threshold number of encoded data slices;

an interface to receive at least a read threshold number of the encoded data slices in response to sending a retrieve command to the target distributed storage units;

the processor further to determine if the revision numbers included in the at least a threshold number of encoded data slices match each other; and

a grid module to decode the at least a read threshold number of encoded data slices to recover the common data portion in response to a favorable match.

20. The distributed storage processing unit of claim 19, further comprising:

the processor further to determine at least one different distributed storage unit from which to request encoded data slices, in response to an at least partially unfavorable match.

21. The distributed storage processing unit of claim 20, further comprising:

the processor further to determine the at least one different distributed storage unit based, at least in part, on which of the target distributed storage units provided encoded data slices with matching revision numbers.

22. The distributed storage processing unit of claim 19, further comprising:

the processor further to send a retrieve command to a number of the target distributed storage units at least as great as a read threshold number, but less than a total number of encoded data slices generated from a common data portion.

23. The distributed storage processing unit of claim 19, further comprising

the processor further to reconstruct the common data portion from encoded data slices having an earlier revision number in response to the interface failing to receive a read threshold number of a current revision number of encoded data slices.