ENCODED DATA SLICE CACHING IN A DISTRIBUTED STORAGE NETWORK

• US 20110107078A1
• Filed: 06/17/2010
• Published: 05/05/2011
• Est. Priority Date: 10/30/2009
• Status: Abandoned Application

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: MOTWANI, MANISH, RESCH, JASON K., DHUSE, GREG
• Application Number:
  US12/817,208
• Publication Number:
  US 20110107078A1
• Time in Patent Office: Days
• US Class Current: 713/150
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed storage processing unit encodes data objects into multiple encoded data slices to prevent reconstruction of the original data object using a single encoded data slice, but to allow reconstruction using at least a threshold number of encoded data slices. The distributed storage processing unit can decide to whether and where to cache frequently requested data slices. When retrieving data slices related to a particular data object, a check can be made to determine if the data slices are cached in a temporary memory associated with the distributed storage processing unit, or elsewhere in the distributed storage network. This check can be facilitated by storing data slices and a hash table identifying the location of stored data slices in the same temporary memory.

Claims

1. A method for use in a distributed storage processing unit, the method comprising:

receiving a request to retrieve requested information stored as encoded data slices in a distributed processing system, each encoded data slice including a data object encoded to prevent reconstruction of the data object using a single encoded data slice, and to permit reconstruction of the data object using at least a threshold number of encoded data slices;

determining whether relevant encoded data slices corresponding to at least part of the requested information are cached in temporary storage associated with a distributed storage processing unit;

if the relevant encoded data slices are cached in the temporary storage, retrieving the relevant encoded data slices from the temporary storage; and

if the relevant encoded data slices are not cached in the temporary storage, retrieving the relevant encoded data slices from a dispersed storage network unit.

2. The method of claim 1, further comprising:

if the relevant encoded data slices are not cached in the temporary storage, determining whether to cache the relevant encoded data slices in the temporary storage; and

if the determining indicates that the encoded data slices are to be cached;

storing the relevant encoded data slices in the temporary storage; and

storing names of the encoded data slices in a hash table.

3. The method of claim 2, further comprising:

determining whether to cache the relevant encoded data slices in the temporary storage based on access tracking information associated with the relevant encoded data slices.

4. The method of claim 3, wherein the access tracking information, on which the decision to cache is based, is selected from a group consisting essentially of:

a frequency of accessing data slices associated with the information, network loading levels, a priority level, and a security level.

5. The method of claim 2, further comprising:

limiting storage of the relevant encoded data slices in the temporary storage to a read threshold number of encoded data slices per data object.

6. The method of claim 1, further comprising:

de-slicing and decoding the relevant encoded data slices to produce the data object; and

sending the data object to a requestor.

7. The method of claim 1, further comprising:

verifying that at least a read threshold number of relevant encoded data slices that have been retrieved are valid.

8. The method of claim 1, further comprising:

removing data slices from the temporary storage in response to a frequency of access dropping below a threshold value.

9. The method of claim 1, further comprising:

removing data slices from the temporary storage in response to at least one of a security level of the requested information, a priority level of the requested information, a network loading level, and a memory loading level.

10. A distributed storage processing unit comprising:

an interface to receive a request to retrieve a data object stored as a plurality of data slices in a distributed processing system, each of the plurality of data slices including an instance of the data object encoded to prevent reconstruction of the data object using a single data slice, and to permit reconstruction of the data object using at least a threshold number of data slices;

a processor to;

determine whether data slices corresponding to the data object are cached in a temporary storage;

retrieve the data slices from the temporary storage if the data slices are cached in the temporary storage; and

retrieve the data slices from a dispersed storage network unit if the data slices are not cached in the temporary storage.

11. The distributed storage processing unit of claim 10, the processor further to:

determine whether to cache the data slices in the temporary storage if the data slices are not cached in the temporary storage; and

if the processor determines that the encoded data slices are to be cached;

store the relevant encoded data slices in the temporary storage; and

store names of the encoded data slices in a hash table.

12. The distributed storage processing unit of claim 11, the processor further to:

determine whether to cache the data slices based on access tracking information associated with the data slices.

13. The distributed storage processing unit of claim 12, wherein the access tracking information used by the processor determine whether to cache the data slices is selected from a group consisting essentially of:

a frequency of accessing data slices associated with the information, network loading levels, a priority level, and a security level.

14. The distributed storage processing unit of claim 10, further comprising:

a grid module to de-slice and decode the data slices to produce the data object; and

an interface to send the data object to a requestor.

15. The distributed storage processing unit of claim 10, the processor further to:

verify a validity of the data slices.

16. The distributed storage processing unit of claim 10, further comprising:

a temporary storage including a slice memory and a hash table.

17. A method comprising:

tracking a retrieval frequency of encoded data slices associated with a requested data object, each encoded data slice including an instance of the data object encoded to prevent reconstruction of the data object using a single encoded data slice, and to permit reconstruction of the data object using at least a read threshold number of data slices;

caching at least the read threshold number of encoded data slices in a local storage based, at least in part, on the retrieval frequency comparing favorably to a frequency threshold;

in response to caching an encoded data slice, updating a hash table stored in the local storage;

accessing the hash table in response to receiving a request for the requested data object, to determine if encoded data slices associated with the requested data object are cached in the local storage; and

retrieving the encoded data slices from the local storage if the hash table indicates that the encoded data slices are cached.

18. The method of claim 17, further comprising:

retrieving the encoded data slices from a memory associated with a dispersed storage network unit if the hash table indicates that the encoded data slices are not cached locally.

19. The method of claim 17, further comprising:

caching at least the read threshold number of encoded data slices in the local storage based, at least in part, on access tracking information.

20. The method of claim 19, wherein the access tracking information is selected from a group consisting essentially of:

network loading levels, a priority level, and a security level.

21. The method of claim 17, further comprising:

de-slicing and decoding the relevant encoded data slices to produce the requested data object; and

sending the data object to a requestor.

22. The method of claim 17, further comprising:

removing data slices from the local storage in response to a frequency of access dropping below a threshold value.