STORING LOW RETENTION PRIORITY DATA IN A DISPERSED STORAGE NETWORK

• US 20150067090A1
• Filed: 06/30/2014
• Published: 03/05/2015
• Est. Priority Date: 08/29/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K.
• Granted Patent: US 9,749,414 B2
• Time in Patent Office: Days
• US Class Current: 709/213
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module of a dispersed storage network (DSN) sending a set of low retention priority write requests to storage units of the DSN, where each low retention priority write request includes a low retention priority query. For each storage unit of the storage units that receives a low retention priority write request of the set of low retention priority write requests, the method continues with the processing module determining a low retention priority response regarding availability for storing low retention priority data based on current storage of low priority data objects and available memory for storing the low retention priority data. The method continues with the processing module sending the low retention priority response. When a threshold number of favorable low retention priority responses have been received, the method continues at the processing module facilitating storage of a low retention priority data object.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

sending a set of low retention priority write requests to storage units of the DSN, wherein each low retention priority write request of the set of low retention priority write requests includes a low retention priority query;

for each storage unit of the storage units that receives a low retention priority write request of the set of low retention priority write requests;

determining a low retention priority response regarding availability for storing low retention priority data based on current storage of low priority data objects and available memory for storing the low retention priority data; and

sending the low retention priority response; and

when a threshold number of favorable low retention priority responses have been received, facilitating storage of a low retention priority data object.

2. The method of claim 1 further comprises:

including a set of encoded data slices within the set of low retention priority write requests when sending the set of low retention priority write requests, wherein a data segment of the low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices; and

for each storage unit of the storage units that receives a low retention priority write request of the set of low retention priority write requests;

determining whether there is available memory for storing the low retention priority data;

when there is not available memory for storing the low retention priority data, comparing retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwriting the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

sending a write response.

3. The method of claim 2, wherein the facilitating the storage of the low retention priority data object comprises:

sending a set of write commit requests to each storage unit sending the write response.

4. The method of claim 2, wherein the determining whether there is available memory for storing the low retention priority data comprises:

determining total available memory within the storage unit;

determining a level of storage of higher priority data objects; and

determining whether there is available memory for storing the low retention priority data based on the total available memory and the level of storage the higher priority data objects.

5. The method of claim 1 further comprises:

the facilitating storage of the low retention priority data object by sending, for storage, a set of encoded data slices to the storage units providing the favorable low retention priority response, wherein a data segment of a low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices; and

for each storage unit receiving an encoded data slice of the set of encoded data slices;

determining whether there is low retention priority memory currently available;

when the low retention priority memory is not currently available, comparing retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwriting the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

sending a write response.

6. The method of claim 1 further comprises:

generating the low retention priority query to include queries regarding different levels of low retention priority data;

for a given low retention priority level of the low retention priority data object to be stored, determining whether the threshold number of favorable low retention priority responses have been received; and

when the threshold number of favorable low retention priority responses have been received for the given low retention priority level, facilitating the storage of the low retention priority data object.

7. A method for execution by a storage unit of a dispersed storage network (DSN), the method comprises:

receiving a low retention priority write request of a set of low retention priority write requests to storage units of the DSN, wherein the low retention priority write request includes a low retention priority query;

determining a low retention priority response regarding availability for storing low retention priority data based on current storage of low priority data objects and available memory for storing the low retention priority data;

sending the low retention priority response; and

receiving, from a computing device, a write facilitating request regarding storage of a portion of a low retention priority data object when the computing device received a threshold number of favorable low retention priority responses.

8. The method of claim 7 further comprises:

receiving an encoded data slice of a set of encoded data slices within the low retention priority write request, wherein a data segment of a low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices;

determining whether there is available memory for storing the low retention priority data;

when there is not available memory for storing the low retention priority data, comparing retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwriting the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

sending a write response.

9. The method of claim 8, wherein the determining whether there is available memory for storing the low retention priority data comprises:

determining total available memory within the storage unit;

determining a level of storage of higher priority data objects; and

determining whether there is available memory for storing the low retention priority data based on the total available memory and the level of storage the higher priority data objects.

10. The method of claim 7 further comprises:

interpreting the low retention priority query to identify queries regarding different levels of low retention priority data;

for a given low retention priority level of the low retention priority data object to be stored, determining whether the threshold number of favorable low retention priority responses have been received; and

when the threshold number of favorable low retention priority responses have been received for the given low retention priority level, facilitating storage of the low retention priority data object.

11. A computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

send a set of low retention priority write requests to storage units of the DSN, wherein each low retention priority write request of the set of low retention priority write requests includes a low retention priority query;

for each storage unit of the storage units that receives a low retention priority write request of the set of low retention priority write requests;

determine a low retention priority response regarding availability for storing low retention priority data based on current storage of low priority data objects and available memory for storing the low retention priority data; and

send the low retention priority response; and

when a threshold number of favorable low retention priority responses have been received, facilitate storage of a low retention priority data object.

12. The computer readable storage medium of claim 11 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

include a set of encoded data slices within the set of low retention priority write requests when sending the set of low retention priority write requests, wherein a data segment of the low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices; and

for each storage unit of the storage units that receives a low retention priority write request of the set of low retention priority write requests;

determine whether there is available memory for storing the low retention priority data;

when there is not available memory for storing the low retention priority data, compare retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwrite the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

send a write response.

13. The computer readable storage medium of claim 12, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to facilitate the storage of the low retention priority data object by:

sending a set of write commit requests to each storage unit sending the write response.

14. The computer readable storage medium of claim 12, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to determine whether there is available memory for storing the low retention priority data object by:

determining total available memory within the storage unit;

determining a level of storage of higher priority data objects; and

determining whether there is available memory for storing the low retention priority data based on the total available memory and the level of storage the higher priority data objects.

15. The computer readable storage medium of claim 11 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

facilitate the storage of the low retention priority data object by sending, for storage, a set of encoded data slices to the storage units providing the favorable low retention priority response, wherein a data segment of a low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices; and

for each storage unit receiving an encoded data slice of the set of encoded data slices;

determine whether there is low retention priority memory currently available;

when the low retention priority memory is not currently available, compare retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwrite the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

send a write response.

16. The computer readable storage medium of claim 11 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

generate the low retention priority query to include queries regarding different levels of low retention priority data;

for a given low retention priority level of the low retention priority data object to be stored, determine whether the threshold number of favorable low retention priority responses have been received; and

when the threshold number of favorable low retention priority responses have been received for the given low retention priority level, facilitate the storage of the low retention priority data object.

17. A storage unit of a dispersed storage network (DSN), the storage unit comprises:

an interface; and

a processing module operable to;

receive, via the interface, a low retention priority write request of a set of low retention priority write requests to storage units of the DSN, wherein the low retention priority write request includes a low retention priority query;

determine a low retention priority response regarding availability for storing low retention priority data based on current storage of low priority data objects and available memory for storing the low retention priority data;

send, via the interface, the low retention priority response; and

receive, from a computing device via the interface, a write facilitating request regarding storage of a portion of a low retention priority data object when the computing device received a threshold number of favorable low retention priority responses.

18. The storage unit of claim 17 further comprises:

the processing module is further operable to;

receive, via the interface, an encoded data slice of a set of encoded data slices within the low retention priority write request, wherein a data segment of a low retention priority data object is dispersed storage error encoded to produce the set of encoded data slices;

determine whether there is available memory for storing the low retention priority data;

when there is not available memory for storing the low retention priority data, compare retention priority of an encoded data slice of another low retention priority data object of the current storage of low priority data objects with a retention priority of an encoded data slice of the set of encoded data slices; and

when the retention priority of the encoded data slice of the set of encoded data slices compares favorably to the retention priority of the encoded data slice of the other low retention priority data object;

overwrite the encoded data slice of the other low retention priority data object with the encoded data slice of the set of encoded data slices; and

send, via the interface, a write response.

19. The storage unit of claim 18, wherein the storage unit functions to determine whether there is available memory for storing the low retention priority data by:

determining total available memory within the storage unit;

determining a level of storage of higher priority data objects; and

determining whether there is available memory for storing the low retention priority data based on the total available memory and the level of storage the higher priority data objects.

20. The storage unit of claim 17 further comprises:

the processing module is further operable to;

interpret the low retention priority query to identify queries regarding different levels of low retention priority data;

for a given low retention priority level of the low retention priority data object to be stored, determine whether the threshold number of favorable low retention priority responses have been received; and

when the threshold number of favorable low retention priority responses have been received for the given low retention priority level, facilitate storage of the low retention priority data object.