Performing a desired manipulation of an encoded data slice based on a metadata restriction and a storage operational condition

• US 10,102,067 B2
• Filed: 07/14/2016
• Issued: 10/16/2018
• Est. Priority Date: 07/14/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin
• Primary Examiner(s): Britt, Cynthia
• Application Number:
  US15/210,162
• Publication Number:
  US 20180018227A1
• Time in Patent Office: 824 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method comprises receiving a set of write fan out requests for a plurality of sets of encoded data slices and metadata regarding storage parameters for the plurality of sets of encoded data slices. The method continues by identifying an encoded data slice of the plurality of sets of encoded data slices based on a desired manipulation of the encoded data slice. The method continues by determining whether the metadata provides a restriction regarding the desired manipulation of the encoded data slice. When the metadata does not provide the restriction regarding the desired manipulation of the encoded data slice, the method continues by determining whether execute the desired manipulation of the encoded data slice based on a storage operational condition. The method continues by executing the desired manipulation of the encoded data slice when the storage unit determines to execute the desired manipulation of the encoded data slice.

Claims

1. A method comprises:

receiving, by a first set of storage units of a dispersed storage network (DSN) and via at least one interface of the first set of storage units and from a device via the DSN, a set of write fan out requests for a plurality of sets of encoded data slices and metadata regarding storage parameters for the plurality of sets of encoded data slices, wherein a data object is segmented into a plurality of data segments by the device, wherein a data segment of the plurality of data segments is dispersed error encoded by the device in accordance with dispersed error encoding parameters to produce a set of encoded data slices of the plurality of sets of encoded data slices;

copying, by the first set of storage units, the plurality of sets of encoded data slices in accordance with the set of write fan out requests to produce one or more copies of the plurality of sets of encoded data slices;

transferring, by the first set of storage units and via the at least one interface of the first set of storage units via the DSN, the one or more copies of the plurality of sets of encoded data slices and the metadata to one or more other sets of storage units to be stored within the one or more other sets of storage units in accordance with the set of write fan out requests;

identifying, by a storage unit of the first set or of the one or more of the other sets of storage units, an encoded data slice of the plurality of sets of encoded data slices stored by the storage unit based on a desired manipulation of the encoded data slice;

determining, by the storage unit, whether the metadata provides a restriction regarding the desired manipulation of the encoded data slice;

based on a first determination that the metadata does not provide the restriction regarding the desired manipulation of the encoded data slice, determining, by the storage unit, whether to execute the desired manipulation of the encoded data slice based on a storage operational condition that is based on at least one of storage reliability, data access rates, disk failure, or available memory of storage units associated with the plurality of sets of encoded data slices; and

based on a second determination by the storage unit to execute the desired manipulation of the encoded data slice, executing, by the storage unit, the desired manipulation of the encoded data slice.

2. The method of claim 1, wherein the identifying the encoded data slices comprises at least one of:

identifying the encoded data slice as being flagged for rebuilding;

identifying the encoded data slice for deletion;

identifying the encoded data slice for overwriting;

identifying the encoded data slice for a revision level update;

or

identifying the encoded data slice for a data transfer.

3. The method of claim 1 further comprises:

determining that the metadata provides the restriction of locked without modification; and

delaying or discarding, by the storage unit, the desired manipulation of the encoded data slice.

4. The method of claim 1 further comprises:

determining that the metadata includes a lock with limited modification restriction for the encoded data slice;

determining, by the storage unit, whether the desired manipulation of the encoded data slice is an acceptable modification in accordance with the limited modification restriction;

when the desired manipulation of the encoded data slice is an acceptable modification in accordance with the limited modification restriction, executing, by the storage unit, the desired manipulation of the encoded data slice; and

when the desired manipulation of the encoded data slice is not an acceptable modification in accordance with the limited modification restriction, delaying or discarding, by the storage unit, the desired manipulation of the encoded data slice.

5. The method of claim 1 further comprises:

identifying the desired manipulation of the encoded data slice to be rebuilding;

when the metadata does not provide restrictions regarding rebuilding of the encoded data slice, determining, by the storage unit, whether to forego, delay, or de-prioritize rebuilding of the encoded data slice based on the storage operational condition; and

foregoing, delaying, or de-prioritizing, by the storage unit, the rebuilding of the encoded data slice when the storage unit determines to forego, delay, or de-prioritize rebuilding of the encoded data slice.

6. The method of claim 1 further comprises:

determining, by the storage unit, an available storage issue;

identifying the desired manipulation of the encoded data slice to be deleted or overwritten in response to the available storage issue; and

when the metadata does not provide restrictions regarding deleting or overwriting of the encoded data slice, deleting or overwriting, by the storage unit, the encoded data slice.

7. The method of claim 1 further comprises:

identifying, by the storage unit, the encoded data slice for data transfer;

determining, by the storage unit, whether to skip the data transfer of the encoded data slice based on the storage operational condition; and

skipping, by the storage unit, the data transfer of the encoded data slice when the storage unit determines to skip the data transfer.

8. The method of claim 1, wherein the storage operational condition further comprises at least one of:

storage unit reliability;

storage unit operational efficiency;

dispersed storage error encoding parameters;

number of copies of the plurality of sets of encoded data slices;

number of overlapping fulfilment of requests for the plurality of sets of encoded data slices;

or

air dates regarding re-run of some encoded data slices of the plurality of sets of encoded data slice.

9. The method of claim 1 further comprises:

identifying, by the first set of storage units, the one or more other sets of storage units in accordance with the set of write fan out requests.

10. A storage unit comprises:

an interface;

memory that stores operational instructions; and

a processing module operably coupled to the interface and the memory, wherein the processing module is configured to execute the operational instructions to;

when the storage unit is in a first set of storage units of a dispersed storage network (DSN);

receive, via the interface and from a device via the DSN, a write fan out request of a set of write fan out requests for a set of encoded data slices and metadata regarding storage parameters for the set of encoded data slices, wherein a data object is segmented into a plurality of data segments by the device, wherein a data segment of the plurality of data segments is dispersed error encoded by the device in accordance with dispersed error encoding parameters to produce a set of encoded data slices of the plurality of sets of encoded data slices, wherein the set of write fan out requests is sent to the first set of storage units;

units from a device via the DSN;

copy an encoded data slice of the set of encoded data slices in accordance with the write fan out request to produce one or more copies of the encoded data slice;

transfer, via the interface and via the DSN, the one or more copies of the encoded data slice and the metadata to one or more other storage units of one or more other sets of storage units to be stored within the one or more other sets of storage units in accordance with the set of write fan out requests;

identify the encoded data slice based on a desired manipulation of the encoded data slice;

determine whether the metadata provides a restriction regarding the desired manipulation of the encoded data slice;

based on a first determination that the metadata does not provide the restriction regarding the desired manipulation of the encoded data slice, determine whether to execute the desired manipulation of the encoded data slice based on a storage operational condition that is based on at least one of storage reliability, data access rates, disk failure, or available memory of storage units associated with the plurality of sets of encoded data slices; and

based on a second determination by the storage unit to execute the desired manipulation of the encoded data slice, execute the desired manipulation of the encoded data slice.

11. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to identify the encoded data slices by at least one of:

identifying the encoded data slice as being flagged for rebuilding;

identifying the encoded data slice for deletion;

identifying the encoded data slice for overwriting;

identifying the encoded data slice for a revision level update;

or

identifying the encoded data slice for a data transfer.

12. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

determine that the metadata provides the restriction of locked without modification; and

delay or discard, by the storage unit, the desired manipulation of the encoded data slice.

13. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

determine that the metadata includes a lock with limited modification restriction for the encoded data slice;

determine, by the storage unit, whether the desired manipulation of the encoded data slice is an acceptable modification in accordance with the limited modification restriction; and

when the desired manipulation of the encoded data slice is an acceptable modification in accordance with the limited modification restriction, execute, by the storage unit, the desired manipulation of the encoded data slice; and

when the desired manipulation of the encoded data slice is not an acceptable modification in accordance with the limited modification restriction, delay or discard, by the storage unit, the desired manipulation of the encoded data slice.

14. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

identify the desired manipulation of the encoded data slice to be rebuilding;

when the metadata does not provide restrictions regarding rebuilding of the encoded data slice, determine, by the storage unit, whether to forego, delay, or de-prioritize rebuilding of the encoded data slice based on the storage operational condition; and

forego, delay, or de-prioritize, by the storage unit, the rebuilding of the encoded data slice when the storage unit determines to forego, delay, or de-prioritize rebuilding of the encoded data slice.

15. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

determine, by the storage unit, an available storage issue;

identify the desired manipulation of the encoded data slice to be deleted or overwritten in response to the available storage issue; and

when the metadata does not provide restrictions regarding deleting or overwriting of the encoded data slice, delete or overwrite, by the storage unit, the encoded data slice.

16. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

identify, by the storage unit, the encoded data slice for data transfer;

determine, by the storage unit, whether to skip the data transfer of the encoded data slice based on the storage operational condition; and

skip, by the storage unit, the data transfer of the encoded data slice when the storage unit determines to skip the data transfer.

17. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to determine the storage operational condition by at least one of:

storage unit reliability;

storage unit operational efficiency;

dispersed storage error encoding parameters;

number of copies of the plurality of sets of encoded data slices;

number of overlapping fulfilment of requests for the plurality of sets of encoded data slices;

or

air dates regarding re-run of some encoded data slices of the plurality of sets of encoded data slice.

18. The storage unit of claim 10, wherein the processing module is further configured to execute the operational instructions to:

identify, by the first set of storage units, the one or more other sets of storage units in accordance with the set of write fan out requests.