List request processing during a dispersed storage network configuration change

• US 10,248,504 B2
• Filed: 06/28/2016
• Issued: 04/02/2019
• Est. Priority Date: 06/30/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Khadiwala, Ravi V., Motwani, Manish, Resch, Jason K.
• Primary Examiner(s): Phillips, III, Albert M
• Application Number:
  US15/194,721
• Publication Number:
  US 20170004156A1
• Time in Patent Office: 1,008 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/33 Synchronisation based on error coding or decoding
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

A method includes identifying resources that are affiliated with a range of slice names of a listing request. The method further includes creating an ordered classification of the resources. The method further includes a resource determining whether it is in a last class of the ordered classification. When the resource is in the last class, it processes the listing request to generate a listing response and sends the listing response to another resource in a next lower class. When the resource is not in the last class, it identifies a second resource for proxying of the listing request and sends the listing request to the second resource. The method further includes receiving a cumulated listing response from the second resource. The method further includes processing the listing request to generate the listing response. The method further includes combining the listing response with the cumulated listing response.

Claims

1. A method for processing and proxying a listing request by resources of a dispersed storage network (DSN) during a system configuration change, the method comprises:

identifying a set of resources that are affiliated with a range of slice names identified by the listing request;

creating an ordered classification of the set of resources based on the system configuration change;

determining, by a resource of the set of resources, whether the resource is in a last class of the ordered classification;

when the resource is in the last class;

processing the listing request to generate a listing response regarding encoded data slices associated with slice names within a sub-range of slice names of the range of slice names, wherein the sub-range of slices names is affiliated with the resource; and

sending the listing response to another resource in a lower higher class of the ordered classification;

when the resource is not in the last class;

identifying a second resource of the set of resource for proxying of the listing request, wherein the second resource is in a next higher class of the ordered classification;

sending the listing request to the second resource;

receiving, in response to the sending, a cumulated listing response from the second resource; and

processing the listing request to generate the listing response regarding encoded data slices associated with slice names within the sub-range of slice names;

combining the listing response with the cumulated listing response to produce an updated cumulated listing response.

2. The method of claim 1 further comprises:

when the resource received the listing request, sending the updated cumulated listing response to a request computing device that issued the listing request.

3. The method of claim 1 further comprises:

when the resource did not receive the listing request, sending the updated cumulated listing response to yet another resource in another next lower class of the ordered classification.

4. The method of claim 1, wherein the creating the ordered classification comprises:

identifying the set resources for the range of slice names in accordance with a distributed agreement protocol based an old system configuration of the DSN and a new system configuration of the DSN;

identifying an old weight factor and a new weight factor for each resource in the set of resources, wherein the old weight factor corresponds to the old system configuration and the new weight factor corresponds to new system configuration;

determining a relative weight change for each resource in the set of resources based on the old weight factor and the new weight factor; and

establishing the ordered classification based on the relative weight changes of the set of resources.

5. The method of claim 1, wherein the listing request comprises:

a request to at least one of;

identify valid encoded data slices having slice names within the range of slice names and to determine location of the valid encoded data slices.

6. The method of claim 5, wherein the valid encoded data slices comprise:

encoded data slices that are stored in one or more storage units;

encoded data slices that are being created or modified by a computing device; and

encoded data slices that being rebuild by a rebuilding agent.

7. The method of claim 1, wherein the system configuration change comprises one or more of:

an addition of a resource;

a deletion of a resource; and

a change in weighting factors of a DAP.

8. The method of claim 1, wherein the resource comprises one of:

a storage location within a storage unit;

a storage unit;

a storage pool;

a computing device; and

a rebuilder.

9. A resource of a dispersed storage network (DSN), the resource comprises:

an interface;

a memory; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

identify a set of the resources that are affiliated with a range of slice names identified by a listing request;

create an ordered classification of the set of resources based on a system configuration change;

determine whether the resource is in a last class of the ordered classification;

when the resource is in the last class;

process the listing request to generate a listing response regarding encoded data slices associated with slice names within a sub-range of slice names of the range of slice names, wherein the sub-range of slices names is affiliated with the resource; and

send, via the interface, the listing response to another resource in a next lower class of the ordered classification;

when the resource is not in the last class;

identify a second resource of the set of resource for proxying of the listing request, wherein the second resource is in a next higher class of the ordered classification;

send, via the interface, the listing request to the second resource;

receive, via the interface in response to the sending, a cumulated listing response from the second resource; and

process the listing request to generate the listing response regarding encoded data slices associated with slice names within the sub-range of slice names;

combine the listing response with the cumulated listing response to produce an updated cumulated listing response.

10. The resource of claim 9, wherein the processing module is further operable to:

when the resource received the listing request, send, via the interface, the updated cumulated listing response to a request computing device that issued the listing request.

11. The resource of claim 9, wherein the processing module is further operable to:

when the resource did not receive the listing request, send, via the interface, the updated cumulated listing response to yet another resource in another next lower class of the ordered classification.

12. The resource of claim 9, wherein the processing module is further operable to create the ordered classification by:

identifying the set resources for the range of slice names in accordance with a distributed agreement protocol based an old system configuration of the DSN and a new system configuration of the DSN;

identifying an old weight factor and a new weight factor for each resource in the set of resources, wherein the old weight factor corresponds to the old system configuration and the new weight factor corresponds to new system configuration;

determining a relative weight change for each resource in the set of resources based on the old weight factor and the new weight factor; and

establishing the ordered classification based on the relative weight changes of the set of resources.

13. The resource of claim 9, wherein the listing request comprises:

a request to at least one of;

identify valid encoded data slices having slice names within the range of slice names and to determine location of the valid encoded data slices.

14. The resource of claim 13, wherein the valid encoded data slices comprise:

encoded data slices that are stored in one or more storage units;

encoded data slices that are being created or modified by a computing device; and

encoded data slices that being rebuild by a rebuilding agent.

15. The resource of claim 9, wherein the system configuration change comprises one or more of:

an addition of a resource;

a deletion of a resource; and

a change in weighting factors of a DAP.

16. The resource of claim 9 comprises one of:

a storage location within a storage unit;

a storage unit;

a storage pool;

a computing device; and

a rebuilder.