Delegating iterative storage unit access in a dispersed storage network

• US 9,965,336 B2
• Filed: 03/04/2015
• Issued: 05/08/2018
• Est. Priority Date: 04/30/2014
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Dhuse, Greg
• Primary Examiner(s): Abad, Farley
• Assistant Examiner(s): Yu, Henry W
• Application Number:
  US14/638,575
• Publication Number:
  US 20150317099A1
• Time in Patent Office: 1,161 Days
• Field of Search: 710 15, 710 62, 710 74
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database

Abstract

A method begins by a computing device of a dispersed storage network (DSN) determining whether a DSN access request involves an iterative storage unit access process. When the DSN access request involves the iterative storage unit access process, the method continues with the computing device determining, based on configuration of a plurality of storage units storing data objects associated with the DSN access request, that a storage unit of the plurality of storage units is capable of executing at least a portion of the iterative storage unit access process better than the computing device, sending the DSN access request and a control command to the storage unit, where the control command instructs the storage unit to perform at least a portion of the iterative storage unit access process, and receiving, from the storage unit, at least a partial response to the DSN access request.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

determining whether a DSN access request involves an iterative storage unit access process; and

when the DSN access request involves the iterative storage unit access process;

determining, based on configuration of a plurality of storage units storing data objects associated with the DSN access request, that a storage unit of the plurality of storage units is capable of executing at least a portion of the iterative storage unit access process better than the computing device, wherein a data object of the data objects is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage (DS) error encoding parameters to produce a set of encoded data slices (EDSs), wherein the set of EDSs are distributedly stored among the plurality of storage units, and wherein a decode threshold number of EDSs are needed to recover the data segment;

sending the DSN access request and a control command to the storage unit, wherein the control command instructs the storage unit to perform at least a portion of the iterative storage unit access process, and wherein the control command includes an instruction indicating the decode threshold number of storage units of the plurality of storage units to access while executing the at least a portion of the iterative storage unit access process; and

receiving, from the storage unit, at least a partial response to the DSN access request.

2. The method of claim 1, wherein the determining that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device comprises:

determining that the storage unit is coupled via a local area network (LAN) connection to one or more other storage units of the plurality of storage units; and

determining that a reduction in wide area network (WAN) communications constitutes better execution of the at least a portion of the iterative storage unit access process.

3. The method of claim 2, wherein the reduction in WAN communications comprises at least one of:

a reduction in WAN traffic; and

a reduction in latency of processing the at least a portion of the iterative storage unit access process.

4. The method of claim 1, wherein the control command comprises:

instructions to perform the iterative storage unit access process and to provide a complete response to the DSN access request.

5. The method of claim 1, wherein the DSN access request comprises one of:

a request for addressing information regarding data having one or more search criteria, wherein the iterative storage unit access process is executed to traverse an index tree to identify the data based on the one or more search criteria; and

a request for the data having the one or more search criteria.

6. The method of claim 1,

wherein at least one of the decode threshold number of storage units is coupled via a wide area network (WAN) connection to the storage unit.

7. The method of claim 1 further comprises:

determining that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device when the storage unit is coupled to a desired number of other storage units of the plurality of storage units via a LAN connection.

8. The method of claim 1 further comprises:

for each iteration of the iterative storage unit access process;

receiving a group iteration partial response from the storage unit that represents a collective response of a sub-set of the plurality of storage units;

receiving iterative partial responses from storage units of a second sub-set of the plurality of storage units; and

processing the group iterative partial response and the iterative partial responses to produce an iterative response.

9. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of a computing device of a dispersed storage network (DSN), causes the computing device to;

determine whether a DSN access request involves an iterative storage unit access process; and

when the DSN access request involves the iterative storage unit access process;

determine, based on configuration of a plurality of storage units storing data objects associated with the DSN access request, that a storage unit of the plurality of storage units is capable of executing at least a portion of the iterative storage unit access process better than the computing device, wherein a data object of the data objects is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage (DS) error encoding parameters to produce a set of encoded data slices (EDSs), wherein the set of EDSs are distributedly stored among the plurality of storage units, and wherein a decode threshold number of EDSs are needed to recover the data segment;

send the DSN access request and a control command to the storage unit, wherein the control command instructs the storage unit to perform at least a portion of the iterative storage unit access process, and wherein the control command includes an instruction indicating the decode threshold number of storage units of the plurality of storage units to access while executing the at least a portion of the iterative storage unit access process; and

receive, from the storage unit, at least a partial response to the DSN access request.

10. The non-transitory computer readable storage medium of claim 9, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the computing device of the DSN to determine that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device by:

determining that the storage unit is coupled via a local area network (LAN) connection to one or more other storage units of the plurality of storage units; and

determining that a reduction in wide area network (WAN) communications constitutes better execution of the at least a portion of the iterative storage unit access process.

11. The non-transitory computer readable storage medium of claim 10, wherein the reduction in WAN communications comprises at least one of:

a reduction in WAN traffic; and

a reduction in latency of processing the at least a portion of the iterative storage unit access process.

12. The non-transitory computer readable storage medium of claim 9, wherein the control command comprises:

instructions to perform the iterative storage unit access process and to provide a complete response to the DSN access request.

13. The non-transitory computer readable storage medium of claim 9, wherein the DSN access request comprises one of:

a request for addressing information regarding data having one or more search criteria, wherein the iterative storage unit access process is executed to traverse an index tree to identify the data based on the one or more search criteria; and

a request for the data having the one or more search criteria.

14. The non-transitory computer readable storage medium of claim 9, wherein the control command comprises:

wherein at least one of the decode threshold number of storage units is coupled via a WAN connection to the storage unit.

15. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the computing device of the DSN to;

determine that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device when the storage unit is coupled to a desired number of other storage units of the plurality of storage units via a LAN connection.

16. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the computing device of the DSN to;

for each iteration of the iterative storage unit access process;

receive a group iteration partial response from the storage unit that represents a collective response of a sub-set of the plurality of storage units;

receive iterative partial responses from storage units of a second sub-set of the plurality of storage units; and

process the group iterative partial response and the iterative partial responses to produce an iterative response.

17. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory;

and a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

determine whether a DSN access request involves an iterative storage unit access process;

and

when the DSN access request involves the iterative storage unit access process;

determine, based on configuration of a plurality of storage units storing data objects associated with the DSN access request, that a storage unit of the plurality of storage units is capable of executing at least a portion of the iterative storage unit access process better than the computing device, wherein a data object of the data objects is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage (DS) error encoding parameters to produce a set of encoded data slices (EDSs), wherein the set of EDSs are distributedly stored among the plurality of storage units, and wherein a decode threshold number of EDSs are needed to recover the data segment;

send, via the interface, the DSN access request and a control command to the storage unit, wherein the control command instructs the storage unit to perform at least a portion of the iterative storage unit access process, and wherein the control command includes an instruction indicating the decode threshold number of storage units of the plurality of storage units to access while executing the at least a portion of the iterative storage unit access process;

and receive, via the interface from the storage unit, at least a partial response to the DSN access request.

18. The computing device of claim 17, wherein the processing module functions to determine that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device by:

determining that the storage unit is coupled via a local area network (LAN) connection to one or more other storage units of the plurality of storage units; and

determining that a reduction in wide area network (WAN) communications constitutes better execution of the at least a portion of the iterative storage unit access process.

19. The computing device of claim 18, wherein the reduction in WAN communications comprises at least one of:

a reduction in WAN traffic; and

a reduction in latency of processing the at least a portion of the iterative storage unit access process.

20. The computing device of claim 17, wherein the control command comprises:

instructions to perform the iterative storage unit access process and to provide a complete response to the DSN access request.

21. The computing device of claim 17, wherein the DSN access request comprises one of:

a request for addressing information regarding data having one or more search criteria, wherein the iterative storage unit access process is executed to traverse an index tree to identify the data based on the one or more search criteria; and

a request for the data having the one or more search criteria.

22. The computing device of claim 17,

wherein at least one of the decode threshold number of storage units is coupled via a WAN connection to the storage unit.

23. The computing device of claim 17, wherein the processing module further functions to:

determine that the storage unit is capable of executing the at least a portion of the iterative storage unit access process better than the computing device when the storage unit is coupled to a desired number of other storage units of the plurality of storage units via a LAN connection.

24. The computing device of claim 17, wherein the processing module further functions to:

for each iteration of the iterative storage unit access process;

receive, via the interface, a group iteration partial response from the storage unit that represents a collective response of a sub-set of the plurality of storage units;

receive, via the interface, iterative partial responses from storage units of a second sub-set of the plurality of storage units; and

process the group iterative partial response and the iterative partial responses to produce an iterative response.