Batching access requests in a dispersed storage network

• US 10,007,444 B2
• Filed: 04/29/2016
• Issued: 06/26/2018
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leggette, Wesley B., Resch, Jason K.
• Primary Examiner(s): Portka, Gary J
• Application Number:
  US15/142,280
• Publication Number:
  US 20170315735A1
• Time in Patent Office: 788 Days
• Field of Search: 709201, 709214, 711148, 711154
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A method begins by a processing system of a dispersed storage network (DSN) selecting a first proper subset of dispersed storage and task (DST) execution units of the DSN. The method continues with the processing system batching access requests over a time period in accordance with the first proper subset of the DST execution units of the DSN to limit the access requests to the first proper subset of DST execution units of the DSN during the time period.

Claims

1. A method for execution by a dispersed storage and task (DST) processing unit of a dispersed storage network (DSN) that includes a processor and a memory, the method comprises:

selecting a first proper subset of dispersed storage and task (DST) execution units of the DSN based on detecting that a second DST processing unit requires access to the first proper subset of DST execution units; and

batching access requests over a time period in accordance with the first proper subset of the DST execution units of the DSN to limit the access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the access requests are issued by the DST processing unit;

wherein the second DST processing unit synchronizes with the DST processing unit to batch second access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN to limit the second access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the second access requests are issued by the second DST processing unit.

2. The method of claim 1 wherein the access requests include a plurality of write requests and wherein batching the access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN includes limiting the plurality of write requests to a limited area of a source name range during the time period.

3. The method of claim 2 wherein batching the access requests over the time period in accordance with the first proper subset of the DST execution units limits a number of the DST execution units of the DSN that load vault metadata during the time period.

4. The method of claim 1 further comprising:

selecting a second proper subset of vaults of the DSN; and

wherein batching the access requests over the time period is further in accordance with the second proper subset of the vaults to further limit the access requests over the time period to the second proper subset of vaults of the DSN.

5. The method of claim 4 wherein batching the access requests over the time period includes limiting a number of vault/DST execution unit combinations during the time period.

6. The method of claim 4 wherein the second proper subset of vaults of the DSN includes a single vault.

7. The method of claim 4 wherein the access request includes one of:

a scanning operation;

or

a repair operation; and

wherein batching the access requests over the time period includes one of;

performing the scanning operation for a single vault;

or

performing the repair operation for the single vault.

8. A dispersed storage and task (DST) processing unit of a dispersed storage network (DSN) comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the DST processing unit to;

select a first proper subset of dispersed storage and task (DST) execution units of the DSN based on detecting that a second DST processing unit requires access to the first proper subset of DST execution units; and

batch access requests over a time period in accordance with the first proper subset of the DST execution units of the DSN to limit the access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the access requests are issued by the DST processing unit;

wherein the second DST processing unit synchronizes with the DST processing unit to batch second access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN to limit the second access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the second access requests are issued by the second DST processing unit.

9. The processing system of claim 8 wherein the access requests include a plurality of write requests and wherein batch the access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN includes limiting the plurality of write requests to a limited area of a source name range during the time period.

10. The processing system of claim 9 wherein batch the access requests over the time period in accordance with the first proper subset of the DST execution units limits a number of the DST execution units of the DSN that load vault metadata during the time period.

11. The processing system of claim 8 wherein the memory further stores operational instructions, that when executed by the at least one processor cause the processing system to:

select a second proper subset of vaults of the DSN; and

wherein batch the access requests over the time period is further in accordance with the second proper subset of the vaults to further limit the access requests over the time period to the second proper subset of vaults of the DSN.

12. The processing system of claim 11 wherein batch the access requests over the time period includes limiting a number of vault/DST execution unit combinations during the time period.

13. The processing system of claim 11 wherein the second proper subset of vaults of the DSN includes a single vault.

14. The processing system of claim 11 wherein the access request includes one of:

a scanning operation;

or

a repair operation; and

wherein batch the access requests over the time period includes one of;

performing the scanning operation for a single vault;

or

performing the repair operation for the single vault.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a dispersed storage and task (DST) processing unit of a dispersed storage network (DSN) that includes a processor and a memory, causes the DST processing unit to;

select a first proper subset of dispersed storage and task (DST) execution units of the DSN based on detecting that a second DST processing unit requires access to the first proper subset of DST execution units; and

batch access requests over a time period in accordance with the first proper subset of the DST execution units of the DSN to limit the access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the access requests are issued by the DST processing unit;

wherein the second DST processing unit synchronizes with the DST processing unit to batch second access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN to limit the second access requests to the first proper subset of DST execution units of the DSN during the time period, wherein the second access requests are issued by the second DST processing unit.

16. The non-transitory computer readable storage medium of claim 15 wherein the access requests include a plurality of write requests and wherein batch the access requests over the time period in accordance with the first proper subset of the DST execution units of the DSN includes limiting the plurality of write requests to a limited area of a source name range during the time period.

17. The non-transitory computer readable storage medium of claim 15 wherein batch the access requests over the time period in accordance with the first proper subset of the DST execution units limits a number of the DST execution units of the DSN that load vault metadata during the time period.

18. The non-transitory computer readable storage medium of claim 15 wherein the at least one memory section further stores operational instructions, that when executed by the processing system, cause the processing system to:

select a second proper subset of vaults of the DSN; and

wherein batch the access requests over the time period is further in accordance with the second proper subset of the vaults to further limit the access requests over the time period to the second proper subset of vaults of the DSN.

19. The non-transitory computer readable storage medium of claim 18 wherein batch the access requests over the time period includes limiting a number of vault/DST execution unit combinations during the time period.

20. The non-transitory computer readable storage medium of claim 18 wherein the access request includes one of:

a scanning operation;

or

a repair operation; and

wherein batch the access requests over the time period includes one of;

performing the scanning operation for a single vault;

or

performing the repair operation for the single vault.