Distributed storage network with coordinated partial task execution and methods for use therewith

• US 9,848,044 B2
• Filed: 05/27/2014
• Issued: 12/19/2017
• Est. Priority Date: 07/31/2013
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Leggette, Wesley, Baptist, Andrew, Dhuse, Greg, Resch, Jason K., Grube, Gary W.
• Primary Examiner(s): Kamran, Mehran
• Application Number:
  US14/287,499
• Publication Number:
  US 20150040134A1
• Time in Patent Office: 1,302 Days
• Field of Search: None
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0668 adopting a particular infrastructure
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes receiving a task for execution by a plurality of distributed storage and task execution units A priority level is determined for the task. A plurality of coordinated partial task requests are generated and sent to the plurality of distributed storage and task execution units, wherein the plurality coordinated partial task requests indicate a plurality of coordinated partial tasks and the priority level. A plurality of partial task results are received in response to performance of the plurality of coordinated partial tasks by the plurality of distributed storage and task execution units. A task result for the task is generated based on the plurality of partial task results.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage and task (DST) network, the method comprises:

receiving a task for execution by a plurality of distributed storage and task execution units;

receiving and processing data associated with the task to produce encoded data slice groupings, wherein the processing partitions the data associated with the task into a plurality of data partitions and, for each data partition, error encodes the data partition to produce encoded data slices and groups the encoded data slices into the encoded data slice groupings;

partitioning the task into partial tasks, where each of the partial tasks includes a corresponding one of the encoded data slice groupings;

determining a priority level for the task based on an estimated performance impact for the task;

generating a plurality of coordinated partial task requests to the plurality of distributed storage and task execution units, wherein the plurality coordinated partial task requests indicate a plurality of coordinated partial tasks, the priority level, and the corresponding one of the encoded data slice groupings, wherein at least one of the plurality of distributed storage and task execution units communicates coordination information with at least one other of the plurality of distributed storage and task execution units and wherein the coordination information relates to execution of another plurality of coordinated partial tasks;

receiving a plurality of partial task results in response to performance of the plurality of coordinated partial tasks by the plurality of distributed storage and task execution units; and

generating a task result for the task based on the plurality of partial task results.

2. The method of claim 1 wherein at least one of the plurality of distributed storage and task execution units communicates at least one of the plurality of partial task results with at least one other of the plurality of distributed storage and task execution units.

3. The method of claim 2 wherein the at least one other of the plurality of distributed storage and task execution units generates at least one other of the plurality of partial task results based on the at least one of the plurality of partial task results received from the at least one of the plurality of distributed storage and task execution units.

4. The method of claim 1 wherein the task includes one of:

aborted transaction clean up from at least one prior aborted task, a data migration, expired data clean up corresponding to stored data that has expired, a snapshot clean up, an index health check, a segment health check, a scan for missing ones of encoded data slices, and a system maintenance task.

5. The method of claim 1 wherein at least one of the plurality of distributed storage and task execution units communicates processing resource availability information with at least one other of the plurality of distributed storage and task execution units;

wherein the processing resource availability information includes at least one of;

a current processing utilization, an estimated processing utilization, indication of at least one pending one of the plurality of coordinated partial tasks, a completion forecast for the at least one pending one of the plurality of coordinated partial tasks, and a commencement forecast for at least one non-pending one of the plurality of coordinated partial tasks.

6. The method of claim 1 wherein each the plurality of distributed storage and task execution units performs its generated partial task upon its corresponding one of the encoded data slice groupings to produce the partial task results.

7. A dispersed storage and task (DST) client device comprises:

a processing system including a processor and a memory that is configured to;

receive a task for execution by a plurality of distributed storage and task execution units;

determine a priority level for the task based on an estimated performance impact for the task;

receive and process data associated with the task to produce encoded data slice groupings, wherein the process partitions the data associated with the task into a plurality of data partitions and, for each data partition, error encodes the data partition to produce encoded data slices and groups the encoded data slices into the encoded data slice groupings;

partition the task into partial tasks, where each of the partial tasks includes a corresponding one of the encoded data slice groupings;

generate a plurality of coordinated partial task requests to the plurality of distributed storage and task execution units, wherein the plurality coordinated partial task requests indicate a plurality of coordinated partial tasks, the priority level, and the corresponding one of the encoded data slice groupings, wherein at least one other of the plurality of distributed storage and task execution units generates at least one other of the plurality of partial task results based on the at least one of the plurality of partial task results received from the at least one of the plurality of distributed storage and task execution units;

receive a plurality of partial task results in response to performance of the plurality of coordinated partial tasks by the plurality of distributed storage and task execution units; and

generate a task result for the task based on the plurality of partial task results.

8. The DST client device of claim 7 wherein the task includes one of:

aborted transaction clean up from at least one prior aborted task, a data migration, expired data clean up corresponding to stored data that has expired, a snapshot clean up, an index health check, a segment health check, a scan for missing ones of encoded data slices, and a system maintenance task.

9. The DST client device of claim 7 wherein at least one of the plurality of distributed storage and task execution units communicates processing resource availability information with at least one other of the plurality of distributed storage and task execution units;

wherein the processing resource availability information includes at least one of;

a current processing utilization, an estimated processing utilization, indication of at least one pending one of the plurality of coordinated partial tasks, a completion forecast for the at least one pending one of the plurality of coordinated partial tasks, and a commencement forecast for at least one non-pending one of the plurality of coordinated partial tasks.

10. The DST client device of claim 7 wherein at least one of the plurality of distributed storage and task execution units communicates coordination information with at least one other of the plurality of distributed storage and task execution units;

wherein the coordination information relates to execution of another plurality of coordinated partial tasks.

11. The dispersed storage and task (DST) client device of claim 7 wherein each the plurality of distributed storage and task execution units performs its generated partial task upon its corresponding one of the encoded data slice groupings to produce the partial task results.

12. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

receive a task for execution by a plurality of distributed storage and task execution units;

determine a priority level for the task based on an estimated performance impact for the task;

receive and process data associated with the task to produce encoded data slice groupings, wherein the processing partitions the data associated with the task into a plurality of data partitions and, for each data partition, error encodes the data partition to produce encoded data slices and groups the encoded data slices into the encoded data slice groupings;

partition the task into partial tasks, where each of the partial tasks includes a corresponding one of the encoded data slice groupings;

generate a plurality of coordinated partial task requests to the plurality of distributed storage and task execution units, wherein the plurality coordinated partial task requests indicate a plurality of coordinated partial tasks, the priority level, and the corresponding one of the encoded data slice groupings, wherein at least one of the plurality of distributed storage and task execution units communicates processing resource availability information with at least one other of the plurality of distributed storage and task execution units and wherein the processing resource availability information includes at least one of;

a current processing utilization, an estimated processing utilization, indication of at least one pending one of the plurality of coordinated partial tasks, a completion forecast for the at least one pending one of the plurality of coordinated partial tasks, and a commencement forecast for at least one non-pending one of the plurality of coordinated partial tasks;

receive a plurality of partial task results in response to performance of the plurality of coordinated partial tasks by the plurality of distributed storage and task execution units; and

generate a task result for the task based on the plurality of partial task results.

13. The non-transitory computer readable storage medium of claim 12 wherein at least one of the plurality of distributed storage and task execution units communicates at least one of the plurality of partial task results with at least one other of the plurality of distributed storage and task execution units and wherein the at least one other of the plurality of distributed storage and task execution units generates at least one other of the plurality of partial task results based on the at least one of the plurality of partial task results received from the at least one of the plurality of distributed storage and task execution units.

14. The non-transitory computer readable storage medium of claim 12 wherein the task includes one of:

aborted transaction clean up from at least one prior aborted task, a data migration, expired data clean up corresponding to stored data that has expired, a snapshot clean up, an index health check, a segment health check, a scan for missing ones of encoded data slices, and a system maintenance task.

15. The non-transitory computer readable storage medium of claim 12 wherein at least one of the plurality of distributed storage and task execution units communicates coordination information with at least one other of the plurality of distributed storage and task execution units;

wherein the coordination information relates to execution of another plurality of coordinated partial tasks.