Storing related data in a dispersed storage network

• US 9,456,035 B2
• Filed: 03/17/2014
• Issued: 09/27/2016
• Est. Priority Date: 05/03/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Storm, Michael Colin, Leggette, Wesley, Motwani, Manish, Dhuse, Greg, Resch, Jason K., Baptist, Andrew
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US14/215,542
• Publication Number:
  US 20140330921A1
• Time in Patent Office: 925 Days
• Field of Search: 709/213, 714/752, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1865 Transactional file systems
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by each of a group of write requesting modules of a dispersed storage network (DSN) generating one or more sets of write requests regarding one of a group of portions of related data, sending a group of the one or more sets of write requests to DSN memory, and sending binding information to a binding module. The method continues with the binding module processing remaining phases of the group of the one or more sets of write requests for writing the related data into the DSN memory as a single set of write requests and notifying the write requesting modules of status of the writing the related data into the DSN memory at completion of the processing of the remaining phases such that the related data is made accessible as a single piece of data when the processing of the remaining phases is successful.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

generating, by each of a group of write requesting modules of the DSN, one or more sets of write requests regarding one of a group of portions of related data, wherein a write request of the one or more sets of write requests includes an encoded data slice and information indicating that the write request corresponds to a portion of the group of portions of the related data, and wherein a data segment of the portion of the group of portions of the related data is encoded to produce a set of encoded data slices, which includes the encoded data slice;

sending, by the group of write requesting modules, a group of the one or more sets of write requests to DSN memory of the DSN;

sending, by the group of write requesting modules, binding information to a binding module of the DSN;

processing, by the binding module, remaining phases of the group of the one or more sets of write requests for writing the related data into the DSN memory as a single set of write requests; and

notifying, by the binding module, the write requesting modules of status of the writing the related data into the DSN memory at completion of the processing of the remaining phases such that, while the related data is written into the DSN memory in pieces over time as groups of portions, the related data is made accessible as a single piece of data when the processing of the remaining phases is successful and, when the processing of the remaining phases is not successful, none of the pieces of the related data are accessible.

2. The method of claim 1 further comprises:

determining, by the group of write requesting modules, that data objects are related to produce the related data.

3. The method of claim 2, wherein the determining comprises at least one of:

receiving a command to relate the data objects into the related data;

identifying a common source of the data objects;

receiving the data objects concurrently within a given time frame;

identifying a common data type of the data objects; and

interpreting metadata of the data objects.

4. The method of claim 1, wherein the information indicating that the write request corresponds to the portion of the group of portions of the related data comprises:

a field within the write request to indicate one or more of;

the write request corresponds to the related data, a total number of sets of write requests regarding the related data, a current number of a set of the sets of write requests regarding the related data, a size of the related data, and an identifier of the binding module.

5. The method of claim 1, wherein the processing the remaining phases by the binding module comprises:

when a favorable response to the group of the one or more sets of write requests is received;

generating, by the binding module, a set of write commit requests for the group of portions of the related data; and

sending, by the binding module, the set of write commit requests to the DSN memory; and

when the favorable response to the group of the one or more sets of write requests is not received;

generating, by the binding module, a set of write rollback requests for the group of portions of the related data;

sending, by the binding module, the set of write rollback requests to the DSN memory; and

notifying, by the binding module, each of the group of write requesting modules of rollback of the writing the related data.

6. The method of claim 5, wherein the processing the remaining phases by the binding module comprises:

when a favorable response to the set of write commit requests is received;

generating, by the binding module, a set of write finalize requests for the group of portions of the related data;

sending, by the binding module, the set of write finalize requests to the DSN memory; and

notifying, by the binding module, each of the group of write requesting modules of finalizing of the writing the related data;

when the favorable response to the set of write commit requests is not received;

generating, by the binding module, a set of write undo requests for the group of portions of the related data;

sending, by the binding module, the set of write undo requests to the DSN memory; and

notifying, by the binding module, each of the group of write requesting modules of undoing of the writing the related data.

7. The method of claim 1 further comprises:

interpreting, by storage units of the DSN memory, the information indicating that the write request corresponds to the portion of the group of portions of the related data; and

communicating, by the storage units, with the binding module regarding the processing the remaining phases.

8. The method of claim 1, wherein the binding information comprises:

transaction information, wherein the transaction information includes, from each of the group of write requesting modules, a different transaction number that indicates a separate write operation and an identifier of the write requesting module.

9. A non-transitory computer readable storage medium comprises:

a first memory section that stores operational instructs that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

generate one or more sets of write requests regarding one of a group of portions of related data, wherein a write request of the one or more sets of write requests includes an encoded data slice and information indicating that the write request corresponds to a portion of the group of portions of the related data, and wherein a data segment of the portion of the group of portions of the related data is encoded to produce a set of encoded data slices, which includes the encoded data slice;

send a group of the one or more sets of write requests to a DSN memory of the DSN; and

send binding information to a binding module of the DSN; and

a second memory section that stores operational instructions that, when executed by the one or more processing modules of the one or more computing devices of the DSN, causes the one or more computing devices to;

process remaining phases of the group of the one or more sets of write requests for writing the related data into the DSN memory as a single set of write requests; and

notify write requesting modules of status of the writing the related data into the DSN memory at completion of the processing of the remaining phases such that, while the related data is written into the DSN memory in pieces over time as groups of portions, the related data is made accessible as a single piece of data when the processing of the remaining phases is successful and, when the processing of the remaining phases is not successful, none of the pieces of the related data are accessible.

10. The non-transitory computer readable storage medium of claim 9 further comprises:

the first memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine that data objects are related to produce the related data.

11. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the further operational instructions stored by the first memory section to cause the one or more computing devices of the DSN to determine that the data objects are related by at least one of:

receiving a command to relate the data objects into the related data;

identifying a common source of the data objects;

receiving the data objects concurrently within a given time frame;

identifying a common data type of the data objects; and

interpreting metadata of the data objects.

12. The non-transitory computer readable storage medium of claim 9, wherein the one or more processing modules functions to execute the operational instructions stored by the second memory section to cause the one or more computing devices of the DSN to process the remaining phases by:

when a favorable response to the group of the one or more sets of write requests is received;

generating a set of write commit requests for the group of portions of the related data; and

sending the set of write commit requests to the DSN memory; and

when the favorable response to the group of the one or more sets of write requests is not received;

generating a set of write rollback requests for the group of portions of the related data;

sending the set of write rollback requests to the DSN memory; and

notifying each of the write requesting modules of rollback of the writing the related data.

13. The non-transitory computer readable storage medium of claim 12, wherein the one or more processing modules functions to execute the operational instructions stored by the second memory section to cause the one or more computing devices of the DSN to process the remaining phases by:

when a favorable response to the set of write commit requests is received;

generating a set of write finalize requests for the group of portions of the related data;

sending the set of write finalize requests to the DSN memory; and

notifying each of the write requesting modules of finalizing of the writing the related data;

when the favorable response to the set of write commit requests is not received;

generating a set of write undo requests for the group of portions of the related data;

sending the set of write undo requests to the DSN memory; and

notifying each of the write requesting modules of undoing of the writing the related data.

14. The non-transitory computer readable storage medium of claim 9, wherein the information indicating that the write request corresponds to the portion of the group of portions of the related data comprises:

a field within the write request to indicate one or more of;

the write request corresponds to the related data, a total number of sets of write requests regarding the related data, a current number of a set of the one or more sets of write requests regarding the related data, a size of the related data, and an identifier of the binding module.

15. The non-transitory computer readable storage medium of claim 9 further comprises:

a third memory section that stores operational instructs that, when executed by the one or more processing modules of the one or more computing devices of the DSN, causes the one or more computing devices to;

interpret the information indicating that the write request corresponds to the portion of the group of portions of the related data; and

communicate with the binding module regarding the processing the remaining phases.

16. The non-transitory computer readable storage medium of claim 9, wherein the binding information comprises:

transaction information, wherein the transaction information includes, from each of the write requesting modules, a different transaction number that indicates a separate write operation and an identifier of the write requesting module.