Zone storage—resilient and efficient storage transactions

• US 10,382,553 B2
• Filed: 02/20/2017
• Issued: 08/13/2019
• Est. Priority Date: 02/20/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Dhuse, Greg R., Khadiwala, Ravi V., Motwani, Manish, Viraraghavan, Praveen, Volvovski, Ilya
• Primary Examiner(s): Zong, Ruolei
• Application Number:
  US15/437,048
• Publication Number:
  US 20180241818A1
• Time in Patent Office: 904 Days
• Field of Search: 709217
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 2212/60 Details of cache memory
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed storage network (DSN) includes a DSN memory and a distributed storage (DS) processing unit. The DS processing unit determines a particular revision of source data for persistence, and generates a protocol message associated with the particular revision of source data. The protocol message is generated based on a balance between durability and persistence associated with the particular revision. The protocol message is transmitted from the DS processing unit to the DSN memory, and indicates to the DSN memory a particular persistence protocol. The DSN memory process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

Claims

1. A method for use in a distributed storage network (DSN) including a DSN memory and distributed storage (DS) processing unit, the method comprising:

selecting, by the DS unit, a particular persistence protocol to be applied to a particular revision of source data, the particular persistence protocol selected from among of a plurality of different persistence protocols, each persistence protocol implementing a different balance between durability and persistence;

determining, by the DS processing unit, the particular revision of source data to be persisted in accordance with the particular persistence protocol;

generating, by the DS processing unit, a protocol message associated with the particular revision of source data, wherein generating the protocol message is based on the particular persistence protocol; and

transmitting the protocol message, from the DS processing unit to the DSN memory, the protocol message indicating to the DSN memory the particular persistence protocol.

2. The method of claim 1, wherein generating the protocol message includes:

generating a protocol message instructing the DSN memory to;

store the particular revision of source data; and

make the particular revision of source data visible to subsequent requests without guaranteeing persistence of the particular revision of source data.

3. The method of claim 2, wherein the protocol message includes:

a dispersal encoded data slice associated with the particular revision of source data.

4. The method of claim 1, wherein generating the protocol message includes:

generating a protocol message instructing the DSN memory to make the particular revision of source data fully durable.

5. The method of claim 4, wherein the protocol message includes:

a versioned slice name of a dispersal encoded data slice associated with the particular revision of source data; and

the protocol message further instructs the DSN memory to transmit an acknowledgement message to the DS processing unit indicating that the particular revision of source data has been flushed to persistent storage.

6. The method of claim 5, wherein the protocol message further instructs the DSN memory to:

remove data slices associated with other revisions of the source data, other than the particular revision of the source data, wherein the other revisions have been stored in a cache memory.

7. The method of claim 1, wherein generating the protocol message includes:

generating a protocol message instructing the DSN memory to;

remove previously stored versions of the source data, other than the particular revision of source data; and

clean up temporary transactions related to the particular revision of source data.

8. A distributed storage (DS) processing unit for use in a distributed storage network (DSN) including at least one DSN memory, the DS processing unit comprising:

a processor;

memory operably associated with the processor;

a non-transitory program of instructions configured to be stored in the memory and executed by the processor, the program of instructions including;

at least one instruction to select a particular persistence protocol to be applied to a particular revision of source data, the particular persistence protocol selected from among of a plurality of different persistence protocols, each persistence protocol implementing a different balance between durability and persistence;

at least one instruction to determine the particular revision of source data to be persisted in accordance with the particular persistence protocol;

at least one instruction to generate a protocol message associated with the particular revision of source data, wherein generating the protocol message is based on the particular persistence protocol; and

at least one instruction to transmit the protocol message to the at least one DSN memory, the protocol message indicating to the at least one DSN memory the particular persistence protocol.

9. The DS processing unit of claim 8, wherein the at least one instruction to generate the protocol message includes:

at least one instruction to generate a protocol message instructing the at least one DSN memory to;

store the particular revision of source data; and

make the particular revision of source data visible to subsequent requests without guaranteeing persistence of the particular revision of source data.

10. The DS processing unit of claim 9, wherein the protocol message includes:

a dispersal encoded data slice associated with the particular revision of source data.

11. The DS processing unit of claim 8, wherein the at least one instruction to generate the protocol message includes:

at least one instruction to generate a protocol message instructing the at least one DSN memory to make the particular revision of source data fully durable.

12. The DS processing unit of claim 11, wherein the protocol message includes:

a versioned slice name of a dispersal encoded data slice associated with the particular revision of source data; and

the protocol message further instructs the at least one DSN memory to transmit an acknowledgement message to the DS processing unit indicating that the particular revision of source data has been flushed to persistent storage.

13. The DS processing unit of claim 12, wherein the protocol message further instructs the at least one DSN memory to:

remove data slices associated with other revisions of the source data, other than the particular revision of the source data, wherein the other revisions have been stored in a cache memory.

14. The DS processing unit of claim 8, wherein at least one instruction to generate the protocol message includes:

at least one instruction to generate a protocol message instructing the DSN memory to;

remove previously stored versions of the source data, other than the particular revision of source data; and

clean up temporary transactions related to the particular revision of source data.

15. A distributed storage network (DSN) comprising:

a DSN memory including a plurality of storage units and a first computing core;

a distributed storage (DS) processing unit including a second computing core, the DS processing unit coupled to the DSN memory via a communications network, and configured to transmit dispersed encoded data slices of source data, including revised data slices representing revisions made to source data, to the DSN memory for storage and later retrieval, the DS processing unit further configured to;

select a particular persistence protocol to be applied to a particular revision of source data, the particular persistence protocol selected from among of a plurality of different persistence protocols, each persistence protocol implementing a different balance between durability and persistence;

determine the particular revision of source data to be persisted in accordance with the particular persistence protocol;

generate a protocol message associated with the particular revision of source data, wherein generating the protocol message is based on the particular persistence protocol;

transmit the protocol message to the DSN memory, the protocol message indicating to the DSN memory the particular persistence protocol; and

the DSN memory configured to process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

16. The DSN of claim 15, wherein generating the protocol message includes:

generating a protocol message instructing the DSN memory to;

store the particular revision of source data; and

make the particular revision of source data visible to subsequent requests without guaranteeing persistence of the particular revision of source data.

17. The DSN of claim 16, wherein the protocol message includes:

a dispersal encoded data slice associated with the particular revision of source data.

18. The DSN of claim 15, wherein generating the protocol message includes:

generating a protocol message instructing the DSN memory to make the particular revision of source data fully durable.

19. The DSN of claim 18, wherein the protocol message includes:

a versioned slice name of a dispersal encoded data slice associated with the particular revision of source data; and

the protocol message further instructs the DSN memory to transmit an acknowledgement message to the DS processing unit indicating that the particular revision of source data has been flushed to persistent storage.

20. The DSN of claim 19, wherein the protocol message further instructs the DSN memory to:

remove data slices associated with other revisions of the source data, other than the particular revision of the source data, wherein the other revisions have been stored in a cache memory.