ZONE STORAGE - QUICKLY RETURNING TO A STATE OF CONSISTENCY FOLLOWING AN UNEXPECTED EVENT

• US 20180239701A1
• Filed: 02/17/2017
• Published: 08/23/2018
• Est. Priority Date: 02/17/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Kaczmarek, Joseph M., Motwani, Manish, Viraraghavan, Praveen, Volvovski, Ilya
• Granted Patent: US 10,552,341 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 2212/1021 Hit rate improvement
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/154 Networked environment

Abstract

Systems and Methods for data storage in a distributed storage network are disclosed. Unexpected errors can adversely affect consistency of both the content of a write (including the slice data), and the synchronicity between the written slices and metadata structures. To maintain consistency between these data structures, a sequencing of the order of writes and flushes to the memory devices for the different data structures may be enforced as follows: First: Slice content data is first written to the volatile memory (e.g. a cache memory) of a DS unit; Second: the Slice content data stored in volatile memory is “flushed” to a non-volatile bin (which bin is associated with a group of physical memory blocks in non-volatile memory); Third: after the flush of the slice content data to the bin (i.e. data is durable on the media device): metadata relating to the data is written.

Claims

1. A method of storing data comprising:

writing slice content data to a volatile memory;

flushing the slice content data to a bin in a non-volatile memory after determining the slice content data was completely written to the volatile memory;

writing missing metadata corresponding to the slice content data after flushing the slice content data to the bin; and

writing missing metadata corresponding to the bin after flushing the slice content data to the bin.

2. The method of claim 1, wherein the slice content data is flushed to the bin in the non-volatile memory using a pattern of append only writes.

3. The method of claim 1, wherein the metadata includes one or more of a pointer to the bin, and slice name and size information.

4. The method of claim 1, wherein the step of writing missing metadata corresponding to the slice content data and the step of writing missing metadata corresponding to the bin are performed sequentially.

5. The method of claim 1, wherein the step of writing missing metadata corresponding to the slice content data and the step of writing missing metadata corresponding to the bin are performed in parallel.

6. The method of claim 1, wherein the slice content data is written to the volatile memory using an atomic write structure.

7. The method of claim 6, wherein the atomic write structure includes one or more of defined headers, length indicators, defined footers and data checksums.

8. The method of claim 1, further comprising the steps of:

writing second slice content data to the volatile memory;

determining that the second slice content data was incompletely written to the volatile memory; and

marking an area of the volatile memory associated with the second slice content data as invalid.

9. The method of claim 1, wherein the missing metadata is identified using by starting a scan of metadata at a safe point marker.

10. The method of claim 9, further comprising updating the safe point marker after the step of writing the missing metadata corresponding to the slice content data and the bin in the non-volatile memory after flushing the slice content data to the bin.

11. A dispersed storage unit for use in a dispersed storage network, the dispersed storage unit comprising:

a communications interface;

a volatile memory;

a non-volatile memory; and

a computer processor;

wherein the non-volatile memory includes instructions for causing the computer processor to;

write slice content data to the volatile memory;

flush the slice content data to a bin in the non-volatile memory after a determination that the slice content data was completely written to the volatile memory;

write missing metadata corresponding to the slice content data after the slice content data has been flushed to the bin; and

write missing metadata corresponding to the bin after the slice content data has been flushed to the bin.

12. The dispersed storage unit of claim 11, wherein the non-volatile memory further comprises instructions for causing the computer processor to flush the slice content data to the bin in the non-volatile memory using a pattern of append only writes.

13. The dispersed storage unit of claim 11, wherein the metadata includes one or more of a pointer to the bin, and slice name and size information.

14. The dispersed storage unit of claim 11, wherein the non-volatile memory further comprises instructions for causing the computer processor to write the missing metadata corresponding to the slice content data and instructions for causing the computer processor to write the missing metadata corresponding to the bin, sequentially.

15. The dispersed storage unit of claim 11, wherein the non-volatile memory further comprises instructions for causing the computer processor to write the missing metadata corresponding to the slice content data and instructions for causing the computer processor to write the missing metadata corresponding to the bin, in parallel.

16. The dispersed storage unit of claim 11, wherein the non-volatile memory further comprises instructions for causing the computer processor to write the slice content data to the volatile memory using an atomic write structure.

17. The dispersed storage unit of claim 16, wherein the atomic write structure includes one or more of defined headers, length indicators, defined footers and data checksums.

18. The dispersed storage unit of claim 11, wherein the non-volatile memory further comprises instructions for causing the computer processor to:

write second slice content data to the volatile memory;

determine that the second slice content data was incompletely written to the volatile memory; and

mark an area of the volatile memory associated with the second slice data as invalid.

19. A dispersed storage network comprising:

a plurality of dispersed storage units;

a dispersed storage processing unit wherein a first dispersed storage unit of the plurality of dispersed storage units includes;

a volatile memory;

a non-volatile memory; and

a computer processor;

wherein the non-volatile memory includes instructions for causing the computer processor to;

write slice content data to the volatile memory;

flush the slice content data to a bin in the non-volatile memory after a determination that the slice content data was completely written to the volatile memory;

write missing metadata corresponding to the slice content data after the slice content data has been flushed to the bin; and

write missing metadata corresponding to the bin after the slice content data has been flushed to the bin.

20. The dispersed storage network of claim 19, wherein the non-volatile memory further comprises instructions for causing the computer processor to flush the slice content data to the bin in the non-volatile memory using a pattern of append only writes.