Virtual memory mapping in a dispersed storage network

• US 9,208,025 B2
• Filed: 06/13/2014
• Issued: 12/08/2015
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Dhuse, Greg, Baptist, Andrew
• Primary Examiner(s): Sadler, Nathan
• Assistant Examiner(s): Namazi, Mehdi
• Application Number:
  US14/304,054
• Publication Number:
  US 20140298137A1
• Time in Patent Office: 543 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method for evolving dispersed storage network (DSN) memory in a DSN begins by a processing module generating tracking information by tracking evolutionary change of storage units of the DSN memory, performance and reliability requirements of the DSN memory, and memory utilization of the DSN memory. The method continues with the processing module, for a given state of evolution of the DSN memory, interpreting the tracking information to produce given state DSN memory data and establishing virtual DSN address boundaries for a set of the storage units for storing data objects in a virtual memory vault of the DSN, where the virtual memory vault is mapped to the set of the storage units, where the data objects are dispersed storage error encoded into pluralities of sets of encoded data slices that are stored in the set of the storage units in accordance with the virtual DSN address boundaries.

Claims

1. A method for evolving dispersed storage network (DSN) memory in a DSN, the method is executable by a processing module and comprises:

as the DSN memory evolves, generating tracking information by tracking;

evolutionary change of storage units of the DSN memory;

performance and reliability requirements of the DSN memory; and

memory utilization of the DSN memory; and

for a given state of evolution of the DSN memory;

interpreting the tracking information for the given state of evolution to produce given state DSN memory data; and

establishing virtual DSN address boundaries for a set of the storage units for storing data objects in a virtual memory vault of the DSN based on the given state DSN memory data,

wherein the virtual memory vault is mapped to the set of the storage units, wherein the data objects are dispersed storage error encoded into pluralities of sets of encoded data slices that are stored in the set of the storage units in accordance with the virtual DSN address boundaries.

2. The method of claim 1, wherein the evolutionary changes of the storage units comprises one or more of:

addition of a new storage unit to the DSN memory;

deletion of one of the storage units; and

change in physical memory of one or more storage units.

3. The method of claim 1, wherein the memory utilization comprises one or more of:

creation of a new virtual memory vault;

modification of the virtual memory vault;

deletion of an existing virtual memory vault;

data retrieval frequencies for the virtual memory vault;

data storage rates for the virtual memory vault;

per storage unit available memory; and

per data object data size.

4. The method of claim 1 further comprises:

detecting a change in the given state of evolution of the DSN memory to a new given state when a change occurs to the virtual memory vault; and

for the new given state;

interpreting the tracking information for the new given state of evolution to produce new given state DSN memory data; and

establishing new virtual DSN address boundaries for the set of the storage units based on the new given state DSN memory data.

5. The method of claim 1 further comprises:

detecting a change in the given state of evolution of the DSN memory to a new given state when a change occurs to one of the storage units or when a new storage unit is added to the DSN memory; and

for the new given state;

interpreting the tracking information for the new given state of evolution to produce new given state DSN memory data;

updating the set of the storage units to produce an updated set of the storage units; and

establishing new virtual DSN address boundaries for the updated set of the storage units based on the new given state DSN memory data.

6. The method of claim 1, wherein the establishing the virtual DSN address boundaries comprises:

establishing the virtual DSN address boundaries in accordance with generations of the virtual memory vault, wherein, for a first generation of the virtual memory vault, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the first generation of the virtual memory vault and wherein, for a second generation of the virtual memory vault, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the second generation of the virtual memory vault.

7. The method of claim 1, wherein the establishing the virtual DSN address boundaries comprises:

establishing the virtual DSN address boundaries in accordance with boundaries of the data objects, wherein, for a first set of the data objects, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data objects and wherein, for a second set of the data objects, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data objects.

8. The method of claim 1, wherein the establishing the virtual DSN address boundaries comprises:

establishing the virtual DSN address boundaries in accordance with boundaries of data segments of the data objects, wherein, for a first set of the data segments, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data segments and wherein, for a second set of the data segments, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data segments.

9. A non-transitory computer readable storage medium having accessible therefrom a set of instructions interpretable by a processing module, the set of instructions being configured to cause the processing module to carry out operations for:

as a dispersed storage network (DSN) memory evolves, generating tracking information by tracking;

evolutionary change of storage units of the DSN memory;

performance and reliability requirements of the DSN memory; and

memory utilization of the DSN memory; and

for a given state of evolution of the DSN memory;

interpreting the tracking information for the given state of evolution to produce given state DSN memory data; and

establishing virtual DSN address boundaries for a set of the storage units for storing data objects in a virtual memory vault of the DSN based on the given state DSN memory data,

wherein the virtual memory vault is mapped to the set of the storage units, wherein the data objects are dispersed storage error encoded into pluralities of sets of encoded data slices that are stored in the set of the storage units in accordance with the virtual DSN address boundaries.

10. The storage medium of claim 9, wherein the evolutionary change of the storage units includes one or more of:

addition of a new storage unit to the DSN memory;

deletion of one of the storage units; and

change in physical memory of one or more storage units.

11. The storage medium of claim 9, wherein the memory utilization includes one or more of:

creation of a new virtual memory vault;

modification of the virtual memory vault;

deletion of an existing virtual memory vault;

data retrieval frequencies for the virtual memory vault;

data storage rates for the virtual memory vault;

per storage unit available memory; and

per data object data size.

12. The storage medium of claim 9, wherein the set of instructions further being configured to cause the processing module to carry out operations for:

detecting a change in the given state of evolution of the DSN memory to a new given state when a change occurs to the virtual memory vault; and

for the new given state;

interpreting the tracking information for the new given state of evolution to produce new given state DSN memory data; and

establishing new virtual DSN address boundaries for the set of the storage units based on the new given state DSN memory data.

13. The storage medium of claim 9, wherein the set of instructions further being configured to cause the processing module to carry out operations for:

detecting a change in the given state of evolution of the DSN memory to a new given state when a change occurs to one of the storage units or when a new storage unit is added to the DSN memory; and

for the new given state;

interpreting the tracking information for the new given state of evolution to produce new given state DSN memory data;

updating the set of the storage units to produce an updated set of the storage units; and

establishing new virtual DSN address boundaries for the updated set of the storage units based on the new given state DSN memory data.

14. The storage medium of claim 9, wherein the set of instructions further causes the processing module to carry out operations for the establishing the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with generations of the virtual memory vault, wherein, for a first generation of the virtual memory vault, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the first generation of the virtual memory vault and wherein, for a second generation of the virtual memory vault, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the second generation of the virtual memory vault.

15. The storage medium of claim 9, wherein the set of instructions further causes the processing module to carry out operations for the establishing the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with boundaries of the data objects, wherein, for a first set of the data objects, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data objects and wherein, for a second set of the data objects, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data objects.

16. The storage medium of claim 9, wherein the set of instructions further causes the processing module to carry out operations for the establishing the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with boundaries of data segments of the data objects, wherein, for a first set of the data segments, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data segments and wherein, for a second set of the data segments, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data segments.

17. A computing device comprises:

an interface;

memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

as a dispersed storage network (DSN) memory evolves, generate tracking information by tracking;

evolutionary change of storage units of the DSN memory;

performance and reliability requirements of the DSN memory; and

memory utilization of the DSN memory; and

for a given state of evolution of the DSN memory;

interpret the tracking information for the given state of evolution to produce given state DSN memory data; and

establish virtual DSN address boundaries for a set of the storage units for storing data objects in a virtual memory vault of the DSN based on the given state DSN memory data, wherein the virtual memory vault is mapped to the set of the storage units, wherein the data objects are dispersed storage error encoded into pluralities of sets of encoded data slices that are stored in the set of the storage units in accordance with the virtual DSN address boundaries.

18. The computing device of claim 17, wherein the evolutionary change of the storage units includes one or more of:

addition of a new storage unit to the DSN memory;

deletion of one of the storage units; and

change in physical memory of one or more storage units.

19. The computing device of claim 17, wherein the memory utilization includes one or more of:

creation of a new virtual memory vault;

modification of the virtual memory vault;

deletion of an existing virtual memory vault;

data retrieval frequencies for the virtual memory vault;

data storage rates for the virtual memory vault;

per storage unit available memory; and

per data object data size.

20. The computing device of claim 17, wherein the processing module further functions to:

detect a change in the given state of evolution of the DSN memory to a new given state when a change occurs to the virtual memory vault; and

for the new given state;

interpret the tracking information for the new given state of evolution to produce new given state DSN memory data; and

establish new virtual DSN address boundaries for the set of the storage units based on the new given state DSN memory data.

21. The computing device of claim 17, wherein the processing module further functions to:

detect a change in the given state of evolution of the DSN memory to a new given state when a change occurs to one of the storage units or when a new storage unit is added to the DSN memory; and

for the new given state;

interpret the tracking information for the new given state of evolution to produce new given state DSN memory data;

update the set of the storage units to produce an updated set of the storage units; and

establish new virtual DSN address boundaries for the updated set of the storage units based on the new given state DSN memory data.

22. The computing device of claim 17, wherein the processing module further functions to establish the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with generations of the virtual memory vault, wherein, for a first generation of the virtual memory vault, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the first generation of the virtual memory vault and wherein, for a second generation of the virtual memory vault, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices generated during the second generation of the virtual memory vault.

23. The computing device of claim 17, wherein the processing module further functions to establish the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with boundaries of the data objects, wherein, for a first set of the data objects, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data objects and wherein, for a second set of the data objects, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data objects.

24. The computing device of claim 17, wherein the processing module further functions to establish the virtual DSN address boundaries by:

establishing the virtual DSN address boundaries in accordance with boundaries of data segments of the data objects, wherein, for a first set of the data segments, a first storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the first set of the data segments and wherein, for a second set of the data segments, a second storage unit of the set of the storage units stores an encoded data slice for each set of encoded data slices of the plurality of sets of encoded data slices corresponding to the second set of the data segments.