Dispersed storage network virtual address generations

• US 8,886,905 B2
• Filed: 08/24/2012
• Issued: 11/11/2014
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Baptist, Andrew, Dhuse, Greg
• Primary Examiner(s): Bragdon, Reginald
• Assistant Examiner(s): Namazi, Mehdi
• Application Number:
  US13/594,199
• Publication Number:
  US 20130006923A1
• Time in Patent Office: 809 Days
• Field of Search: 711/114, 711/170, 711/171, 711/172, 711/173, 718/105
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity

Abstract

A dispersed storage device within a dispersed storage network includes a processing module for determining whether to add a new generation for a vault, in which the vault identifies at least one user having data to be stored. When the new generation is to be added to the vault, the processing module further assigns a vault generation identifier to the new generation and reserves memory for the new generation.

Claims

1. A dispersed storage device for use within a dispersed storage network, comprising:

a processing module operable to;

determine whether to add a new generation for a vault, the vault identifying at least one user having data to be stored, the vault including a current generation; and

when the new generation is to be added to the vault;

assign a vault generation identifier to the new generation; and

reserve memory for the new generation;

wherein data stored in the current generation is different from data stored the new generation.

2. The dispersed storage device of claim 1, wherein the processing module is further operable to reserve memory for the new generation by:

assigning a virtual address range of a virtual memory associated with the dispersed storage network to the new generation; and

mapping the virtual address range to a physical memory for storage of the data therein.

3. The dispersed storage device of claim 2, wherein the physical memory includes a plurality of storage units and further comprising:

a table mapping the respective virtual address range of each generation of the vault to corresponding ones of the plurality of storage units.

4. The dispersed storage device of claim 3, wherein:

the data associated with the current generation is stored in different ones of the plurality of storage units than the data associated with the new generation.

5. The dispersed storage device of claim 3, wherein the processing module is further operable to:

detect a new storage unit of the plurality of storage units;

compare storage utilization within the physical memory of a current generation of the vault to a generation utilization threshold;

when the storage utilization compares unfavorably to the generation utilization threshold, add the new generation.

6. The dispersed storage device of claim 5, wherein the processing module is further operable to:

when the storage utilization compares favorably to the generation utilization threshold;

transfer at least a portion of the data from an existing storage unit of the plurality of storage units to the new storage unit; and

update the table.

7. The dispersed storage device of claim 3, further comprising an interface, and wherein the processing module is further operable to:

determine a vault generation of a data object;

slice a data segment of the data object into data slices, a number of the data slices corresponding to a number of pillars for storing the data object;

create a slice name for each of the data slices, the slice name including a virtual memory address within a corresponding virtual address range of the vault generation, the virtual memory address including a pillar identifier that identifies a pillar associated with the data slice, the virtual address range being assigned to the pillar; and

output, via the interface, a data slice of the data slices and the respective slice name to a corresponding one of the plurality of storage units for storage therein.

8. The dispersed storage device of claim 7, wherein the number of pillars is based on the vault generation and each of the pillars of the vault generation is assigned to a different one of the plurality of storage units.

9. The dispersed storage device of claim 7, wherein the processing module is further operable to:

determine storage utilization within the physical memory of each generation of the vault; and

select the vault generation for the data object based on the storage utilization of each generation.

10. The dispersed storage device of claim 3, wherein the processing module is further operable to randomly assign the virtual address range to the new generation.

11. The dispersed storage device of claim 3, wherein the processing module is further operable to utilize a sequential assignment algorithm to assign the virtual address range to the new generation in a sequential manner and to update the table to assign different ones of the plurality of storage units to each generation of the vault.

12. The dispersed storage device of claim 3, wherein the processing module is further operable to utilize an even distribution algorithm to assign the virtual address range to the new generation such that adjacent vault generations are automatically assigned to different ones of the plurality of storage units within the table.

13. The dispersed storage device of claim 2, wherein the processing module is further operable to determine whether to add the new generation by:

comparing storage utilization within the physical memory of a current generation of the vault to a generation utilization threshold; and

when the storage utilization compares unfavorably to the generation utilization threshold, adding the new generation.

14. A method for managing data storage within a dispersed storage network, the method comprising:

determining whether to add a new generation for a vault, the vault identifying at least one user having data to be stored, the vault including a current generation; and

when the new generation is to be added to the vault;

assigning a vault generation identifier to the new generation; and

reserving memory for the new generation;

wherein data stored in the current generation is different from data stored in the new generation.

15. The method of claim 14, wherein the reserving memory comprises:

assigning a virtual address range of a virtual memory associated with the dispersed storage network to the new generation; and

mapping the virtual address range to a physical memory for storage of the data therein.

16. The method of claim 15, wherein the physical memory includes a plurality of storage units and further comprising:

maintaining a table mapping the respective virtual address range of each generation of the vault to corresponding ones of the plurality of storage units.

17. The method of claim 16, further comprising:

storing the data associated with the current generation in different ones of the plurality of storage units than the data associated with the new generation.

18. The method of claim 16, wherein the determining whether to add the new generation further comprises:

detecting a new storage unit of the plurality of storage units;

comparing storage utilization within the physical memory of a current generation of the vault to a generation utilization threshold;

when the storage utilization compares unfavorably to the generation utilization threshold, adding the new generation.

19. The method of claim 16, further comprising:

determining a vault generation of a data object;

slicing a data segment of the data object into data slices, a number of the data slices corresponding to a number of pillars for storing the data object;

creating a slice name for each of the data slices, the slice name including a virtual memory address within a corresponding virtual address range of the vault generation, the virtual memory address including a pillar identifier that identifies a pillar associated with the data slice, the virtual address range being assigned to the pillar; and

outputting a data slice of the data slices and the respective slice name to a corresponding one of the plurality of storage units for storage therein.

20. The method of claim 19, wherein the number of pillars is based on the vault generation and further comprising:

assigning each of the pillars of the vault generation to a different one of the plurality of storage units.