Block consolidation in a direct-mapped flash storage system

• US 10,452,290 B2
• Filed: 07/31/2017
• Issued: 10/22/2019
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Kirkpatrick, Peter E., Karr, Ronald
• Primary Examiner(s): Lane, John A
• Application Number:
  US15/665,134
• Publication Number:
  US 20180173442A1
• Time in Patent Office: 813 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

In one implementation, a method includes maintaining a list of available allocation units across a plurality of flash devices of a flash storage system, wherein the flash devices map erase blocks as directly addressable storage, and wherein erase blocks are categorized by the flash storage system as available for use, in use, or unusable, and wherein at least a portion of an erase block can be assigned as an allocation unit. The method further includes receiving data from a plurality of sources, wherein the data is associated with processing a dataset, the dataset comprising multiple file systems and associated metadata. The method further includes determining a plurality of subsets of the data such that each subset is capable of being written in parallel with the remaining subsets, mapping each subset of the plurality of subsets to an available allocation unit, and writing the plurality of subsets in parallel.

Claims

1. A storage system comprising:

a plurality of embedded storage devices, wherein each embedded storage device of the plurality of embedded storage devices comprises a direct-mapped solid state drive (SSD) storage portion, wherein the direct-mapped SSD storage has erase blocks mapped as directly addressable storage of the SSD; and

a plurality of storage system controllers, operatively coupled to the plurality of embedded storage devices via a bus, the plurality of storage system controllers to;

maintain a list of a plurality of available allocation units across the respective direct-mapped SSD storage portions of the plurality of embedded storage devices;

receive data from a plurality of sources, wherein the data is associated with processing a dataset, the dataset comprising multiple volumes or file systems and associated metadata;

determine a plurality of shards of the data such that each shard of the plurality of shards is capable of being written in parallel with the remaining shards;

map the plurality of shards to a subset of the plurality of available allocation units, respectively; and

write the plurality of shards to the subset of the plurality of available allocation units in parallel.

2. The storage system of claim 1, wherein the plurality, of storage system controllers is further to:

determine an erasure code corresponding to the data; and

write the erasure code to the subset of the plurality of available allocation units.

3. The storage system of claim 2, wherein the erasure code comprises a first portion and a second portion, and wherein to write the erasure code the plurality of storage system controllers is further to:

write the first portion to a first allocation unit of the subset of the plurality of available allocation units; and

write the second portion to a second allocation unit of the subset of the plurality of available allocation units in parallel with the first portion.

4. The storage system of claim 1, wherein the erase blocks are mapped directly to allocation units on a one-to-one basis, a multiple-to-one basis, or a one-to-multiple basis, and wherein the erase blocks are categorized as available for use, in use, or unusable.

5. The storage system of claim 1, further comprising a non-volatile random-access memory (NVRAM) portion to journal the data, wherein the direct-mapped SSD portion and the NVRAM portion are separately addressable, and wherein the NVRAM portion is smaller than the direct-mapped SSD portion.

6. The storage system of claim 5, wherein the NVRAM portion comprises:

a random access memory (RAM) device;

a stored energy device; and

a processing device.

7. The storage system of claim 1, wherein the data is compressed data.

8. A storage system comprising:

a plurality of embedded storage devices, wherein each embedded storage device of the plurality of embedded storage devices comprises a direct-mapped solid state drive (SSD) storage portion, wherein the direct-mapped SSD storage has erase blocks mapped as directly addressable storage of the SSD; and

a storage system controller, operatively coupled to the plurality embedded storage devices via a bus, the storage system controller to;

maintain a list of a plurality of available allocation nits across the respective direct-mapped SSD storage portions of the plurality of embedded storage devices;

receive data to be stored to the plurality of available allocation units;

determine a plurality of groups of the data such that each group of the plurality of groups is capable of being written in parallel with the remaining groups;

map the plurality of groups to a subset of the plurality of available allocation units, respectively; and

write the plurality of groups to the subset of the plurality of available allocation units in parallel.

9. The storage system of claim 8, wherein the data is received from a plurality of sources, and wherein the data is associated with processing a dataset, the dataset comprising multiple file systems and associated metadata.

10. The storage system of claim 8, wherein the storage system controller is further to:

determine an erasure code corresponding to the data; and

write the erasure code to the subset of the plurality of available allocation units.

11. The storage system of claim 10, wherein the erasure code comprises a first portion and a second portion, and wherein to write the erasure code the storage system controller is further to:

write the first portion to a first allocation unit of the subset of the plurality of available allocation units; and

write the second portion to a second allocation unit of the subset of the plurality of available allocation units in parallel with the first portion.

12. The storage system of claim 8, wherein the erase blocks are mapped directly to allocation units on a one-to-one basis, a multiple-to-one basis, or a one-to-multiple basis, and wherein the erase blocks are categorized as available for use, in use, or unusable.

13. The storage system of claim 8, further comprising a non-volatile random, access memory (NVRAM) portion to journal the data, wherein the direct-mapped SSD portion and the NVRAM portion are separately addressable, and wherein the NVRAM portion is smaller than the direct-mapped SSD portion.

14. The storage system of claim 13, wherein the NVRAM portion comprises:

a random access memory (RAM) device;

a capacitor; and

a processing device.

15. The storage system of claim 8, wherein the data is compressed data.

16. A method comprising:

receive data to be stored to a plurality of available allocation units across a corresponding plurality of direct-mapped solid-state drive (SSD) portions of a plurality of storage devices, wherein the direct-mapped SSD storage has erase blocks mapped as directly addressable storage of the SSD;

determine, by a processing device of a storage controller associated with the plurality of storage devices, a plurality of groups of the data such that each group of the plurality of groups is capable of being written in parallel with the remaining groups;

map, by the processing device, the plurality of groups to a subset of the plurality of available allocation units, respectively; and

write the plurality of groups to the subset of the plurality of available allocation units in parallel.

17. The method of claim 16, wherein the data is received from a plurality of sources, and wherein the data is associated with processing a dataset, the dataset comprising multiple file systems and associated metadata.

18. The method of claim 16, further comprising:

determining an erasure code corresponding to the data; and

writing the erasure code to the subset of the plurality of available allocation units.

19. The method of claim 16, wherein the erase blocks are mapped directly to allocation units on a one-to-one basis, a multiple-to-one basis, or a one-to-multiple basis, and wherein the erase blocks are categorized as available for use, in use, or unusable.

20. The method of claim 16, further comprising journaling the data in a non-volatile random-access memory (NVRAM) portion of the plurality of storage devices, respectively, and

wherein the direct-mapped SSD portion and the NVRAM portion are separately addressable.