Optimize data protection layouts based on distributed flash wear leveling

• US 9,672,905 B1
• Filed: 10/12/2016
• Issued: 06/06/2017
• Est. Priority Date: 07/22/2016
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Gold, Brian, Hayes, John, Lee, Robert
• Primary Examiner(s): Le, Vu
• Application Number:
  US15/292,004
• Time in Patent Office: 237 Days
• Field of Search: 36523003, 365189011, 36518915, 36518916
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/7211 Wear leveling
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0616 in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G11C 11/5628 Programming or writing circuits; Data input circuits
• CPC Class Codes: G11C 16/3495 Circuits or methods to detect or delay wearout of nonvolatile EPROM or EEPROM memory devices, e.g. by counting numbers of erase or reprogram cycles, by using multiple memory areas serially or cyclically

Abstract

A method for storing data in a storage system having solid-state memory is provided. The method includes determining portions of the solid-state memory that have a faster access rate and portions of the solid-state memory that have a slower access rate, relative to each other or to a threshold. The method includes writing data bits of erasure coded data to the portions of the solid-state memory having the faster access rate, and writing one or more parity bits of the erasure coded data to the portions of the solid-state memory having the slower access rate. A storage system is also provided.

Claims

1. A storage system, comprising:

a plurality of storage nodes having solid-state memory, each storage node of the plurality of storage nodes configurable to write data across the plurality of storage nodes using erasure coding, each portion of data having a plurality of data bits and one or more parity bits; and

each storage node of the plurality of storage nodes configurable to identify portions of the solid-state memory having a first access rate and portions of the solid state memory having a second access rate, the first access rate faster than the second access rate; and

each storage node of the plurality of storage nodes configurable to write the data bits to the portions of the solid-state memory having the first access rate and write the one or more parity bits to the portions of the solid-state memory having the second access rate.

2. The storage system of claim 1, wherein each storage node of the plurality of storage nodes is configurable to determine amounts or rates of error associated with differing portions of the solid-state memory, wherein the portions of the solid-state memory having the first access rate include portions of the solid-state memory having lesser amounts or rates of error and the portions of memory having the second access rate include portions of the solid-state memory having greater amounts or rates of error.

3. The storage system of claim 1, wherein the solid state memory includes triple level cell (TLC) flash memory, and wherein each storage node of the plurality of storage nodes is configurable to identify access rates associated with a first portion of the TLC flash memory and a second portion of the TLC flash memory, the first portion of the TLC flash memory having a faster access rate than the second portion of the TLC flash memory, wherein the portions of the solid-state memory having the first access rate include the first portion of the TLC flash memory and the portions of the solid-state memory having the second access rate include the second portion of the TLC flash memory.

4. The storage system of claim 1, wherein each storage node of the plurality of storage nodes is configurable to determine which portions of the solid-state memory have a type of flash memory having faster access and which portions of the solid-state memory have a type of flash memory having slower access, with the portions of the solid-state memory having the first access rate including the portions of the solid-state memory having the type of flash memory having faster access, and the portions of the solid-state memory having the second access rate including the portions of the solid-state memory having the type of flash memory having slower access.

5. The storage system of claim 1, wherein each storage node of the plurality of storage nodes is configurable to identify sections of the solid-state memory having fewer program or erase cycles, wherein the portions of the solid-state memory having the first access rate include the sections of the solid-state memory having fewer program or erase cycles, and the portions of the solid-state memory having the second access rate include remaining sections of the solid-state memory.

6. The storage system of claim 1, further comprising:

each of the plurality of storage nodes configurable to further subdivide each of the portions of the solid-state memory according to access rate and direct writing of the data bits and the parity bits to corresponding subdivisions based on frequency of access.

7. The storage system of claim 1, wherein:

the solid-state memory includes triple level cell flash memory having slower access pages and faster access pages;

the triple level cell flash memory includes slower access blocks and faster access blocks; and

each of the plurality of storage nodes is configurable to;

write the data bits of a first portion of data to the faster access pages of the faster access blocks, the first portion of data being more recently accessed or more frequently accessed than a second portion of data;

write the one or more parity bits of the first portion of data to the faster access pages of the slower access blocks;

write the data bits of the second portion of data to the slower access pages of the faster access blocks; and

write the one or more parity bits of the second portion of data to the slower access pages of the slower access blocks.

8. A solid-state memory storage system, comprising:

a plurality of storage nodes, at least a portion of the plurality of storage nodes having triple level cell flash memory for storage, the triple level cell flash memory having first pages with a first access rate and second pages with a second access rate, the first access rate faster than the second access rate;

each of the plurality of storage nodes configurable to associate the first access rate to first data and associate the second access rate to second data; and

each of the plurality of storage nodes configurable to write the first data to the first pages having the first access rate and write the second data to the second pages having the second access rate.

9. The storage system of claim 8, wherein each of the plurality of storage nodes is configurable to associate the first access rate to the first data that is more frequently accessed or more recently accessed data relative to the second data.

10. The storage system of claim 8, wherein:

data to be written across the plurality of storage nodes is erasure coded and includes data bits and one or more parity bits;

the first data is the data bits; and

the second data is the one or more parity bits.

11. The storage system of claim 8, wherein each of the plurality of storage nodes is configurable to associate the first access rate to the first data that is more frequently accessed or more recently accessed data relative to the second data;

and

each of the plurality of storage nodes is configurable to;

write data bits of the first data to a first subset of the first pages having fewer program or erase cycles than a second subset of the first pages;

write one or more parity bits of the first data to the second subset of the first pages;

write data bits of the second data to a first subset of the second pages having fewer program or erase cycles than a second subset of the second pages; and

write one or more parity bits of the second data to the second subset of the second pages.

12. The storage system of claim 8, wherein each of the plurality of storage nodes is configurable to associate the first access rate to the first data that is more frequently accessed or more recently accessed data relative to the second data;

and

each of the plurality of storage nodes is configurable to;

write data bits of the first data to a first subset of the first pages having fewer program or erase cycles than a second subset of the first pages;

write one or more parity bits of the first data to a first subset of the second pages having fewer program or erase cycles than a second subset of the second pages;

write data bits of the second data to the second subset of the first pages; and

write one or more parity bits of the second data to the second subset of the second pages.

13. The storage system of claim 8, wherein each of the plurality of storage nodes is configurable to associate the first access rate to the first data that is more frequently accessed or more recently accessed data relative to the second data;

and

each of the plurality of storage nodes is configurable to;

write data bits of the first data to a first subset of the first pages on a first subset of the plurality of storage nodes;

write one or more parity bits of the first data to a second subset of the first pages on a second subset of the plurality of storage nodes;

write data bits of the second data to a first subset of the second pages on the first subset of the plurality of storage nodes; and

write one or more parity bits of the second data to a second subset of the second pages on the second subset of the plurality of storage nodes.

14. A method for storing data in a storage system having solid-state memory, comprising:

determining portions of the solid-state memory that have a faster access rate and portions of the solid-state memory that have a slower access rate, relative to each other or to a threshold;

writing data bits of erasure coded data to the portions of the solid-state memory having the faster access rate; and

writing one or more parity bits of the erasure coded data to the portions of the solid-state memory having the slower access rate.

15. The method of claim 14, wherein the determining comprises:

determining first portions of the solid-state memory that have lesser amounts or rates of error and second portions of the solid-state memory that have greater amounts or rates of error, relative to each other or to an error threshold.

16. The method of claim 14, wherein the determining comprises:

identifying first portions of the solid-state memory having the faster access rate as a first characteristic of triple level cell flash memory and second, differing portions of the solid-state memory having the slower access rate as a second characteristic of triple level cell flash memory.

17. The method of claim 14, wherein the determining comprises:

determining first portions of the solid-state memory that have a faster access type or design of flash memory and second portions of the solid-state memory that have a slower access type or design of flash memory.

18. The method of claim 14, wherein the determining comprises:

determining first portions of the solid-state memory that have fewer program or erase cycles and second portions of the solid-state memory that have more program or erase cycles.

19. The method of claim 14, further comprising:

determining faster access sub portions and slower access sub portions of the portions of the solid-state memory having the faster access rate, and faster access sub portions and slower access sub portions of the portions of the solid-state memory having the slower access rate;

writing the data bits of more frequently accessed or more recently accessed data to the faster access sub portions of the portions of the solid-state memory having the faster access rate; and

writing the one or more parity bits of the more frequently accessed or more recently accessed data to the faster access sub portions of the portions of the solid-state memory having the slower access rate.

20. The method of claim 14, further comprising:

writing the data bits of more frequently accessed or more recently accessed data to faster access pages of triple level cell flash memory in the portions of the solid-state memory having the faster access rate; and

writing the one or more parity bits of the more frequently accessed or more recently accessed data to faster access pages of the triple level cell flash memory in the portions of the solid-state memory having the slower access rate.