Self-describing data format for DMA in a non-volatile solid-state storage

• US 9,747,229 B1
• Filed: 07/03/2014
• Issued: 08/29/2017
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Hayes, John, Gupta, Shantanu, Davis, John, Gold, Brian, Tan, Zhangxi
• Primary Examiner(s): Phan, Raymond
• Application Number:
  US14/323,894
• Time in Patent Office: 1,153 Days
• Field of Search: 710260-269, 710104-110, 710308, 710 22- 28, 711147-153
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0625 Power saving in storage systems
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

Claims

1. A method of applying a data format in a direct memory access transfer, comprising:

distributing user data throughout a plurality of storage nodes through erasure coding, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage;

reading a self-describing data portion that is one of a plurality of self-describing data portions from a first memory of the nonvolatile solid-state memory, each of the plurality of self-describing data portions comprising a self-contained combination of data, a destination section having information regarding a destination of the data, and a status section having information about transfer of the data from the first memory to a second memory of the nonvolatile solid-state memory;

extracting the destination from the self-describing data portion; and

writing the data, from the self-describing data portion, to the second memory of the nonvolatile solid-state memory according to the destination and the information in the status section of the self-describing data portion.

2. The method of claim 1, further comprising:

writing to the status section of the self-describing data portion in the first memory to confirm the writing data to the second memory.

3. The method of claim 1, further comprising:

writing an identifier of the data to a header of a non-volatile memory page, wherein the second memory includes non-volatile memory having the non-volatile memory page containing the destination.

4. The method of claim 1, further comprising:

extracting status information from a status section of the self-describing data portion, the extracting responsive to a loss of power, and wherein the writing data to the second memory is responsive to the loss of power and responsive to the status information indicating the data has not yet been written to the second memory.

5. The method of claim 1, further comprising:

extracting status information from the status section of the self-describing data portion; and

determining whether the data has been written to the second memory.

6. The method of claim 1, further comprising:

writing the self-describing data portion to the first memory, wherein the destination section specifies the destination.

7. The method of claim 1, further comprising:

writing an indication that the data in the self-describing data portion will not be used and need not be transferred to the second memory to the status section of the self-describing data portion in the first memory.

8. A method of moving data by direct memory access (DMA), comprising:

directing a DMA unit to move data from a first memory to a second memory;

extracting, using the DMA unit, destination information from a self-describing data portion in the first memory, wherein a plurality of self-describing data portions are within the first memory, each of the self-describing data portions comprising a self-contained combination of data, a destination section having information regarding a destination of the data, and a status section having information about transfer of the data from the first memory to a second memory of the nonvolatile solid-state memory;

extracting, using the DMA unit, the data from the self-describing data portion; and

writing, using the DMA unit, the data to a destination in the second memory according to the destination information and the information in the status section of the self-describing data portion.

9. The method of claim 8, further comprising:

writing, using the DMA unit, to the status section an indication of successful data transfer, responsive to writing the data to the destination in the second memory.

10. The method of claim 8, wherein the destination information includes one of an address, a destination pointer, or an offset.

11. The method of claim 8, further comprising:

writing an identifier to a header of a non-volatile memory page, wherein the identifier identifies the data as to an inode identifier (ID), a medium address, or a segment ID, and wherein the non-volatile memory page is in the second memory.

12. The method of claim 8, further comprising:

reading status information from the self-describing data portion in the first memory; and

determining from the status information that the data has not been written to the second memory, wherein the writing the data to the destination in the second memory is responsive to the determining.

13. The method of claim 8, further comprising:

reading status information from the self-describing data portion in the first memory, responsive to a power loss; and

determining from the status information that the data has been written to the second memory;

writing, using the DMA unit, further data from the first memory to the second memory, responsive to the power loss, without rewriting the data that has been written to the second memory according to the status information.

14. The method of claim 8, wherein the first memory includes non-volatile random-access memory (NVRAM) and the second memory includes non-volatile memory.

15. A storage cluster, comprising:

a plurality of storage nodes within a single chassis, each of the plurality of storage nodes having nonvolatile solid-state memory for storage of user data, the nonvolatile solid-state memory comprising;

a first memory, configured to hold a plurality of self-describing data portions, each comprising a self-contained combination of data, a destination section having information regarding a destination of the data, and a status section having information about transfer of the data from the first memory to a second memory of the nonvolatile solid-state memory;

a second memory; and

a direct memory access (DMA) unit, configured to perform actions including;

extracting a destination from one of the plurality of self-describing data portions in the first memory; and

writing data, from the one of the plurality of self-describing data portions, to the destination in the second memory.

16. The storage cluster of claim 15, further comprising:

the first memory includes non-volatile random access memory (NVRAM).

17. The storage cluster of claim 15, further comprising:

the second memory includes non-volatile memory; and

an energy reserve coupled to the first memory, the second memory and the DMA unit, the energy reserve configured to provide sufficient power for the DMA unit to transfer contents of the first memory to the second memory responsive to a power loss.

18. The storage cluster of claim 15, further comprising:

each of the plurality of data portions includes data, a destination for the data, and a status of the data.

19. The storage cluster of claim 15, wherein:

the actions further include writing to the status section of the one of the plurality of self-describing data portions to confirm transfer of the data from the first memory to the second memory, responsive to writing the data to the destination in the second memory.

20. The storage cluster of claim 15, further comprising:

the one of the plurality of self-describing data portions in the first memory includes a status of the data indicating one of whether the data of the one of the plurality of self-describing data portions has been written to the second memory or whether the data of the one of the plurality of self-describing data portions will be used.