MASTERLESS RAID FOR BYTE-ADDRESSABLE NON-VOLATILE MEMORY

• US 20200133777A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Lesartre, Gregg B., Herrell, Russ W., Brueggen, Chris Michael
• Granted Patent: US 10,922,178 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 12/0238 Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/7207 management of metadata or control data
• CPC Class Codes: G06F 2212/7208 Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

A system includes byte-addressable non-volatile memory (NVM) modules. The system includes media controllers communicatively connected to one another over a memory semantic fabric. Each media controller is responsible for a corresponding NVM module to which the media controller is attached. The media controllers cooperatively provide redundant array of independent disks (RAID) functionality at a granularity at which the NVM modules are byte-addressable without employing a master RAID controller.

Claims

1. A system comprising:

a plurality of byte-addressable non-volatile memory (NVM) modules; and

a plurality of media controllers communicatively connected to one another over a memory semantic fabric, each media controller responsible for a corresponding NVM module to which the media controller is attached,

wherein the media controllers cooperatively provide redundant array of independent disks (RAID) functionality at a granularity at which the NVM modules are byte-addressable without employing a master RAID controller.

2. The system of claim 1, wherein the granularity at which the media controllers provide the RAID functionality is a cache line granularity at which the processing nodes cache data persistently stored within the NVM modules.

3. The system of claim 1, wherein the granularity at which media controllers provide the RAID functionality is a granularity of individual load and store operations issued by processing nodes communicatively connected to the memory semantic fabric.

4. The system of claim 1, wherein the processing nodes use the NVM modules as persistent main memory.

5. The system of claim 1, wherein a memory address space comprises a plurality of memory addresses that are each assigned to a corresponding media controller,

wherein each memory address assigned to each media controller is part of a RAID group including a memory address assigned to each other media controller module,

and wherein the memory addresses assigned to some of the media controller of the RAID group store data accessible by processing nodes issuing load and store operations over the memory semantic fabric, and the memory address assigned to another media controller of the RAID group not storing the data accessible by the processing nodes stores parity for the data.

6. The system of claim 5, wherein each media controller maintains a RAID group mapping table storing, for each memory address assigned to the media controller:

whether the memory address stores the data accessible by the processing node issuing load and store operations or whether the memory address stores the parity for the data;

identification of the media controllers for other of the memory addresses within the RAID group, and whether the other of the memory addresses store the data accessible by the processing nodes issuing load and store operations or the parity for the data; and

a status of the RAID group as normal or degraded.

7. The system of claim 5, wherein the media controller responsible for a selected memory address of the RAID group directly receives the load operations for the data stored at the selected memory address from over the memory semantic fabric,

and wherein the media controller responsible for the selected memory address retrieves the data from the NVM module to which the media controller is attached and responsively returns the data, without involving other of the media controllers, when the media controller does not encounter a read error for the selected memory address.

8. The system of claim 7, wherein, when the media controller encounters the read error for the selected memory address, the media controller responsible for the selected memory address requests that the media controller responsible for the memory address of the RAID group storing the parity for the data reconstruct the data,

wherein the media controller responsible for the memory address storing the parity reconstructs the data at the selected memory address from the parity and the data stored at other of the memory addresses of the RAID group as retrieved from the media controllers responsible for the other of the memory addresses.

9. The system of claim 5, wherein the media controller responsible for a selected memory address of the RAID group directly receives the store operations to write new data at the selected memory address from over the memory semantic fabric,

wherein the media controller responsible for the selected memory address, when not encountering an error, retrieves existing data stored at the selected memory address from the NVM module to which the media controller is attached, writes the new data to the NVM module, and sends an exclusive-or (XOR) operation result of the existing data and the new data to the media controller responsible for the memory address of the RAID group storing the parity,

wherein the media controller responsible for the memory address storing the parity responsively updates the parity using the XOR operation result, without involving other of the media controllers.

10. The system of claim 9, wherein, the media controller responsible for the selected memory address, when encountering the error, sends the new data to the media controller responsible for the memory address of the RAID group storing the parity,

wherein the media controller responsible for the memory address storing the parity responsively updates the parity from the new data and from the data stored at other of the memory addresses of the RAID group as retrieved from the media controllers responsible for the other of the memory addresses.

11. A media controller comprising:

a non-volatile memory (NVM) controller to attach to a selected byte-addressable NVM module of a plurality of byte-addressable NVMs;

a link layer device to communicatively connect to a memory semantic fabric to which other media controllers physically attached to other byte-addressable NVMs of the plurality; and

a fabric bridge to provide redundant array of independent disks (RAID) functionality,

wherein for a selected memory address assigned to the media controller and for which the RAID functionality is provided by the fabric bridge, the media controller directly receives store operations to write new data stored at the selected memory address over the memory semantic fabric,

wherein, when the fabric bridge does not encounter an error for the selected memory address, the fabric bridge retrieves existing data stored at the selected memory address from the NVM module, writes the new data to the NVM module, and sends an exclusive-or (XOR) operation result of the existing data and the new data to the other media controller for the other NVM module storing parity for the existing data,

wherein, when the fabric bridge encounters the error, the fabric bridge sends the new data to the other media controller for the other NVM module storing the parity for the existing data.

12. The media controller of claim 11, wherein for a second selected memory address assigned to the memory controller and for which the RAID functionality is provided by the fabric bridge, the media controller directly receives load operations for data stored at the second selected memory address from over the memory semantic fabric,

wherein, when the fabric bridge does not encounter a read error for the second selected memory address, the fabric bridge retrieves the data from the NVM module and responsively returns the data over the memory semantic fabric, without involving the other media controllers,

and wherein, when the fabric bridge encounters the read error, the fabric bridge requests that the other media controller for the other NVM module storing parity for the data reconstruct the data.

13. The media controller of claim 11, wherein for a second selected memory address assigned to the media controller and for which the RAID functionality is provided by the fabric bridge, the media controller receives a request to reconstruct data for a different memory address from the other media controller responsible for storing the data at the different memory address and that encountered a read error for the different memory address,

wherein, the fabric bridge retrieves at the second selected memory address of the second selected byte-addressable NVM module parity for the data, retrieves different data stored at the other NVM modules not storing the parity for the data, and reconstructs the data from the parity and the different data.

14. The media controller of claim 11, wherein for a second selected memory address assigned to the media controller and for which the RAID functionality is provided by the fabric bridge together, the media controller receives new data for a different memory address from the other media controller responsible for storing the new data at the different memory address and that encountered an error for the different memory address,

wherein, the fabric bridge responsively updates parity at the second selected memory address of the selected byte-addressable NVM module from the new data and from existing data, including reconstructing the existing data from the parity and different data stored at the other NVM modules not storing the parity.

15. A media controller comprising:

a non-volatile memory (NVM) controller to attach to a selected byte-addressable NVM module of a plurality of byte-addressable NVMs;

a link layer device to communicatively connect to a memory semantic fabric to which other media controllers physically attached to other byte-addressable NVMs of the plurality; and

a fabric bridge to provide redundant array of independent disks (RAID) functionality,

wherein for a selected memory address assigned to the memory controller and for which the RAID functionality is provided by the fabric bridge, the media controller directly receives load operations for data stored at the selected memory address from over the memory semantic fabric,

wherein, when the fabric bridge does not encounter a read error for the selected memory address, the fabric bridge retrieves the data from the NVM module and responsively returns the data over the memory semantic fabric, without involving the other media controllers,

and wherein, when the fabric bridge encounters the read error, the fabric bridge requests that the other media controller for the other NVM module storing parity for the data reconstruct the data.

16. A method comprising:

receiving from over a memory semantic fabric, by a media controller for a non-volatile memory (NVM) module of a plurality of byte-addressable NVMs including other NVMs, a load operation for data stored at a selected memory address, the media controller cooperatively providing, with other media controllers for the other NVMs, redundant array of independent disks (RAID) functionality at a granularity at which the NVMs are byte-addressable;

when the media controller does not encounter a read error for the selected memory address;

retrieving, by the media controller, the data from the NVM module; and

responsively returning, by the media controller, the data over the memory semantic fabric, without involving the other media controllers;

receiving from over the memory semantic fabric, a store operation to write new data at the selected memory address; and

when the media controller does not encounter an error for the selected memory address;

retrieving, by the media controller, the data from the NVM module;

writing, by the media controller, the new data to the NVM module;

performing, by the media controller, an exclusive-or (XOR) operation on the data and the new data; and

sending, by the media controller, a result of the XOR operation to the other media controller for the other NVM module storing parity for the data.

17. The method of claim 16, further comprising:

when the media controller encounters the read error;

requesting, by the media controller, the other media controller for the other NVM module storing the parity for the data reconstruct the data;

receiving, by the media controller, the data as reconstructed by the other media controller for the other NVM module storing the parity for the data; and

responsively returning, by the media controller, the received reconstructed data over the memory semantic fabric.

18. The method of claim 16, further comprising:

when the media controller encounters the error;

sending, by the media controller, the new data to the other media controller for the other NVM module storing the parity for the data.

19. The method of claim 16, wherein the selected memory address is a first selected memory address, and the method further comprises:

receiving, by the media controller, a request to reconstruct data for a different memory address from the other media controller responsible for storing data at the different memory address and that encountered the read error for the different memory address;

retrieving, by the media controller from the NVM module, parity for the data at a second selected memory address;

retrieving, by the media controller from the other media controllers for the other NVM modules not storing the parity, different data;

reconstructing, by the media controller, the data at the different memory address from the parity for the data and from the different data; and

returning, by the media controller to the other media controller responsible for storing the data at the different memory address, the reconstructed data.

20. The method of claim 16, wherein the selected memory address is a first selected memory address, and the method further comprises:

receiving, by the media controller, new data for a different memory address from the other media controller responsible for storing data at the different memory address and that encountered the error for the different memory address;

retrieving, by the media controller from the NVM module, parity for the data at a second selected memory address;

retrieving, by the media controller from the other media controllers for the other NVM modules not storing the parity, different data; and

updating, by the media controller, the parity at the second selected memory address from the different data and from the new data.