Scatter-gather approach for parallel data transfer in a mass storage system

• US 9,971,524 B1
• Filed: 03/17/2014
• Issued: 05/15/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: BiTMICRO, LLC (BiTMICRO Networks, Inc.)
• Original Assignee: BiTMICRO Networks, Inc.
• Inventors: Bruce, Ricardo H., Santos, Avnher Villar, Verdan, Marlon Basa, Villapana, Elsbeth Lauren Tagayo
• Primary Examiner(s): Oberly, Eric
• Application Number:
  US14/217,249
• Time in Patent Office: 1,520 Days
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

An embodiment of the invention provides a method for optimizing flash device accesses, comprising: interleaving and striping, in tandem, for a transfer of data the other portions of the data.

Claims

1. A method for optimizing flash device accesses, comprising:

data striping and memory interleaving in a storage array in order to permit a data transfer, wherein the storage array comprises a first flash module, a second flash module and a third flash module;

wherein the data striping and memory interleaving comprise;

data striping, by a plurality of Direct Memory Access (DMA) controllers, a data into a first data stripe, a second data stripe, and a third data stripe, wherein the plurality of DMA controllers comprises a first DMA controller, a second DMA controller, and a third DMA controller,

and storing the first data stripe into a first data buffer in the first DMA controller, storing the second data stripe into a second data buffer in the second DMA controller, and storing the third data stripe into a third data buffer in the third DMA controller;

wherein the first flash module is coupled via a first high speed bus to the first Direct Memory Access (DMA) controller, wherein the second flash module is coupled via a second high speed bus to the second DMA controller, and wherein the third flash module is coupled via a third high speed bus to the third DMA controller, respectively;

wherein the first DMA controller, the second DMA controller, and third DMA controller are coupled to a local bus;

wherein the first flash module comprises a first flash buffer chip and a first flash bank, wherein the first flash bank comprises a first plurality of flash devices that are coupled via a first flash memory bus to the first flash buffer chip and wherein the first flash buffer chip is coupled via the first high speed bus to the first DMA controller;

wherein the second flash module comprises a second flash buffer chip and a second flash bank, wherein the second flash bank comprises a second plurality of flash devices that are coupled via a second flash memory bus to the second flash buffer chip and wherein the second flash buffer chip is coupled via the second high speed bus to the second DMA controller;

wherein the third flash module comprises a third flash buffer chip and a third flash bank, wherein the third flash bank comprises a third plurality of flash devices that are coupled via a third flash memory bus to the third flash buffer chip and wherein the third flash buffer chip is coupled via the third high speed bus to the third DMA controller;

wherein the data striping and memory interleaving comprise;

storing, in a first flash device in the first plurality of flash devices in the first flash bank in the first flash module, the first data stripe of the data;

storing, in a second flash device in the second plurality of flash devices in the second flash bank in the second flash module, the second data stripe of the data; and

storing, in a third flash device in the third plurality of flash devices in the third flash bank in the third flash module, the third data stripe of the data;

wherein the first DMA controller comprises a first engine and the first data buffer, the second DMA controller comprises a second engine and the second data buffer, the third DMA controller comprises a third engine and the third data buffer;

wherein the first engine transfers the first data stripe between the first flash device in the first flash bank and the first data buffer in the first Direct Memory Access (DMA) controller;

wherein the second engine transfers the second data stripe between the second flash device in the second flash bank and the second data buffer in the second DMA controller;

wherein the third engine transfers the third data stripe between the third flash device in the third flash bank and the third data buffer in the third DMA controller.

2. The method of claim 1 wherein the first flash device and a fourth flash device are in the first flash bank.

3. The method of claim 1 wherein the second flash bank further comprises a fifth flash device.

4. The method of claim 1 wherein the first flash device and a fourth flash device are in the first flash module.

5. The method of claim 1 wherein the second memory module further comprises a fifth flash device.

6. The method of claim 1 wherein the first flash buffer chip is coupled to the first flash bank and a fourth flash bank in the first flash module.

7. The method of claim 1 wherein the second flash buffer chip is coupled to the second flash bank and a fifth flash bank in the second flash module.

8. The method of claim 1 wherein the DMA controllers transfer the data stripes between a cache and the storage array.

9. The method of claim 1, wherein the first engine transfers the first data stripe between the first flash device and the first DMA controller and the second engine transfers the second data stripe between the second flash device and the second DMA controller in parallel.

10. The method of claim 1, wherein the first engine transfers the first data stripe between the first flash device and the first DMA controller and the second engine transfers the second data stripe between the second flash device and the second DMA controller in sequence.

11. An article of manufacture, comprising:

a non-transitory computer-readable medium having stored thereon instructions operable to permit an apparatus to perform a method comprising;

data striping and memory interleaving in a storage array in order to permit a data transfer, wherein the storage array comprises a first flash module, a second flash module and a third flash module;

wherein the data striping and memory interleaving comprise;

data striping, by a plurality of Direct Memory Access (DMA) controllers, a data into a first data stripe, a second data stripe, and a third data stripe, wherein the plurality of DMA controllers comprises a first DMA controller, a second DMA controller, and a third DMA controller,

and storing the first data stripe into a first data buffer in the first DMA controller, storing the second data stripe into a second data buffer in the second DMA controller, and storing the third data stripe into a third data buffer in the third DMA controller;

wherein the first flash module is coupled via a first high speed bus to the first Direct Memory Access (DMA) controller, wherein the second flash module is coupled via a second high speed bus to the second DMA controller, and wherein the third flash module is coupled via a third high speed bus to the third DMA controller, respectively;

wherein the first DMA controller, the second DMA controller, and third DMA controller are coupled to a local bus;

wherein the first flash module comprises a first flash buffer chip and a first flash bank, wherein the first flash bank comprises a first plurality of flash devices that are coupled via a first flash memory bus to the first flash buffer chip and wherein the first flash buffer chip is coupled via the first high speed bus to the first DMA controller;

wherein the second flash module comprises a second flash buffer chip and a second flash bank, wherein the second flash bank comprises a second plurality of flash devices that are coupled via a second flash memory bus to the second flash buffer chip and wherein the second flash buffer chip is coupled via the second high speed bus to the second DMA controller;

wherein the third flash module comprises a third flash buffer chip and a third flash bank, wherein the third flash bank comprises a third plurality of flash devices that are coupled via a third flash memory bus to the third flash buffer chip and wherein the third flash buffer chip is coupled via the third high speed bus to the third DMA controller;

wherein the data striping and memory interleaving comprise;

storing, in a first flash device in the first plurality of flash devices in the first flash bank in the first flash module, the first data stripe of the data;

storing, in a second flash device in the second plurality of flash devices in the second flash bank in the second flash module, the second data stripe of the data;

storing, in a third flash device in the third plurality of flash devices in the third flash bank in the third flash module, the third data stripe of the data;

wherein the first DMA controller comprises a first engine and the first data buffer, the second DMA controller comprises a second engine and the second data buffer, the third DMA controller comprises a third engine and the third data buffer;

wherein the first engine transfers the first data stripe between the first flash device in the first flash bank and the first data buffer in the first Direct Memory Access (DMA) controller;

wherein the second engine transfers the second data stripe between the second flash device in the second flash bank and the second data buffer in the second DMA controller;

wherein the third engine transfers the third data stripe between the third flash device in the third flash bank and the third data buffer in the third DMA controller.

12. An apparatus, comprising:

a system comprising a storage array, and a plurality of Direct Memory Access (DMA) controllers comprising a first Direct Memory Access (DMA) controller, a second DMA controller, and a third DMA controller;

wherein the system is configured to perform data striping and memory interleaving in the storage array in order to permit a data transfer, wherein the storage array comprises a first flash module, a second flash module and a third flash module;

wherein the system is configured to perform the data striping and memory interleaving comprising;

data striping, by the plurality of Direct Memory Access (DMA) controllers, a data into a first data stripe, a second data stripe, and a third data stripe,

and storing the first data stripe into a first data buffer in the first DMA controller, storing the second data stripe into a second data buffer in the second DMA controller, and storing the third data stripe into a third data buffer in the third DMA controller;

wherein the first flash module is coupled via a first high speed bus to the first DMA controller, wherein the second flash module is coupled via a second high speed bus to the second DMA controller, and wherein the third flash module is coupled via a third high speed bus to the third DMA controller, respectively;

wherein the first DMA controller, the second DMA controller, and third DMA controller are coupled to a local bus;

wherein the first flash module comprises a first flash buffer chip and a first flash bank, wherein the first flash bank comprises a first plurality of flash devices that are coupled via a first flash memory bus to the first flash buffer chip and wherein the first flash buffer chip is coupled via the first high speed bus to the first DMA controller;

wherein the second flash module comprises a second flash buffer chip and a second flash bank, wherein the second flash bank comprises a second plurality of flash devices that are coupled via a second flash memory bus to the second flash buffer chip and wherein the second flash buffer chip is coupled via the second high speed bus to the second DMA controller;

wherein the third flash module comprises a third flash buffer chip and a third flash bank, wherein the third flash bank comprises a third plurality of flash devices that are coupled via a third flash memory bus to the third flash buffer chip and wherein the third flash buffer chip is coupled via the third high speed bus to the third DMA controller;

wherein the first DMA controller comprises a first engine and the first data buffer, the second DMA controller comprises a second engine and the second data buffer, the third DMA controller comprises a third engine and the third data buffer;

wherein the first engine transfers the first data stripe between a first flash device in the first flash bank and the first data buffer in the first Direct Memory Access (DMA) controller;

wherein the second engine transfers the second data stripe between a second flash device in the second flash bank and the second data buffer in the second DMA controller;

wherein the third engine transfers the third data stripe between a third flash device in the third flash bank and the third data buffer in the third DMA controller.

13. The apparatus of claim 12 wherein the first flash device and a fourth flash device are in the first flash bank.

14. The apparatus of claim 12 wherein the second flash bank further comprises a fifth flash device.

15. The apparatus of claim 12 wherein the first flash device and a fourth flash device are in the first flash module.

16. The apparatus of claim 12 wherein the second memory module further comprises in a fifth flash device.

17. The apparatus of claim 12 wherein the first flash buffer chip is coupled to the first flash bank and a fourth flash bank in the first flash module.

18. The apparatus of claim 12 wherein the second flash buffer chip is coupled to the second flash bank and a fifth flash bank in the second flash module.

19. The apparatus of claim 12 wherein the DMA controllers transfer the data stripes between a cache and the storage array.

20. The apparatus of claim 12, wherein the first engine transfers the first data stripe between the first flash device and the first DMA controller and the second engine transfers the second data stripe between the second flash device and the second DMA controller in parallel.