Scatter-gather approach for parallel data transfer in a mass storage system

• US 10,489,318 B1
• Filed: 04/17/2015
• Issued: 11/26/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: BiTMICRO, LLC (BiTMICRO Networks, Inc.)
• Original Assignee: BiTMICRO Networks, Inc.
• Inventors: Bruce, Ricardo H., Santos, Avnher Villar, Verdan, Marlon Basa, Villapana, Elsbeth Lauren Tagayo
• Primary Examiner(s): Oberly, Eric T
• Application Number:
  US14/690,243
• Time in Patent Office: 1,684 Days
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 13/4265 on a point to point bus G06F13/4247, G06F13/4282 take precedence
• CPC Class Codes: G06F 13/4282 on a serial bus, e.g. I2C bus, SPI bus on daisy chain buses G06F13/4247
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

In an embodiment of the invention, an apparatus comprises: a first flash module comprising a first flash device; and a second flash module comprising a second flash device; wherein the first flash module and second flash module are coupled by a flash interconnect; wherein the first flash device is configured to store a first data stripe of a data and wherein the second flash device is configured to store a second data stripe of the data. In another embodiment of the invention, a method comprises: storing, in a first flash device in a first flash module, a first data stripe of a data; and storing, in a second flash device in a second flash module, a second data stripe of the data; wherein the first flash module and second flash module are coupled by a flash interconnect. In yet another embodiment of the invention, an article of manufacture comprises a non-transient computer-readable medium having stored thereon instructions that permit a method comprising: storing, in a first flash device in a first flash module, a first data stripe of a data; and storing, in a second flash device in a second flash module, a second data stripe of the data; wherein the first flash module and second flash module are coupled by a flash interconnect.

Claims

1. An apparatus, comprising:

a volatile memory configured to store data;

a Direct Memory Access (DMA) controller directly coupled by a local bus to the volatile memory;

a first flash module comprising a first flash buffer chip, a first flash memory bus, and a first flash bank comprising a first plurality of flash devices including a first flash device, wherein the first plurality of flash devices are coupled by the first flash memory bus to the first flash buffer chip in the first flash module; and

a second flash module comprising a second flash buffer chip, a second flash memory bus, and a second flash bank comprising a second plurality of flash devices including a second flash device, wherein the second plurality of flash devices are coupled by the second flash memory bus to the second flash buffer chip in the second flash module;

wherein the first flash buffer chip in the first flash module and the second flash buffer chip in the second flash module are directly coupled by a flash interconnect to the DMA controller;

wherein the flash interconnect comprises a first plurality of high speed buses that is directly coupled to the first flash buffer chip in the first flash module and to the DMA controller;

wherein the flash interconnect comprises a second plurality of high speed buses that is directly coupled to the second flash buffer chip in the second flash module and to the first flash buffer chip in the first flash module;

wherein the first flash device is configured to store a first data stripe of the data and wherein the second flash device is configured to store a second data stripe of the data;

wherein the DMA controller comprises a DMA controller data buffer, a first DMA engine, and a second DMA engine;

wherein the DMA controller data buffer comprises a first buffer location and a second buffer location;

wherein the first DMA engine transfers the first data stripe between the first flash device and the first buffer location when a predetermined portion of data stripes is stored in the first flash buffer chip; and

wherein the second DMA engine transfers the second data stripe between the second flash device and the second buffer location when a predetermined portion of data stripes is stored in the second flash buffer chip.

2. The apparatus of claim 1, wherein the flash interconnect comprises a fabric interconnect.

3. The apparatus of claim 1, wherein the flash interconnect comprises a point-to-point serial bus topology.

4. The apparatus of claim 1, wherein the flash interconnect comprises a plurality of flash memory bus controllers and at least one flash memory bus that connects at least two of the plurality of flash memory bus controllers.

5. The apparatus of claim 1, further comprising:

a second DMA controller coupled by a second flash interconnect to a third flash module.

6. The apparatus of claim 1, further comprising a plurality of independent DMA controllers.

7. The apparatus of claim 1 wherein each of the plurality of independent DMA controllers includes a plurality of DMA engines and a data buffer.

8. The apparatus of claim 1, wherein each of the plurality of DMA controllers is coupled with a shared data cache.

9. A method, comprising:

storing data in a volatile memory;

wherein the volatile memory is directly coupled by a local bus to a Direct Memory Access (DMA) controller;

wherein the DMA controller is directly coupled by a flash interconnect to a first flash module and to a second flash module;

wherein the first flash module comprises a first flash buffer chip, a first flash memory bus, and a first flash bank comprising a first plurality of flash devices including a first flash device, wherein the first plurality of flash devices are coupled by the first flash memory bus to the first flash buffer chip in the first flash module;

wherein the second flash module comprises a second flash buffer chip, a second flash memory bus, and a second flash bank comprising a second plurality of flash devices including a second flash device, wherein the second plurality of flash devices are coupled by the second flash memory bus to the second flash buffer chip in the second flash module;

wherein the first flash buffer chip in the first flash module and the second flash buffer chip in the second flash module are directly coupled by the flash interconnect to the DMA controller;

wherein the flash interconnect comprises a first plurality of high speed buses that is directly coupled to the first flash buffer chip in the first flash module and to the DMA controller;

wherein the flash interconnect comprises a second plurality of high speed buses that is directly coupled to the second flash buffer chip in the second flash module and to the first flash buffer chip in the first flash module;

storing, in the first flash device in the first flash module, a first data stripe of the data; and

storing, in the second flash device in the second flash module, a second data stripe of the data;

wherein the DMA controller comprises a DMA controller data buffer, a first DMA engine, and a second DMA engine;

wherein the DMA controller data buffer comprises a first buffer location and a second buffer location;

wherein the first DMA engine transfers the first data stripe between the first flash device and the first buffer location when a predetermined portion of data stripes is stored in the first flash buffer chip; and

wherein the second DMA engine transfers the second data stripe between the second flash device and the second buffer location when a predetermined portion of data stripes is stored in the second flash buffer chip.

10. The method of claim 9, wherein the flash interconnect comprises a fabric interconnect.

11. The method of claim 9, wherein the flash interconnect comprises a point-to-point serial bus topology.

12. The method of claim 9, wherein the flash interconnect comprises a plurality of flash memory bus controllers and at least one flash memory bus that connects at least two of the plurality of flash memory bus controllers.

13. The method of claim 9, wherein a second DMA controller is coupled by a second flash interconnect to a third flash module.

14. The method of claim 9 wherein the first DMA engine transfers the first data stripe between the first flash device and the first buffer location when a single data stripe is stored in the first flash buffer chip.

15. An article of manufacture, comprising:

a non-transitory computer-readable medium having stored thereon instructions to permit an apparatus to perform a method comprising;

storing data in a volatile memory;

wherein the volatile memory is directly coupled by a local bus to a Direct Memory Access (DMA) controller;

wherein the DMA controller is directly coupled by a flash interconnect to a first flash module and to a second flash module;

wherein the first flash module comprises a first flash buffer chip, a first flash memory bus, and a first flash bank comprising a first plurality of flash devices including a first flash device, wherein the first plurality of flash devices are coupled by the first flash memory bus to the first flash buffer chip in the first flash module;

wherein the second flash module comprises a second flash buffer chip, a second flash memory bus, and a second flash bank comprising a second plurality of flash devices including a second flash device, wherein the second plurality of flash devices are coupled by the second flash memory bus to the second flash buffer chip in the second flash module;

wherein the first flash buffer chip in the first flash module and the second flash buffer chip in the second flash module are directly coupled by the flash interconnect to the DMA controller;

wherein the flash interconnect comprises a first plurality of high speed buses that is directly coupled to the first flash buffer chip in the first flash module and to the DMA controller;

wherein the flash interconnect comprises a second plurality of high speed buses that is directly coupled to the second flash buffer chip in the second flash module and to the first flash buffer chip in the first flash module;

storing, in the first flash device in the first flash module, a first data stripe of the data; and

storing, in the second flash device in the second flash module, a second data stripe of the data;

wherein the DMA controller comprises a DMA controller data buffer, a first DMA engine, and a second DMA engine;

wherein the DMA controller data buffer comprises a first buffer location and a second buffer location;

wherein the first DMA engine transfers the first data stripe between the first flash device and the first buffer location when a predetermined portion of data stripes is stored in the first flash buffer chip; and

wherein the second DMA engine transfers the second data stripe between the second flash device and the second buffer location when a predetermined portion of data stripes is stored in the second flash buffer chip.

16. The article of manufacture of claim 15, wherein the flash interconnect comprises a fabric interconnect.