Systems and methods for storage parallelism

• US 9,454,492 B2
• Filed: 12/28/2012
• Issued: 09/27/2016
• Est. Priority Date: 12/06/2006
• Status: Active Grant

• Current Assignee: Unification Technologies LLC (Acacia Research Corporation)
• Original Assignee: Longitude Enterprise Flash S.à r.l. (Vector Capital Corporation)
• Inventors: Flynn, David, Lagerstedt, Bert, Strasser, John, Thatcher, Jonathan, Walker, John, Zappe, Michael
• Primary Examiner(s): Tsai, Henry
• Assistant Examiner(s): Shyu, Jing-Yih
• Application Number:
  US13/730,602
• Publication Number:
  US 20130124791A1
• Time in Patent Office: 1,369 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/183 Internal mounting support structures, e.g. for printed circuit boards, internal connecting means for buses G06F13/409
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 12/123 with age lists, e.g. queue, most recently used [MRU] list or least recently used [LRU] list
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 13/426 using an embedded synchronisation, e.g. Firewire bus, Fibre Channel bus, SSA bus
• CPC Class Codes: G06F 2211/002 Bus
• CPC Class Codes: G06F 2211/103 Hybrid, i.e. RAID systems with parity comprising a mix of RAID types
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/2022 Flash memory
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 2212/70 Details relating to dynamic memory management
• CPC Class Codes: G06F 2212/7203 Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 2212/7208 : Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 : Saving storage space on storage systems
• CPC Class Codes: G06F 3/0613 : in relation to throughput
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0643 : Management of files
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0652 : Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0656 : Data buffering arrangements
• CPC Class Codes: G06F 3/0659 : Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 : Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0688 : Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 9/52 : Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/54 : Interprogram communication
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H05K 7/1444 : Complex or three-dimensional-arrangements; Stepped or dual mother boards
• CPC Class Codes: H05K 7/1487 : Blade assemblies, e.g. blade cases or inner arrangements within a blade
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

One method includes streaming a data segment to a write buffer corresponding to a virtual page including at least two physical pages. Each physical page is defined within a respective solid-state storage element. The method also includes programming contents of the write buffer to the virtual page, such that a first portion of the data segment is programmed to a first one of the physical pages, and a second portion of the data segment is programmed to a second one of the physical pages.

Claims

1. A method, comprising:

streaming a data segment to a single write buffer corresponding to a virtual page comprising at least two physical pages, each physical page defined within a respective solid-state storage element of a plurality of solid-state storage elements;

programming contents of the single write buffer to the virtual page, such that a first portion of the dataⁱ segment is programmed to a first one of the physical pages, and a second portion of the data segment is programmed to a second one of the physical pages; and

initializing a storage division comprising a plurality of virtual pages by;

relocating valid data stored on the virtual pages of the storage division to virtual pages of another storage division of the plurality of solid-state storage elements; and

erasing the storage division.

2. The method of claim 1, wherein streaming the data segment to the single write buffer comprises interleaving data of the data segment within program buffers of two or more solid-state storage elements of the plurality of solid-state storage element.

3. The method of claim 1, further comprising reading data from another virtual page of the plurality of solid-state storage elements concurrently with streaming the data segment.

4. The method of claim 1, wherein relocating the valid data comprises transferring the valid data to the single write buffer with the data segment.

5. The method of claim 1, wherein programming the virtual page comprises issuing a single write command in parallel to the plurality of solid-state storage elements comprising the virtual page.

6. An apparatus, comprising:

a storage controller configured to perform storage operations on at least two solid-state storage elements of a plurality of solid-state storage elements;

a storage bus configured to couple the at least two solid-state storage elements to the storage controller in parallel; and

a single write data module configured to store a data segment on two or more solid-state storage elements of the plurality of solid-state storage elements, wherein the single write data module is configured to stream a plurality of data segments to a virtual page, the virtual page comprises a respective page of each solid-state storage element of the plurality of solid-state storage elements, each solid-state storage element comprising respective storage divisions, each storage division comprising a respective page of the virtual page, and, in response to a failure of one of the storage divisions of the virtual page, the storage controller is configured to map a replacement storage division into the virtual page.

7. The apparatus of claim 6, wherein the single write data module is configured to interleave data of the data segment on the two or more solid-state storage elements with data of one or more other data segments.

8. The apparatus of claim 6, wherein one or more data segments of the plurality of data segments streamed to the virtual page comprise data segments being relocated from other storage divisions of the plurality of solid-state storage elements.

9. The apparatus of claim 6, wherein the storage controller is configured to program data of the data segment to the two or more solid-state storage elements by use of a single write command.

10. The apparatus of claim 6, wherein the storage controller is configured to erase respective erase blocks of the plurality of solid-state storage elements by use of a single erase command.

11. The apparatus of claim 6, further comprising an error-correcting code (ECC) generator module configured to generate two or more ECC blocks comprising data of the data segment, and wherein the single write data module is configured to store data of each of the ECC blocks on the two or more solid-state storage elements.

12. The apparatus of claim 11, wherein one of the two or more ECC blocks comprising data of the data segment comprises data of another, different data segment.

13. A computer-program product comprising program code configured for execution by a computing device to cause the computing device to perform operations, comprising:

transferring data of a data block from a single write buffer to at least two different storage units of respective, independent flash storage elements of a plurality of flash storage elements;

generating a single write command configured to store the data of the data block on the at least two storage units of the respective, independent flash storage elements; and

initializing a storage division comprising a plurality of virtual pages by;

relocating valid data stored on the virtual pages of the storage division to virtual pages of another storage division of the plurality of flash storage elements; and

erasing the storage division.

14. The computer-program product of claim 13, wherein the data of the data block is interleaved with data of one or more other data blocks within the storage units of the respective, independent flash storage elements.

15. The computer-program product of claim 13, wherein transferring the data of the data block comprises buffering a data segment within program buffers of the respective, independent flash storage elements.

16. The computer-program product of claim 13, the operations further comprising encapsulating the data of the data block in a packet comprising a header, wherein the header indicates a logical address associated with a data segment.

17. The computer-program product of claim 13, the operations further comprising issuing the single write command to the respective, independent flash storage elements.