Multiprocessor storage controller

• US 8,762,620 B2
• Filed: 04/08/2008
• Issued: 06/24/2014
• Est. Priority Date: 12/27/2007
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: SanDisk Enterprise IP LLC (Western Digital Corporation)
• Inventors: Prins, Douglas A., Olbrich, Aaron K.
• Primary Examiner(s): Kindred, Alford W.
• Assistant Examiner(s): Mercado, Ramon A
• Application Number:
  US12/082,221
• Publication Number:
  US 20090172261A1
• Time in Patent Office: 2,268 Days
• Field of Search: 711/5, 711/140, 711/149, 711/169, 711/200, 712245-248, 712/216, 712/212
• US Class Current: 711/103
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 13/1657 Access to multiple memories
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 2212/7201 Logical to physical mapping or translation of blocks or pages
• CPC Class Codes: G06F 2212/7203 Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G11C 16/10 Programming or data input circuits
• CPC Class Codes: G11C 7/1051 Data output circuits, e.g. read-out amplifiers, data output buffers, data output registers, data output level conversion circuits
• CPC Class Codes: G11C 7/1063 Control signal output circuits, e.g. status or busy flags, feedback command signals

Abstract

A storage controller containing multiple processors. The processors are divided into groups, each of which handles a different stage of a pipelined process of performing host reads and writes. In one embodiment, the storage controller operates with a flash memory module, and includes multiple parallel pipelines that allow plural host commands to be handled simultaneously.

Claims

1. A storage system comprising:

a first and a second flash memory groups, each group comprising a plurality of flash memory devices;

a storage controller including a plurality of groups of processors, each processor group comprising one or more processors for handling a different stage of a pipelined execution of host storage commands, the processor groups including;

a host interface circuit coupled to receive a first host command from a first host through a host interface and a second host command from a second host through the host interface, the host interface circuit including;

a first processor of the host interface circuit operably connected to one or more buffers that hold information received through the host interface, the first processor of the host interface circuit including software or hardware control disposed to format such information for use by downstream elements of the storage system; and

a second processor of the host interface circuit operably connected to one or more buffers that hold information received from other storage system elements, the second processor of the host interface circuit including software or hardware control disposed to format such information for transmission through the host interface;

a first processor group of the plurality of groups of processors including a first processor of the first processor group and a second processor of the first processor group, the first processor of the first processor group associated with the first flash memory group and the second processor of the first processor group associated with the second flash memory group, each such processor configured for controlling at least some operations of the flash memory group associated therewith; and

a command processing circuit configured to provide the first host command to the first processor of the first processor group and the second host command to the second processor of the first processor group, the first flash memory group being configured to carry out, under control of the first processor of the first processor group, flash read or write operations relating to the first host command and the second flash memory group being configured to carry out, under control of the second processor of the first processor group, flash read or write operations relating to the second host command, the storage controller configured to cause the first and second host commands to be carried out substantially simultaneously.

2. The system as in claim 1, wherein the command processing circuit further includes:

one or more processors, each including software or hardware control disposed to directly or indirectly receive information generated by the first processor of the host interface circuit and directly or indirectly transmit information related to one or more received host commands to one or more processors in the first processor group.

3. The system as in claim 2, wherein:

the one or more processors of the command processing circuit includes a first processor of the command processing circuit and a second processor of the command processing circuit; and

the first processor of the command processing circuit includes a memory that stores information related to a first host command, but does not store information related to a second host command; and

the second processor of the command processing circuit includes a memory that stores information related to the second host command, but does not store information related to the first host command.

4. The system as in claim 3, wherein:

the first processor of the host interface circuit includes a memory that stores information related to a third host command, but does not store information related to the first or second host command or to a fourth, fifth or sixth host command;

the second processor of the host interface circuit includes a memory that stores information related to the fourth host command, but does not store information related to the first, second, third, fifth or sixth host commands;

the first processor of the first processor group includes a memory that stores information related to the fifth host command, but does not store information related to the first, second, third, fourth or sixth host commands;

the second processor of the first processor group includes a memory that stores information related to a sixth host command, but does not store information related to the first, second, third, fourth or fifth host commands; and

the memories from the first and second processors of the command processing circuit do not store information related to the third, fourth, fifth or sixth host commands.

5. A method of operating a system including a storage controller disposed on a single integrated circuit device, and a plurality of groups of flash memory devices, the storage controller including a first processor group, a second processor group and a third processor group, each processor group comprising one or more processors for handling a different stage of a pipelined execution of host storage commands, the method comprising the following steps:

(a) receiving a first host command in a first processor of the first processor group, the first host command including a first logical address or range, wherein the first host is one of at least two hosts coupled to the storage controller through a host interface, wherein the first processor is operably connected to one or more buffers that hold information relating to the first host command, the first processor including software or hardware control disposed to format such information for use by downstream elements of the storage system;

(b) the first processor performing a task related to the first host command;

(c) the first processor directly or indirectly passing information related to the first host command to a second processor, the second processor being part of the second processor group;

(d) the second processor evaluating the first logical address or range;

(e) based at least in part on the first logical address or range, the second processor directly or indirectly passing information related to the first host command to a third processor, the third processor being part of the third processor group;

(f) the third processor initiating a flash memory read in a first flash group, based at least in part on information directly or indirectly received from the second processor;

(g) receiving a second host command through the host interface from a second host at a fourth processor, the fourth processor being part of the first processor group, the second host command including a second logical address or range, wherein the fourth processor is operably connected to one or more buffers that hold information relating to the second host command, the fourth processor including software or hardware control disposed to format such information for use by downstream elements of the storage system; and

(h) the fourth processor performing a task related to the second host command, this step (h) overlapping step (b) in time.

6. The method as in claim 5, further including:

(i) the fourth processor directly or indirectly passing information related to the second host command to a fifth processor, the fifth processor being part of the second processor group;

(j) the fifth processor evaluating the second logical address or range;

(k) based at least in part on the second logical address or range, the fifth processor directly or indirectly passing information related to the second host command to a sixth processor, the sixth processor being part of the third processor group;

(l) the sixth processor initiating a flash memory read in a second flash group, based at least in part on information directly or indirectly received from the fifth processor, the flash memory read overlapping in time the flash memory read of step (f).

7. The method as in claim 6, wherein step (e) further includes:

(e)(i) based at least in part on the first logical address or range, the second processor directly or indirectly passing information related to the first host command to a seventh processor, the seventh processor being part of the third processor group;

and step (f) further includes;

(f)(i) the seventh processor initiating a flash memory read in a third flash group, based at least in part on information directly or indirectly received from the second processor, the flash memory read overlapping in time the flash memory read initiated by the third processor.

8. The method as in claim 7, wherein the storage controller further includes a fourth processor group made up of one or more processors, the method further including:

(m) at an eighth processor that is part of the fourth processor group, receiving information relating to the data read in step (f);

(n) the eighth processor performing an action relating to the information received in step (m); and

(o) transmitting the data read in step (f) to the first host.

9. The method as in claim 8, further including:

(p) at a ninth processor that is part of the fourth processor group, receiving information relating to the data read in step (l);

(q) the ninth processor performing an action relating to the information received in step (p); and

(r) transmitting the data read in step (l) to the second host.

10. The method as in claim 8, further including:

(p) receiving a third host command at the first processor, the third host command including a third logical address or range;

(q) the first processor performing a task related to the third host command;

(r) the third processor directly or indirectly passing information related to the third host command to a processor in the second processor group, steps (p)-(r) starting before step (f) is completed.

11. The method as in claim 5, wherein:

the direct or indirect passage of information in step (c) from the first processor to the second processor includes passing information to a tenth processor, the tenth processor evaluating the information and selecting the second processor.

12. The method as in claim 11, wherein:

the step (c) selection of the second processor is based at least in part on an evaluation of the current workload of the second processor relative to the current workloads of other second group processors.

13. The method as in claim 12, further including:

(g) receiving a second host command from a second host at a fourth processor, the fourth processor being part of the first processor group, the second host command including a second logical address or range;

(h) the fourth processor performing a task related to the second host command, this step (h) overlapping step (b) in time;

(i) the tenth processor evaluating information relating to the second host command and selecting an eleventh processor to directly or indirectly receive information relating to the second host command, the selection being based at least in part on a determination that the second host command does not include a storage read or write.

14. The method as in claim 13, further including:

(j) the eleventh processor evaluating the second host command and executing software to at least in part carry out one or more tasks related to the second host command.

15. A method of operating a storage system comprising the following steps:

(a) at a first host port, receiving a first communication including a first host command, wherein the first host port is one of at least two host ports coupled to a storage controller, the storage controller including a plurality of groups of processors including a first processor group, each processor group comprising one or more processors for handling a different stage of a pipelined execution of host storage commands;

(b) a first processor of the first processor group performing an action relating to the first host command, wherein the first processor is operably connected to one or more buffers that hold information relating to the first host command, the first processor including software or hardware control disposed to format such information for use by downstream elements of the storage system;

(c) transmitting information relating to the first host command to a second processor;

(d) the second processor evaluating the first host command;

(e) the second processor routing information relating to a second processor portion of the first host command based at least in part on the evaluation;

(f) receiving information related to a third processor portion of the first host command at a third processor;

(g) at the third processor, evaluating a first logical address associated with the first host command;

(h) based at least in part on the first logical address, routing information relating to a fourth processor portion of the first host command to a fourth processor;

(i) at the fourth processor, evaluating the received information relating to the fourth processor portion of the first host command;

(j) at the fourth processor, generating a first flash read command based on the first host command;

(k) communicating the first flash read command to a first flash group;

(l) the first flash group executing the first flash read command;

(m) communicating information relating to the status of the first flash read command to the fourth processor;

(n) following completion of the first flash read command, communicating information relating to the first flash read command to a fifth processor;

(o) the fifth processor taking an action relating to formatting the information read in the first flash read command for transmission to a first host;

(p) transmitting the information read in the first flash read command to the first host through the first host port;

(q) at a second host port, receiving a second communication from a second host including a second host command;

(r) a sixth processor of the first processor group performing an action relating to the second host command;

the action being performed prior to completion of step (p), wherein the sixth processor is operably connected to one or more buffers that hold information relating to the second host command, the sixth processor including software or hardware control disposed to format such information for use by downstream elements of the storage system.

16. The method as in claim 15, further including:

(q) based at least in part on the first logical address, routing information relating to the first host command to a seventh processor;

(r) at the seventh processor, evaluating the received information relating to the first host command;

(s) at the seventh processor, generating a second flash read command based on the first host command;

(t) communicating the second flash read command to a second flash group;

(u) the second flash group executing the second flash read command;

(v) communicating information relating to the status of the second flash read command to the fourth processor;

wherein;

step (n) includes, following completion of the second flash read command, communicating information relating to the second flash read command to the fifth processor;

the step (o) action relates to formatting information read in the second flash read command in combination with information read in the first flash read command;

and the step (p) transmission includes information read in the second flash read command.

17. The method as in claim 15, wherein:

the step (d) evaluation includes evaluating the nature of the first host command and taking a first action if the command is a host read or write, and a second action if the first host command is other than a host read or write.

18. The method as in claim 15, wherein:

the third processor is one of a group made up of at least two processors; and

the step (f) further includes choosing the third processor for receipt of the information at least in part based on the current workload of the third processor in relation to other processors in the group.

19. A method of operating a storage system including the following steps:

(a) a first processor receiving first information relating to a first host command from a first host which is one of at least two hosts coupled to a storage controller, the storage controller including a plurality of groups of processors, each processor group comprising one or more processors for handling a different stage of a pipelined execution of host storage commands;

(b) based at least in part on such information, the first processor generating a first flash command and a second flash command;

(c) the first processor transmitting the first flash command to a first flash memory group comprising a plurality of flash memory devices;

(d) the first flash memory group carrying out a first flash operation based on the first flash command;

(e) the first processor transmitting the second flash command to the first flash memory group, the second command being transmitted to the first flash memory group prior to completion of the first flash operation;

(f) the first flash memory group carrying out a second flash operation based on the second flash command;

(g) a second processor receiving second information relating to a second host command;

(h) based at least in part on such information, the second processor generating a third flash command and a fourth flash command;

(i) the second processor transmitting the third flash command to a second flash memory group comprising a plurality of flash memory devices;

(j) the second flash memory group carrying out a third flash operation based on the third flash command;

(k) the second processor transmitting the fourth flash command to the second flash memory group, the fourth command being transmitted to the second flash memory group prior to completion of the third flash operation;

(l) the second flash memory group carrying out a fourth flash operation based on the fourth flash command;

the storage system adapted to execute multiple flash commands relating to multiple host commands in an overlapped manner and the first host command and the second host command are provided by a command processing circuit.

20. The method as in claim 19, further including:

prior to step (a), a third processor receiving information relating to the first host command and based on said information, the third processor selecting the first processor for receipt of information relating to the first host command.

21. The method as in claim 20, further including:

the selection of the first processor being made based on one or more addresses associated with the first host command.

22. The method as in claim 21, further including:

(m) the third processor further transmitting information relating to the first host command to a fourth processor;

(n) based at least in part on such information, the fourth processor generating a fifth flash command;

(o) the fourth processor transmitting the fifth flash command to a third flash memory group comprising a plurality of flash memory devices;

(p) the third flash memory group carrying out a fifth flash operation based on the fifth flash command.

23. The method as in claim 22, further including:

(q) a sixth processor performing an operation relating to the results of the first, second and fifth flash operations; and

(r) data read as part of the first, second and fifth flash operations being transmitted to a host as part of an organized data structure.