Disk array controller with automated processor which routes I/O data according to addresses and commands received from disk drive controllers

• US 6,138,176 A
• Filed: 03/04/1998
• Issued: 10/24/2000
• Est. Priority Date: 11/14/1997
• Status: Expired due to Term

• Current Assignee: Summit Data Systems LLC (Acacia Research Corporation)
• Original Assignee: 3WARE
• Inventors: McDonald, James Arthur, Herz, John Peter, Altman, Mitchell Allen, Smith, III, William Edward
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): WANG, ALBERT C
• Application Number:
  US09/034,248
• Time in Patent Office: 965 Days
• Field of Search: 711/4, 711/114, 710/52, 710/129, 710/74, 710/241, 710/6
• US Class Current: 710/6
• CPC Class Codes: G06F 13/124 where hardware is a sequential transfer control unit, e.g. microprocessor, peripheral processor or state-machine
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0658 Controller construction arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A high-performance RAID system for a PC comprises a controller card which controls an array of ATA disk drives. The controller card includes an array of automated disk drive controllers, each of which controls one respective disk drive. The disk drive controllers are connected to a microcontroller by a control bus and are connected to an automated coprocessor by a packet-switched bus. The coprocessor accesses system memory and a local buffer. In operation, the disk drive controllers respond to controller commands from the microcontroller by accessing their respective disk drives, and by sending packets to the coprocessor over the packet-switched bus. The packets carry I/O data (in both directions, with the coprocessor filling-in packet payloads on I/O writes), and carry transfer commands and target addresses that are used by the coprocessor to access the buffer and system memory. The packets also carry special completion values (generated by the microcontroller) and I/O request identifiers that are processed by a logic circuit of the coprocessor to detect the completion of processing of each I/O request. The coprocessor grants the packet-switched bus to the disk drive controllers using a round robin arbitration protocol which guarantees a minimum I/O bandwidth to each disk drive. This minimum I/O bandwidth is preferably greater than the sustained transfer rate of each disk drive, so that all drives of the array can operate at the sustained transfer rate without the formation of a bottleneck.

Claims

1. A disk array controller which operatively connects a host computer to an array of disk drives, the host computer including a system memory, the disk array controller comprising:

a plurality of disk drive controllers, each disk drive controller connected to and configured to control at least one disk drive of the array;

a microcontroller which dispatches controller commands to the disk drive controllers over a control bus to initiate transfers of input/output (I/O) data between the disk drives and the host computer, at least some of the controller commands including system memory addresses for performing said transfers, the microcontroller responsive to I/O requests generated by the host computer; and

an automated processor which transfers I/O data between at least the disk drive controllers and the system memory, the automated processor connected to the plurality of disk drive controllers by a packet bus which is separate from the control bus, and connected as a bus master to a bus of the host computer, the automated processor responsive to transfer commands and target addresses received from the disk drive controllers over the packet bus, at least some of the transfer commands specifying transfers of I/O data between the disk drive controllers and the system memory.

2. The disk array controller according to claim 1, wherein the disk drive controllers generate packets that are transferred to the automated processor over the packet bus, at least some of the packets including (i) a block of I/O data read from the disk drive array, and (ii) a transfer command and a target address which specify how the block of I/O data is to be routed by the automated processor.

3. The disk array controller according to claim 1, wherein each disk drive controller is configured to generate a packet that includes a system memory write command, a target write address, and a block of I/O data, and wherein the automated processor responds to the system memory write command by writing the block of I/O data to the system memory based on the target write address.

4. The disk array controller according to claim 1, wherein at least one of the disk drive controllers is configured to generate a packet that includes a transfer completion command, the transfer completion command indicating that the disk drive controller has finished processing an I/O request, and wherein the automated processor responds to the transfer completion command by determining whether all disk drive controllers invoked by the I/O request have finished processing the I/O request.

5. The disk array controller according to claim 1, further comprising a buffer coupled to the automated processor, wherein the automated processor stores I/O data in the buffer and is responsive to transfer commands which specify transfers of I/O data to and from the buffer.

6. The disk array controller according to claim 5, wherein each disk drive controller is configured to generate a packet that includes a system memory read command, and wherein the automated processor responds to the system memory read command by transferring I/O data from the system memory to the buffer.

7. The disk array controller according to claim 5, wherein each disk drive controller is configured to generate a packet that includes a buffer read command, and the automated processor responds to the buffer read command by transferring I/O data from the buffer to the disk drive controller that generates the buffer read command.

8. The disk array controller according to claim 1, wherein the automated processor controls all accesses to by the disk drive controllers to the packet bus.

9. The disk array controller according to claim 8, wherein the automated processor grants the packet bus to individual disk drive controllers using a round robin arbitration protocol.

10. The disk array controller according to claim 8, wherein the automated processor implements a bus arbitration protocol which guarantees that at least 1/N of the I/O bandwidth of the packet bus is available to each disk drive controller, where N is the number of disk drive controllers.

11. The disk array controller according to claim 1, wherein each disk drive controller is an automated controller which controls a single, respective disk drive of the array.

12. The disk array controller according to claim 1, wherein each disk drive controller is an automated ATA controller which controls a single, respective ATA disk drive.

13. The disk array controller according to claim 12, wherein at least one of the automated ATA controllers includes a command buffer which buffers multiple disk drive commands, so that a new disk drive command can be dispatched to the respective ATA disk drive immediately upon completion of a pending disk drive command.

14. The disk array controller according to claim 1, wherein microcontroller and the automated processor are integrated within a common semiconductor device.

15. The disk array controller according to claim 1, wherein the microcontroller runs a control program which implements a RAID configuration.

16. The disk array controller as in claim 1, wherein the disk array controller processes multiple I/O requests at a time, and the automated processor and transfers the I/O data to the system memory without regard to an identity of the I/O request to which the I/O data corresponds.

17. A disk array controller which operatively connects a host computer to an array of disk drives, the host computer including a system memory, the disk array controller comprising:

a plurality of disk drive controllers, each disk drive controller connected to and configured to control at least one disk drive of the array;

a microcontroller which dispatches controller commands to the disk drive controllers over a first bus to initiate transfers of input/output (I/O) data between the disk drives and the host computer, at least some of the controller commands including system memory addresses for performing said transfers, the microcontroller responsive to I/O requests generated by the host computer; and

an automated processor which transfers I/O data between at least the disk drive controllers and the system memory in response to transfer commands and target system memory addresses received from the disk drive controllers, the automated processor connected to the plurality of disk drive controllers by a second bus which is separate from the first bus.

18. The disk array controller according to claim 17, wherein the disk drive controllers generate packets that are transferred to the automated processor over the second bus, at least some of the packets including (i) a block of I/O data read from the disk drive array, (ii) a transfer command which specifies that the block of I/O data is to be written to the system memory address, and (iii) a target system memory address for writing the block of I/O data to the system memory.

19. The disk array controller according to claim 17, wherein the automated processor controls all accesses by the disk drive controllers to the second bus.

20. The disk array controller according to claim 19, wherein the automated processor grants the second bus to individual disk drive controllers using a round robin arbitration protocol.

21. The disk array controller according to claim 17, wherein the automated processor and the disk drive controllers implement a bus arbitration protocol which guarantees that at least 1/N of the I/O bandwidth of the second bus is available to each disk drive controller, where N is the number of disk drive controllers.

22. The disk array controller according to claim 17, wherein each disk drive controller is configured to generate a transfer completion command which indicates that the respective disk drive controller has finished processing a pending I/O request, and wherein the automated processor responds to the transfer completion command by determining whether all disk drive controllers invoked by the pending I/O request have finished processing the I/O request.

23. The disk array controller according to claim 17 wherein each disk drive controller is an automated controller which controls a single, respective disk drive of the array.

24. The disk array controller according to claim 23, wherein each disk drive of the array is an ATA drive.

25. The disk array controller as in claim 17, wherein the disk array controller processes multiple PO requests at a time, and the automated processor transfers the I/O data between the disk drive controllers and the system memory without regard to identities of the I/O requests to which the I/O data corresponds.

26. In a computer system which comprises a host computer and an array of disk drives, a method of processing an input/output (I/O) request generated by the host computer, the method comprising:

(a) dispatching at least one controller command to a disk drive controller to initiate a transfer of I/O data from a disk drive controlled by the disk drive controller to a system memory of the host computer, the controller command including a system memory address which specifies a location to which the I/O data is to be transferred;

(b) dispatching controller commands to a plurality of other disk drive controllers to initiate transfers of I/O data from other disk drives of the array to the system memory;

(c) responding to the controller command dispatched in step (a) with the disk drive controller by (i) reading the I/O data from the disk drive, and (ii) generating and sending a sequence of packets to an automated processor, the sequence of packets including the I/O data and at least one target system memory address, the at least one target system memory address based on the system memory address included within the controller command; and

(d) processing the sequence of packets with the automated processor by writing the I/O data included therein to said location in system memory.

27. The method according to claim 26, wherein step (a) comprises sending the controller command to the disk drive controller over a first bus, and step (c) comprises sending the sequence of packets from the controller to the automated processor over a second bus which is separate from the first bus.

28. The method according to claim 26, wherein step (a) comprises dispatching a plurality of controller commands to the disk drive controller, the plurality of controller commands specifying a plurality of transfer operations to be performed by the disk drive controller.

29. The method according to claim 26, wherein at least one packet of the sequence of packets includes a transfer command that instructs the automated processor to write the I/O data to the system memory.

30. The method according to claim 26, wherein the last packet of the sequence of packets includes a completion command which informs the automated processor that the controller has finished processing the I/O request.

31. The method according to claim 30, wherein the automated processor responds to the completion command by determining whether all disk drive controllers of the disk array system have finished processing the I/O request.

32. The method according to claim 26, wherein the controller command dispatched in step (a) includes an I/O request identifier, and wherein the disk drive controller sends the I/O request identifier and a completion command to the automated processor upon finishing processing of the controller command.

33. The method according to claim 26, wherein step (a) is performed by a microcontroller which runs a control program, the control program implementing a RAID storage configuration.

34. The method according to claim 26, wherein the automated processor is coupled to a buffer which temporarily stores I/O data, and wherein at least some of the packets of the sequence of packets include transfer commands which specify transfers of I/O data to and from the buffer.

35. In a disk array controller which operatively connects a host computer to an array of disk drives, and which processes multiple input/output (I/O) requests from the host computer concurrently, a method of processing an I/O request from the host computer, comprising, with an automated processor:

receiving packets over a bus from each of a plurality of disk drive controllers invoked by the I/O request, at least some of the packets including I/O data read from the disk drives and including transfer commands and target system memory addresses that fully specify to the automated processor transfers of the I/O data to a system memory of the host computer; and

transferring the I/O data to the system memory of the host computer according to the transfer commands and the target system memory addresses contained within the packets without regard to an identity of the I/O request.

36. The method according to claim 35, wherein receiving packets over a bus comprises receiving from one of the disk drive controllers a sequence of packets that each fully specify an operation to be performed by the automated processor.

37. The method according to claim 35, wherein receiving packets comprises receiving a completion value from each of the disk drive controllers invoked by the I/O request when the respective disk drive controller has finished processing the I/O request, and wherein the method further comprises combining the completion values to determine whether all of the invoked disk drive controllers have finished the I/O request, the automated processor thereby detecting completion of the I/O request without prior knowledge of the number of disk drive controllers invoked.

38. The method according to claim 35, wherein receiving packets over the bus comprises granting the bus to the disk drive controllers according to a round robin bus arbitration protocol.

39. The method according to claim 35, wherein receiving packets over the bus comprises granting the bus to the disk drive controllers according to a least-recently-accessed bus arbitration protocol.

40. The method according to claim 35, wherein receiving packets over the bus comprises granting the bus to the disk drive controllers according to a bus arbitration protocol that guarantees a minimum I/O bandwidth of substantially 1/N of the I/O bandwidth of the bus to each disk drive controller, where N is the total number of disk drive controllers connected to the bus.

41. In a disk array controller which operatively connects a host computer to an array of disk drives, a method of processing a disk drive specific portion of an I/O read request, comprising, with an automated disk drive controller:

receiving a controller command that specifies at least a disk read operation to be performed and a target system memory address for writing resulting I/O data to the system memory;

reading the I/O data from the disk drive according to the command;

transmitting the I/O data on a bus within a sequence of packets, wherein at least some of the packets include transfer commands and target system memory addresses that fully specify transfer operations for transferring the I/O data to the system memory.

42. The method as in claim 41, wherein the disk drive is an ATA disk drive, and reading the I/O data from the disk drive comprises implementing an ATA protocol within automated circuitry of the disk drive controller.

43. The method as in claim 42 further comprising receiving and buffering multiple disk drive commands for the ATA disk drive within a buffer of the disk drive controller.

44. The method as in claim 41, wherein the controller command is received over a bus which is separate from the bus used to transfer the I/O data.

45. The method as in claim 41, wherein transmitting the I/O data comprises transmitting a packet that contains a block of I/O data, a transfer command, and a target system memory address, wherein the transfer command and target system memory address fully specify a transfer operation to be performed with the block of I/O data.

46. The method according to claim 41, further comprising transmitting a completion value on the bus to indicate completion of the I/O request by the disk drive controller, wherein the completion value is adapted to be combined with completion values transmitted by other disk drive controllers to determine whether processing of the I/O request is complete.

47. The method according to claim 41, wherein transmitting the I/O data on the bus comprises transmitting each packet during an assigned timeslot.