Method of operating a data storage system having plural data pipes

• US 8,090,789 B1
• Filed: 06/28/2007
• Issued: 01/03/2012
• Est. Priority Date: 06/28/2007
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: O'Shea, John, Kinne, Jeffrey, Sgrosso, Michael, McClure, Steven T.
• Primary Examiner(s): Avellino, Joseph E.
• Assistant Examiner(s): BARON, JAMES T
• Application Number:
  US11/769,745
• Time in Patent Office: 1,650 Days
• Field of Search: 709/211
• US Class Current: 709/211
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A data storage system having protocol controller for converting packets between PCIE format used by a storage processor and Rapid IO format used by a packet switching network. The controller includes a PCIE end point for transferring atomic operation (DSA) requests, a data pipe section having a plurality of data pipes for passing user data; and a message engine section for passing messages among the plurality of storage processors. An acceleration path controller bypasses a DSA buffer in the absence of congestion on the network. Packets fed to the PCIE end point include an address portion having code indicating an atomic operation. An encoder converts the code from a PCIE format into the same atomic operation in SRIO format. Each one of a plurality of CPUs is adapted to perform a second DSA request during execution of a first DSA request.

Claims

1. A method for transferring data between a host computer/server and a bank of disk drives through a system interface, such system interface having a plurality of storage processors, one portion of the storage processors having a user data port coupled to the host computer/server and another portion of the storage processors having a user data port coupled to the bank of disk drives, the method comprising:

passing packets of user data between the plurality of storage processors via a packet switching network coupled to the plurality of storage processors, each one of the plurality of storage processors comprising;

a CPU section; and

a data pipe section coupled between the user data port and the packet switching network, such data pipe section comprising;

a plurality of data pipes, wherein the data pipes are initially void of any apriori assignment of priority; and

wherein the packets of user data at the user data port passes through the data pipes to the packet switching network;

placing the descriptors on one or more request rings each one of the descriptors being associated with a corresponding one of a plurality of user data I/O transfers, and

wherein the ring manager;

selects one of the request rings to service next using a prioritized scheduling algorithm;

retrieves one of the descriptors from the selected one of the request rings;

determines which one or ones of the plurality of data pipes is available to process the selected descriptor;

selects the determined available one of the plurality of data pipes for a corresponding one of the plurality of user data I/O transfers;

wherein the selecting determined available one of the plurality of data pipes for a corresponding one of the plurality of user data I/O transfers comprises selecting the determined available one of the plurality of data pipes for a corresponding one of the plurality of user data I/O transfers; and

programming the selected one of the data pipes in accordance with the retrieved descriptors to control the flow of the user data associated with the corresponding user data I/O transfer passing through such selected one of the data pipes.

2. The method recited in claim 1 wherein the ring manager manages transfer of user data passing through the selected data pipe and reprograms such selected one of the data pipes as required to complete the user data transfer.

3. The method recited in claim 2 wherein when the user data transfer is complete, the ring manager collects status of the user data I/O transfer and places a descriptor on a response ring.

11. The method recited in claim 1 wherein one type of the descriptors is a plurality of scatter gather lists (SGLs), wherein each SGL is a linked data structure list having at a head entry of the list a request descriptor and remaining entries linked via pointers contained within each SGL entry, wherein each one of the SGLs is associated with a corresponding one of the data pipes, each one of the SGLs providing control for passing user data through the data pipe associated with such one SGLs, and wherein entries in any one of the SGLs are executed by the data pipe associated therewith in order and wherein there is an absence of ordering in executing entries on different SGLs.

14. The method recited in claim 1 including assigning work to available data pipes in a successive fashion until all data pipes are busy.

4. A method for transferring data between a host computer/server and a bank of disk drives through a system interface, such system interface having a plurality of storage processors, one portion of the storage processors having a user data port coupled to the host computer/server and another portion of the storage processors having a user data port coupled to the bank of disk drives, the method comprising:

passing packets of user data between the plurality storage processors via a packet switching network coupled to the plurality of storage processors, each one of the plurality of storage processors comprising;

a CPU section;

a ring manager; and

a plurality of data pipes with non-pre-assigned priorities coupled between the user data port and the packet switching network, each one of such data pipes comprising;

a data pipe manager controlling such one of the data pipes in response to descriptors produced by the CPU section and passed to the ring manager, the user data at the user data port passing through such one of the data pipes, such one of the data pipes being controlled by the data pipe manager in response to the descriptors passed to the ring manager;

operating the ring manager to;

select one of the request rings to service next using a prioritized scheduling algorithm;

retrieve one of the descriptors from the selected one of the request rings;

determine which one or ones of the plurality of data pipes is available to process the selected descriptor;

select the determined available one of the plurality of data pipes with non-pre-assigned priorities for a corresponding one of the plurality of user data I/O transfers;

pass the retrieved one of the descriptors associated with the corresponding one of the user data I/O transfers to the data pipe manager of the selected one of the data pipes; and

wherein the data pipe manger of the selected one of the data pipes examines such one of the retrieved descriptors and controls the selected one of the data pipes in accordance with the one of the retrieved descriptors.

5. The method recited in claim 4 wherein the data pipe manager manages transfer of user data passing through the selected data pipe.

6. The method recited in claim 5 wherein when the user data I/O transfer is complete, the ring manager collects status of the user data I/O transfer and places a descriptor on a response ring.

7. The method recited in claim 4 wherein the data pipe manager include registers to manage the transfer of user data passing through the selected data pipe and wherein the ring manager reprograms the registers in such selected one of the data pipes as required to complete the user data transfer.

8. The method recited in claim 4 including a router and wherein the method includes passing the user data I/O transfers passing through the plurality of data pipes to the packet switching network through the router.

9. The method recited in claim 4 wherein each one of the data pipes operates in response to the data pipe manager therein independently of another one of the data pipes responding to the data pipe manager therein.

10. The method recited in claim 4 wherein one type of the descriptors is a plurality of scatter gather lists (SGLs), wherein each SGL is a linked data structure list having at a head entry of the list a request descriptor and remaining entries linked via pointers contained within each SGL entry, wherein each one of the SGLs is associated with a corresponding one of the data pipes, each one of the SGLs providing control for passing user data through the data pipe associated with such one SGLs, and wherein entries in any one of the SGLs are executed by the data pipe associated therewith in order and wherein there is an absence of ordering in executing entries on different SGLs.

12. A method for transferring data between a host computer/server and a bank of disk drives through a system interface, such system interface having a plurality of storage processors, one portion of the storage processors having a user data port coupled to the host computer/server and another portion of the storage processors having a user data port coupled to the bank of disk drives, the method comprising:

passing packets of user data between the plurality storage processors via a packet switching network coupled to the plurality of storage processors, each one of the plurality of storage processors comprising;

a CPU section; and

a data pipe section coupled between the user data port and the packet switching network, such data pipe section comprising;

a plurality of data pipes with non-pre-assigned priorities,

a ring manager,

wherein the packets of user data at the user data port passes through the data pipes to the packet switching network;

placing the descriptors on one or more request rings, each one of the descriptors being associated with a corresponding one of a plurality of user data I/O transfers;

determining which one or ones of the plurality of data pipes is available to process a selected one of the descriptors;

selecting the determined available one of the plurality of data pipes with non-pre-assigned priorities for a corresponding one of the plurality of user data I/O transfers;

configuring the selected one of the data pipes in accordance with the one of the selected descriptors; and

controlling the flow of the user data associated with the user data I/O transfer passing through such selected one of the data pipes in accordance with the selected descriptor.

13. A method for transferring data between a host computer/server and a bank of disk drives through a system interface, such system interface having a plurality of storage processors, one portion of the storage processors having a user data port coupled to the host computer/server and another portion of the storage processors having a user data port coupled to the bank of disk drives, the method comprising:

passing packets of user data between the plurality storage processors via a packet switching network coupled to the plurality of storage processors, each one of the plurality of storage processors comprising;

a CPU section; and

a data pipe section coupled between the user data port and the packet switching network, such data pipe section comprising;

a plurality of data pipes with non-pre-assigned priorities, and

wherein the packets of user data at the user data port passes through the data pipes to the packet switching network;

placing the descriptors on one or more request rings, each one of the descriptors being associated with a corresponding one of a plurality of user data I/O transfers;

determining which one or ones of the plurality of data pipes is available to process a selected one of the descriptors;

selecting the determined available one of the plurality of the non-pre-assigned prioritized data pipes for a corresponding one of the plurality of user data I/O;

configuring the selected one of the data pipes in accordance with the one of the selected descriptor; and

controlling the flow of the user data associated with the user data I/O transfer passing through such selected one of the data pipes in accordance with the selected descriptor.