System and method for peer-to-peer accelerated I/O shipping between host bus adapters in clustered computer network
First Claim
1. In a clustered computing network having a plurality of computers connected to a plurality of storage devices through a plurality of adapters, a system for shipping of I/O requests including:
- a communication medium interconnecting said plurality of adapters for peer to peer communication therebetween;
a directory stored in a memory within at least one of said plurality of adapters for storing location information regarding blocks of data stored within said plurality of storage devices; and
managing means operable within said at least one adapter of said plurality of adapters and associated with said communication medium and with said directory for selecting which of said plurality of adapters should process a received I/O request from one of said plurality of computers and for shipping said received I/O request via said communication medium to another of said plurality of adapters in response to the selection, wherein said managing means is operable within said at least one adapter independent of said plurality of computers and is operable to ship a received I/O request to said another of said Plurality of adapters transparently with respect to said plurality of computers.
7 Assignments
0 Petitions
Accused Products
Abstract
The present invention retrieves data across independent computer nodes of a server cluster by providing for I/O shipping of block level requests to peer intelligent host-bus adapters (hereinafter referred to as HBA). This peer-to-peer distribution of block I/O requests is transparent to the host. The HBA has the intelligence to decide whether to satisfy a block I/O request locally or remotely. Each HBA driver utilizes the I2O protocol, which allows peer-to-peer communication independent of the operating system or hardware of the underlying network. In a first embodiment of the present invention, local and remote storage channels, within a node, are supported by a single HBA. In a second embodiment of the present invention, local storage channels, within a node, are supported by one HBA, and the remote storage channel, within a node, is supported by a separate HBA.
-
Citations
31 Claims
-
1. In a clustered computing network having a plurality of computers connected to a plurality of storage devices through a plurality of adapters, a system for shipping of I/O requests including:
-
a communication medium interconnecting said plurality of adapters for peer to peer communication therebetween;
a directory stored in a memory within at least one of said plurality of adapters for storing location information regarding blocks of data stored within said plurality of storage devices; and
managing means operable within said at least one adapter of said plurality of adapters and associated with said communication medium and with said directory for selecting which of said plurality of adapters should process a received I/O request from one of said plurality of computers and for shipping said received I/O request via said communication medium to another of said plurality of adapters in response to the selection, wherein said managing means is operable within said at least one adapter independent of said plurality of computers and is operable to ship a received I/O request to said another of said Plurality of adapters transparently with respect to said plurality of computers. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
updating means within said at least one adapter for revising said directory with information regarding the location of blocks of data within at least one of said plurality of storage devices.
-
-
3. The system of claim 2 wherein said updating means is operable for updating said directory in response to receiving location information from said plurality of adapters regarding blocks of data.
-
4. The system of claim 1 wherein said directory includes:
-
a local portion corresponding to data stored in a portion of said plurality of storage devices connected to said one of said plurality of adapters; and
a remote portion corresponding to a portion of said plurality of storage devices connected to others of said plurality of adapters.
-
-
5. The system of claim 4 wherein said managing means includes:
-
searching means to search said local portion and said remote portion of said directory to select which of said plurality of adapters should process the received I/O request;
first determining means for deciding whether said at least one adapter can process the received I/O request in response to said searching means locating the requested blocks in said local portion; and
second determining means for deciding whether another of said plurality of adapters can process the received I/O request in response to said searching means locating the requested blocks in said remote portion and in response to said first determining means.
-
-
6. The system of claim 1 wherein said storage devices include disk arrays.
-
7. The system of claim 6 wherein said storage devices further include RAID management means to store data and redundancy information.
-
8. The system of claim 1 wherein at least one computer of said plurality of computers creates said directory and distributes said directory to said plurality of adapters.
-
9. The system of claim 8 wherein said system further comprises:
updating means in said at least one computer to update said directory and to distribute the updated directory to said plurality of adapters.
-
10. The system of claim 1 wherein said at least one of said plurality of adapters creates said directory and distributes said directory to others of said plurality of adapters.
-
11. The system of claim 10 wherein said managing means further comprises:
updating means to update said directory and to distribute the updated directory to others of said plurality of adapters.
-
12. A method for shipping of data block I/O requests among a plurality of adapters which connect a plurality of storage devices to a plurality of computers in a clustered computer network, said method comprising the steps of:
-
determining, within said plurality of adapters, which of said plurality of adapters should process a received I/O request from one of said plurality of computers; and
communicating among said plurality of adapters via said communication medium to ship said received I/O request from one of said plurality of adapters to another of said plurality of adapters for processing, wherein the method is operable within said plurality of adapters independent of said plurality of computers and is operable to ship a received I/O request to said another of said plurality of adapters transparently with respect to said plurality of computers. - View Dependent Claims (13, 14, 15, 16, 17)
searching a directory, within said plurality of adapters, that includes location information regarding blocks of data within said plurality of storage devices.
-
-
14. The method of claim 13 wherein said directory includes a local portion corresponding to data stored in a portion of said plurality of storage devices connected to said one of said plurality of adapters and a remote portion corresponding to a portion of said plurality of storage devices connected to others of said plurality of adapters, and wherein the step of searching includes the steps of:
-
searching said local portion to determine whether said one of said plurality of adapters can process said I/O request; and
searching said remote portion to determine whether another of said plurality of adapters can process said I/O request in response to the determination that said one of said plurality of adapters cannot process said I/O request.
-
-
15. The method of claim 12 further including the step of:
creating a directory which maps the location of blocks of data within at least one of said plurality of storage devices.
-
16. The method of claim 15 wherein the step of creating a directory further includes the step of:
distributing said directory to said plurality of adapters.
-
17. The method of claim 16 wherein the step of creating a directory further includes the steps of:
-
coalescing information regarding the location of blocks of data within said plurality of storage devices; and
building said directory using said coalesced information.
-
-
18. A system for I/O shipping of an I/O request comprising:
-
a plurality of host adapters for connecting at least one host computer that generates said I/O request to at least one I/O device;
a communication medium interconnecting said plurality of host adapters, wherein said plurality of host adapters connect said at least one host computer to said communication medium; and
shipping means within said plurality of host adapters and coupled with said communication medium, for shipping said I/O request to an identified one of said plurality of host adapters via said communication medium, wherein said shipping means is operable within said plurality of adapters independent of said at least one host computer and is operable to ship a received I/O request to said identified one of said plurality of adapters transparently with respect to said at least one host computer. - View Dependent Claims (19, 20, 21)
a directory mapping the location of data in I/O devices of said at least one I/O device associated with said plurality of host adapters; and
search means, coupled with said directory, for searching said directory to locate data requested by said I/O request to thereby identify said identified one of said plurality of host adapters capable of processing said I/O request.
-
-
20. The system of claim 19 wherein said directory includes:
-
a local portion corresponding to data stored in a portion of said at least one I/O device connected to a corresponding one of said plurality of host adapters; and
a remote portion corresponding to a portion of said plurality of said at least one I/O device connected to others of said plurality of host adapters.
-
-
21. The system of claim 20 wherein said search means includes:
-
searching means, within at least one adapter of said plurality of host adapters, for searching said local portion and said remote portion of said directory to select which of said plurality of host adapters should process said I/O request;
first determining means for deciding whether said at least one adapter can process the I/O request in response to said searching means locating the requested blocks in said local portion; and
second determining means for deciding whether another of said plurality of host adapters can process the I/O request in response to said searching means locating the requested blocks in said remote portion and in response to said first determining means.
-
-
22. A clustered computer network, comprising:
-
a first node, including;
a first host computer with a first processor bus;
a first bus bridge connected to said first processor bus;
a first host bus adapter;
a first peripheral bus connected to said first bus bridge and said first host bus adapter, wherein said first bus bridge and said first peripheral bus provide communication between said first host computer and said first host bus adapter; and
a first storage device that is accessed by said first host bus adapter;
a second node, remote to said first node, including;
a second host computer with a second processor bus;
a second bus bridge connected to said second processor bus;
a second host bus adapter;
a second peripheral bus connected to said second bus bridge and said second host bus adapter, wherein said second bus bridge bus and said second peripheral bus provide communication between said second host computer and said second host bus adapter; and
a second storage device that is accessed by said second host bus adapter; and
an adapter communication medium that provides communication between said first and second host bus adapters, wherein said first host bus adapter routes I/O requests from said first host computer for said first storage device to said first storage device and ships I/O requests from said first host computer for said second storage device across said adapter communication medium to said second host bus adapter independent of said first host computer, thereby relieving said first host computer of determining which of said storage devices said I/O requests from said first host computer are directed to, and said second host bus adapter routes I/O requests from said second host computer for said second storage device to said second storage device and ships I/O requests from said second host computer for said first storage device across said adapter communication medium to said first host bus adapter independent of said second host computer, thereby relieving said second host computer of determining which of said storage devices said I/O requests from said second host computer are directed to wherein the shipping of I/O requests is transparent with respect to said first host computer and said second host computer. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31)
said first host bus adapter permanently physically owns said first storage device for I/O requests in said network for said first storage device and permanently logically shares said second storage device for I/O requests from said first host computer for said second storage device; - and
said second host bus adapter permanently physically owns said second storage device for I/O requests in said network for said second storage device and permanently logically shares said first storage device for I/O requests from said second host computer for said first storage device.
-
-
28. The network of claim 22 wherein
said first host computer includes first processors, said first processor bus provides communication between said first processors, and said first peripheral bus provides communication between said first processors and said first host bus adapter without providing communication between said first processors; - and
said second host computer includes second processors, said second processor bus provides communication between said second processors, and said second peripheral bus provides communication between said second processors and said second host bus adapter without providing communication between said second processors.
- and
-
29. The network of claim 22 wherein
said first host bus adapter consists essentially of a first peripheral bus interface for interfacing with said first peripheral bus, a first internal bus connected to said first peripheral bus interface, a first internal bus bridge connected to said first internal bus, a first internal processor bus connected to said first internal bus bridge, a first central processing unit and a first memory connected to said first internal processor bus, a first storage device interface connected said first internal bus for interfacing with said first storage device, and a first adapter communication medium interface connected to said first internal bus for interfacing with said adapter communication medium; - and
said second host bus adapter consists essentially of a second peripheral bus interface for interfacing with said second peripheral bus, a second internal bus connected to said second peripheral bus interface, a second internal bus bridge connected to said second internal bus, a second internal processor bus connected to said second internal bus bridge, a second central processing unit and a second memory connected to said second internal processor bus, a second storage device interface connected said second internal bus for interfacing with said second storage device, and a second adapter communication medium interface connected to said second internal bus for interfacing with said adapter communication medium.
- and
-
30. The network of claim 22 wherein
said first host bus adapter includes a first routing host bus adapter for routing I/O requests for said first data storage device to said first data storage device, and includes a first shipping host bus adapter for shipping I/O requests from said first host computer for said second storage device to said second host bus adapter via said adapter communication medium and for transferring 1/O requests from said second host bus adapter via said adapter communication medium to said first routing host bus adapter for transfer to said first storage device; - and
said second host bus adapter includes a second routing host bus adapter for routing I/O requests for said second data storage device to said second data storage device, and includes a second shipping host bus adapter for shipping 1/O requests from said second host computer for said first storage device to said first host bus adapter via said adapter communication medium and for transferring I/O requests from said first host bus adapter via said adapter communication medium to said second routing host bus adapter for transfer to said second storage device.
- and
-
31. The network of claim 30 wherein
said first routing host bus adapter consists essentially of a first routing peripheral bus interface for interfacing with said first peripheral bus, a first routing bus connected to said first routing peripheral bus interface, a first routing bus bridge connected to said first routing bus, a first routing processor bus connected to said first routing bus bridge, a first routing central processing unit and a first routing memory connected to said first routing processor bus, and a first routing storage device interface connected to said first routing bus for interfacing with said first storage device; -
said first shipping host bus adapter consists essentially of a first shipping peripheral bus interface for interfacing with said first peripheral bus, a first shipping bus connected to said first shipping peripheral bus interface, a first shipping bus bridge connected to said first shipping bus, a first shipping processor bus connected to said first shipping bus bridge, a first shipping central processing unit and a first shipping memory connected to said first shipping processor bus, and a first shipping adapter communication medium interface connected to said first shipping bus for interfacing with said adapter communication medium;
said second routing host bus adapter consists essentially of a second routing peripheral bus interface for interfacing with said second peripheral bus, a second routing bus connected to said second routing peripheral bus interface, a second routing bus bridge connected to said second shipping bus, a second routing processor bus connected to said second routing bus bridge, a second routing central processing unit and a second routing memory connected to said second routing processor bus, and a second routing storage device interface connected to said second routing bus for interfacing with said second storage device; and
said second shipping host bus adapter consists essentially of a second shipping peripheral bus interface for interfacing with said second peripheral bus, a second shipping bus connected to said second shipping peripheral bus interface, a second shipping bus bridge connected to said second shipping bus, a second shipping processor bus connected to said second shipping bus bridge, a second shipping central processing unit and a second shipping memory connected to said second shipping processor bus, and a second shipping adapter communication medium interface connected to said second shipping bus for interfacing with said adapter communication medium.
-
Specification