Stream-oriented interconnect for networked computer storage

• US 20040158623A1
• Filed: 11/14/2003
• Published: 08/12/2004
• Est. Priority Date: 05/17/2001
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Decru Incorporated (NetApp, Inc.)
• Inventors: Plotkin, Serge, Avida, Dan
• Granted Patent: US 7,069,375 B2
• Time in Patent Office: Days
• US Class Current: 709/221
• CPC Class Codes: H04L 45/10 Routing in connection-oriented networks, e.g. X.25 or ATM
• CPC Class Codes: H04L 45/46 Cluster building
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling
• CPC Class Codes: H04L 49/30 Peripheral units, e.g. input or output ports
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04Q 11/0005 Switch and router aspects
• CPC Class Codes: H04Q 11/0071 Provisions for the electrical-optical layer interface
• CPC Class Codes: H04Q 2011/0015 using splitting combining
• CPC Class Codes: H04Q 2011/0024 using space switching
• CPC Class Codes: H04Q 2011/003 using switches based on microelectro-mechanical systems [MEMS]
• CPC Class Codes: H04Q 2011/0069 using dedicated optical channels
• CPC Class Codes: H04Q 2011/0084 Quality of service aspects
• CPC Class Codes: Y10S 370/912 Packet communications

Abstract

An apparatus and method for connecting a plurality of computing devices (14), e.g. web servers, database servers, etc., to a plurality of storage devices (16), such as disks, disk arrays, tapes, etc., by using a stream-oriented (circuit oriented) switch (12) that has high throughput, but that requires non-negligible time for reconfiguration is disclosed. An example of such stream-oriented switch is an optical switch (11). The preferred embodiment comprises a plurality of communication ports for connection to servers, and plurality of ports for connection to storage devices. The system decodes the requests from the computing devices and uses this information to create circuits, e.g. optical paths in embodiments where the stream-oriented switch (12) is an optical switch (11), through the stream-oriented switch (12). The system uses these circuits to route traffic between the computing devices (14) and the storage devices (16). Buffering of data and control in the device memory is used to improve overall throughput and reduce the time spent on reconfigurations. An alternative embodiment of the system uses two interconnects, in which all devices are connected to one or both interconnects. One of the interconnects is the stream-oriented switch described above. The second interconnect is a conventional packet switch which can switch traffic on a packet-by-packet basis, e.g. electronic packet switch. In contrast to the stream-oriented switch, the packet switch has much smaller overall throughput, but requires much less time for reconfiguration. The stream-oriented switch is used for switching relatively large data streams, e.g. reading a large file, while the packet switch is used for control an the rest of the data traffic.

Claims

1. An apparatus for interconnecting any of plurality of computing devices and a plurality of storage devices, comprising:

a stream-oriented switch having a high throughput, but that requires non-negligible time for reconfiguration;

said stream-oriented switch further comprising;

a plurality of communication ports for connection to any of said computing devices and a plurality of ports for connection to said storage devices;

means for decoding requests from any of said computing devices and/or storage devices and for using information from said decoded requests to create circuits through said stream-oriented switch;

means for using said circuits to route traffic between any of said computing devices and said storage devices; and

means for buffering data and control information to improve overall throughput and reduce time spent on reconfiguration.

2. The apparatus of claim 1, wherein said stream-oriented switch is an optical switch.

3. The apparatus of claim 1, further comprising:

at least two interconnects;

wherein all devices are connected to at least one of said interconnects;

wherein one of said interconnects comprises said stream-oriented switch;

wherein at least a second interconnect comprises a conventional packet switch which can switch traffic on a packet-by-packet basis; and

wherein said stream-oriented switch is used for switching relatively large data streams, while said packet switch is used for control and the rest of data traffic.

4. The apparatus of claim 3, further comprising any of:

a statistical prediction algorithm for predicting traffic flows; and

a statistical decision algorithm for deciding on how to reconfigure said stream-oriented switch in real time.

5. The apparatus of claim 4, wherein certain selected measures are optimized by configuring said stream-oriented switch for routing said predicted traffic flow, and wherein a remainder of predicted traffic that could not be accommodated by said stream-oriented switch, as well as unpredicted traffic, through said packet switch.

6. The apparatus of claim 4, wherein said statistical prediction algorithm comprises a Hidden Markov Model;

and wherein said statistical decision algorithm comprises a Markov Decision Process.

7. An apparatus for interconnecting any of plurality of computing devices and a plurality of storage devices, comprising:

an optical switch;

a switch control unit for modifying said optical switch configuration, wherein once said optical switch is reconfigured, said switch control unit generates information indicating said reconfiguration, and wherein any of said computing devices and said storage devices commence using circuits that are available through said reconfiguration for communicating;

a scheduling and load balancing unit for determining a performance enhancing packet schedule and routing, for generating information used in determining switch configuration, for distributing a communication load among various available links between any of said computing devices and said storage devices, and for ordering packet flows; and

a system controller for communicating with said computing devices and said storage devices, for gathering requests, for monitoring status information, for distributing a packet schedule, for routing information, and for effecting stream-oriented switch configuration.

8. The apparatus of claim 7, further comprising:

a

packet scheduling unit for sending a transmission schedule to said computing devices and said storage devices, as well as to said system controller.

9. The apparatus of claim 7, said optical switch comprising a multiple fabric switch for connecting said computing devices and said storage devices.

10. The apparatus of claim 9, wherein all said devices are connected to at least one of an electronic packet switch and an optical switch;

wherein said packet switch is used for control and a portion of data traffic;

wherein said optical switch is used for a remainder of said data traffic; and

wherein said multiple fabrics are controlled by said system control unit which receives and issues control and status information via said packet switch, and which is responsible for configuring said optical switch.

11. The apparatus of claim 10, wherein said scheduling and load balancing unit distributes a communication load between available links through said fabrics which connect said computing devices and said storage devices by recommending to said system controller unit a configuration of said optical switch and said packet switch that divides traffic between said fabrics based upon selected criteria.

12. The apparatus of claim 11, wherein said selected criteria comprise any of throughput and delay.

13. The apparatus of claim 7, further comprising any of:

a buffer for aggregating multiple data write requests; and

a data caching and predictive data read-ahead mechanism;

wherein amount of data that can be transferred through said optical switch between reconfigurations is increased.

14. The apparatus of claim 7, further comprising:

a transaction prediction system that predicts future behavior of said computing devices and said storage devices based on prior transactions and known behavior patterns.

15. The apparatus of claim 14, wherein said transaction prediction system builds and continually adapts a statistical model, which optionally comprises a Hidden Markov Model;

wherein said transaction prediction system improves said statistical model from time to time and uses said statistical model to enhance performance of said optical switch by making predictions about future transactions.

16. The apparatus of claim 14, wherein a decision of when and how to reconfigure said optical switch is made using a Markov Decision Process.

17. The apparatus of claim 14, wherein a maintained statistical model is used to predict an expected benefit of each possible reconfiguration decision at every step, and to choose a preferred reconfiguration or reconfiguration sequence.

18. The apparatus of claim 7, further comprising:

means for detecting instances where data are distributed among said storage devices in a less than optimal way; and

means for dynamically rearranging data stored on said storage devices.

19. The apparatus of claim 18, further comprising:

an adaptive statistical access model for analyzing data redistribution advantages and disadvantages and for deciding whether to relocate said data and, if so, where to relocate said data.

20. A method for interconnecting any of plurality of computing devices and a plurality of storage devices, comprising the steps of:

providing a stream-oriented switch having a high throughput, but that requires non-negligible time for reconfiguration;

providing a plurality of communication ports for connection to any of said computing devices and said storage devices;

decoding requests from any of said computing devices and said storage devices, and for using information from said decoded requests to create circuits through said stream-oriented switch;

using said circuits to route traffic between any of said computing devices and said storage devices; and

buffering data and control information to improve overall throughput and reduce time spent on reconfiguration.

21. The method of claim 20, wherein said stream-oriented switch is an optical switch.

22. The method of claim 20, further comprising the step of:

providing at least two interconnects;

wherein all devices are connected to at least one of said interconnects;

wherein one of said interconnects comprises said stream-oriented switch;

wherein at least a second interconnect comprises a conventional packet switch which can switch traffic on a packet-by-packet basis; and

wherein said stream-oriented switch is used for switching relatively large data streams, while said packet switch is used for control and the rest of data traffic.

23. The method of claim 22, further comprising the step of providing any of:

a statistical prediction algorithm for predicting traffic flows; and

a statistical decision algorithm for deciding on how to reconfigure said stream-oriented switch in real time.

24. The method of claim 23, wherein certain selected measures are optimized by configuring said stream-oriented switch for routing said predicted traffic flow, and wherein a remainder of predicted traffic that could not be accommodated by said stream-oriented switch, as well as unpredicted traffic, through said packet switch.

25. The method of claim 23, wherein said statistical prediction algorithm comprises a Hidden Markov Model;

and wherein said statistical decision algorithm comprises a Markov Decision Process.

26. A method for interconnecting any of plurality of computing devices and a plurality of storage devices, comprising the steps of:

providing an optical switch;

providing a switch control unit for modifying said optical switch configuration, wherein once said optical switch is reconfigured, said switch control unit generates information indicating said reconfiguration, and wherein said any of computing devices and said storage devices commence using circuits that are available through said reconfiguration for communicating;

providing a scheduling and load balancing unit for determining a performance enhancing packet schedule and routing, for generating information used in determining switch configuration, for distributing a communication load among various available links between any of said computing devices and said storage devices, and for ordering packet flows; and

providing a system controller for communicating with any of said computing devices and said storage devices, for gathering requests, for monitoring status information, for distributing a packet schedule, for routing information, and for effecting stream-oriented switch configuration.

27. The method of claim 26, further comprising the step of:

providing a packet scheduling unit for sending a transmission schedule to said computing devices and said storage devices, as well as to said system controller.

28. The method of claim 26, said optical switch comprising a multiple fabric switch for connecting said computing devices and said storage devices.

29. The method of claim 28, wherein all said devices are connected to at least one of an electronic packet switch and an optical switch;

wherein said packet switch is used for control and a portion of data traffic;

wherein said optical switch is used for a remainder of said data traffic; and

wherein said multiple fabrics are controlled by said system control unit which receives and issues control and status information via said packet switch, and which is responsible for configuring said optical switch.

30. The method of claim 29, wherein said scheduling and load balancing unit distributes a communication load between available links through said fabrics which connect said computing devices and said storage devices by recommending to said system controller unit a configuration of said optical switch and said packet switch that divides traffic between said fabrics based upon selected criteria.

31. The method of claim 30, wherein said selected criteria comprise any of throughput and delay.

32. The method of claim 26, further comprising the step of providing any of:

a buffer for aggregating multiple data write requests; and

a data caching and predictive data read-ahead mechanism;

wherein amount of data that can be transferred through said optical switch between reconfigurations is increased.

33. The method of claim 26, further comprising the step of:

providing a transaction prediction system that predicts future behavior of said computing devices and said storage devices based on prior transactions and known behavior patterns.

34. The method of claim 33, wherein said transaction prediction system builds and continually adapts a statistical model, which optionally comprises a Hidden Markov Model;

wherein said transaction prediction system improves said statistical model from time to time and uses said statistical model to enhance performance of said optical switch by making predictions about future transactions.

35. The method of claim 33, wherein a decision of when and how to reconfigure said optical switch is made using a Markov Decision Process.

36. The method of claim 33, wherein a maintained statistical model is used to predict an expected benefit of each possible reconfiguration decision at every step, and to choose a preferred reconfiguration or reconfiguration sequence.

37. The method of claim 26, further comprising the step of:

detecting instances where data are distributed among said storage devices in a less than optimal way; and

means for dynamically rearranging data stored on said storage devices.

38. The apparatus of claim 37, further comprising the step of:

providing an adaptive statistical access model for analyzing data redistribution advantages and disadvantages and for deciding whether to relocate said data and, if so, where to relocate said data.