Switching system

• US 20040078419A1
• Filed: 09/03/2003
• Published: 04/22/2004
• Est. Priority Date: 11/02/2001
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Oracle America, Inc. (Oracle Corporation)
• Inventors: Stronge, William, Hall, Howard, Merhar, Milan, Ferrari, Stephen, Cochrane, Christopher
• Granted Patent: US 7,958,199 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1027 Persistence of sessions during load balancing
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/10 Streamlined, light-weight or high-speed protocols, e.g. express transfer protocol [XTP] or byte stream
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures
• CPC Class Codes: H04L 69/164 Adaptation or special uses of UDP protocol
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Disclosed are improved methods, devices and systems for storage management in digital networks.

Claims

1. In a digital processing system comprising at least two NAS servers, a workload distribution method whereby multiple NAS servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

replicating a least a subset of data on a NAS server on at least one other NAS server to create a replicated storage domain,

receiving file system requests from at least one client,

if the file system request is a read request, directing the read request to a server in the replicated storage domain, the server being selected in accordance with a workload distribution function,

if the file system request is a write request,

(1) multicasting the write request to each of the individual servers included in the replicated storage domain, to maintain substantial data synchronization across the replicated storage domain, and

(2) receiving, from at least one of the servers in the replicated storage domain, a response to the received file system request.

2. The method of claim 1 wherein the response is the first successful completion response from any of the servers in the replicated storage domain.

3. The method of claim 1 further comprising tracking each outstanding write request and response from each of the servers in the replicated storage domain, to enable detection of conditions associated with unsynchronized file system states.

4. The method of claim 1 wherein the workload distribution function includes a round robin load balancing function.

5. The method of claim 1 wherein the workload distribution function includes a weighted access-based load balancing function.

6. The method of claim 1 wherein the workload distribution function is file handle-based.

7. The method of claim 6 wherein the weighted access-based workload distribution function includes any of fixed or dynamic server-to-server weighting factors.

8. The method of claim 7 wherein the fixed weighting factors include percentage distribution of requests.

9. The method of claim 7 wherein the dynamic weighting factors include server response time.

10. In a digital processing system comprising at least two NAS servers, a workload distribution method whereby multiple NAS servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

replicating a least a subset of data on a NAS server on at least one other NAS server to create a replicated storage domain,

receiving file system requests from at least one client,

if the file system request is a read request, directing the read request to a server in the replicated storage domain, the server being selected in accordance with a file handle workload distribution function,

the file handle workload distribution function directing requests for previously-accessed files back to the server that previously satisfied the requests;

if the file system request is a write request,

(1) broadcasting the write request to each of the individual servers included in the replicated storage domain, to maintain substantial data synchronization across the replicated storage domain, and

(2) receiving, from at least one of the servers in the replicated storage domain, a response to the received file system request.

11. In a digital processing system comprising at least two NAS servers, a workload distribution method whereby multiple NAS servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

replicating a least a subset of data on a NAS server on at least one other NAS server to create a replicated storage domain;

receiving file system requests from at least one client;

if the file system request is a read request,

directing the read request to a server in the replicated storage domain, the server being selected in accordance with a workload distribution function that;

(1) initially directs requests for a given file to a single server, then

(2) if the rate at which the file is being requested exceeds a threshold limit, designates an additional server to serve the file, and then

(3) distributes successive requests for the file in round robin fashion between the servers designated to serve the file;

if the file system request is a write request,

(1) multicasting the write request to each of the individual servers included in the replicated storage domain, to maintain substantial data synchronization across the replicated storage domain, and

(2) receiving, from at least one of the servers in the replicated storage domain, a response to the received file system request.

12. In a digital processing system comprising at least two NAS servers, a workload distribution method whereby multiple NAS servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

providing a switching system including

(1) a first configurable set of processor elements to process storage resource connection requests,

(2) a second configurable set of processor elements capable of communications with the first configurable set of processor elements to receive, from the first configurable set of processor elements, file system requests, and to route the requests to selected NAS servers; and

(3) a configurable switching fabric interconnected between the first and second sets of processor elements;

designating a workload distribution group including at least two NAS servers;

receiving a file system request from a client via one of the first set of processor elements;

if the file system request is a read request, directing the request to a selected NAS server in the workload distribution group, via the switching system, the server being selected in accordance with a workload distribution function; and

if the file system request is a write request, then (1) multicasting the request to each of the NAS servers in the workload distribution group, via the switching system.

13. The method of claim 12 wherein the workload distribution function comprises examining the contents of a received request to select a NAS server to which to direct the request.

14. The method of claim 12 wherein the workload distribution function includes selecting a NAS server to which to direct the request based on any of load on the server, server availability, and whether the server contains, in its cache, the requested file.

15. The method of claim 14 further comprising:

directing requests for a first file system subdirectory to a first NAS server, and

directing requests for a second file system subdirectory to a second NAS server.

16. In a digital processing system comprising at least two servers, a method of distributing requests to the servers whereby multiple NAS servers are made to appear as a single, virtual, server to an client making a file system request, the method comprising:

receiving, from the client, a packet containing a read request;

using file handle information contained in the request to execute a lookup in a file handle hash table, the file handle hash table containing addresses for servers to which the request may be directed;

obtaining, as a result of the hash table lookup, an address for a server to which the request is to be directed;

updating, in the packet, the destination address with the obtained server address; and

forwarding the packet to the server specified by the obtained server address.

17. In a digital processing system comprising at least two servers, a method of distributing requests to the servers whereby multiple servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

providing a switching system including;

(1) a first configurable set of processor elements to process storage resource connection requests,

(2) a second configurable set of processor elements capable of communications with the first configurable set of processor elements to receive, from the first configurable set of processor elements, file system requests, and to route the requests to selected servers, and

(3) a configurable switching fabric interconnected between the first and second sets of processor elements;

designating a workload distribution group including at least two servers;

receiving a file system request from the client;

if the file system request is a read request, directing the request to a selected server in the workload distribution group, via the switching system, the server being selected in accordance with a workload distribution function,

the workload distribution function including

(1) receiving, from the client, a packet containing the read request,

(2) using file handle information contained in the request to execute a lookup in a file handle hash table, the file handle hash table containing addresses for servers to which the request may be directed,

(3) obtaining, as a result of the hash table lookup, an address for a server to which the request is to be directed,

(4) updating, in the packet, the destination address with the obtained server address, and

(5) forwarding the packet to the server specified by the obtained server address; and

if the file system request is a write request, then multicasting the request to each of the servers in the workload distribution group, via the switching system.

18. In a digital processing system comprising a plurality of servers, a workload distribution method whereby multiple servers are made to appear as a single, virtual, server to a client making a file system request, the method comprising:

providing a switching system including

(1) a first configurable set of processor elements to process storage resource connection requests,

(2) a second configurable set of processor elements capable of communications with the first configurable set of processor elements to receive, from the first configurable set of processor elements, file system requests, and to route the requests to selected servers; and

(3) a configurable switching fabric interconnected between the first and second sets of processor elements;

designating a workload distribution group including at least two servers;

receiving a file system request from a client;

if the file system request is a read request, directing the request to a selected server in the workload distribution group, via the switching system, the server being selected in accordance with a workload distribution function, the workload distribution function including

(1) first directing the request to a selected subset of the servers, via the switching system, the subset being selected in response to examination of the contents of the request, and

(2) then directing the request to one or more selected servers within the selected subset of servers, the servers being selected in accordance with a load balancing function; and

if the file system request is a write request, then (1) multicasting the request to each of the servers in the workload distribution group, via the switching system.

19. The method of claim 18 wherein selecting the subset of servers further comprises

directing requests for a first file system subdirectory to a first subset of servers, and

directing requests for a second file system subdirectory to a second subset of servers.

20. The method of claim 18 wherein entries in the hast table can have more than one server address, so that a file can exist in more than one server cache and be served up by more than one server.

21. The method of claim 20 further comprising using a selection algorithm to determine which of multiple servers to direct a read request to.

22. The method of claim 21 wherein the selection algorithm is round robin.

23. The method of claim 21 wherein frequently accessed files are served by more than one server.

24. The method of claim 18 wherein file handle entries can be placed on multiply linked lists, wherein each list has contains file handle entries for files served by a specific number of servers.

25. The method of claim 24 wherein the multiply linked lists include a list for file handle entries of files having only one server serving them;

a list for file handle entries of files having two servers serving them; and

a list for file handle entries of files having four servers serving them.

26. The method of claim 25 further comprising promotion and demoting file handle entries, the promoting and demoting comprising moving file handle entries from list to list as frequency of access requests for the corresponding files increases or decreases.

27. The method of claim 26 wherein:

file handle entries initially are present on a single server list denoting files served by one server,

when a read request is received, the file handle in the request is used to access the hash table, and if there is no entry for that file handle, a free entry is taken from an entry free list and a single server is designated to serve the file, based on a selected criterion;

if no entries are free, a server is selected and the request directed to the selected server without creating an entry;

when a new entry is created it is added to the hash table and placed at the top of the single server list.

28. The method of claim 27 wherein the criterion is any of least loaded, fastest responding or round robin.

29. The method of claim 27 wherein entries are promoted to a next higher-order list when frequency of access requests for the corresponding file exceeds a threshold.

30. The method of claim 29 wherein entries are demoted to a next lower-order list when frequency of access requests for the corresponding file falls below a threshold.

31. The method of claim 18 further comprising dynamically modifying the contents of the servers by creating additional copies of frequently requested files.

32. In a digital processing system including a plurality of NAS servers, a method for distributing processing of file system requests over the servers, the method including:

configuring a workload distribution function to distribute workload across multiple servers storing identical file data multicast to them, and

configuring a workload distribution function to distribute workload across servers storing unique data.

33. The method of claim 32 wherein in the case of servers storing unique data, the further step of configuring a virtual domain configuration based on file type and file requested, including location in a file system tree.

34. The method of claim 32 wherein, in the case of servers that have identical data, the further step of defining a virtual storage domain and a workload distribution policy.

35. The method of claim 32 wherein the workload distribution is based on any of virtual IP addresses, content and flows.

36. The method of claim 32 wherein the configuring includes defining a virtual storage domain and assigning to the virtual storage domain a virtual IP address (VIP).

37. The method of claim 36 wherein external entities use the VIP to address the NAS servers.

38. The method of claim 32 further comprising

monitoring health of servers across a virtual domain, and,

in the event a detectable issue arises with a server in the virtual domain, directing no further requests to the server having such issues while continuing to load balance across remaining servers in the virtual domain.

39. The method of claim 38 wherein if one server fails, requests can be switched from one server to another transparently, such that an file system session can be maintained if a server fails.

40. The method of claim 32 wherein multiple load balancing algorithms can be defined for a given set of servers, and an order of operations can be defined to select among the multiple algorithms.

41. The method of claim 40 wherein load balancing algorithms are dependent upon any of server loading information, percentage of loading on the server'"'"'s Ethernet, percentage of loading on the server'"'"'s ports, round trip time, response time, packet rate, completion rate, round robin, least connections, or random.

42. A digital processing system comprising:

a plurality of NAS servers,

means for configuring a workload distribution function to distribute workload across multiple servers storing identical file data multicast to them, and

means for configuring a workload distribution function to distribute workload across servers storing unique data.