Method and apparatus for providing scalable services using a packet distribution table

• US 6,667,980 B1
• Filed: 01/10/2000
• Issued: 12/23/2003
• Est. Priority Date: 10/21/1999
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Ramamoorthi, Sankar, Fox, Kevin C., Mani, Mahalingam, Modi, Sohrab F., Oki, Brian M., Mankude, Hariprasad B.
• Primary Examiner(s): Yao, Kwang Bin
• Application Number:
  US09/480,147
• Time in Patent Office: 1,443 Days
• Field of Search: 370/216, 370/217, 370/218, 370/219, 370/220, 370/221, 370/395.1, 370/395.3, 370/395.31, 370/395.32, 370/428, 370/429, 709/201, 709/203, 709/227, 709/228, 709/229, 709/239
• US Class Current: 370/395.32
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

One embodiment of the present invention provides a system that uses a packet distribution table to distribute packets to server nodes in a cluster of nodes that operate in concert to provide at least one service. The system operates by receiving a packet at an interface node in the cluster of nodes. This packet includes a source address specifying a location of a client that the packet originated from, and a destination address specifying a service provided by the cluster of nodes (and possibly a protocol). The system uses the destination address to lookup a packet distribution table. The system then performs a function that maps the source address to an entry in the packet distribution table, and retrieves an identifier specifying a server node from the entry in the packet distribution table. Next, the system forwards the packet to the server node specified by the identifier so that the server node can perform a service for the client. In this way, packets directed to a service specified by a single destination address are distributed across multiple server nodes in a manner specified by the packet distribution table. In one embodiment of the present invention, the system allows the server node to send return communications directly back to the client without forwarding the communications through the interface node. In one embodiment of the present invention, the function includes a hash function that maps different source addresses to different entries in the packet distribution table in a substantially random manner. Note that this hash function always maps a given source address to the same entry in the packet distribution table.

Claims

1. A method for distributing packets to server nodes in a cluster of nodes in order to provide at least one service, the method comprising:

receiving a packet at an interface node in the cluster of nodes, the packet including a source address specifying a location of a client that the packet originated from, and a destination address specifying a service provided by the cluster of nodes;

mapping the source address to an entry in a packet distribution table, the packet distribution table including entries containing identifiers for server nodes in the cluster of nodes, wherein mapping the source address includes using a hash function that maps different source addresses to different entries in the packet distribution table in a substantially random manner, wherein the hash function always maps a given source address to the same entry in the packet distribution table, and wherein a policy for distributing packets between server nodes in the cluster of nodes is enforced by varying a number of entries in the packet distribution table for each server node so that a server node with more entries receives packets more frequently than a server node with fewer entries;

retrieving an identifier specifying a server node from the entry in the packet distribution table; and

forwarding the packet to the server node specified by the identifier so that the server node can perform the service for the client specified by the destination address.

2. The method of claim 1, wherein packets directed to a service specified by a single destination address are distributed across multiple server nodes in a manner specified by the packet distribution table.

3. The method of claim 1, further comprising allowing the server node to send return communications directly to the client without forwarding the return communications through the interface node.

4. The method of claim 1, further comprising using the destination address to select the packet distribution table associated with the service from a plurality of packet distribution tables;

and

wherein each packet distribution table in the plurality of packet distribution tables is associated with at least one service provided by the cluster of nodes.

5. The method of claim 1, further comprising:

periodically sending checkpointing information from a primary packet distribution table (PDT) server to a secondary PDT server so that the secondary PDT server is kept in a consistent state with the primary PDT server; and

if the primary PDT server fails, allowing the secondary PDT server to take over for the primary PDT server.

6. The method of claim 1, further comprising periodically sending checkpointing information from a master packet distribution table (PDT) server to a slave PDT server located on the interface node.

7. The method of claim 1, further comprising:

examining the destination address to determine whether a service specified by the destination address is a scalable service that is provided by multiple server nodes in the cluster of nodes, or a non-scalable service that is provided by a single server node; and

if the service is a non-scalable service, sending the packet to a service instance on the interface node.

8. The method of claim 1, further comprising if a new server from the cluster of nodes becomes available, adding at least one entry for the new server in the packet distribution table.

9. A computer readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for distributing packets to server nodes in a cluster of nodes in order to provide at least one service, the method comprising:

receiving a packet at an interface node in the cluster of nodes, the packet including a source address specifying a location of a client that the packet originated from, and a destination address specifying a service provided by the cluster of nodes;

mapping the source address to an entry in a packet distribution table, the packet distribution table including entries containing identifiers for server nodes in the cluster of nodes, wherein mapping the source address includes using a hash function that maps different source addresses to different entries in the packet distribution table in a substantially random manner, wherein the hash function always maps a given source address to the same entry in the packet distribution table, and wherein a policy for distributing packets between server nodes in the cluster of nodes is enforced by varying a number of entries in the packet distribution table for each server node so that a server node with more entries receives packets more frequently than a server node with fewer entries;

retrieving an identifier specifying a server node from the entry in the packet distribution table; and

forwarding the packet to the server node specified by the identifier so that the server node can perform the service for the client specified by the destination address.

10. The computer-readable storage medium of claim 9, wherein packets directed to a service specified by a single destination address are distributed across multiple server nodes in a manner specified by the packet distribution table.

11. The computer-readable storage medium of claim 9, wherein the method further comprises allowing the server node to send return communications directly to the client without forwarding the return communications through the interface node.

12. The computer-readable storage medium of claim 9, wherein mapping the source address includes using a hash function that maps different source addresses to different entries in the packet distribution table in a substantially random manner, and wherein the hash function always maps a given source address to the same entry in the packet distribution table.

13. The computer-readable storage medium of claim 9, wherein the method further comprises using the destination address to select the packet distribution table associated with the service from a plurality of packet distribution tables;

and

wherein each packet distribution table in the plurality of packet distribution tables is associated with at least one service provided by the cluster of nodes.

14. The computer-readable storage medium of claim 9, wherein the method further comprises:

periodically sending checkpointing information from a primary packet distribution table (PDT) server to a secondary PDT server so that the secondary PDT server is kept in a consistent state with the primary PDT server; and

if the primary PDT server fails, allowing the secondary PDT server to take over for the primary PDT server.

15. The computer-readable storage medium of claim 9, wherein the method further comprises periodically sending checkpointing information from a master packet distribution table (PDT) server to a slave PDT server located on the interface node.

16. An apparatus that distributes packets to server nodes in a cluster of nodes in order to provide at least one service, the apparatus comprising:

a receiving mechanism, within an interface node in the cluster of nodes, that is configured to receive a packet including a source address specifying a location of a client that the packet originated from, and a destination address specifying a service provided by the cluster of nodes;

a server selection mechanism that is configured to,

map the source address to an entry in a packet distribution table, the packet distribution table including entries containing identifiers for server nodes in the cluster of nodes, and to

retrieve an identifier specifying a server node from the entry in the packet distribution table,

wherein the server selection mechanism is configured to map source address using a hash function that maps different source addresses to different entries in the packet distribution table in a substantially random manner, and wherein the hash function always maps a given source address to the same entry in the packet distribution table;

a policy implementing mechanism that is configured to implement a policy for distributing packets between server nodes in the cluster of nodes by varying a number of entries in the packet distribution table for each server node so that a server node with more entries receives packets more frequently than a server node with fewer entries; and

a forwarding mechanism that is configured to forwarding the packet to the server node specified by the identifier so that the server node can perform the service for the client specified by the destination address.

17. The apparatus of claim 16, wherein packets directed to a service specified by a single destination address are distributed across multiple server nodes in a manner specified by the packet distribution table.

18. The apparatus of claim 16, further comprising a communication mechanism on the server node that is configured to communicate directly with the client without forwarding the communications through the interface node.

19. The apparatus of claim 16, further comprising a packet distribution table selection mechanism that is configured to use the destination address to select the packet distribution table associated with the service from a plurality of packet distribution tables;

and

wherein each packet distribution table in the plurality of packet distribution tables is associated with at least one service provided by the cluster of nodes.

20. The apparatus of claim 16, further comprising:

a checkpointing mechanism that is configured to periodically send checkpointing information from a primary packet distribution table (PDT) server to a secondary PDT server so that the secondary PDT server is kept in a consistent state with the primary PDT server; and

a failover mechanism that is configured to allow the secondary PDT server to take over for the primary PDT server if the primary PDT server fails.

21. The apparatus of claim 20, wherein the checkpointing mechanism is additionally configured to periodically send checkpointing information from the primary PDT server to a slave PDT server located on the interface node.