Extensible computing system

• US 6,779,016 B1
• Filed: 02/11/2000
• Issued: 08/17/2004
• Est. Priority Date: 08/23/1999
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Terraspring, Inc. (Oracle Corporation)
• Inventors: Patterson, Martin, Markson, Tom, Aziz, Ashar
• Primary Examiner(s): EL HADY, NABIL M
• Application Number:
  US09/502,170
• Time in Patent Office: 1,649 Days
• Field of Search: 709/201, 709/202, 709/205, 709/226, 709/107, 709/104, 709/105, 709/208, 709/209
• US Class Current: 709/201
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 20/102 Bill distribution or payments
• CPC Class Codes: H04L 12/4679 Arrangements for the registration or de-registration of VLAN attribute values, e.g. VLAN identifiers, port VLAN membership
• CPC Class Codes: H04L 49/205 Quality of Service based
• CPC Class Codes: H04L 49/351 for local area network [LAN], e.g. Ethernet switches
• CPC Class Codes: H04L 49/354 for supporting virtual local area networks [VLAN]
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and apparatus providing a dynamically sized, highly scalable and available server farm are disclosed. A Virtual Server Farm (VSF) is created out of a wide scale computing fabric (“Computing Grid”) which is physically constructed once and then logically divided up into VSFs for various organizations on demand. Each organization retains independent administrative control of a VSF. A VSF is dynamically firewalled within the Computing Grid. A allocation and control of the elements in the VSF is performed by a Control Plane connected to all computing, networking, and storage elements in the computing grid through special control ports. The internal topology of each VSF is under control of the Control Plane. No physical rewiring is necessary in order to construct VSFs in many different configurations, including single-tier Web server or multi-tier Web-server, application server, database server configurations. Each tier of a multi-tier VSF (e.g. Web server tier, application server tier, database server tier, etc) can be dynamically sized based on the load on the servers in that particular tier. Storage devices may include a plurality of pre-defined logical blueprints that are associated with roles that may be assumed by the computing grid elements. Initially, no computing element is dedicated to any particular role or task such as Web server, application server, database server, etc. The role of each computing element is acquired from one of a plurality of pre-defined, stored blueprints, each of which defines a boot image for the computing elements that are associated with that role.

Claims

1. A data processing method, comprising the steps of:

selecting, from among a set of processors, a subset of the processors;

generating instructions that cause a virtual local area network switch to logically couple together each of the processors in the subset of the processors;

selecting, from among a set of storage devices, a subset of the storage devices; and

generating instructions that cause a second switching system to logically couple together each of the storage devices in the subset of the storage devices to one another and to the subset of processors.

2. A method as recited in claim 1, wherein selecting, from among a set of processors, a subset of the processors comprises the steps of selecting subset of central processing units from among a pool of available central processing units.

3. A method as recited in claim 2, further comprising the steps of:

selecting, from among the subset of processors, a particular processor to be removed from the subset;

generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the subset of the processors;

logically placing the particular processor in the pool of available processors.

4. A method as recited in claim 1,

wherein the step of selecting, from among a set of processors, a subset of the processors comprises the step of selecting subset of central processing units from among a pool of available central processing units, and

wherein each of the central processing units includes first and second network interfaces configured for receiving instructions from a virtual local area network switch and a storage interface configured for connection to the subset of the storage devices through a storage area network switch.

5. A method as recited in claim 1, wherein the step of selecting, from among a set of storage devices, a subset of the storage devices comprises the steps of selecting the subset of storage devices from among a pool of available storage devices.

6. A method as recited in claim 1,

wherein the step of selecting, from among a set of storage devices, a subset of the storage devices comprises the steps of selecting the subset of storage devices from among a pool of available storage devices;

and

wherein each of the storage devices includes a switching interface configured for receiving instructions from a virtual storage area network switch.

7. A method as recited in claim 1, wherein generating instructions that cause a second switching system to logically couple together each of the storage devices comprises the step of generating instructions to a virtual storage area network switch coupled to the storage devices that cause the virtual storage area network switch to logically couple together the storage devices in the subset.

8. A method as recited in claim 1, wherein the step of selecting, from among a set of processors, a subset of the processors is carried out by a controller that is coupled to and controls the virtual local area network switch the second switching system, and the set of processors.

9. A method as recited in claim 8, further comprising the step of logically assigning one or more ports of the second switching system, which ports are associated with one of the storage devices that is assigned exclusively to the controller, to a private storage area network zone for use by the controller.

10. A method as recited in claim 1, further comprising the steps of:

creating a first virtual server farm for use in a first data processing operation by selecting, from among the set of processors, a first subset of the processors to process the first data processing operation;

generating instructions that cause the virtual local area network switch to logically couple together each of the processors in the first subset of the processors in a first virtual local area network;

selecting, from among the set of storage devices, a first subset of the storage devices to store information for the first data processing problem; and

generating instructions that cause a second switching system to logically couple together each of the storage devices in the first subset of the storage devices to one another and to the first subset of processors in a first storage area network zone;

creating a second virtual server farm for use in a second data processing operation by selecting, from among the set of processors, a second subset of the processors to process the second data processing operation;

generating instructions that cause the virtual local area network switch to logically couple together each of the processors in the second subset of the processors in a second virtual local area network;

selecting, from among the set of storage devices, a second subset of the storage devices to store information for the second data processing problem; and

generating instructions that cause the second switching system to logically couple together each of the storage devices in the second subset of the storage devices to one another and to the second subset of processors in a second storage area network zone;

wherein the instructions securely isolate the first subset of processors from the second subset of processors and from the second subset of storage devices.

11. A method as recited in claim 10, further comprising the steps of:

selecting, from among the first subset of processors, a particular processor to be removed from the first subset;

generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the first subset of the processors;

generating instructions that cause the virtual local area network switch to logically add the particular processor to the second subset of the processors.

12. A method as recited in claim 10, further comprising the step of dynamically logically adding an additional processor to the subset of processors in response to real-time loading experienced by the subset of processors, by:

adding to the virtual local area network those interface ports of the virtual local area network switch that are associated with the additional processor;

adding a storage area port of the additional processor to the storage area network zone.

13. A method as recited in claim 10, further comprising the step of dynamically logically removing one processor from the subset of processors in response to real-time loading experienced by the subset of processors, by:

removing from the virtual local area network those interface ports of the virtual local area network switch that are associated with the additional processor;

removing a storage area port of the one processor from the storage area network zone.

14. A method as recited in claim 1, further comprising the steps of:

selecting, from among the set of processors, an additional processor;

generating instructions that cause the virtual local area network switch to logically couple the additional processor to the processors in the subset of the processors.

15. A method as recited in claim 1, further comprising the steps of:

selecting, from among the subset of processors, a particular processor to be removed from the subset;

generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the subset of the processors.

16. A method as recited in claim 1, further comprising the step of initially assigning all the processors to an idle pool of available processors.

17. A method as recited in claim 1, further comprising the step of dynamically logically adding or removing one or more processors to or from the subset of processors in response to real-time loading experienced by the subset of processors.

18. A method as recited in claim 17, further comprising the steps of:

logically adding an additional processor to the subset of processors in response to real-time loading experienced by the subset of processors;

re-starting the additional processor;

loading into the additional processor, an operating system image obtained from a pre-defined location of one of the storage devices in the subset of storage devices.

19. A method as recited in claim 1, further comprising the step of dynamically logically adding or removing one or more storage devices to or from the subset of storage devices in response to real-time loading experienced by the subset of storage devices.

20. A method as recited in claim 1, further comprising coupling a interface of the virtual local area network switch to an external network whereby the subset of processors is responsive to requests from the external network.

21. A method as recited in claim 1, further comprising the steps of:

logically coupling a load balancer to the processors in the subset of processors;

instructing the load balancer to load balance processing that is carried out by the processors in the subset of processors.

22. A method as recited in claim 1, further comprising the step of

associating each of the processors in the subset of processors with a stored, pre-defined blueprint that associates one of a plurality of processing roles with a boot image of one of the storage devices in the subset of storage devices;

generating instructions that cause each of the processors in the subset of processors to load and execute the boot image from the storage device associated with the processing role of that processor.

23. A method of data processing comprising the steps of:

creating a first virtual server farm for use in a first data processing operation by selecting, from among a set of processors, a first subset of the processors to process the first data processing operation;

generating instructions that cause a virtual local area network switch to logically couple together each of the processors in the first subset of the processors in a first virtual local area network;

selecting, from among a set of storage devices, a first subset of the storage devices to store information for the first data processing operation; and

generating instructions that cause a second switching system to logically couple together each of the storage devices in the first subset of the storage devices to one another and to the first subset of processors in a first storage area network zone;

creating a second virtual server farm for use in a second data processing operation by selecting, from among the set of processors, a second subset of the processors to process the second data processing operation;

generating instructions that cause the virtual local area network switch to logically couple together each of the processors in the second subset of the processors in a second virtual local area network;

selecting, from among the set of storage devices, a second subset of the storage devices to store information for the second data processing operation; and

generating instructions that cause the second switching system to logically couple together each of the storage devices in the second subset of the storage devices to one another and to the second subset of processors in a second storage area network zone;

wherein the instructions securely isolate the first subset of processors from the second subset of processors and from the second subset of storage devices.

28. A data processing system as recited in claim 23, further comprising:

means for selecting, from among the first subset of processors, a particular processor to be removed from the first subset;

means for generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the first subset of the processors;

means for generating instructions that cause the virtual local area network switch to logically add the particular processor to the second subset of the processors.

24. A data processing system, comprising:

a plurality of processors;

a virtual local area network switch coupled to the plurality of processors;

a plurality of storage devices;

a second switching system coupled to the plurality of storage devices;

a controller coupled to the virtual local area network switch and the second switching system;

means in the controller for selecting, from among the plurality of processors, a subset of the processors;

means in the controller for generating instructions that cause the virtual local area network switch to logically couple together each of the processors in the subset of processors;

means in the controller for selecting, from among the plurality of storage devices, a subset of the storage devices; and

means in the controller for generating instructions that cause the second switching system to logically couple together each of the storage devices in the subset of the storage devices to one another and to the subset of processors.

25. A data processing system as recited in claim 24, wherein the controller further comprises means for selecting, from among a set of processors, a subset of the processors by selecting subset of central processing units from among a pool of available central processing units.

26. A data processing system as recited in claim 25, further comprising:

means for selecting, from among the subset of processors, a particular processor to be removed from the subset;

means for generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the subset of the processors;

means for logically placing the particular processor in the pool of available processors.

27. A data processing system as recited in claim 24,

wherein the means for selecting a subset of the processors comprises means for selecting subset of central processing units from among a pool of available central processing units, and

wherein each of the central processing units includes first and second network interfaces configured for receiving instructions from a virtual local area network switch and a storage interface configured for connection to the subset of the storage devices through a storage area network switch.

29. A data processing system as recited in claim 24, wherein the means for selecting a subset of the storage devices comprises means for selecting the subset of storage devices from among a pool of available storage devices.

30. A data processing system as recited in claim 24,

wherein the means for selecting a subset of the storage devices comprises means for selecting the subset of storage devices from among a pool of available storage devices;

and

wherein each of the storage devices includes a switching interface configured for receiving instructions from a virtual storage area network switch.

31. A data processing system as recited in claim 24, wherein the means for generating instructions that cause a second switching system to logically couple together each of the storage devices comprises means for generating instructions to a virtual storage area network switch coupled to the storage devices that cause the virtual storage area network switch to logically couple together the storage devices in the subset.

32. A data processing system as recited in claim 24, further comprising:

a first virtual server farm for use in a first data processing operation that is created by selecting, from among the set of processors, a first subset of the processors to process the first data processing operation;

generating instructions that cause the virtual local area network switch to logically couple together each of the processors in the first subset of the processors in a first virtual local area network;

selecting, from among the set of storage devices, a first subset of the storage devices to store information for the first data processing problem; and

generating instructions that cause a second switching system to logically couple together each of the storage devices in the first subset of the storage devices to one another and to the first subset of processors in a first storage area network zone;

a second virtual server farm for use in a second data processing operation that is created by selecting, from among the set of processors, a second subset of the processors to process the second data processing operation;

generating instructions that cause the system virtual local area network switch to logically couple together each of the processors in the second subset of the processors in a second virtual local area network;

selecting, from among the set of storage devices, a second subset of the storage devices to store information for the second data processing problem; and

generating instructions that cause the second switching system to logically couple together each of the storage devices in the second subset of the storage devices to one another and to the second subset of processors in a second storage area network zone;

wherein the instructions securely isolate the first subset of processors from the second subset of processors and from the second subset of storage devices.

33. A data processing system as recited in claim 24, further comprising:

means for selecting, from among the set of processors, an additional processor;

means for generating instructions that cause the virtual local area network switch to logically couple the additional processor to the processors in the subset of the processors.

34. A data processing system as recited in claim 24, further comprising:

means for selecting, from among the subset of processors, a particular processor to be removed from the subset;

means for generating instructions that cause the virtual local area network switch to logically decouple the particular processor from the subset of the processors.

35. A data processing system as recited in claim 24, further comprising means for initially assigning all the processors to an idle pool of available processors.

36. A data processing system as recited in claim 24, further comprising means for dynamically logically adding or removing one or more processors to or from the subset of processors in response to real-time loading experienced by the subset of processors.

37. A data processing system as recited in claim 24, further comprising means for dynamically logically adding or removing one or more storage devices to or from the subset of storage devices in response to real-time loading experienced by the subset of storage devices.

38. A data processing system as recited in claim 24, wherein the controller further comprises a load monitor that generates information representing processing load of each of the processors in the subset.

39. A data processing system as recited in claim 24, wherein the virtual local area network switch comprises a virtual local area network switch having non-forgeable port identifiers.

40. A data processing system as recited in claim 24, wherein the second switching system comprises a storage area network switch, wherein the subset of storage devices is logically organized in a storage area network zone and wherein the storage area network switch permits access to the subset of storage devices only to those processors in the subset of processors.

41. A data processing system as recited in claim 24, wherein the second switching system comprises a storage area network switch, wherein the subset of storage devices is logically organized in a storage area network zone, and wherein the storage area network switch permits access to the subset of storage devices only to those processors in the subset of processors using one or more Fibre Channel switches.

42. A data processing system as recited in claim 24, wherein the second switching system comprises a storage area network switch, wherein the subset of storage devices is logically organized in a storage area network zone, and wherein the subset of processors is coupled to an external network through a load balancer or firewall.

43. A data processing system as recited in claim 24, further comprising a plurality of controllers logically interconnected in a subnetwork that includes control ports of the virtual local area network switch and that includes control ports of the second switching system.

44. A data processing system as recited in claim 43, further comprising the step of logically assigning one or more ports of the second switching system, which ports are associated with one of the storage devices that is assigned exclusively to the controller, to a private storage area network zone for use by the controller.

45. A data processing system as recited in claim 43, further comprising an agent controller configured to periodically poll each processor in the subset of processors for receiving real-time information representing then-current processor load, network load, or storage load, and configured to communicate the information to each of the controllers.

46. A data processing system as recited in claim 24, further comprising:

a plurality of stored, pre-defined blueprints, each of which associates one of a plurality of processing roles with a boot image of one of the storage devices in the subset of storage devices;

means for associating each of the processors in the subset of processors with one of the blueprints and for causing each of the processors in the subset of processors to load and execute the boot image from the storage device associated with the processing role of that processor.

47. A virtual computing system, comprising:

a plurality of processors logically organized in a plurality of subsets of the processors, each of the subsets logically organized as one of a plurality of virtual local area networks;

a plurality of storage devices logically organized in a storage area network coupled to the plurality of virtual local area networks for storing data and instructions used by one of the subsets; and

a control plane coupled to the plurality of virtual local area networks and to the storage area network, and configured to dynamically add or remove processors from the subsets and to dynamically add or remove storage devices from the storage area network in response to changing processing load conditions and changing storage needs that occur in real time.

48. A virtual computing system as recited in claim 47, further comprising:

a plurality of stored, pre-defined blueprints, each of which associates one of a plurality of processing roles with a boot image of one of the storage devices in the subset of storage devices;

means for associating each of the processors one of the subsets of processors with one of the blueprints and for causing each of the processors in that subset of processors to load and execute the boot image from the storage device associated with the processing role of that processor.

49. A computer-readable medium carrying one or more sequences of instructions for data processing using an extensible computing system, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

selecting, from among a set of processors, a subset of the processors;

generating instructions that cause a virtual local area network switch to logically couple together each of the processors in the subset of the processors;

selecting, from among a set of storage devices, a subset of the storage devices;

generating instructions that cause a second switching system to logically couple together each of the storage devices in the subset of the storage devices to one another and to the subset of processors.