Method and apparatus for controlling an extensible computing system

• US 6,597,956 B1
• Filed: 08/02/2000
• Issued: 07/22/2003
• Est. Priority Date: 08/23/1999
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Terraspring, Inc. (Oracle Corporation)
• Inventors: Gray, Mark, Aziz, Ashar, Patterson, Martin, Markson, Tom
• Primary Examiner(s): Patel, Ramesh
• Assistant Examiner(s): PHAM, THOMAS K
• Application Number:
  US09/630,440
• Time in Patent Office: 1,084 Days
• Field of Search: 700/3, 700/99, 700/101, 709/104, 709/105, 709/208, 709/209, 712/31
• US Class Current: 700/3
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 9/45504 Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A Virtual Server Farm (VSF) is created out of a wide scale computing fabric (“Computing Grid”) which is physically constructed once and then logically divided up into VSFs for various organizations on demand. Allocation and control of the elements in the VSF is performed by a control plane connected to all computing, networking, and storage elements in the computing grid through special control ports. The control plane is comprised of a control mechanism hierarchy that includes one or more master control process mechanisms communicatively coupled to one or more slave control process mechanisms. The one or more master control process mechanisms instruct the slave control process mechanisms to establish VSFs by selecting subsets of processing and storage resources.

Claims

1. A control apparatus comprising:

a master control mechanism; and

one or more slave control mechanisms communicatively coupled to the master control mechanism and being configured to, in response to one or more instructions from the master control mechanism, establish a first logical that contains a first subset of processing resources and a first subset of storage resources and is capable of operating independent of the master control mechanism and the one or more slave control mechanisms, wherein the first logical computing entity is established by;

selecting the first subset of processing resources from a set of processing resources,

selecting the first subset of storage resources from a set of storage resources, and

causing the first subset of processing resources to be communicatively coupled to the first subset of storage resources.

2. A control apparatus as recited in claim 1, wherein the master control mechanism is a master control process executing on one or more processors and the one or more slave control mechanisms are one or more slave processes executing on the one or more processors.

3. A control apparatus as recited in claim 1, wherein the master control mechanism is one or more master processors and the one or more slave control mechanisms are one or more slave processors.

4. A control apparatus as recited in claim 1, wherein the master control mechanism is configured to, based upon slave control process mechanism loading, dynamically reassign control, between the one or more slave control mechanisms, of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources.

5. A control apparatus as recited in claim 1, wherein the master control mechanism is configured to, based upon slave control process mechanism loading, dynamically allocate one or more additional slave control mechanisms, and assign control of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources to the one or more additional slave control mechanisms.

6. A control apparatus as recited in claim 1, wherein the master control mechanism is configured to, based upon slave control process mechanism loading, reassign control to one or more other slave control mechanisms from the one or more slave control mechanisms of one or more particular processing resources from the subset of processing resources and one or more particular storage resources from the subset of storage resources that were previously assigned to one or more particular slave control mechanisms from the one or more slave control mechanisms, and

dynamically de-allocate the one or more particular slave control mechanisms.

7. A control apparatus as recited in claim 1, wherein the master control mechanism is configured to:

determine a status of the one or more slave control mechanisms,

if one or more particular slave control mechanisms from the one or more slave control mechanisms are not responding or functioning correctly, then

attempting to restart the one or more particular slave control mechanisms, and

if the one or more particular slave control mechanisms cannot be restarted, then

initiating one or more new slave control mechanisms, and

reassigning control of processing resources and storage resources from the one or more particular slave control mechanisms to the one or more new slave control mechanisms.

8. A control apparatus as recited in claim 1, wherein the one or more slave control mechanisms are configured to:

determine a status of the master control mechanism, and

if the master control mechanism has failed or is no longer functioning properly, elect a new master control mechanism from the one or more slave control mechanisms.

9. A control apparatus as recited in claim 1, wherein the one or more instructions from the master control mechanism are generated based upon expected processing and storage requirements for the first logical computing entity.

10. A control apparatus as recited in claim 1, wherein the one or more slave control mechanisms are further configured to, in response to the one or more instructions from the master control mechanism, perform the following:

dynamically change the number of processing resources in the first subset of processing resources,

dynamically change the number of storage resources in the first subset of storage resources,

dynamically change the communicative coupling between the first subset of processing resources and the first subset of storage resources to reflect changes in the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources.

11. A control apparatus as recited in claim 1, wherein the one or more slave control mechanisms are further configured to, in response to the one or more instructions from the master control mechanism, establish a second logical computing entity that contains a second subset of processing resources and a second subset of storage resources and is capable of operating independent of the master control mechanism the one or more slave control mechanisms and the first logical computing entity, wherein the second logical computing entity is communicatively isolated from the first logical computing entity by:

selecting the second subset of processing resources from the set of processing resources,

selecting the second subset of storage resources from the set of processing resources, and

causing the second subset of processing resources to be communicatively coupled to the second subset of storage resources.

12. A control apparatus as recited in claim 1, wherein:

the master control mechanism is communicatively coupled to a central control mechanism,

the master control mechanism is configured to provide loading information for the first logical computing entity to the central control mechanism, and

the master control mechanism is further configured to generate the one or more instructions for the one or more slave control mechanisms based upon one or more central control instructions received from the central control mechanism.

13. A control apparatus as recited in claim 10, wherein changes to the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources is instructed by the master control mechanism based upon actual loading of the first subset of processing resources and first subset of storage resources.

14. A control apparatus as recited in claim 11, wherein:

the first subset of processing resources is communicatively coupled to the first subset of storage resources using one or more storage area network (SAN) switches,

the second subset of processing resources is communicatively coupled to the second subset of storage resources using the one or more SAN switches, and

the second logical resource group is communicatively isolated from the first logical resource group using tagging and SAN zoning.

15. A control apparatus as recited in claim 14, wherein SAN zoning is performed using port-level SAN zoning or LUN level SAN zoning.

16. A method for managing processing resources comprising the steps of:

initiating a master control mechanism; and

initiating one or more slave control mechanisms communicatively coupled to the master control mechanism and being configured to, in response to one or more instructions from the master control mechanism, establish a first logical computing entity that contains a first subset of processing resources and a first subset of storage resources and is capable of operating independent of the master control mechanism and the one or more slave control mechanisms, wherein the first logical computing entity is established by;

selecting the first subset of processing resources from a set of processing resources,

selecting the first subset of storage resources from a set of storage resources, and

causing the first subset of processing resources to be communicatively coupled to the first subset of storage resources.

17. A method as recited in claim 16, wherein:

initiating a master control mechanism includes initiating a master control process executing on one or more processors, and

initiating one or more slave control mechanisms includes initiating one or more slave processes executing on the one or more processors.

18. A method as recited in claim 16, wherein:

initiating a master control mechanism includes initiating one or more master control processors, and

initiating one or more slave control mechanisms includes initiating one or more slave processors.

19. A method as recited in claim 16, further comprising the master control mechanism dynamically reassigning control, based upon slave control process mechanism loading, between the one or more slave control mechanisms, of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources.

20. A method as recited in claim 16, further comprising the master control mechanism, based upon slave control process mechanism loading,

dynamically allocating one or more additional slave control mechanisms, and

assigning control of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources to the one or more additional slave control mechanisms.

21. A method as recited in claim 16, further comprising the master control mechanism, based upon slave control process mechanism loading,

reassigning control to one or more other slave control mechanisms from the one or more slave control mechanisms of one or more particular processing resources from the subset of processing resources and one or more particular storage resources from the subset of storage resources that were previously assigned to one or more particular slave control mechanisms from the one or more slave control mechanisms, and

dynamically de-allocating the one or more particular slave control mechanisms.

22. A method as recited in claim 16, further comprising the master control mechanism:

determining a status of the one or more slave control mechanisms, if one or more particular slave control mechanisms from the one or more slave control mechanisms are not responding or functioning correctly, then

attempting to restart the one or more particular slave control mechanisms, and if the one or more particular slave control mechanisms cannot be restarted, then

initiating one or more new slave control mechanisms, and

reassigning control of processing resources and storage resources from the one or more particular slave control mechanisms to the one or more new slave control mechanisms.

23. A method as recited in claim 16, further comprising the one or more slave control mechanisms:

determining a status of the master control mechanism, and

if the master control mechanism has failed or is no longer functioning properly, electing a new master control mechanism from the one or more slave control mechanisms.

24. A method as recited in claim 16, wherein the one or more instructions from the master control mechanism are generated based upon expected processing and storage requirements for the first logical computing entity.

25. A method as recited in claim 16, further comprising the one or more slave control mechanisms, in response to the one or more instructions from the master control mechanism, performing the following:

dynamically changing the number of processing resources in the first subset of processing resources,

dynamically changing the number of storage resources in the first subset of storage resources,

dynamically changing the communicative coupling between the first subset of processing resources and the first subset of storage resources to reflect changes in the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources.

26. A method as recited in claim 16, further comprising the one or more slave control mechanisms, in response to the one or more instructions from the master control mechanism, establishing a second logical computing entity that contains a second subset of processing resources and a second subset of storage resources and is capable of operating independent of the master control mechanism, the one or more slave control mechanisms and the first logical computing entity, wherein the second logical computing entity is communicatively isolated from the first logical computing entity, by:

selecting the second subset of processing resources from the set of processing resources,

selecting the second subset of storage resources from the set of processing resources, and

causing the second subset of processing resources to be communicatively coupled to the second subset of storage resources.

27. A method as recited in claim 16, wherein:

the master control mechanism is communicatively coupled to a central control mechanism,

the master control mechanism is configured to provide loading information for the first logical computing entity to the central control mechanism, and

the master control mechanism is further configured to generate the one or more instructions for the one or more slave control mechanisms based upon one or more central control instructions received from the central control mechanism.

28. A method as recited in claim 25, wherein changes to the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources is instructed by the master control mechanism based upon actual loading of the first subset of processing resources and first subset of storage resources.

29. A method as recited in claim 26, wherein:

the first subset of processing resources is communicatively coupled to the first subset of storage resources using one or more storage area network (SAN) switches,

the second subset of processing resources is communicatively coupled to the second subset of storage resources using the one or more SAN switches, and

the second logical resource group is communicatively isolated from the first logical resource group using tagging and SAN zoning.

30. A method as recited in claim 29, wherein SAN zoning is performed using port-level SAN zoning or LUN level SAN zoning.

31. A computer-readable medium carrying one or more sequences of one or more instructions for managing processing resources, wherein execution of the one or more sequences of one or more instructions by one or more processors causes the one or more processors to perform the steps of:

initiating a master control mechanism; and

initiating one or more slave control mechanisms communicatively coupled to the master control mechanism and being configured to, in response to one or more instructions from the master control mechanism, establish a first logical computing entity that contains a first subset of processing resources and a first subset of storage resources and is capable of operating independent of the master control mechanism and the one or more slave control mechanisms, wherein the first logical computing entity is established by;

selecting the first subset of processing resources from a set of processing resources,

selecting the first subset of storage resources from a set of storage resources, and

causing the first subset of processing resources to be communicatively coupled to the first subset of storage resources.

32. A computer-readable medium as recited in claim 31, wherein:

initiating a master control mechanism includes initiating a master control process executing on one or more processors, and

initiating one or more slave control mechanisms includes initiating one or more slave processes executing on the one or more processors.

33. A computer-readable medium as recited in claim 31, wherein:

initiating a master control mechanism includes initiating one or more master control processors, and

initiating one or more slave control mechanisms includes initiating one or more slave processors.

34. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the master control mechanism to dynamically reassign control, based upon slave control process mechanism loading, between the one or more slave control mechanisms, of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources.

35. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the master control mechanism to, based upon slave control process mechanism loading,

dynamically allocate one or more additional slave control mechanisms, and assign control of one or more processing resources from the subset of processing resources and one or more storage resources from the subset of storage resources to the one or more additional slave control mechanisms.

36. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the master control mechanism to, based upon slave control process mechanism loading,

reassign control to one or more other slave control mechanisms from the one or more slave control mechanisms of one or more particular processing resources from the subset of processing resources and one or more particular storage resources from the subset of storage resources that were previously assigned to one or more particular slave control mechanisms from the one or more slave control mechanisms, and

dynamically de-allocate the one or more particular slave control mechanisms.

37. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the master control mechanism to:

determine a status of the one or more slave control mechanisms, and

if one or more particular slave control mechanisms from the one or more slave control mechanisms are not responding or functioning correctly, then

attempt to restart the one or more particular slave control mechanisms, and

if the one or more particular slave control mechanisms cannot be restarted, then

initiate one or more new slave control mechanisms, and

reassign control of processing resources and storage resources from the one or more particular slave control mechanisms to the one or more new slave control mechanisms.

38. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the one or more slave control mechanisms to:

determine a status of the master control mechanism, and

if the master control mechanism has failed or is no longer functioning properly, elect a new master control mechanism from the one or more slave control mechanisms.

39. A computer-readable medium as recited in claim 31, wherein the one or more instructions from the master control mechanism are generated based upon expected processing and storage requirements for the first logical computing entity.

40. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the one or more slave control mechanisms, in response to the one or more instructions from the master control mechanism, performing the following:

dynamically changing the number of processing resources in the first subset of processing resources,

dynamically changing the number of storage resources in the first subset of storage resources,

dynamically changing the communicative coupling between the first subset of processing resources and the first subset of storage resources to reflect changes in the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources.

41. A computer-readable medium as recited in claim 31, further comprising one or more additional sequences of instructions which, when executed by the one or more processors, cause the one or more slave control mechanisms to, in response to the one or more instructions from the master control mechanism, establish a second logical computing entity that contains a second subset of processing resources and a second subset of storage resources and is capable of operating independent of the master control mechanism, the one or more slave control mechanisms and the first logical computing entity, wherein the second logical computing entity is communicatively isolated from the first logical computing entity by:

selecting the second subset of processing resources from the set of processing resources,

selecting the second subset of storage resources from the set of processing resources, and

causing the second subset of processing resources to he communicatively coupled to the second subset of storage resources.

42. A computer-readable medium as recited in claim 31, wherein:

the master control mechanism is communicatively coupled to a central control mechanism,

the master control mechanism is configured to provide loading information for the first logical computing entity to the central control mechanism, and

the master control mechanism is further configured to generate the one or more instructions for the one or more slave control mechanisms based upon one or more central control instructions received from the central control mechanism.

43. A computer-readable medium as recited in claim 40, wherein changes to the number of processing resources in the first subset of processing resources and the number of storage resources in the first subset of storage resources is instructed by the master control mechanism based upon actual loading of the first subset of processing resources and first subset of storage resources.

44. A computer-readable medium as recited in claim 41, wherein:

the first subset of processing resources is communicatively coupled to the first subset of storage resources using one or more storage area network (SAN) switches,

the second subset of processing resources is communicatively coupled to the second subset of storage resources using the one or more SAN switches, and

the second logical resource group is communicatively isolated from the first logical resource group using tagging and SAN zoning.

45. A computer-readable medium as recited in claim 44, wherein SAN zoning is performed using port-level SAN zoning or LUN level SAN zoning.