Automatically configuring a distributed computing system according to a hierarchical model

• US 7,685,148 B2
• Filed: 01/31/2005
• Issued: 03/23/2010
• Est. Priority Date: 01/31/2005
• Status: Active Grant

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: Nordby, Johnathan D., Engquist, James D., Jackson, Jerry R., Lindley, Craig A., Resch, David L., Church, Dann M., Vosburgh, Craig A., Berliner, Brian, Collins, Doreen E.
• Primary Examiner(s): Vital; Pierre M
• Assistant Examiner(s): Nofal; Christopher P
• Application Number:
  US11/047,468
• Publication Number:
  US 20060173895A1
• Time in Patent Office: 1,877 Days
• Field of Search: None
• US Class Current: 709/201
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

A distributed computing system conforms to a multi-level, hierarchical organizational model. One or more control nodes provide for the efficient and automated allocation and management of computing functions and resources within the distributed computing system in accordance with the organization model. The model includes four distinct levels: fabric, domains, tiers and nodes that provide for the logical abstraction and containment of the physical components as well as system and service application software of the enterprise. A user, such as a system administrator, interacts with the control nodes to logically define the hierarchical organization of distributed computing system. The control nodes are responsible for all levels of management in accordance with the model, including fabric management, domain creation, tier creation and node allocation and deployment.

Claims

1. A method comprising:

receiving input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain, and wherein each computing node comprises one or more processors for executing software instructions;

storing the model defined by the received input within a database; and

automatically configuring the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database, wherein automatically configuring the distributed computing system comprises executing software on one or more control nodes for automatically;

mapping software images associated with the computing functions to the node slots in accordance with the model stored within the database;

receiving input that defines specific node requirements of each of the tiers of the fabric;

assigning the computing nodes to the node slots of the tiers in accordance with the node requirements; and

deploying the software images on the assigned computing nodes, wherein assigning the computing nodes to the node slots of the tiers comprises;

selecting one of the tiers;

identifying the node requirements of the selected tier;

retrieving from a database computing node attributes associated with an unallocated set of the computing nodes;

comparing the node attributes of the unallocated set of computing nodes to the node requirements of the selected tier;

calculating a processing energy for each of the unallocated computing nodes based on differences between the node attributes of the unallocated computing nodes and the node requirements of the selected tier;

recording the processing energy of the unallocated computing nodes;

selecting the unallocated computing node having a lowest positive processing energy; and

assigning the selected unallocated computing node to a node slot of the selected tier.

2. The method of claim 1, wherein the node requirements include at least one of a central processing unit (CPU) count, a CPU speed, an amount of memory or local disk characteristics.

3. The method of claim 1, wherein assigning the computing nodes to the node slots of the tiers further comprises assigning the computing nodes having attributes that most closely match the node requirements defined for the tiers.

4. The method of claim 1, wherein assigning the computing nodes to the node slots of the tiers further comprises:

determining whether the node attributes for each of the unallocated computing nodes meet a set of minimum node requirements of the selected tier; and

removing the unallocated computing nodes from a list of candidate computing nodes for the selected tier when the node attributes do not meet the minimum node requirements of the selected tier.

5. The method of claim 1, wherein assigning the computing nodes to the node slots of the tiers further comprises assigning the unallocated computing nodes to the node slots of the tier when the node attributes of the unallocated computing nodes and the node requirements of the selected tier are an exact match.

6. The method of claim 1, further comprising:

identifying a need for additional processing capacity on one of the tiers; and

assigning an additional computing node to the tier.

7. The method of claim 6, wherein assigning an additional computing node comprises:

selecting an unallocated computing node from a free pool based the node requirements associated with the tier in need of the additional processing capacity; and

assigning the selected unallocated computing node to the tier in need of processing capacity.

8. The method of claim 1, further comprising:

receiving input that defines a re-provisioning policy for one or more of the tiers of the fabric identifying a need for additional processing capacity on one of the tiers; and

transferring a node from one of the other tiers to the tier in need of additional processing capacity according to the re-provisioning policy.

9. The method of claim 8, wherein transferring the node comprises:

identifying tiers with a priority lower than the tier in need of additional processing capacity;

determining which of the computing nodes associated with the lower priority tiers satisfy node requirements of the tier in need of processing capacity;

calculating a processing energy of each of the nodes of the lower priority tiers that satisfy the node requirements based on differences between the nodes that satisfy the node requirements of the tier in need of processing capacity and the node requirements of the tier in need of processing capacity; and

assigning the computing node with a lowest processing energy to the tier in need of additional processing capacity.

10. The method of claim 1, wherein the node slots represent a data set that describes specific information for a corresponding node.

11. The method of claim 1, wherein the model defines a priority for each of the tiers, and wherein assigning the computing nodes to the node slots of the tiers further comprises assigning unallocated computing nodes to the tiers from a free pool based on the priority and node requirements for each of the tiers.

12. The method of claim 1, wherein each of the software images comprises a complete set of software that can be loaded onto a node including an operating system and all boot code, middleware and application files necessary to provide a computing environment for execution of user software applications.

13. A method comprising:

receiving input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain, wherein the tiers are assigned priority values and wherein each computing node comprises one or more processors for executing software instructions;

storing the model defined by the received input within a database; and

automatically configuring the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database, wherein automatically configuring the distributed computing system comprises executing software on one or more control nodes for automatically;

mapping software images associated with the computing functions to the node slots in accordance with the model stored within the database;

assigning the computing nodes to the node slots of the tiers; and

deploying the software images on the assigned computing nodes, wherein the method further comprises;

receiving input that defines a re-provisioning policy for one or more of the tiers of the fabric;

identifying a need for additional processing capacity on one of the tiers; and

transferring a node from one of the other tiers to the tier in need of additional processing capacity according to the re-provisioning policy, wherein transferring the node comprises;

identifying tiers with a priority lower than the tier in need of additional processing capacity;

determining which of the computing nodes associated with the lower priority tiers satisfy node requirements of the tier in need of processing capacity;

calculating a processing energy of each of the nodes of the lower priority tiers that satisfy the node requirements based on differences between the nodes that satisfy the node requirements of the tier in need of processing capacity and the node requirements of the tier in need of processing capacity; and

assigning the computing node with a lowest processing energy to the tier in need of additional processing capacity.

14. The method of claim 13, wherein the node requirements include at least one of a central processing unit (CPU) count, a CPU speed, an amount of memory or local disk characteristics.

15. The method of claim 13, wherein assigning the computing nodes to the node slots of the tiers comprises:

receiving input that defines specific node requirements of each of the tiers of the fabric; and

assigning the computing nodes to the node slots in accordance with the node requirements.

16. The method of claim 15, wherein assigning the computing nodes comprises:

selecting one of the tiers;

identifying the node requirements of the selected tier;

retrieving from a database computing node attributes associated with an unallocated set of the computing nodes; and

comparing the node attributes of the unallocated set of computing nodes to the node requirements of the selected tier.

17. The method of claim 13, wherein the node slots represent a data set that describes specific information for a corresponding node.

18. The method of claim 13, wherein the model defines a priority for each of the tiers, and wherein assigning the computing nodes to the node slots of the tiers further comprises assigning unallocated computing nodes to the tiers from a free pool based on the priority and node requirements for each of the tiers.

19. The method of claim 13, wherein each of the software image instances comprises a complete set of software that can be loaded onto a computing node including an operating system and all boot code, middleware and application files necessary to provide a computing environment for execution of user software applications.

20. A distributed computing system comprising:

a plurality of application nodes interconnected via a communications network, wherein each application node comprises one or more programmable processors for executing software instructions;

a control node that receives input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes; and

a memory comprising a database that stores the model defined by the received input, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain,

wherein the control node automatically configures the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database,

wherein the control node automatically maps software images associated with the computing functions to the node slots in accordance with the model stored within the database, receives input that defines specific node requirements of each of the tiers of the fabric, assigns the computing nodes to the node slots of the tiers in accordance with the node requirements, and deploys the software images on the assigned computing nodes, and

wherein the control node selects one of the tiers, identifies the node requirements of the selected tier, retrieves from a database computing node attributes associated with an unallocated set of the computing nodes, compares the node attributes of the unallocated set of computing nodes to the node requirements of the selected tier, calculates a processing energy for each of the unallocated computing nodes based on differences between the node attributes of the unallocated computing nodes and the node requirements of the selected tier, records the processing energy of the unallocated computing nodes, selects the unallocated computing node having a lowest positive processing energy, and assigns the selected unallocated computing node to a node slot of the selected tier.

21. The distributed computing system of claim 20, further comprising:

an image repository storing master software images and software image instances generated from the master software images.

22. The distributed computing system of claim 20, wherein the node requirements include at least one of a central processing unit (CPU) count, a CPU speed, an amount of memory or local disk characteristics.

23. The distributed computing system of claim 20, wherein the control node assigns the computing nodes having attributes that most closely match the node requirements defined for the tiers.

24. The distributed computing system of claim 20, wherein the control node determines whether the node attributes for each of the unallocated computing nodes meet a set of minimum node requirements of the selected tier, and removes the unallocated computing nodes from a list of candidate computing nodes for the selected tier when the node attributes do not meet the minimum node requirements of the selected tier.

25. The distributed computing system of claim 20, wherein the control node assigns the unallocated computing nodes to the node slots of the tier when the node attributes of the unallocated computing nodes and the node requirements of the selected tier are an exact match.

26. The distributed computing system of claim 20, wherein the control node identifies a need for additional processing capacity on one of the tiers, and assigns an additional computing node to the tier.

27. The distributed computing system of claim 20, wherein each of the software images comprises a complete set of software that can be loaded onto a computing node including an operating system and all boot code, middleware and application files necessary to provide a computing environment for execution of user software applications.

28. A distributed computing system comprising:

a plurality of application nodes interconnected via a communications network, wherein each application node comprises one or more programmable processors for executing software instructions;

a control node that receives input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes; and

a memory comprising a database that stores the model defined by the received input, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain, wherein the tiers are assigned priority values,

wherein the control node automatically configures the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database,

wherein the control node automatically maps software images associated with the computing functions to the node slots in accordance with the model stored within the database, assigns the computing nodes to the node slots of the tiers, and deploys the software images on the assigned computing nodes,

wherein the control node receives input that defines a re-provisioning policy for one or more of the tiers of the fabric, identifies a need for additional processing capacity on one of the tiers, and transfers a node from one of the other tiers to the tier in need of additional processing capacity according to the re-provisioning policy, and

wherein the control node identifies tiers with a priority lower than the tier in need of additional processing capacity, determines which of the computing nodes associated with the lower priority tiers satisfy node requirements of the tier in need of processing capacity, calculates a processing energy of each of the nodes of the lower priority tiers that satisfy the node requirements based on differences between the nodes that satisfy the node requirements of the tier in need of processing capacity and the node requirements of the tier in need of processing capacity, and assigns the computing node with a lowest processing energy to the tier in need of additional processing capacity.

29. The distributed computing system of claim 28, further comprising:

an image repository storing master software images and software image instances generated from the master software images.

30. The distributed computing system of claim 28, wherein the node requirements include at least one of a central processing unit (CPU) count, a CPU speed, an amount of memory or local disk characteristics.

31. The distributed computing system of claim 28, wherein the control node receives input that defines specific node requirements of each of the tiers of the fabric, and assigns the computing nodes to the node slots in accordance with the node requirements.

32. The distributed computing system of claim 28, wherein the control node selects one of the tiers, identifies the node requirements of the selected tier, retrieves from a database computing node attributes associated with an unallocated set of the computing nodes, and compares the node attributes of the unallocated set of computing nodes to the node requirements of the selected tier.

33. The distributed computing system of claim 28, wherein the node slots represent a data set that describes specific information for a corresponding node.

34. The distributed computing system of claim 28, wherein the model defines a priority for each of the tiers, and wherein the control node assigning unallocated computing nodes to the tiers from a free pool based on the priority and node requirements for each of the tiers.

35. The distributed computing system of claim 28, wherein each of the software images comprises a complete set of software that can be loaded onto a computing node including an operating system and all boot code, middleware and application files necessary to provide a computing environment for execution of user software applications.

36. A computer readable medium storing instructions that cause a processor to:

receive input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain, and wherein each computing node comprises one or more processors for executing software instructions;

store the model defined by the received input within a database; and

automatically configure the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database, wherein the instructions that cause the processor to automatically configure the distributed computing system comprise instructions that automatically;

map software images associated with the computing functions to the node slots in accordance with the model stored within the database;

receive input that defines specific node requirements of each of the tiers of the fabric; and

assign the computing nodes to the node slots of the tiers in accordance with the node requirements; and

deploy the software images on the assigned computing nodes, wherein the instructions that cause the processor to assign the computing nodes to the node slots of the tiers comprise instructions that cause the processor to;

select one of the tiers;

identify the node requirements of the selected tier;

retrieve from a database computing node attributes associated with an unallocated set of the computing nodes;

compare the node attributes of the unallocated set of computing nodes to the node requirements of the selected tier;

calculate a processing energy for each of the unallocated computing nodes based on differences between the node attributes of the unallocated computing nodes and the node requirements of the selected tier;

record the processing energy of the unallocated computing nodes;

select the unallocated computing node having a lowest positive processing energy; and

assign the selected unallocated computing node to a node slot of the selected tier.

37. A computer readable medium storing instructions that cause a processor to:

receive input that defines a model for a hierarchical organization of a distributed computing system having a plurality of computing nodes, wherein the model specifies a fabric having one or more domains, wherein each domain has at least one tier that includes at least one node slot, wherein each of the domains corresponds to one of an enterprise application or an enterprise service, wherein, for each domain, the model defines a different computing function for each of the tiers of the domain, wherein the tiers are assigned priority values and wherein each computing node comprises one or more processors for executing software instructions;

store the model defined by the received input within a database; and

automatically configure the computing nodes of the distributed computing system in accordance with the hierarchical organization defined by the model within the database, wherein the instructions that cause the processor to automatically configure the distributed computing system comprise instructions that automatically;

map software images associated with the computing functions to the node slots in accordance with the model stored within the database;

assign the computing nodes to the node slots of the tiers; and

deploy the software images on the assigned computing nodes, wherein the computer readable medium stores instructions that cause the processor to;

receive input that defines a re-provisioning policy for one or more of the tiers of the fabric;

identify a need for additional processing capacity on one of the tiers; and

transfer a node from one of the other tiers to the tier in need of additional processing capacity according to the re-provisioning policy, wherein the instructions that cause the processor to transfer the node comprise instructions that cause the processor to;

identify tiers with a priority lower than the tier in need of additional processing capacity;

determine which of the computing nodes associated with the lower priority tiers satisfy node requirements of the tier in need of processing capacity;

calculate a processing energy of each of the nodes of the lower priority tiers that satisfy the node requirements based on differences between the nodes that satisfy the node requirements of the tier in need of processing capacity and the node requirements of the tier in need of processing capacity; and

assign the computing node with a lowest processing energy to the tier in need of additional processing capacity.