Autonomic control of a distributed computing system using an application matrix to control application deployment

• US 7,571,154 B2
• Filed: 07/28/2005
• Issued: 08/04/2009
• Est. Priority Date: 01/31/2005
• Status: Active Grant

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Cassatt (Broadcom, Inc.)
• Inventors: Jackson, Jerry R., Emeis, Mark W.
• Primary Examiner(s): Cottingham; John R.
• Assistant Examiner(s): Arjomandi; Noosha
• Application Number:
  US11/191,882
• Publication Number:
  US 20060173994A1
• Time in Patent Office: 1,468 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 2209/5021 Priority
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database

Abstract

A distributed computing system conforms to a multi-level, hierarchical organizational model. One or more control nodes provide for the efficient and automated allocation and management of computing functions and resources within the distributed computing system in accordance with the organization model. A user, such as a system administrator, interacts with the control nodes to logically define the hierarchical organization of distributed computing system. The control node includes an automation subsystem having one or more rule engines that provide autonomic control of the application nodes in accordance with a set of one or more rules. A pluggable, application-specific application matrix is selected and installed within the control node. The application matrix contains a logical definition of the applications, and parameters for controlling the deployment of the applications within the distributed computing system.

Claims

1. A distributed computing system comprising:

a plurality of application nodes interconnected via a communications network; and

a control node that comprises;

a set of two or more different applications to be executed on the application nodes;

an application matrix that includes an application matrix entry for each of the applications in the set of two or more different applications, wherein each application matrix entry contains application-specific parameters for controlling the deployment, undeployment, and execution of the corresponding application within the distributed computing system, wherein the parameters for each of the application matrix entries include parameters that are based on information that is specific to an installation instance of the corresponding application that has been previously deployed within a staging environment and customized for subsequent deployment within the distributed computing system; and

an automation infrastructure having one or more application-neutral rule engines that provide autonomic control of the application nodes and autonomic control over the deployment, undeployment, and execution of each of the applications in accordance with a set of one or more application-neutral rules and the application-specific parameters stored within the corresponding entry within the application matrix,

wherein each of the one or more application-neutral rule engines comprises a working memory that includes one or more local objects, wherein each rule in the set of one or more application-neutral rules includes a match condition and an implied action, wherein the set of one or more application-neutral rules is compiled into a discrimination network that includes a translation of the match conditions of the set of one or more rules such that redundant tests are avoided during rule execution, wherein the one or more application-neutral rule engines further comprises an execution engine that matches the set of one or more compiled rules within the discrimination network against the one or more local objects within the working memory,

wherein the automation infrastructure detects new rule attributes within application entries of the application matrix that specify application-specific deployment constraints for the corresponding applications with respect to one or more application nodes, and creates local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically control the deployment of the applications to the application nodes in accordance with a current state of the application matrix, and

wherein the automation infrastructure detects new rule attributes within application entries of the application matrix that specify application-specific service levels for the corresponding applications, and creates local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically monitor the execution of the applications when deployed to the application nodes in accordance with a current state of the application matrix.

2. The distributed computing system of claim 1, further comprising an application governor that accesses the application matrix to communicate instructions from the automation infrastructure to the applications.

3. The distributed computing system of claim 1, wherein the automation infrastructure automatically updates the application-neutral rule engines to autonomically control the deployment of the applications to the application nodes in accordance with a current state of the application matrix.

4. The distributed computing system of claim 1, wherein the automation infrastructure automatically updates the application-neutral rule engines to monitor the execution of the applications when deployed to the application nodes in accordance with a current state of the application matrix.

5. The distributed computing system of claim 1, further comprising:

a monitoring subsystem that collects the status data from the application nodes and communicates the status data to the automation subsystem, wherein the status data represents an actual state for the application nodes;

wherein the application-neutral rule engines analyze the status data from the monitoring subsystem and apply the set of application-neutral rules to control the applications.

6. The distributed computing system of claim 1, wherein the application matrix is described in a data description language.

7. The distributed computing system of claim 3, wherein the data description language is extensible markup language (XML).

8. The distributed computing system of claim 1, wherein the application matrix comprises a set of application descriptors that define a set of rules and parameters for each of the applications.

9. The distributed computing system of claim 1, wherein the application-specific deployment constraints define criteria for use by the application-neutral rule engine when selecting a set of the application nodes to which to deploy the applications.

10. The distributed computing system of claim 1, wherein the application matrix includes elements that set a desired minimum or maximum numbers of the application nodes onto which the control node may deploy each of the applications.

11. The distributed computing system of claim 1, wherein the application-neutral rule engines are forward-chaining rule engines.

12. The distributed computing system of claim 1, wherein the automation subsystem includes a rule compiler that compiles the rules into one or more discrimination networks.

13. The distributed computing system of claim 1, wherein the control node further comprises:

a monitoring subsystem that collects the status data from the application nodes and communicates the status data to the automation subsystem, wherein the status data represents an actual state for the application nodes; and

a business logic tier that provides expected state data to the automation subsystem, wherein the expected state data represents an expected state for the application nodes,

wherein the application-neutral rule engines analyze the status data from the monitoring subsystem and apply the set of rules to produce action requests to the business logic tier to control the application nodes to reduce any difference between the actual state and the expected state.

14. The distributed computing system of claim 1, wherein the application nodes are organized into tiers, and wherein each of the information collectors collects the status data from the application nodes of different tiers.

15. The distributed computing system of claim 1, further comprising:

a database storing data that defines a model for a hierarchical organization of the distributed computing system, wherein the model specifies a fabric having a one or more domains, and wherein each domain has at least one tier that includes at least one of the application nodes,

wherein the application-neutral rule engines automatically produce action requests to control the deployment of the applications in accordance with a hierarchical organization of the distributed computing system defined by the model.

16. The distributed computing system of claim 15, wherein each tier represents a different computing function.

17. The distributed computing system of claim 1, wherein the rules are units of code that conform with a rule language.

18. The distributed computing system of claim 1, wherein the new applications may be added to the application matrix while the distributed computing system is operating.

19. The distributed computing system of claim 1,

wherein the application matrix is a pluggable module that is specific to the set of applications, and

wherein the distributed computing system includes a control node providing a framework to receive any of a plurality of pluggable application matrices that can be installed and removed from the control node to provide autonomic control over different groups of applications within the distributed computing system.

33. The distributed computing system of claim 1, wherein the information that is specific to the installation instance of the corresponding application is derived by deploying a desired image for the corresponding application, creating a domain, and specifying any services and external system connectivity for the corresponding application within the domain.

34. The distributed computing system of claim 1, wherein the execution engine collects all matching rules and matching local objects, selects a particular rule instantiation to fire, executes the selected instantiated rule, and propagates any changes to working memory, and repeats the collection, selection, execution, and propagation processes until no more matching rule instantiations can be found.

35. The distributed computing system of claim 1, further comprising:

a monitoring subsystem that collects the status data from the application nodes and communicates the status data to the automation subsystem, wherein the status data represents an actual state for the application nodes,

wherein the one or more application-neutral rule engines analyze the status data from the monitoring subsystem and apply the set of one or more application-neutral rules derived from the application matrix to control the applications,

wherein the automation infrastructure detects new application-specific service monitored attributes within application entries of the application matrix that specify attributes of services that are to be monitored after the corresponding application is deployed, and updates the monitoring system to monitor the attributes specified by the service monitored attributes and provide status data to the application-neutral rule engines based on the monitored attributes, and

wherein the automation infrastructure detects new application-specific node monitored values within application entries of the application matrix that specify attributes of one or more application nodes that are to be monitored after the corresponding application is deployed, and updates the monitoring system to monitor the attributes specified by the node monitored values and provide status data to the application-neutral rule engines based on the monitored attributes.

20. A method comprising:

generating an application matrix that includes an application matrix entry for each of the applications in a set of two or more different applications to be executed on a plurality of application nodes interconnected via a communications network, wherein each application matrix entry contains application-specific parameters for controlling the deployment, undeployment, and execution of the corresponding applications within a distributed computing system, and wherein the parameters for each of the application matrix entries include parameters that are based on information that is specific to an installation instance of the corresponding application that has been previously deployed within a staging environment and customized for subsequent deployment within the distributed computing system;

providing, with one or more application-neutral rule engines, autonomic control of the application nodes and autonomic control over the deployment, undeployment, and execution of each of the applications within the distributed computing system in accordance with a set of one or more application-neutral rules and the application-specific parameters stored within the corresponding entry within the application matrix, wherein each of the one or more application-neutral rule engines comprises a working memory that includes one or more local objects, wherein each rule in the set of one or more application-neutral rules includes a match condition and an implied action, wherein the set of one or more application-neutral rules is compiled into a discrimination network that includes a translation of the match conditions of the set of one or more rules such that redundant tests are avoided during rule execution, wherein providing, with one or more application-neutral rule engines, autonomic control comprises matching the set of one or more compiled rules within the discrimination network against the one or more local objects within the working memory;

detecting new rule attributes within application entries of the application matrix that specify application-specific deployment constraints for the corresponding applications with respect to one or more application nodes;

creating local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically control the deployment of the applications to the application nodes in accordance with a current state of the application matrix;

detecting new rule attributes within application entries of the application matrix that specify application-specific service levels for the corresponding applications; and

creating local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically monitor the execution of the applications when deployed to the application nodes in accordance with a current state of the application matrix.

21. The method of claim 20, wherein providing, with one or more application-neutral rule engines, autonomic control comprises:

receiving status data for the distributed computing system, wherein the status data represents an actual state of the application nodes;

processing the status data with the application-neutral rules in the one or more application-neutral rule engines to determine operations for reducing any difference between an expected state and the actual state of the distributed computing system; and

performing the operations to provide autonomic control over the deployment of a set of applications within the distributed computing system in accordance with the application-neutral rules.

22. The method of claim 20, further comprising accessing parameters of the application matrix for use by the one or more application-neutral rules in autonomically controlling the deployment of the applications.

23. The method of claim 22, further comprising processing the application-neutral rules with a plurality of forward-chaining rule engines.

24. The method of claim 20, further comprising selecting the application matrix from a plurality of pluggable application matrices that can be installed and removed from a control node of the distributed computing system to provide autonomic control over different groups of applications.

25. The method of claim 20, further comprising capturing attributes from the application matrix to autonomically control the deployment of the applications based on the parameters of the application matrix.

26. The method of claim 25, wherein generating one or more attributes comprises capturing one or more of the attributes for use in monitoring the execution of the applications when deployed.

27. The method of claim 20, wherein generating an application matrix comprises generating the application matrix to conform to a data description language.

28. The method of claim 20, wherein the application matrix comprises a set of application descriptors that define a set of rules and parameters for each of the applications.

29. The method of claim 20, wherein the deployment constraints define criteria for use by the application-neutral rule engine when selecting a set of the application nodes to which to deploy the applications.

30. The method of claim 20, wherein the application matrix includes elements that set a desired minimum or maximum numbers of the application nodes onto which the control node may deploy each of the applications.

31. The method of claim 20,

wherein generating the application matrix comprises adding applications to the application matrix after the distributed computing system has begun operation;

and

wherein providing, with one or more application-neutral rule engines, autonomic control over the deployment of the applications comprises performing operations to provide autonomic control over the deployment, undeployment, and execution of the application added to the application matrix after the distributed computing system has begun operation.

32. A computer-readable medium comprising instructions that cause a programmable processor to:

generate an application matrix that includes an application matrix entry for each of the applications in a set of two or more different applications to be executed on a plurality of application nodes interconnected via a communications network, wherein each application matrix entry contains application-specific parameters for controlling the deployment, undeployment, and execution of the corresponding applications within a distributed computing system, and wherein the parameters for each of the application matrix entries include parameters that are based on information that is specific to an installation instance of the corresponding application that has been previously deployed within a staging environment and customized for subsequent deployment within the distributed computing system;

provide, with one or more application-neutral rule engines, autonomic control of the application nodes and autonomic control over the deployment, undeployment, and execution of each of the applications within the distributed computing system in accordance with a set of one or more application-neutral rules and the application-specific parameters stored within the corresponding entry within the application matrix, wherein each of the one or more application-neutral rule engines comprises a working memory that includes one or more local objects, wherein each rule in the set of one or more application-neutral rules includes a match condition and an implied action, wherein the set of one or more application-neutral rules is compiled into a discrimination network that includes a translation of the match conditions of the set of one or more rules such that redundant tests are avoided during rule execution, wherein the instructions that cause the programmable processor to provide, with one or more application-neutral rule engines, autonomic control comprise instructions that cause the processor to match the set of one or more compiled rules within the discrimination network against the one or more local objects within the working memory;

detect new rule attributes within application entries of the application matrix that specify application-specific deployment constraints for the corresponding applications with respect to one or more application nodes;

create local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically control the deployment of the applications to the application nodes in accordance with a current state of the application matrix;

detect new rule attributes within application entries of the application matrix that specify application-specific service levels for the corresponding applications; and

create local objects reflecting the new rule attributes within the working memories of the one or more application-neutral rule engines without having to recompile the set of application-neutral rules in order to autonomically monitor the execution of the applications when deployed to the application nodes in accordance with a current state of the application matrix.