Autonomic control of a distributed computing system using rule-based sensor definitions

• US 7,454,427 B2
• Filed: 07/07/2005
• Issued: 11/18/2008
• Est. Priority Date: 01/31/2005
• Status: Active Grant

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Cassatt (Broadcom, Inc.)
• Inventors: Jackson, Jerry R.
• Primary Examiner(s): Vital; Pierre M.
• Assistant Examiner(s): Vo; Truong V
• Application Number:
  US11/176,161
• Publication Number:
  US 20060173857A1
• Time in Patent Office: 1,230 Days
• Field of Search: 707/10, 713/201, 455/445
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

A distributed computing system conforms to a multi-level, hierarchical organizational model. One or more control nodes provide for the efficient and automated allocation and management of computing functions and resources within the distributed computing system in accordance with the organization model. A user, such as a system administrator, interacts with the control nodes to logically define the hierarchical organization of distributed computing system. The control node includes an automation subsystem having one or more rule engines that provide autonomic control of the application nodes in accordance with a set of one or more rules. For example, the rules may define sensors or methods of configuring the distributed computing system. More specifically, the sensor rules define the conditions under which the automation subsystem may update a model of the distributed computing system based on data received from the distributed computing system.

Claims

1. A distributed computing system comprising:

a plurality of application nodes interconnected via a communications network, wherein each of the application nodes comprises one or more programmable processors; and

a control node that comprises;

(i) a working memory that stores a model that reflects an actual state of the distributed computing system;

(ii) an automation subsystem having a rules engine that provides autonomic control of the application nodes in accordance with a set of one or more rules;

(iii) a monitoring subsystem that collects status data from the application nodes and communicates the status data to the automation subsystem, wherein the status data represents an actual state for the application nodes; and

(iv) a business logic tier that performs operations to control the application nodes and provides expected state data to the automation subsystem, wherein the expected state data represents an expected state for the application nodes,

wherein the rules engine applies a first subset of the set of rules to update a model of the actual state of the application nodes based on the status data received from the monitoring subsystem, wherein each rule in the first subset of the set of rules defines a sensor for the automation subsystem,

wherein the rules engine determines whether the model satisfies a condition specified in a condition portion of each rule in the first subset of the set of rules,

wherein the rules engine activates a rule within the first subset of the set of rules when the model satisfies the condition specified in the condition portion of the rule,

wherein the rules engine deactivates the rule when the model does not satisfy the condition specified in the condition portion of the rule,

wherein the rules engine determines whether the status data from the monitoring subsystem includes a data pattern specified in an action portion of the rule when the rule is activated,

wherein the rules engine ceases to determine whether the status data from the monitoring subsystem includes the data pattern specified in the action portion of the rule when the rule is deactivated,

wherein the rule engine updates the model in accordance with an operation specified in the action portion of the rule when the rules engine determines that the status data includes the data pattern specified in the action portion of the rule,

wherein the rules engine analyzes the model of the actual state of the application nodes analyzes the expected state data from the business logic tier, and applies a second subset of the set of rules to produce action requests to the business logic tier, wherein the action requests indicate one or more operations for the business logic tier to perform to reduce any difference between the model of the actual state of the application nodes and the expected state for the application nodes.

2. The distributed computing system of claim 1, wherein the rule engine is a forward-chaining rule engine.

3. The distributed computing system of claim 1,

wherein the automation subsystem includes a rule compiler that compiles the rules into one or more discrimination networks;

and

wherein the rules engine uses the one or more discrimination networks to determine whether the model satisfies the conditions specified in the condition portions of each rules in the first subset of the set of rules.

4. The distributed computing system of claim 1, wherein the monitoring subsystem comprises a set of modular information collectors that collect the status data from the application nodes.

5. The distributed computing system of claim 4, wherein each of the information collectors conforms to a different communication protocol.

6. The distributed computing system of claim 4, wherein the application nodes are organized into tiers, and wherein each of the information collectors collects the status data from the application nodes of different tiers.

7. The distributed computing system of claim 4, wherein the automation subsystem registers collection requests with the monitoring subsystem, and the monitoring subsystem includes a monitoring engine that invokes the collectors in response to the collection requests.

8. The distributed computing system of claim 1, wherein the business logic tier includes a set of web services that control the application nodes.

9. The distributed computing system of claim 1, wherein the business logic tier presents interfaces by which a user defines one or more goals for the expected state of the distributed computing system.

10. The distributed computing system of claim 9, wherein the business logic tier registers the goals with the automation subsystem for achievement by the rules engine in accordance with the set of rules.

11. The distributed computing system of claim 9, wherein the automation subsystem automatically formulates dependencies between the goals.

12. The distributed computing system of claim 9, wherein the business logic tier further comprises:

a task manager that invokes a task that controls the operation; and

an event subsystem that tracks the progress of the task.

13. The distributed computing system of claim 9, wherein the goals include at least one of software image management goals, node allocation goals, goals for harvesting nodes, tier capacity goals, asset requirement goals and goals for gathering the status data

14. The distributed computing system of claim 1, wherein the rules are units of code that conform with a rule language.

15. The distributed computing system of claim 1, wherein the control node presents interfaces by which a user defines the rules in the set of rules.

16. The distributed computing system of claim 1, further comprising:

a database storing data that defines a model for a hierarchical organization of the distributed computing system, wherein the model specifies a fabric having one or more domains, and wherein each domain has at least one tier that includes at least one node slot that is associated with a software image,

wherein when the business logic tier applies the operation to the distributed computing system, the business logic tier configures one of the application nodes to execute the software image associated with the node slot.

17. The distributed computing system of claim 16, wherein each tier represents a different computing function.

18. The distributed computing system of claim 1, wherein the rules engine comprises:

a rule compiler to generate a discrimination network based on a set of match conditions supplied by the set of rules;

a memory to store the model of the actual state of the application nodes; and

an execution engine to determine whether each rule in the set of rules matches a current state of the memory, collect all rules that match the current state of the memory, select a rule within the set of rules to fire, fire the selected rule, and propagate any changes to the memory.

19. A computer-implemented method comprising:

receiving input that defines an expected state for a distributed computing system having a plurality of application nodes interconnected via a communications network;

receiving status data from the distributed computing system regarding an actual state of the distributed computing system;

determining, with a rules engine, whether a model of the actual state of the application nodes satisfies a condition specified in a condition portion of each rule in a first subset of a set of rules;

activating a rule within the first subset of the set of rules when the model satisfies the condition specified in the condition portion of the rule;

deactivating the rule when the model does not satisfy the condition specified in the condition portion of the rule;

determining, with the rules engine, whether the status data includes a data pattern specified in an action portion of the rule when the rule is activated;

ceasing to determine, with the rules engine, whether the status data includes the data pattern specified in the action portion of the rule when the rule is deactivated;

updating the model in accordance with an operation specified in the action portion of the rule when the rules engine determines that the status data includes the data pattern specified in the action portion of the rule;

processing rules within a second subset of the set of rules to determine operations for reducing any difference between the expected state and a model of the actual state of the nodes;

applying the operations for reducing any difference between the expected state and the model to the distributed computing system to control the application nodes in accordance with the rules.

20. The method of claim 19,

wherein the rules in the set of rules are forward-chaining rules;

wherein the method further comprises compiling the rules in the set of rules into one or more discrimination networks; and

wherein determining whether the model satisfies conditions specified in the condition portions of the rules in the first subset of the set of rules comprises using the one or more discrimination networks to determine whether the model satisfies conditions specified in the condition portions of the rules in the first subset of the set of rules.

21. The method of claim 19, further comprising collecting the status data from the application nodes with a plurality of modular information collectors.

22. The method of claim 21, further comprising:

registering collection requests with a monitoring subsystem; and

executing a monitoring engine that invokes the collectors in response to the collection requests.

23. The method of claim 19, further comprising:

presenting an interface by which a user defines the expected state as one or more goals; and

wherein receiving the input that defines the expected state comprises receiving the goals.

24. The method of claim 23, further comprising:

storing data that defines a model for a hierarchical organization of the distributed computing system, wherein the model specifies a fabric having a one or more domains, and wherein each domain has at least one tier that includes at least one node slot that is associated with a software image; and

wherein applying the operation comprises configuring one of the application nodes to execute the software image associated with the node slot.

25. The method of claim 19, further comprising:

presenting an interface by which a user inputs the rules in the set of rules.