Real-time database exception monitoring tool using instance eviction data

• US 9,027,025 B2
• Filed: 04/17/2007
• Issued: 05/05/2015
• Est. Priority Date: 04/17/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Zoll, Michael, Chan, Wilson Wai Shun, Pruscino, Angelo, Wang, Tak Fung
• Primary Examiner(s): Wai, Eric C
• Application Number:
  US11/736,132
• Publication Number:
  US 20080263556A1
• Time in Patent Office: 2,940 Days
• US Class Current: 718/100
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

Techniques for monitoring resources of a computer system are provided. A monitoring process collects and reports utilization data for one or more resources of a computer system, such as CPU, memory, disk I/O, and network I/O. Instead of reporting just an average of the collected data over a period of time (e.g., 10 seconds), the monitoring process at least reports individually collected resource utilization values. If one or more of the utilization values exceed specified thresholds for the respective resources, then an alert may be generated. In one approach, the monitoring process is made a real-time priority process in the computer system to ensure that the memory used by the monitoring process is not swapped out of memory. Also, being a real-time priority process ensures that the monitoring process obtains a CPU in order collect resource utilization data even when the computer system is in a starvation mode.

Claims

1. A method for monitoring a particular node of a first cluster database system that comprises a database, a database server that executes on a plurality of nodes that includes the particular node, wherein each node of the plurality of nodes executes one or more database instances, wherein the database of the first cluster database system is shared by multiple database instances of the database server, the method comprising:

establishing, on the particular node of the plurality of nodes of the first cluster database system, a monitoring process;

the monitoring process collecting a plurality of values, wherein each value of the plurality of values indicates a measure of utilization or responsiveness of a first resource of the particular node at a different instant;

determining that a database instance that was executing on a node of a second cluster database system was evicted from the second cluster database system;

in response to determining that the database instance was evicted from the second cluster database system, storing eviction data that indicates a measure of utilization or responsiveness of a second resource of the node of the second cluster database system;

wherein the eviction data reflects the utilization or responsiveness that the second resource was experiencing when the database instance was evicted from the second cluster database system;

determining, based on (a) one or more values of the plurality of values and (b) the eviction data, a probability that the utilization or responsiveness of the first resource indicated by the one or more values will lead to performance problems for the particular node; and

based on the probability, performing one or more specified actions;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein a memory used by the monitoring process is never swapped out of memory while the particular node is operating.

3. The method of claim 1, wherein the monitoring process obtains a CPU of the particular node whenever the monitoring process requests the CPU.

4. The method of claim 1, wherein performing one or more specified actions includes issuing an alert to a user of the first cluster database system.

5. The method of claim 1, wherein:

the particular node is connected to a network; and

performing one or more specified actions includes denying one or more requests from other nodes in the network for one or more resources of the particular node.

6. The method of claim 1, wherein performing one or more specified actions includes issuing an alert to one or more other processes, of the first cluster database system, that perform one or more other actions in response to the alert.

7. The method of claim 1, wherein performing one or more specified actions includes generating one or more suggestions of possible actions that should be taken by a user of the first cluster database system to avoid a decrease in performance of the particular node.

8. The method of claim 1, wherein the first resource is one of a CPU, a memory, a network queue, or a disk queue.

9. The method of claim 1, further comprising, for each resource of one or more additional resources, in the particular node, that are different than the first resource:

the monitoring process collecting a second plurality of values, wherein each value of the second plurality of values indicate a measure of utilization or responsiveness of said each resource at a different instant, wherein a different threshold value is associated with said each resource; and

determining based on (c) one or more second plurality of values and (d) the eviction data, a second probability that the utilization or responsiveness of said each resource will lead to performance problems for the particular node;

based on the second probability, performing the one or more specified actions.

10. The method of claim 1, wherein:

determining comprises comparing the one or more of the plurality of values with one or more second values indicated in historical data;

the one or more second values were collected by a second monitoring process prior to the database instance of the node of the second cluster database system experiencing one or more performance problems;

the one or more second values are associated with the one or more performance problems.

11. The method of claim 1, wherein the first resource is a first memory area that is different than a second memory area of the particular node, the method further comprising:

the monitoring process collecting a second plurality of values, wherein each value of the second plurality of values indicates a measure of utilization or responsiveness of the second memory area of the particular node at a different instant.

12. The method of claim 11, wherein the first memory area is a private memory area of the particular node and the second memory area is a shared memory area of the particular node.

13. The method of claim 1, wherein the second cluster database system is the first cluster database system.

14. The method of claim 1, wherein the monitoring process is a real-time priority process.

15. One or more non-transitory machine-readable storage media storing instructions for monitoring a particular node of a first cluster database system that comprises a database, a database server that executes on a plurality of nodes that includes the particular node, wherein each node of the plurality of nodes executes one or more database instances, wherein the database of the first cluster database system is shared by multiple database instances of the database server, wherein the instructions, when executed by one or more processors, cause:

establishing, on the particular node of the plurality of nodes of the first cluster database system, a monitoring process;

the monitoring process collecting a plurality of values, wherein each value of the plurality of values indicates a measure of utilization or responsiveness of a first resource of the particular node at a different instant;

determining that a database instance that was executing on a node of a second cluster database system was evicted from the second cluster database system;

in response to determining that the database instance was evicted from the second cluster database system, storing eviction data that indicates a measure of utilization or responsiveness of a second resource of the node of the second cluster database system;

wherein the eviction data reflects the utilization or responsiveness that the second resource was experiencing when the database instance was evicted from the second cluster database system;

determining, based on (a) one or more values of the plurality of values and (b) the eviction data, a probability that the utilization or responsiveness of the first resource indicated by the one or more values will lead to performance problems for the particular node; and

based on the probability, performing one or more specified actions.

16. The one or more non-transitory machine-readable storage media of claim 15, wherein a memory used by the monitoring process is never swapped out of memory while the particular node is operating.

17. The one or more non-transitory machine-readable storage media of claim 15, wherein the monitoring process obtains a CPU of the particular node whenever the monitoring process requests the CPU.

18. The one or more non-transitory machine-readable storage media of claim 15, wherein performing one or more specified actions includes issuing an alert to a user of the first cluster database system.

19. The one or more non-transitory machine-readable storage media of claim 15, wherein:

the particular node is connected to a network; and

performing one or more specified actions includes denying one or more requests from other nodes in the network for one or more resources of the particular node.

20. The one or more non-transitory machine-readable storage media of claim 15, wherein performing one or more specified actions includes issuing an alert to one or more other processes, of the first cluster database system, that perform one or more other actions in response to the alert.

21. The one or more non-transitory machine-readable storage media of claim 15, wherein performing one or more specified actions includes generating one or more suggestions of possible actions that should be taken by a user of the first cluster database system to avoid a decrease in performance of the particular node.

22. The one or more non-transitory machine-readable storage media of claim 15, wherein the first resource is one of a CPU, a memory, a network queue, or a disk queue.

23. The one or more non-transitory machine-readable storage media of claim 15, wherein the plurality of nodes collectively manage a single database.

24. The one or more non-transitory machine-readable storage media of claim 15, wherein the instructions, when executed by the one or more processors, further cause, for each resource of one or more additional resources, in the particular node, that are different than the first resource:

the monitoring process collecting a second plurality of values, wherein each value of the second plurality of values indicate a measure of utilization or responsiveness of said each resource at a different instant, wherein a different threshold value is associated with said each resource; and

determining based on (c) one or more second plurality of values and (d) the eviction data, a second probability that the utilization or responsiveness of said each resource will lead to performance problems for the particular node;

based on the second probability, performing one or more specified actions.

25. The one or more non-transitory machine-readable storage media of claim 15, wherein:

determining comprises comparing the one or more of the plurality of values with one or more second values indicated in historical data;

the one or more second values were collected by a second monitoring process prior to the database instance of the node of the second cluster database system experiencing one or more performance problems;

the one or more second values are associated with the one or more performance problems.

26. The one or more non-transitory machine-readable storage media of claim 15, wherein the first resource is a first memory area that is different than a second memory area of the particular node, wherein the instructions, when executed by the one or more processors, further cause:

the monitoring process collecting a second plurality of values, wherein each value of the second plurality of values indicates a measure of utilization or responsiveness of the second memory area of the particular node at a different instant.

27. The one or more non-transitory machine-readable storage media of claim 26, wherein the first memory area is a private memory area of the particular node and the second memory area is a shared memory area of the particular node.

28. The one or more non-transitory machine-readable storage media of claim 15, wherein the second cluster database system is the first cluster database system.

29. The one or more non-transitory machine-readable storage media of claim 15, wherein the monitoring process is a real-time priority process.