User-centric measurement of quality of service in a computer network

• US 6,959,265 B1
• Filed: 10/07/2003
• Issued: 10/25/2005
• Est. Priority Date: 10/07/2003
• Status: Expired due to Fees

• Current Assignee: Interact Software
• Original Assignee: Serden Technologies, Inc.
• Inventors: Tosoni, Stéphane, Candela, Eric Eva, Duthoit, Marc, Andreo, Blaise, Le Roux, Jean-Noël
• Primary Examiner(s): Nghiem, Michael
• Assistant Examiner(s): VO, HIEN XUAN
• Application Number:
  US10/680,048
• Time in Patent Office: 749 Days
• Field of Search: 702/186, 702182-183, 709/200, 709/232, 709/220, 709/223, 709/224, 370/254, 370351-352, 370/462, 370/400
• US Class Current: 702/186
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

In a business network environment, each workstation or information access point (IAP) has installed thereon a quality of service (QoS) module to monitor the IAP'"'"'s performance. The QoS module declares exceptions when it perceives that one or more of a predetermined plurality of functions or conditions of the computer has entered into a state indicative of degraded performance. At the time that an exception is declared, a snapshot is taken of the software applications running on the IAP, the amount of resources that they are using, and presence or absence of the user. As used in calculating a QoS index representative of the operational state of the IAP, exceptions are weighted to take into account the relative importance of these exceptions and the users which they affect. Because the QoS indices are time-normalized, they may be aggregated or compared across the network by an IT administrator.

Claims

1. A method for assessing the performance of a computer, comprising the steps of:

storing, on the computer, a profile which defines criteria for a plurality of states for each of a plurality of computer functions or conditions, said states representing degraded levels of performance of progressive severity;

using a quality of service (QoS) module of the computer to monitor the performance of each of a predetermined plurality of computer functions or conditions;

using the QoS module to detect each of one or more exceptions of predetermined kinds, an exception being detected when one of the predetermined computer functions or conditions enters into one of said states;

using the QoS module to record the occurrence of exceptions for a predetermined period of time;

using the QoS module to attribute a value to the exception based on the sensed state and kind of the exception;

using the QoS module to aggregate the values of the exceptions for the predetermined period of time according to a predetermined function in order to derive a quality of service index; and

storing the quality of service index as an indicium of the quality of service rendered by the computer during said predetermined period of time.

2. The method of claim 1, wherein the step of monitoring the predetermined plurality of computer functions or conditions includes monitoring two or more of the following:

memory usage, processor usage, the rate of input/output operations, availability of printing, disk space, status of applications running on the computer, status of computer system services, time which the computer takes to respond to user input, events noted by an operating system of the computer, network availability, and a manual entry by a user of the computer that an exception exists.

3. The method of claim 2, wherein the QoS module measures the rate of input/output operations to and from at least one hard drive of the computer.

4. The method of claim 1, and further comprising the steps of:

programming a general purpose processor of the computer with a computer program in order to create the QoS module.

5. The method of claim 1, and further comprising the steps of:

when an exception occurs, using the QoS module to automatically determine whether a user is using the computer; and

weighting the exception in said predetermined function according to whether the user was detected as using the computer.

6. The method of claim 1, and further comprising the steps of:

when an exception occurs, using the QoS module to determine the time of day;

using the QoS module to record the determined time of day as attributed to the exception; and

weighting the exception in the predetermined function as a function of the time of day when the exception occurred.

7. The method of claim 1, and further comprising the steps of:

when an exception occurs, using the QoS module to ascertain the current day of the week;

using the QoS module to record the ascertained day of the week as attributed to the exception; and

weighting the exception in the predetermined function as a function of the recorded day of the week attributed to the exception.

8. The method of claim 1, wherein the computer is capable of running one or more of a plurality of different software applications or tasks available for execution by the computer, the method comprising the further steps of:

when an exception occurs, using the QoS module to identify which of a predetermined plurality of the software applications or tasks are running; and

using the QoS module to record which of the plurality of software applications or tasks are running at the time that the exception occurred.

9. The method of claim 1, wherein the computer is a node on a network, the method further comprising the step of uploading a record of QoS indices and related data to a central location for analysis by a system administrator.

10. The method of claim 1, and further comprising the steps of:

automatically and periodically recording performance data for each software application or task then running on the IAP; and

using the periodically recorded performance data to heuristically derive the criteria for ones of the predetermined states as a function of the recorded performance data.

11. The method of claim 10, wherein said performance data include data relating to the consumption of IAP resources by the running software applications or tasks.

12. The method of claim 10, wherein said performance data include one or more of the following:

CPU usage, memory usage, amounts of network and mass storage usage, handle usage, number of threads and number of page faults.

13. The method of claim 10, wherein the computer is a node in a network including a plurality of computers, said criteria for ones of said states being derived from periodically recorded measurements made at a plurality of IAPs on the network.

14. The method of claim 1, wherein the computer is a node in a network including a plurality of computers usable by different ones of a plurality of users each having an assigned user type, the method further including the steps of:

determining which user type is currently using the computer;

comparing the monitored performance of each of the functions or conditions to respective criteria read from a table associated with the user type; and

determining whether at least one exception exists based on said step of comparing.

15. The method of claim 1, and further comprising the steps of:

assigning a base numerical value to the exception based on its attributed state; and

using the base numerical value of the exception in the predetermined function in order to derive the quality of service index.

16. The method of claim 1, wherein the plurality of states for which predetermined criteria are stored include degraded, severely degraded, frozen and crashed.

17. The method of claim 1, wherein the computer is a node on a network including a plurality of computers, the method including the steps of:

normalizing the quality of service index by dividing the aggregated values of the exceptions by the period of time during which the exceptions occurred; and

calculating a quality of service index for a group of computers on the network based on quality of service indices of individual computers.

18. A method for assessing the quality of service present on an information access point in a computer network, the method comprising the steps of:

at the information access point, providing a quality of service module for monitoring respective performance levels of each of a plurality of functions or conditions of the information access point;

for each of the plurality of functions or conditions, storing predetermined criteria the presence of any of which is indicative of a state of degraded performance;

declaring an exception when a sensed performance level of a function or condition enters into a respective stored state;

using the quality of service module to determine whether a user is present at the information access point at the time that the exception occurred;

weighting the exception with a user presence factor having a value which is a function of whether the user was present at the time the exception occurred; and

calculating a numerical quality of service index indicative of the quality of service rendered by the computer from one or more of the weighted exceptions.

19. The method of claim 18, and further comprising the steps of:

using a probe to declare an attended status when a user has an input at user interface;

starting a clock from the time that the user made said input;

determining when the clock reaches a predetermined timeout period;

declaring an unattended status if no further user input is sensed by the expiration of the predetermined timeout period; and

restarting the clock and declaring an attended status upon the next occurrence of user input.

20. A method for assessing the quality of service present on an information access point (IAP) in a computer network, comprising the steps of:

at the IAP, providing a quality of service module for monitoring a plurality of functions or conditions of the IAP;

at the IAP, storing predetermined criteria the presence of any of which are determinative of an exception to normal operation;

monitoring the plurality of functions or conditions to determine whether any of said criteria are present;

using the quality of service module to declare at least one exception when one or more of said criteria are sensed to exist at the IAP;

using the quality of service module to determine which of a predetermined plurality of software applications were running at IAP at the time the criteria were sensed; and

using the quality of service module to record the identity of the sensed application(s) and the occurrence of an exception in a log.

21. The method of claim 20, and further comprising the steps of:

periodically uploading the log to a server;

compiling, at the server, exceptions and sensed applications to discern patterns in which applications were running at the time that the exceptions occurred; and

attributing responsibility to one or more of the sensed applications based on the compiled data.

22. The method of claim 20, and further including the steps of:

calculating a quality of service (QoS) index for an interval during which the at least one exception occurred; and

distributing the value of the QoS index among ones of the software applications then running to derive a plurality of QoS application indices.

23. The method of claim 22, and further including the steps of:

at the time the at least one exception occurs, using the quality of service module to determine the amounts of consumption of at least one predetermined resource of the IAP by each of the running applications; and

distributing the value of the QoS index to a preselected number of the applications consuming the largest amounts of the at least one predetermined resource.

24. The method of claim 23, wherein said at least one predetermined resource is selected from the group consisting of memory and processor usage.

25. Apparatus for assessing the quality of service (QoS) present at an information access point (IAP), comprising:

an operational user profile stored at the IAP, the operational user profile storing predetermined criteria for each of a plurality of states for each of a plurality of predetermined IAP functions or conditions, the states being indicative of different degrees of degraded performance;

an exception capture component of a quality of service (QoS) module of the IAP periodically sensing the performance of each predetermined function or condition of the IAP, comparing the performance to the respective criterion for the function or condition stored in the operational user profile, the exception capture component declaring an exception when a sensed performance of any predetermined function or condition enters into one of said states; and

a QoS calculator of the QoS module for calculating a single QoS index for the IAP based on one or more declared exceptions.

26. The apparatus of claim 25, wherein the operational user profile stores a numerical value for each state, said numerical value used by the QoS calculator in calculating the QoS index.

27. The apparatus of claim 26, wherein the operational user profile stores a plurality of weighting factors selectively used by the QoS calculator to modify the numerical value of the state of each declared exception, the weighting factors applied as a function of the kind of exception, the identity of the user and/or the circumstances under which the exception occurred.

28. The apparatus of claim 27, wherein the QoS module further includes a user attendance probe to detect whether the IAP is in attended or nonattended status, the attended/nonattended status being used to determine one of the weighting factors applied to each declared exception.

29. The apparatus of claim 27, wherein the QoS module further determines the time of day when the exception occurred, the time of day being used to determine one of the weighting factors applied to each declared exception.

30. The apparatus of claim 27, wherein the QoS module further determines the day of the week when the exception occurred, the day of the week being used to determine one of the weighting factors applied to each declared exception.

31. The apparatus of claim 25, wherein said predetermined functions or conditions of the IAP include two or more of:

memory usage, processor usage, a rate of input/output operations, availability of printing, disk space, status of applications running on the IAP, status of computer system services, time which the computer takes to respond to a user input, status of a network to which the IAP is connected, events noted by an operating system of the IAP, and a manual entry by the user of the IAP that an exception exists.

32. The apparatus of claim 31, wherein the exception capture component of the QoS module monitors input/output operations to and from at least one hard disk of the IAP.

33. The apparatus of claim 25, wherein the IAP is a workstation.

34. The apparatus of claim 25, wherein the quality of service module is created by programming a general purpose processor of the IAP.

35. The apparatus of claim 25, wherein the quality of service module further includes a snapshot component which, at the time that an exception is declared, records which of a predetermined plurality of applications is running on the IAP, the snapshot component recording the identity of ones of the running applications to a log together with the fact that an exception has occurred.

36. The apparatus of claim 35, wherein the snapshot component determines the amount of at least one IAP resource that each of the running applications is using, the QoS module recording the identities of the highest users of said resource among said running applications to the log.

37. The apparatus of claim 35, wherein the predetermined plurality of applications is stored locally on the IAP.

38. The apparatus of claim 25, wherein the exception capture component records the time at which the exception occurred, the QoS index calculator calculating the QoS index as normalized for the period of time in which said one or more exceptions occurred.

39. The apparatus of claim 25, wherein an application performance meter periodically senses and records performance data of each of a plurality of predetermined applications or tasks then running on the IAP, a QoS baseline calculator heuristically calculating ones of said predetermined criteria for said states based on the recorded performance data of said predetermined applications or tasks.

40. The apparatus of claim 39, wherein said recorded performance data include data concerning each application'"'"'s or task'"'"'s consumption of IAP resources.

41. The apparatus of claim 39, wherein said performance data indicate one or more of the following:

CPU consumption, memory consumption, time to respond to user input, amounts of network and disk traffic, handle usage, number of threads and number of page faults.

42. The apparatus of claim 39, wherein the IAP is a node on a network which also includes a QoS server, the recorded performance levels of each of a plurality of IAPs periodically uploaded to the QoS server, said QoS baseline calculator resident at said QoS server and calculating said predetermined criteria as a function of the recorded performance levels of each of a plurality of said applications or tasks, said QoS baseline calculator periodically downloading said predetermined criteria to a plurality of IAPs on the network.

43. Apparatus for assessing the quality of service delivered by an information access point (IAP) to a user, comprising:

an exception capture component of a quality of service (QoS) module for monitoring each of a plurality of predetermined functions or conditions of the IAP, the exception capture component comparing a sensed performance level of each predetermined function or condition against respective criteria stored on the IAP, each stored criterion defining a state of degraded performance for a respective function or condition, the exception capture component declaring an exception if a sensed performance level of any of the predetermined functions or conditions enters into one of said states;

a user presence probe of the QoS module for determining the presence or absence of a user at the IAP at the time for which the exception is declared; and

a quality of service calculator of the QoS module coupled to the exception capture component and the user presence probe for calculating a quality of service index as a function of the exception and whether the user is present at the time the exception occurred.

44. The apparatus of claim 43, wherein the user presence probe declares an attended status upon sensing a user input and resets a clock at that time, the probe declaring an unattended status if no further user input is received within a predetermined period of time, the user presence probe reestablishing an attended status upon sensing the next user input.

45. The apparatus of claim 43, wherein the predetermined functions or conditions include at least two from the following:

memory usage, processor usage, rate of input/output operations, disk space, status of applications running on the IAP, status of computer system services, time with which the IAP takes to respond to a user input, events noted by an operating system of the IAP, availability of a network to which the IAP is connected, printing availability, and manual entry by a user of the IAP that an exception exists.

46. The apparatus of claim 43, wherein the IAP is a workstation.

47. The apparatus of claim 43, wherein the QoS module is a general purpose processor programmed by a QoS computer program.

48. Apparatus for assessing the quality of service delivered by an information access point (IAP), comprising:

a quality of service (QoS) module for periodically monitoring performance levels of each of a plurality of predetermined functions or conditions of the IAP;

an application performance meter for periodically recording performance data for each software application or process running on the IAP;

a QoS baseline calculator coupled to the application performance meter for periodically receiving the recorded application performance data, the QoS baseline calculator heuristically calculating a plurality of baseline application performance criteria based on the recorded application performance data, the baseline application performance criteria being made available to an exception capture component of the QoS module for comparison with the sensed application performance levels, the exception capture component of the QoS module declaring an exception upon a sensed application performance level deviating from a respective application performance criterion by more than a predetermined amount.

49. The apparatus of claim 48, wherein the IAP is a node in a network which also includes a QoS server, the QoS server having mounted thereon the QoS baseline calculator, the application performance meter of the IAP periodically uploading the recorded application performance data, the QoS baseline calculator receiving the recorded application performance data from each of a plurality of IAPs on the network and calculating the baseline performance criteria based on the recorded application performance data from multiple IAPs, the QoS baseline calculator downloading the application baseline criteria to each of a plurality of IAPs on the network.

50. The apparatus of claim 48, wherein the IAP is a workstation.

51. The apparatus of claim 48, wherein the application performance meter is a general purpose processor of the IAP which has been programmed with a computer program.

52. Apparatus for assessing the quality of service (QoS) present at an information access point (IAP), comprising:

a QoS module of the IAP for periodically monitoring performance levels of each of a plurality of predetermined functions or conditions of the IAP;

a file of predetermined performance criteria against which respective performance levels of the functions or conditions of the IAP may be compared by the IAP, each stored criterion indicative of a state of degraded performance on the part of one of said predetermined functions or conditions, the IAP declaring an exception whenever at least one of the predetermined functions or conditions enters into one of said states; and

the IAP running one or more of a plurality of predetermined software applications, the identities of the software applications then running being noted by a snapshot component of the QoS module at the time that an exception is declared, ones of said identities being recorded in an exception log coupled to the QoS module.

53. The apparatus of claim 52, wherein the QoS module further includes a QoS calculator which calculates a QoS index as a predetermined function of the sensed states associated with one or more exceptions, the QoS calculator further calculating a QoS application index for each of said software applications for which the identity has been recorded in the exception log, the QoS calculator distributing the value of the QoS index to selected ones of the applications then running to derive one or more respective QoS application indices for the last said applications.

54. The apparatus of claim 53, wherein the snapshot component of the QoS module reads, at the time that it notes the identities of the applications then running, the amount of at least one IAP resource being consumed by each noted application, the largest consumers of said at least one IAP resource being selected by the QoS module for recordation in the exception log.

55. The apparatus of claim 53, wherein the snapshot component determines, at the time that an exception is declared, one or more of the following for each application then running:

memory usage and CPU usage.

56. The apparatus of claim 53, wherein the IAP is a node on a network to which a QoS server and at least one other IAP having a QoS module are connected, the QoS modules of the IAPs periodically uploading the QoS application indices to the QoS server, the QoS server deriving QoS application indices across a plurality of IAPs from the QoS application indices uploaded from the IAPs.

57. The apparatus of claim 52, wherein the QoS module is a general purpose processor of the IAP which has been programmed with a QoS computer program.

58. The apparatus of claim 52, wherein the IAP is a workstation.

59. The apparatus of claim 52, wherein the predetermined functions or conditions monitored by the QoS module include at least two of the following:

memory usage, processor usage, rate of input/output operations, availability of printing, disk space, status of IAP system services, status of applications running on the IAP, time which the IAP takes to respond to user input, events noted by an operating system of the IAP, network availability and a manual entry by a user of the computer that an exception exists.

60. A machine-readable medium preprogrammed with a computer program with instructions which, when executed by a processor of a computer, creates a quality of service (QoS) module which performs the following steps:

periodically monitoring performance levels of each of a plurality of predetermined functions or conditions of the computer;

comparing the sensed performance levels to stored criteria each indicative of a state of said functions or conditions, the states for which said criteria are stored each defining a level of degraded performance for a respective function or condition;

declaring an exception whenever at least one of the monitored functions or conditions enters into one of said states; and

calculating, as a function of all such declared exceptions occurring within a predetermined period of time, a QoS index indicative of the relative operational state of the computer during that period of time.

61. The medium of claim 60, wherein the functions or conditions include at least two of the following:

processor usage, memory usage, rate of input/output operations, availability of printing, disk space, status of applications running on the computer, status of computer system services, time which the computer takes to respond to a user input, status of a network to which the computer is connected, events noted by an operating system of the computer, and a manual entry by the user that an exception exists.

62. The medium of claim 61, wherein the functions or conditions include a rate of input/output operations to one or more hard disks on the computer.

63. The medium of claim 61, wherein predetermined criteria are stored for each of a plurality of states for each of the monitored functions or conditions, the states representing different degrees of degradation in performance.

64. The medium of claim 63, wherein the QoS module attributes a stored base value to each of the states, each declared exception having attributed to it one of the base values according to the state which the respective function or condition has entered and being weighted by one or more factors according to kind of exception, the identity of the user and/or the circumstances under which the exception occurred.

65. The medium of claim 64, wherein the factors include a factor for the time of day during which the exception occurred.

66. The medium of claim 64, wherein the factors include a factor for the day of the week during which the exception occurred.

67. The medium of claim 61, wherein the QoS index is normalized for the period of time for which it is calculated.

68. A machine-readable medium preprogrammed with a computer program with instructions which, when executed by a processor of a computer, creates a quality of service (QoS) assessment module which performs the following steps:

declaring an exception at a time when the performance of one of a predetermined number of computer functions or conditions enters into a state indicative of a degraded level of performance;

determining whether, at such time, a user was present at the computer;

attributing one of a plurality of stored base values to the exception based on its kind and severity;

weighting the base value of the exception with a value indicative of the presence or absence of the user at the computer at such time; and

calculating, as a function of all such weighted exceptions occurring within a predetermined period of time, a quality of service index indicative of the relative operational state of the computer.

69. The medium of claim 68, wherein the QoS assessment module further includes a user attendance probe which performs the following steps:

counting a period of time since the last sensed user input;

if the counted period of time exceeds a predetermined limit, declaring the computer to be in an unattended status; and

declaring the computer to be in an attended status if a new user input is sensed.

70. The medium of claim 68, wherein the functions or conditions include at least two of the following:

processor usage, memory usage, rate of input/output operations, availability of printing, disk space, status of applications running on the IAP, status of computer system services, time which the computer takes to respond to a user input, status of a network to which the computer is connected, events noted by an operating system of the computer, and a manual entry by a user of the computer that an exception exists.

71. A machine-readable medium preprogrammed with a computer program having instructions which, when executed by a processor of a computer, creates a quality of service assessment module that performs the following steps:

periodically monitoring performance levels of each of a plurality of predetermined computer functions or conditions;

comparing the performance levels to stored, predetermined criteria which define predetermined states for each of the functions or conditions, each of said states indicative of a degraded level of performance for a respective function or condition;

declaring an exception whenever a monitored performance level of one of the functions or conditions is found to have entered into one of said states;

for the time for which the exception is declared, determining which of a predetermined plurality of software applications was running on the computer; and

recording the exception as referenced to the identities of ones of the software applications.

72. The medium of claim 71, wherein the quality of service assessment module performs the following additional steps:

for the time for which the exception is declared, determining which of the running ones of the identified computer software applications were the highest users of at least one kind of computer resource; and

tracking responsibility for the exception by recording the identities of said highest users among the running software applications together with the fact that an exception has occurred.

73. The medium of claim 71, wherein the quality of service assessment module performs the following additional steps:

calculating, for a predetermined period of time, a quality of service index as a function of all exceptions recorded for the period of time; and

distributing the quality of service index to each of said ones of said software applications to derive one or more respective QoS application indices.

74. The medium of claim 71, wherein the predetermined functions or conditions include at least two of the following:

processor usage, memory usage, rate of input/output operations, availability of printing, disk space, status of applications running on the IAP, status of computer system services, time which the computer takes to respond to a user input, status of a network to which the computer is connected, events noted by an operating system of the computer, and a manual entry by the user of the computer that an exception exists.

75. The medium of claim 71, wherein predetermined criteria for each of several states are stored for each of said functions or conditions, the states being indicative of different levels of degradation of performance.

76. A machine-readable medium preprogrammed with a computer program having instructions which, when executed by a processor of a computer, creates a quality of service (QoS) assessment module and an application performance meter which perform the following steps:

using the application performance meter to periodically record performance data for one or more software applications then running on the computer;

using the recorded performance data to derive a set of baseline performance criteria for each of the software applications;

using the QoS module to monitor performance levels of software applications running on the computer;

using the QoS module to compare the monitored performance levels to respective ones of the baseline performance criteria; and

declaring an exception whenever at least one of the monitored performance levels departs from a respective performance criterion by more than a predetermined amount.