Method, system, and product for assessing a server application performance

• US 5,812,780 A
• Filed: 05/24/1996
• Issued: 09/22/1998
• Est. Priority Date: 05/24/1996
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Simpson, Russell Lee Jr., Durasoff, Matthew David, Chen, John Yun-Kuang, Shakib, Darren Arthur, Lockard, Eric Neil
• Primary Examiner(s): Rinehart, Mark H.
• Application Number:
  US08/653,412
• Time in Patent Office: 851 Days
• Field of Search: 395/200.54, 395/674, 395/675, 395/241, 395/200.38, 395/200.57, 395/200.33, 395/200.53, 395/500, 370/252, 370/253, 702/122, 702/188
• US Class Current: 709/224
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Method and system for assessing the performance of a server application that acquires performance information from the perspective of a simulated user and has significantly reduced hardware requirements. Particularly, actual user behavior is modeled so that accurate determinations can be made as to the number of users a given server application can adequately support.

User behavior is modeled in a client profile that contains user parameters corresponding to the nature, timing, and frequency of user activities in operating a client that in turn corresponds to client tasks. A plurality of processes and process threads are initiated to contact a server as a plurality of simulated clients from a single client computer, each simulated client making a separate logical connection to the server. A task scheduler will schedule the simulated client tasks that are determined for each simulated user by reference to the user parameters in the client profile throughout a work day. The scheduler also introduces a random element so that the tasks simulate natural variability in user behavior.

User receivable response times for the task corresponding to simulated user activity are maintained in a log file and the 95th percentile time or score for each task type is calculated. The individual task type scores may be weighted and averaged together to arrive at a weighted average response time. The weighted average response time can then be used as a threshold value to determine the total number of users a server application can adequately support.

Claims

1. A method for assessing server application software performance under a simulated load generated by at least one client computer connected to at least one server running the server application software via a communications network without having to create actual clients of the server application software in an actual configuration while retaining actual user behavior characteristics, the method comprising the steps of:

defining at least one client profile based on one or more user parameters representing actual user behavior characteristics;

establishing a plurality of logical connections between the at least one client computer and the server application software, said logical connections simulating a plurality of logically simulated clients of the server application software and each client computer having many logical connections with the server application software;

generating periodically, for said logically simulated clients, one or more tasks for transmission to the server application software, the nature, timing and frequency of such tasks for said logically simulated clients being controlled by the client profile; and

communicating the tasks to the server application software from the at least one client computer so as to simulate the real time generation and transmission of messages from multiple actual clients of the server application software and without having to create actual clients of the server application software in an actual configuration.

2. A method for assessing server application software performance as recited in claim 1 further comprising the step of monitoring at the client computer responses received from the server application software to the tasks and response times therefor.

3. A method for assessing server application software performance as recited in claim 2 further comprising the step of computing the weighted average response time for all responses received from the server application software.

4. A method for assessing server application software performance as recited in claim 3 further comprising the step of determining the maximum number of clients the server application software can process at a desired weighted average response time.

5. A method for assessing server application software performance as recited in claim 1 further comprising the step of determining the maximum number of clients the server application software can process based on a predetermined criteria.

6. A method for assessing server application software performance as recited in claim 1 wherein the step of defining the client profile comprises defining a separate client profile for each logically simulated client.

7. A method for assessing server application software performance as recited in claim 1 wherein the step of establishing a plurality of logical connections comprises running a multitasking operating system on the at least one client computer such that said logical connections are established by a plurality of processes of said operating system.

8. A method for assessing server application software performance as recited in claim 1 wherein the step of establishing a plurality of logical connections comprises running a multithreaded, multitasking operating system on the client computer such that said logical connections are established by a plurality of threads of a single process of said operating system.

9. A method for assessing server application software performance as recited in claim 1 wherein the step of establishing a plurality of logical connections comprises running a multithreaded, multitasking operating system on the client computer such that said logical connections are established by a plurality of threads under each of a plurality processes of said operating system.

10. A method for assessing server application software performance as recited in claim 1 wherein the step of communicating the tasks to the server application software comprises scheduling each task for said logically simulated clients throughout a time period with an introduced random element to represent the natural variability in user behavior.

11. A method for assessing server application software performance as recited in claim 1 wherein the server application software is an E-mail server and the user parameters used in the client profile correspond to the nature, timing, and frequency of a user operating an E-mail client.

12. A method for assessing server application software performance as recited in claim 11 wherein the tasks comprise sending mail messages and further comprising a means for causing mail message task generation to converge to a steady state by use of a damping factor applied to existing mail messages.

13. A method for assessing E-mail server application software performance under a simulated load generated by at least one client computer connected to at least one server running the server application software via a communications network without having to create actual E-mail clients of the E-mail server application software in an actual configuration while retaining actual E-mail user behavior characteristics, the method comprising the steps of:

defining at least one client profile based on one or more user parameters representing actual E-mail user behavior characteristics;

establishing a plurality of logical connections between the at least one client computer and the E-mail server application software said logical connections simulating a plurality of logically simulated E-mail clients of the E-mail server application software and each client computer having many logical connections with the E-mail server application software, said logical connections comprising individual threads of a process in a multithreaded, multitasking operating system running on the client computer making each of the logical connections with the server;

generating periodically, for said logically simulated E-mail clients, one or more tasks for transmission to the E-mail server application software, the nature, timing and frequency of such tasks for said logically simulated clients being controlled by the client profile and such tasks including E-mail messages;

causing E-mail message task generation to converse to a steady state by use of a damping factor applied to existing E-mail messages to prevent E-mail message task generation from diverging in an unbounded manner,

communicating the tasks to the E-mail server application software from the at least one client computer so as to simulate the real time generation and transmission of messages from multiple actual E-mail clients of the E-mail server application software by scheduling each task for said logically simulated E-mail clients throughout a time period with an introduced random element to represent the natural variability in user behavior and without having to create actual E-mail clients of the E-mail server application software in an actual configuration;

monitoring at the at least one client computer responses received from the E-mail server application software to the tasks and response times therefor,

computing the weighted average response time for all responses from the E-mail server application software; and

determining the maximum number of actual E-mail clients the E-mail server application software can process at a desired weighted average response time.

14. A computer program product for use on at least one client computer connected to at least one server computer via a communications network comprising:

a computer usable medium having computer readable program code means embodied in said medium for assessing the performance of server application software running on a server computer under a simulated load generated by at least one client computer, said computer readable program code means comprising;

program code means for causing the client computer to store at least one client profile based on one or more user parameters representing actual user behavior characteristics;

program code means for causing the client computer to establish a plurality of logical connections between the client computer and the server application software, said logical connections simulating a plurality of logically simulated clients of the server application software and the client computer having many such logical connections;

program code means for causing the client computer to generate periodically, for said logically simulated clients, one or more tasks for transmission to the server application software, the nature, timing and frequency of such tasks for said logically simulated clients being controlled by the client profile; and

program code means for causing the client computer to communicate the tasks to the server application software from the client computer so as to simulate the real time generation and transmission of messages from multiple actual clients of the server application software and without having to create actual clients of the server application software in an actual configuration.

15. A computer program product as recited in claim 14 wherein the computer readable program means further comprises program code means for causing the client computer to monitor responses received from the server application software to the tasks and response times therefor.

16. A computer program product as recited in claim 15 wherein the computer readable program means further comprises program code means for causing the client computer to compute the weighted average response time for all responses received from the server application software.

17. A computer program product as recited in claim 16 wherein the computer readable program means further comprises program code means for using the weighted average response time for all responses previously computed to determine the maximum number of clients the server application software can process at a desired weighted average response time.

18. A computer program product as recited in claim 14 wherein the computer readable program means further comprises program code means for causing the client computer to compute the maximum number of clients the server application software can process based on a predetermined criteria.

19. A computer program product as recited in claim 14 wherein the computer readable program means further comprises program code means for storing client profile information for each logically simulated client.

20. A computer program product as recited in claim 14 wherein the client profile based on user parameters corresponds to a user operating an E-mail client.

21. A computer program product as recited in claim 20 wherein the tasks comprise sending mail messages and further comprising program code means for causing mail message task generation to converge to a steady state by use of a damping factor applied to existing mail messages.

22. A computer program product as recited in claim 14 wherein the computer readable program means for establishing the plurality of logical connections to the server application software comprises initiating a plurality of processes under a multitasking operating system, each process to make and control each logical connection.

23. A computer program product as recited in claim 14 wherein the computer readable program means for establishing the plurality of logical connections to the server application software comprises initiating a plurality of process threads under a process of a multithreaded, multitasking operating system, each process thread to make and control each logical connection.

24. A computer program product as recited in claim 14 wherein the computer readable program means for establishing the plurality of logical connections to the server application software comprises initiating a plurality of process threads, said plurality of process threads running under each of a plurality of processes of a multithreaded multitasking operating system, each process thread to make and control each logical connection.

25. A computer program product as recited in claim 14 wherein the computer readable program means for causing the client computer to communicate tasks to the server application software further comprises scheduling the tasks throughout a time period and introducing a random element to the scheduling of tasks to represent the natural variations in user behavior.

26. A computer program product for use on at least one client computer connected to at least one server computer via a communications network comprising:

a computer usable medium having computer readable program code means embodied in said medium for assessing the performance of E-mail server application software running on a server computer under a simulated load generated by at least one client computer, said computer readable program code means comprising;

program code means for causing the client computer to store at least one client profile based on one or more user parameters representing actual E-mail user behavior characteristics;

program code means for causing the client computer to establish a plurality of logical connections between the client computer and the E-mail server application software, said logical connections simulating a plurality of logically simulated E-mail clients of the E-mail server application software and the client computer having many such logical connections, said logical connections comprising program code means for initiating a separate process thread under a process of a multithreaded, multitasking operating system to make and control each logical connection;

program code means for causing the client computer to generate periodically, for said logically simulated E-mail clients, one or more tasks for transmission to the E-mail server application software the nature, timing and frequency of such tasks for said logically simulated E-mail clients being controlled by the client profile and such tasks including E-mail messages;

program code means for causing the client computer to communicate the tasks to the E-mail server application software so as to simulate the real time generation and transmission of messages from multiple actual E-mail clients of the E-mail server application software and without having to create actual E-mail clients of the E-mail server application software in an actual configuration;

program code means for causing E-mail message task generation to converge to a steady state by use of a damping factor applied to existing E-mail messages to prevent E-mail message task generation from diverging in an unbounded manner;

program code means for scheduling the tasks throughout a time period and introducing a random element to the scheduling of tasks to represent the natural variations in E-mail user behavior;

program code means for causing the client computer to monitor responses received from the E-mail server application software to the tasks and response times therefor;

program code means for causing the client computer to compute the weighted average response time for all responses received from the E-mail server application software.

27. A system for assessing server application software performance under a simulated load comprising:

at least one server computer running server application software; and

at least one client computer connected to the server computer via a communications network, said client computer comprising;

a CPU;

storage means, electronically coupled and responsive to said CPU, wherein said storage means is partitioned into at least a first memory partition and includes at least one client profile based on one or more user parameters representing actual user behavior characteristics stored in said first memory partition;

interface means, electronically coupled and responsive to said CPU, for establishing a plurality of logical connections between the client computer and the server application software, said logical connections simulating a plurality of logically simulated clients of the server application software and the client computer having many such logical connections;

means, electronically coupled and responsive to said CPU, for generating periodically, for said logically simulated clients, one or more tasks for transmission to the server application software, the nature, timing and frequency of such tasks for said logically simulated clients being controlled by the client profile; and

output means, electronically coupled and responsive to said CPU, for communicating the tasks to the server application software so as to simulate the real time generation and transmission of messages from multiple actual clients of the server application software and without having to create actual clients of the server application software in an actual configuration.

28. A system for assessing server application software performance under a simulated load as recited in claim 27 further comprising said storage means divided into a second memory partition and input means, electronically coupled and responsive to said CPU, for receiving responses from the server application software to the tasks and for storing said responses and response times therefor in said second memory partition.

29. A system for assessing server application software performance under a simulated load as recited in claim 28 further comprising said storage means divided into a third memory partition and means, electronically coupled and responsive to said CPU, for computing the weighted average response time for all responses received from the server application software and for storing said weighted average response time in said third memory partition.

30. A system for assessing server application software performance under a simulated load as recited in claim 29 further comprising said storage means divided into a fourth memory partition and means, electronically coupled and responsive to said CPU, for determining the maximum number of clients the server application software can adequately support based on the weighted average response time stored in said third memory partition and for storing said maximum number of clients in said fourth memory partition.

31. A system for assessing server application software performance under a simulated load as recited in claim 27 further comprising said storage means divided into a second memory partition and means, electronically coupled and responsive to said CPU, for determining the maximum number of clients the server application software can adequately support based on a predetermined criteria and for storing said maximum number or clients in said second memory partition.

32. A system for assessing server application software performance under a simulated load as recited in claim 27 further comprising said storage means first memory partition is subdivided into a plurality of memory partitions to store individual client profiles based on one or more user parameters for each logical connection representing a simulated client.

33. A system for assessing server application software performance under a simulated load as recited in claim 27 wherein the user parameters stored in the client profile pertain to the nature, timing, and frequency of a user operating an E-mail client.

34. A system for assessing server application software performance under a simulated load as recited in claim 33 wherein the tasks comprise sending mail messages and further comprising means, electronically coupled and responsive to said CPU, for causing mail message task generation to converge to a steady state by use of a damping factor applied to existing mail messages.

35. A system for assessing server application software performance under a simulated load as recited in claim 27 wherein the interface means establishes said logical connections by initiating a plurality of processes on a multitasking operating system, each process to make and control each logical connection.

36. A system for assessing sever application software performance under a simulated load as recited in claim 27 wherein the interface means establishes said logical connections by initiating a plurality of process threads on a multitasking, multithreaded operating system, each process thread to make and control each logical connection.

37. A system for assessing server application software performance under a simulated load as recited in claim 27 wherein the interface means establishes said logical connections by initiating a plurality of process threads, said plurality of process threads running under each of a plurality of processes on a multitasking, multithreaded operating system, each process thread to make and control each logical connection.

38. A system for assessing server application software performance under a simulated load as recited in claim 27 wherein the output means further comprises means for scheduling the tasks for execution throughout a given time period and for introducing a random clement to simulate natural user variability.

39. A system for assessing server application software performance under a simulated load comprising:

at least one server computer running server application software; and

at least one client computer connected to the server computer via a communications network, said client computer comprising;

a CPU;

storage means, electronically coupled and responsive to said CPU, wherein said storage means is partitioned into at least first, second and third memory partitions and includes at least one client profile based on one or more user parameters representing actual user behavior characteristics stored in said first memory partition;

interface means, electronically coupled and responsive to said CPU, for establishing a plurality of logical connections between the client computer and the server application software, said logical connections simulating a plurality of logically simulated clients of the server application software and the client computer having many such logical connections, wherein each logical connection is initiated by a separate process thread on a multitasking, multithreaded operating system to make and control the logical connection;

means, electronically coupled and responsive to said CPU, for generating periodically, for said logically simulated clients, one or more tasks for transmission to the server application software, the nature, timing and frequency of such tasks for said logically simulated clients being controlled by the client profile and such tasks including E-mail messages, for scheduling the execution of said tasks throughout a given time period, and for introducing a random element to said execution to simulate natural E-mail user variability;

output means, electronically coupled and responsive to said CPU, for communicating the tasks to the E-mail server application software so as to simulate the real time generation and transmission of messages from multiple actual E-mail clients of the E-mail server application software and without having to create actual E-mail clients of the E-mail server application software in an actual configuration,

means, electronically coupled and responsive to said CPU, for causing E-mail message task generation to converge to a steady state by use of a damping factor applied to existing E-mail messages to prevent E-mail message task generation from diverging in an unbounded manner;

input means, electronically coupled and responsive to said CPU, for receiving responses from the E-mail server application software to the tasks and for storing said responses and response times therefor in said second memory partition; and

means, electronically coupled and responsive to said CPU, for computing the weighted average response time for all responses received from the E-mail server application software and for storing said weighted average response time in said third memory partition.