Simulation of network traffic using non-deterministic user behavior models

• US 7,376,550 B1
• Filed: 10/26/2005
• Issued: 05/20/2008
• Est. Priority Date: 10/26/2005
• Status: Expired due to Fees

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Wang, Yong, Gao, Yue, Sidebottom, Greg, Bokaemper, Martin
• Primary Examiner(s): Shah; Kamini S
• Assistant Examiner(s): Luu; Cuong V
• Application Number:
  US11/258,985
• Time in Patent Office: 937 Days
• Field of Search: 714/32, 714/43, 717/124, 717/104, 370/389, 370/394, 703/21, 703/22
• US Class Current: 703/27
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 43/50 Testing arrangements

Abstract

A network testing environment includes a control server and a testing cluster composed of one or more load generating devices. The load generating devices output network communications in a non-deterministic manner to model real-world network users and test a network system. The load generating devices operate in accordance with probabilistic state machines distributed by the control server. The probabilistic state machines model patterns of interaction between users and the network system.

Claims

1. A method for simulating behavior of a network user comprising:

collecting data representing actual user behavior when interacting with a network;

generating a non-deterministic behavior model of a behavior of a network user, wherein generating a non-deterministic behavior model comprises specifying potential network communications that the network user could send and specifying a non-deterministic delay for each of the network communications, the non-deterministic delay comprising an average time that the user waits before or after generating the network communication and a distribution for how much the user is likely to deviate from the average time, wherein the average time and the distribution being determined based on the collected data; and

generating and outputting network communications to a network stem based on the generated non-deterministic behavior model to simulate the behavior of the network user and test the network system.

2. The method of claim 1, wherein the behavior model describes behavior that is realistic for a user interacting with a network system when the network system is deployed in a non-testing environment.

3. The method of claim 2, wherein the behavior model is based on empirically collected data.

4. The method of claim 1,

wherein generating a non-deterministic behavior model comprises generating a state machine having a plurality of nodes interconnected by a set of directed edges,

wherein each of the nodes represents an action that the user may perform on the network system, and

wherein each of the directed edges represents a transition from, a first node in the state machine to a second node in the state machine.

12. The method of claim 1, wherein the non-deterministic behavior model generates more than one sequence of network communications.

13. The method of claim 1, further comprising:

distributing simulation instructions to the load generating devices, wherein the simulation instructions comprise the behavior model and a number of users; and

executing the simulation instructions to produce the network communications that simulate the behavior for the specified number of users based on the behavior model.

14. The method of claim 13, further comprising compiling the behavior model into a series of commands.

15. The method of claim 13, wherein the specified number equals or exceeds 500,000 users.

16. The method of claim 13, wherein the specified number equals or exceeds 1,000,000 users.

17. The method of claim 1, wherein generating network communications comprises simulating the behavior of multiple concurrent network users by generating network communications based on the behavior model.

18. The method of claim 1, further comprising:

generating a second non-deterministic model of a behavior of a network user; and

generating network communications based on the second model to simulate behavior of a network user concurrently with a first model.

19. The method of claim 1, further comprising analyzing network communications from the network system to verify functionality of the network system.

20. The method of claim 19, wherein analyzing network communications comprises:

collecting actual network communications sent from the network system in response to network communications generated based on the model; and

comparing the actual network communications from the network system with expected network communications from the network system.

21. The method of claim 19, wherein analyzing network communications comprises:

collecting end user perceived response times to network requests from the network system; and

using the response times to analyze performance of the network system.

22. The method of claim 1, wherein the network system is any device or set of devices connected by a network, such that the device or the devices can receive and process network communications and can send network communications.

5. A method comprising:

collecting data representing actual user behavior when interacting with a network;

based on the collected data, generating a non-deterministic behavior model of a behavior of a network user as a state machine having a plurality of nodes interconnected by a set of directed edges, wherein each of the nodes represents an action that the user may perform on the network system, and wherein each of the directed edges represents a transition from a first node in the state machine to a second node in the state machine, and

generating and outputting network communications based on the generated non-deterministic behavior model to simulate the behavior of the network user and test a network system,

wherein each node in the plurality of nodes specifies an average amount of time before performing the action associated with the node.

6. The method of claim 5, wherein each node in the plurality of nodes describes a distribution of delay that describes how much the user is likely to deviate from the average time.

7. The method of claim 6, wherein the distribution of delay is a Poisson distribution.

8. The method of claim 7, wherein each of the directed edges is associated with a probability that the user performs an action associated with the second node.

9. The method of claim 5, wherein outputting the network communications comprises:

waiting a duration based on the average amount of time specified by a first node in the plurality of nodes;

performing the action specified by the first node;

selecting a directed edge in the plurality of directed edges based on a weight associated with the directed edge, wherein the directed edge is a member of a set of directed edges having the first node as a source node; and

transitioning to an end node of the directed edge.

10. The method of claim 9, wherein waiting a duration comprises waiting the duration based on a distribution for user deviation from the average time.

11. The method of claim 9, wherein performing the action comprises sending a network communication to the network system.

23. A system comprising:

a network system; and

a cluster of testing devices coupled to the network system via a network,

wherein the cluster of testing devices outputs network communications based on a non-deterministic behavior model that simulates the behavior of a network user to test the network system,

wherein the model specifies a delay for each of the network communications,

wherein the delay comprises an average time that the user waits before performing the action and a probability distribution for how the user is likely to deviate from the average time, and

wherein the average time and probability distribution is based on collected data representing actual user behavior.

24. The system of claim 23, wherein the model describes behavior that is realistic for a user interacting with a network system when the network system is deployed in a non-testing environment.

25. The method of claim 24, wherein the behavior model is based on empirically collected data.

26. The system of claim 23, wherein the model specifies potential network communications that a network user could send.

30. The system of claim 23, wherein the non-deterministic model generates more than one sequence of network communications.

31. The system of claim 23,

wherein the control server distributes simulation instructions to the cluster of testing devices, wherein the simulation instructions comprise the behavior model and a number of users;

and

wherein the cluster of testing devices executes the simulation instructions to produce the network communications that simulate the behavior for the specified number of users based on the behavior model

32. The system of claim 31, wherein the behavior model is compiled into a series of commands that the cluster of testing devices can process.

33. The system of claim 23, wherein the cluster of testing devices simulates the behavior of multiple concurrent network users by generating network communications based on the behavior model.

34. The system of claim 23, wherein the testing cluster of testing devices generates additional network communications based on a second model to simulate behavior of a network user while the cluster of testing devices is also generating the network communications based on a first behavior model.

35. The system of claim 23, further comprising a control server that distributes behavior model to the cluster of testing devices.

36. The system of claim 23, further comprising an analysis module that analyzes network communications from the network system to verify the functionality of the network system.

37. The system of claim 36,

wherein the cluster of testing devices collects actual network communications from the network system in response to network communications generated based on the model;

and

wherein the analysis module compares the actual network communications from the network system with expected network communications from the network system.

38. The system of claim 36,

wherein the cluster of testing devices collects end user perceived response times to network requests from the network system;

and

wherein the analysis module uses the end user perceived response times to analyze performance of the network system.

27. A system comprising:

a network system; and

a cluster of testing devices coupled to the network system via a network,

wherein the cluster of testing devices outputs network communications based on a non-deterministic behavior model that simulates the behavior of a network user to test the network system,

wherein the model comprises a state machine having a plurality of nodes interconnected by a set of directed edges,

wherein, each of the nodes in the plurality of nodes represents an action that the users may perform with respect to the network system given a particular state of the network system,

wherein, each of the directed edges represents a transition from a first node in the state machine to a second node in the state machine,

wherein each node in the plurality of nodes specifies an average amount of time to wait before performing the action associated with the node, and

wherein the average time is based on data representing actual user behavior.

28. The system of claim 27, wherein each node in the plurality of node specifies a distribution for how much the user is likely to deviate from the average time.

29. The system of claim 27, wherein each of the directed edges is associated with a probability that the user performs the action associated with the second node after performing the action associated with the first node.

39. A computer-readable storage medium containing instructions, wherein the instructions cause a programmable processor to:

generate a non-deterministic model of a behavior of a network user, wherein the non-deterministic behavior model specifies potential network communications that a network user could send and a non-deterministic delay for each of the network communications, the non-deterministic delay comprising an average time that the user waits before or after generating the network communication and a distribution for how much the user is likely to deviate from the average time, wherein the average time and distribution are determined from data representing actual user behavior; and

generate and output network communications to a network system based on the generated non-deterministic behavior model to simulate the behavior of the network user and test the network system.

40. The method of claim 39, wherein the model describes behavior that is realistic for a user interacting with a network system when the network system is deployed in a non-testing environment.