Method and apparatus for analyzing communications on different threads

• US 6,493,754 B1
• Filed: 11/05/1998
• Issued: 12/10/2002
• Est. Priority Date: 08/10/1995
• Status: Expired due to Term

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: Moore, Steven J., Rosborough, James M.
• Primary Examiner(s): Burgess, Glenton B.
• Assistant Examiner(s): SALAD, ABDULLAHI ELMI
• Application Number:
  US09/186,906
• Time in Patent Office: 1,496 Days
• Field of Search: 709/224, 709/236, 370/241, 370/252, 370/253, 370/394, 700/226, 700/227, 700/203
• US Class Current: 709/224
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/349 for interfaces, buses
• CPC Class Codes: G06F 2201/87 Monitoring of transactions
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 43/0864 Round trip delays
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]

Abstract

The present invention provides a method and apparatus for measuring transaction response times. The method and apparatus can identify service request sequences corresponding to a transaction and the start and stop times for the transaction. The invention can be applied non-intrusively/non-invasively to the service packets communicated between a source and destination node.

Claims

1. A method for analyzing a plurality of data packets communicated between a first computational component and at least one second computational component, comprising:

generating a communications data set including information relating to the plurality of data packets;

sorting the plurality of data packets based on at least one of source and destination to form a plurality of communications data subsets containing data packets indexed by the at least one of source and destination; and

analyzing the data packets in each of the plurality of communications data subsets based at least partially on a timing of at least some of the data packets.

2. The method as claimed in claim 1, wherein the communications data set includes information relating to a timing in communicating a collection of data packets from the plurality of data packets and wherein the analyzing step comprises:

determining a pattern for grouping at least some of said plurality of data packets, wherein said grouping corresponds to an occurrence of said pattern, and wherein said pattern is dependent on an ordering of data packets of said plurality of data packets; and

comparing said communications data set with said pattern for determining whether at least some of the collection of data packets correspond to an occurrence of said pattern.

3. The method as claimed in claim 2, wherein said generating step comprises:

detecting with a detecting device said data packets communicated along a communications line extending between the first and second computational components;

determining, for each of said data packets, a corresponding received time when the data packet is detected; and

recording an identity of each of said data packets and said corresponding received time.

4. The method as claimed in claim 3, further comprising:

adding to said corresponding received time for a data packet, a transit time corresponding substantially to the time required by a data packet to travel from said detecting device to said first computational component.

5. The method as claimed in claim 2, wherein each of a majority of said data packets has at least one of a corresponding node address and a corresponding port number and said communications data set includes for each of said data packets, a corresponding received time when the data packet is detected on said communication line, and said generating step comprises:

reading with a detecting device each of said data packets from said communications line;

filtering said data packets, read by said detecting device, based on at least one of node address and port number to form a selected set of data packets; and

recording for each data packet in said selected set of data packets, said corresponding received time.

6. The method as claimed in claim 2, wherein said data packets correspond to a plurality of threads with each thread corresponding to thread identification information and said comparing step comprises:

sorting said data packets in said communications data set into a plurality of thread data sets wherein the data packets in each thread data set have the same thread identification information.

7. The method as claimed in claim 2, wherein said data packets include service request packets and service results packets, each said service request packet corresponds to a service request from the first computational component, and said comparing step comprises:

identifying said service request packets in said data packets using contents of said data packets.

8. The method as claimed in claim 2, wherein said data packets of said collection includes service request packets and corresponding service results packets with each of the service request packets and corresponding service results packets corresponding to a service request and said comparing step comprises:

identifying said service request packets in said collection using contents of said data packets in said collection;

correlating, each service results packet of said collection of service results packets, with a corresponding one of said service request packets; and

determining a corresponding start and a corresponding stop time for each of the corresponding service requests, the corresponding start time being related to a corresponding service request packet and the corresponding stop time related to a corresponding service results packet.

9. The method as claimed in claim 8, further comprising:

computing a response time for said grouping using at least some of said start times and said stop times.

10. The method as claimed in claim 8, wherein the determining step comprises:

comparing a time interval between the stop of a first data packet of said plurality of data packets and the start of a second data packet of said plurality of data packets with a predetermined value for said time interval to identify a sequence of data packets corresponding to an occurrence of the grouping.

11. The method as claimed in claim 2, wherein at least some of said data packets in said collection are service request packets for requesting service from the first computational component, and, said comparing step comprises:

first matching a first service request packet in said collection with a representation of a service request packet in a pattern characterization data set;

second matching a second service request packet in said collection with another representation of a service request packet in said pattern characterization data set wherein a time interval between said first and second service request packets is no more than a predetermined value.

12. The method as claimed in claim 2, wherein said data packets of said collection correspond to a plurality of service requests from the first computational component to the second computational component, said service requests correspond to a plurality of thread data sets, and said comparing step comprises:

first matching a first service request corresponding to a first thread with a representation of a service request packet in a pattern characterization data set; and

second matching a second service request corresponding to a second thread with another representation of a service request in said pattern characterization data set, wherein a time interval between said first and second service requests is no more than a predetermined value.

13. The method as claimed in claim 2, wherein said collection is substantially identical to said plurality of data packets.

14. The method as claimed in claim 2, wherein said collection includes said grouping of data packets.

15. The method as claimed in claim 2, wherein said grouping and said collection have substantially different data packets of said plurality of data packets.

16. An apparatus for analyzing a plurality of data packets communicated between a first computational component and at least one second computational component, comprising:

means for generating a communications data set including information relating to the plurality of data packets;

means for sorting the plurality of data packets based on at least one of source and destination to form a plurality of communications data subsets containing data packets indexed by the at least one of source and destination; and

means for analyzing the data packets in each of the plurality of communications data subsets based at least partially on a timing of at least some of the data packets.

17. The apparatus as claimed in claim 16, wherein the communications data set includes information relating to a timing in communicating a collection of data packets from the plurality of data packets and wherein the means for analyzing further includes:

means for determining a pattern for grouping at least some of said plurality of data packets, wherein said grouping corresponds to an occurrence of said pattern, wherein said pattern is dependent on an ordering of data packets of said plurality of data packets, and wherein the means for determining is in communication with the means for generating;

means for comparing said communications data set with said pattern for determining whether at least some of the collection of data packets correspond to an occurrence of said pattern, wherein the means for comparing is in communication with the means for determining.

18. The apparatus as claimed in claim 17, wherein said comparing means comprises means for contrasting said plurality of data packets and the order in which said data packets are received by said generating means against a pattern characterization data set.

19. The apparatus as claimed in claim 16, wherein said generating means includes a detecting means for detecting said data packets, said detecting means located on at least a portion of a communications line between said first and second computational components.

20. A method for identifying a pattern of communication data packets from a plurality of data packets communicated between a first computational component and at least a second computational component, comprising:

generating a communications data set comprising (i) information relating to a plurality of data packets and (ii) a timing for each data packet;

sorting the information based on at least one of source and destination to form a plurality of communications data subsets containing data packets indexed by the at least one of source and destination; and

analyzing the timing of a first data packet and the timing of a second data packet to identify an ordering of said data packets.

21. The method as claimed in claim 20, wherein, in the analyzing step, a time interval between the timing of the first and second data packets is compared with a predetermined value for said time interval.

22. The method as claimed in claim 21, wherein a portion of said data packets correspond to a thread and at least two data packets correspond to different threads and said data packets comprise:

(a) a plurality of service request packets, and (b) a plurality of service result packets, wherein each said service request packet corresponds to at least one of the service result packets and said sorting step comprises;

identifying each of said service request packets in said plurality of data packets;

correlating each of said service result packets with the corresponding service request packet for forming a service data subset, wherein said service data subset includes the correlated service request packets and the service result packets; and

sorting said service data subsets by thread to form a plurality of thread data sets with the data packets in said thread data set having the same thread addresses.

23. The method as claimed in claim 20, wherein a plurality of sequences of the data packets correspond to a plurality of transactions and said analyzing step comprises:

recording each of said sequences of data packets and the number of occurrences of each sequence in a pattern characterization data set.

24. The method as claimed in claim 21, further comprising:

recording each occurrence of said ordering of data packets and the total number of occurrences thereof in a first communications data set;

selecting a second predetermined value;

comparing said time interval against said second predetermined value to identify a second ordering of said data packets; and

recording each occurrence of said second ordering of data packets and the total number of occurrences thereof in a second communications data set.

25. The method as claimed in claim 24, further comprising:

selecting a third predetermined value based on a relationship between (i) the number of occurrences of said ordering of data packets and said predetermined value and (ii) the number of occurrences of said second ordering of data packets and said second predetermined value.

26. The method as claimed in claim 25, further comprising:

comparing said time interval against said third predetermined value for said time interval to identify a third ordering of said data packets.

27. The method as claimed in claim 26, further comprising:

comparing said third ordering against said communications data set to determine whether at least a portion of said plurality of data packets correspond to said third ordering.

28. The method as claimed in claim 20, further comprising:

computing a response time for said ordering.

29. The method as claimed in claim 20, wherein said analyzing step produces a pattern characterization data set having a representation of a plurality of orderings of data packets and further comprising:

comparing said data packets from said analyzing step with said pattern characterization data set to determine if said data packets are contained in said pattern characterization data set.

30. The method as claimed in claim 20, wherein the first data packet corresponds to a first thread and the second data packet corresponds to a second thread.

31. An apparatus for identifying a pattern of communication data packets from a plurality of data packets communicated between a first computational component and at least a second computational component, comprising:

means for generating a communications data set comprising (i) information relating to a plurality of data packets and (ii) a timing for each data packet;

means for sorting the information based on at least one of source and destination to form a plurality of communications data subsets containing data packets indexed by the at least one of source and destination; and

means for analyzing a timing of a first data packet and a timing of a second data packet to identify an ordering of said data packets, wherein the means for generating is in communication with the means for sorting and the means for comparing.

32. The apparatus as claimed in claim 31, wherein said means for analyzing compares a time interval between the timing of the first data packet and the timing of the second data packet with a predetermined value for said time interval and wherein said comparing means comprises means for contrasting said plurality of data packets and the ordering thereof against a predetermined ordering of data packets.

33. The apparatus as claimed in claim 31, wherein at least a portion of said plurality of data packets relates to different service request packets, each service request packet corresponding to a service request, said generating means provides a first data set including (i) said plurality of data packets and (ii) a start time and a stop time for each service request corresponding to at least some of said plurality of data packets, and said analyzing means comprises means for contrasting a time interval between the stop time of a first service request and the start time of a second service request against a predetermined value for said time interval to identify a sequence of said service requests corresponding to a transaction.

34. A system for analyzing a plurality of data packets communicated between a first component and a second component, comprising:

a recorder operable to detect the plurality of data packets communicated between the first and second components; and

a monitor coupled to the recorder and operable to;

sort the plurality of data packets based on at least one of a source and a destination to form a plurality of communications data subsets containing data packets indexed by the at least one of the source and the destination; and

analyze the data packets in each of the plurality of communications data subsets based at least partially on a timing of at least some of the data packets.

35. The system of claim 34, wherein the monitor is operable to analyze the data packets by:

determining a pattern for grouping at least some of the plurality of data packets, the pattern dependent on an ordering of at least some of the data packets; and

comparing the plurality of data packets with the pattern to determine whether at least some of the data packets correspond to an occurrence of the pattern.

36. The system of claim 35, wherein the monitor is operable to compare the plurality of data packets with the pattern by comparing the plurality of data packets and an order in which the data packets are received against a pattern characterization data set.

37. The system of claim 34, wherein the recorder comprises a probe.

38. A system for identifying a pattern of data packets in a plurality of data packets communicated between a first component and at least one second component, comprising:

a recorder operable to detect the plurality of data packets communicated between the first and second components; and

a monitor coupled to the recorder and operable to;

sort the plurality of data packets based on at least one of a source and a destination to form a plurality of communications data subsets containing data packets indexed by the at least one of the source and the destination; and

analyze a timing of a first data packet and a timing of a second data packet to identify an ordering of the first and second data packets.

39. The system of claim 38, wherein the monitor is operable to analyze the first and second data packets by:

comparing a time interval between the timing of the first and second data packets with a predetermined value; and

comparing the ordering of the data packets with a predetermined order.

40. The system of claim 38, wherein:

the first and second data packets relate to different service request packets, each service request packet corresponding to a service request; and

the monitor compares a time interval between the stop time of a first service request and the start time of a second service request against a predetermined value to identify a sequence of service requests corresponding to a transaction.

41. A method for analyzing communication between a first component and at least one second component, comprising:

receiving a plurality of data packets communicated over a plurality of threads between the first component and at least one second component, the data packets associated with a plurality of service requests;

determining a start time and a stop time for at least some of the service requests associated with at least some of the data packets;

sorting the service requests into a plurality of thread data subsets based on the thread over which the data packets associated with the service requests were communicated; and

analyzing the service requests in each of the thread data subsets.

42. The method of claim 41, wherein the data packets comprise service request packets and service completion packets, each service request associated with at least one service request packet and at least one service completion packet.

43. The method of claim 42, further comprising generating a communications data set by:

receiving a first of the data packets;

determining that the first data packet comprises a service request packet;

examining at least one second data packet to identify a service completion packet; and

recording a receive time of the first data packet and a receive time of the second data packet, the receive time of the first data packet comprising the start time of the service request associated with the first data packet, the receive time of the second data packet comprising the stop time of the service request associated with the first data packet.

44. The method of claim 41, further comprising sorting the service requests in each thread data subset by at least one of the start time and the stop time.

45. The method of claim 41, wherein analyzing the service requests comprises:

determining a time interval between the stop time of a first service request and the start time of a second service request, the first and second service requests associated with a common thread data subset;

comparing the time interval to a predetermined value; and

adding at least one of the first and second service requests to a pattern list when the time interval is less than the predetermined value, the pattern list identifying at least a portion of a service request sequence associated with a transaction.

46. The method of claim 45, wherein:

the time interval comprises a first time interval; and

analyzing the service requests further comprises;

determining a second time interval between the stop time of the first service request and the start time of a third service request, the first and third service requests associated with different thread data subsets;

comparing the second time interval to the predetermined value; and

transferring the pattern list to the thread data subset of the third service request when the second time interval is less than the predetermined value.

47. The method of claim 46, wherein analyzing the service requests further comprises:

comparing the pattern list to one or more identified patterns in a pattern characterization data set when the first and second time intervals exceed the predetermined value;

updating a number of occurrences of one of the identified patterns when the pattern list matches that identified pattern; and

recording the pattern list as a new pattern in the pattern characterization data set when the pattern list does not match the identified patterns in the pattern characterization data.

48. The method of claim 47, wherein analyzing the service requests in each of the thread data subsets comprises analyzing the service requests in each of the thread data subsets in parallel.

49. The method of claim 47, wherein analyzing the service requests further comprises:

repeating processing of the service requests using a plurality of additional predetermined values;

using a total number of patterns identified for each of the predetermined values to select an optimal predetermined value; and

repeating processing of the service requests using the optimal predetermined value to generate a second pattern characterization data set comprising an optimal listing of patterns.

50. A system for analyzing communication between a first component and at least one second component, comprising:

at least one recorder operable to detect a plurality of data packets communicated between the first and second components over a plurality of threads; and

a monitor coupled to the at least one recorder and operable to;

receive the plurality of data packets, the data packets associated with a plurality of service requests;

determine a start time and a stop time for at least some of the service requests associated with at least some of the data packets;

sort the service requests into a plurality of thread data subsets based on the thread over which the data packets associated with the service requests were communicated; and

analyze the service requests in each of the thread data subsets.

51. The system of claim 50, wherein the data packets comprise service request packets and service completion packets, each service request associated with at least one service request packet and at least one service completion packet.

52. The system of claim 51, wherein the monitor is operable to determine the start time and the stop time for at least some of the service requests by:

receiving a first of the data packets;

determining that the first data packet comprises a service request packet;

examining at least one second data packet to identify a service completion packet; and

recording a receive time of the first data packet and a receive time of the second data packet, the receive time of the first data packet comprising the start time of the service request associated with the first data packet, the receive time of the second data packet comprising the stop time of the service request associated with the first data packet.

53. The system of claim 50, wherein the monitor is further operable to sort the service requests in each thread data subset by at least one of the start time and the stop time.

54. The system of claim 50, wherein the monitor is operable to analyze the service requests by:

determining a time interval between the stop time of a first service request and the start time of a second service request, the first and second service requests associated with a common thread data subset;

comparing the time interval to a predetermined value; and

adding at least one of the first and second service requests to a pattern list when the time interval is less than the predetermined value, the pattern list identifying at least a portion of a service request sequence associated with a transaction.

55. The system of claim 54, wherein:

the time interval comprises a first time interval; and

the monitor is further operable to analyze the service requests by;

determining a second time interval between the stop time of the first service request and the start time of a third service request, the first and third service requests associated with different thread data subsets;

comparing the second time interval to the predetermined value; and

transferring the pattern list to the thread data subset of the third service request when the second time interval is less than the predetermined value.

56. The system of claim 55, the monitor is further operable to analyze the service requests by:

comparing the pattern list to one or more identified patterns in a pattern characterization data set when the first and second time intervals exceed the predetermined value;

updating a number of occurrences of one of the identified patterns when the pattern list matches that identified pattern; and

recording the pattern list as a new pattern in the pattern characterization data set when the pattern list does not match the identified patterns in the pattern characterization data.

57. The system of claim 56, wherein the monitor is operable to analyze the service requests in each of the thread data subsets in parallel.

58. The system of claim 56, wherein the monitor is further operable to analyze the service requests by:

repeating processing of the service requests using a plurality of additional predetermined values;

using a total number of patterns identified for each of the predetermined values to select an optimal predetermined value; and

repeating processing of the service requests using the optimal predetermined value to generate a second pattern characterization data set comprising an optimal listing of patterns.

59. A method for identifying a pattern in communication between a first component and at least one second component, comprising:

receiving a plurality of data packets communicated over a plurality of threads between the first component and at least one second component, the data packets associated with a plurality of service requests;

determining a start time and a stop time for at least some of the service requests associated with at least some of the data packets;

sorting the service requests into a plurality of thread data subsets based on the thread over which the data packets associated with the service requests were communicated; and

analyzing the service requests in each of the thread data subsets to identify one or more transactions involving at least at portion of the service requests.

60. The method of claim 59, further comprising sorting the service requests in each thread data subset by at least one of a start time and a stop time associated with each service request.

61. The method of claim 59, wherein analyzing the service requests comprises comparing the service requests in the thread data subsets against a plurality of transaction patterns in a pattern characterization data set.

62. The method of claim 61, wherein comparing the service requests in the thread data subsets against the transaction patterns comprises:

selecting a first service request in a first thread data subset;

comparing the first service request to an initial service request in each of the transaction patterns;

adding the first service request to a transaction list when the first service request matches the initial service request in at least one transaction pattern; and

discarding the first service request and selecting at least one additional service request in the first thread data subset when the first service request does not match the initial service request in any of the transaction patterns.

63. The method of claim 62, wherein comparing the service requests in the thread data subsets against the transaction patterns further comprises:

selecting a second service request in the first thread data subset when the first service request matches at least one initial service request in at least one transaction pattern;

comparing the second service request to a secondary service request in the at least one transaction pattern; and

adding the second service request to the transaction list when the second service request matches the secondary service request in at least one transaction pattern.

64. The method of claim 63, wherein comparing the service requests in the thread data subsets against the transaction patterns further comprises:

comparing a third service request in a second thread data subset to the secondary service request in the at least one transaction pattern when the second service request does not match the secondary service request in any of the at least one transaction pattern; and

transferring the transaction list to the second thread data subset when the third service request matches the secondary service request in the at least one transaction pattern.

65. The method of claim 63, wherein comparing the service requests in the thread data subsets against the transaction patterns further comprises:

determining whether the secondary service request in the at least one transaction pattern is a final service request in the at least one transaction pattern when the second service request matches the secondary service request in the at least one transaction pattern; and

recording a start time and a stop time associated with the transaction list when the secondary service request in the at least one transaction pattern is the final service request in the at least one transaction pattern.

66. A system for identifying a pattern in communication between a first component and at least one second component, comprising:

at least one recorder operable to detect a plurality of data packets communicated between the first and second components over a plurality of threads; and

a monitor coupled to the at least one recorder and operable to;

receive the plurality of data packets, the data packets associated with a plurality of service requests;

determine a start time and a stop time for at least some of the service requests associated with at least some of the data packets;

sort the service requests into a plurality of thread data subsets based on the thread over which the data packets associated with the service requests were communicated; and

analyze the service requests in each of the thread data subsets to identify one or more transactions involving at least at portion of the service requests.

67. The system of claim 66, wherein the monitor is further operable to sort the service requests in each thread data subset by at least one of a start time and a stop time associated with each service request.

68. The system of claim 66, wherein the monitor is operable to analyze the service requests by comparing the service requests in the thread data subsets against a plurality of transaction patterns in a pattern characterization data set.

69. The system of claim 68, wherein the monitor is operable to compare the service requests in the thread data subsets against the transaction patterns by:

selecting a first service request in a first thread data subset;

comparing the first service request to an initial service request in each of the transaction patterns;

adding the first service request to a transaction list when the first service request matches the initial service request in at least one transaction pattern; and

discarding the first service request and selecting at least one additional service request in the first thread data subset when the first service request does not match the initial service request in any of the transaction patterns.

70. The system of claim 69, wherein the monitor is further operable to compare the service requests in the thread data subsets against the transaction patterns by:

selecting a second service request in the first thread data subset when the first service request matches at least one initial service request in at least one transaction pattern;

comparing the second service request to a secondary service request in the at least one transaction pattern; and

adding the second service request to the transaction list when the second service request matches the secondary service request in at least one transaction pattern.

71. The system of claim 70, wherein the monitor is operable to compare the service requests in the thread data subsets against the transaction patterns by:

comparing a third service request in a second thread data subset to the secondary service request in the at least one transaction pattern when the second service request does not match the secondary service request in any of the at least one transaction pattern; and

transferring the transaction list to the second thread data subset when the third service request matches the secondary service request in the at least one transaction pattern.

72. The system of claim 70, wherein the monitor is operable to compare the service requests in the thread data subsets against the transaction patterns by:

determining whether the secondary service request in the at least one transaction pattern is a final service request in the at least one transaction pattern when the second service request matches the secondary service request in the at least one transaction pattern; and

recording a start time and a stop time associated with the transaction list when the secondary service request in the at least one transaction pattern is the final service request in the at least one transaction pattern.