Method and apparatus for measuring latency in web services

• US 7,685,270 B1
• Filed: 03/31/2005
• Issued: 03/23/2010
• Est. Priority Date: 03/31/2005
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Scharf, James E. Jr., Vermeulen, Allan H.
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US11/095,919
• Time in Patent Office: 1,818 Days
• Field of Search: 709223-224, 709/203, 710/36, 370/252
• US Class Current: 709/224
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5083 wherein the managed service relates to web hosting

Abstract

Method and apparatus for measuring true end-to-end latency for calls to Web services are described. In embodiments, a Web service client and a Web service provider may collaborate to collect timing/latency data for calls to the Web service. This data may be collected, stored, and analyzed by a latency measurement service to generate displays and/or reports on true end-to-end latency measurements for Web service calls. Embodiments may collect Internet/network infrastructure latency for Web service calls up to and including the “last mile” to the Web service client and the Web service processing time. Additionally, by analyzing latency data collected from a number of Web services clients and/or Web service providers, embodiments may provide a macro-level view into overall Internet performance. In one embodiment, the latency measurement service may be a Web service.

Claims

1. A system, comprising:

a processor; and

a memory comprising program instructions, wherein the program instructions are executable by the processor to implement a latency measurement service configured to;

collect Web service request processing time data sent to the latency measurement service from a Web service, the processing time data including both a respective unique request identifier and a server-side processing time for each of one or more Web service requests received by the Web service from a Web service client via a network, wherein the latency measurement service is a separate service from the Web service, and wherein the request processing time data is sent separate from the one or more Web service requests;

collect Web service request latency data sent to the latency measurement service from the Web service client, the service request latency data including both a respective client-side request latency and unique request identifier for each of the Web service requests sent to the Web service by the Web service client via the network, wherein the service request latency data is sent separate from the one or more Web service requests; and

correlate and store the Web service request processing time data and the Web service request latency data for each Web service request according to the respective unique request identifier associated with each of the one or more Web service requests.

2. The system as recited in claim 1, wherein the latency measurement service is further configured to generate one or more Web service latency reports from the correlated and stored data.

3. The system as recited in claim 2, wherein the latency measurement service is further configured to provide the Web service latency reports to the Web service.

4. The system as recited in claim 2, wherein the latency measurement service is further configured to provide the Web service latency reports to the Web service and the Web service client.

5. The system as recited in claim 1, wherein the latency measurement service is further configured to generate a Web service latency report from the correlated and stored data, wherein the Web service latency report indicates the client processing time, the Web service processing time, and a network latency for Web service requests in a given time period, wherein the network latency for a Web service request is the client processing time for the Web service request minus the Web service processing time for the Web service request.

6. The system as recited in claim 1, wherein the Web service request processing time for a Web service request indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending a Web service response to the Web service client via the network.

7. The system as recited in claim 1, wherein Web service request latency data collected from the Web service client for a Web service request indicates elapsed time between the Web service client sending the Web service request to the Web service via the network and the Web service client receiving a Web service response from the Web service via the network.

8. The system as recited in claim 1, wherein the latency measurement service is a Web service.

9. The system as recited in claim 8,

wherein, to collect Web service request processing time data from a Web service for Web service requests received by the Web service from a Web service client via a network, the latency measurement service is further configured to receive Web service request messages each including the Web service request processing time data for a particular Web service request from the Web service;

and

wherein, to collect Web service request latency data from the Web service client for the Web service requests sent to the Web service by the Web service client via the network, the latency measurement service is further configured to receive Web service request messages each including the Web service request latency data for a particular Web service request from the Web service client.

10. The system as recited in claim 1, wherein the latency measurement service is further configured to:

collect Web service request processing time data from a plurality of Web services for Web service requests received by the Web services from Web service clients via the network;

collect Web service request latency data from the Web service clients for the Web service requests sent to the Web services by the Web service clients via the network; and

correlate and store the Web service request processing time data and the Web service request latency data according to a unique request identifier associated with one or more of the Web service requests.

11. The system as recited in claim 10, wherein the latency measurement service is further configured to generate one or more Web service latency reports from the stored data for particular Web service clients and particular Web services, wherein at least one of the reports indicates the Web service request latency, the Web service request processing time, and a network latency for at least a subset of the Web service requests, wherein the network latency for a Web service request is the Web service request latency minus the Web service request processing time.

12. The system as recited in claim 10, wherein the latency measurement service is further configured to generate network performance reports from at least a subset of the stored data, wherein a network performance report indicates network latencies on one or more regions of the network, wherein the network latency for a region is calculated from the Web service request processing time data and the Web service request latency data collected from Web services and Web service clients within that region, and wherein the network latency for a particular Web service request is the Web service request latency minus the Web service request processing time.

13. A method, comprising:

a latency measurement service collecting Web service request processing time data sent to the latency measurement service from a Web service, the processing time data including both a respective unique request identifier and a server-side processing time for each of one or more Web service requests received by the Web service from a Web service client via a network, wherein the latency measurement service is a separate service from the Web service, and wherein the request processing time data is sent separate from the one or more Web service requests;

the latency measurement service collecting Web service request latency data sent to the latency measurement service from the Web service client, the service request latency data comprising both a respective client-side request latency and unique request identifier for each of the one or more Web service requests sent to the Web service by the Web service client via the network, wherein the service request latency data is sent separate from the one or more Web service requests; and

the latency measurement service correlating and storing the Web service request processing time data and the Web service request latency data for each Web service request according to the respective unique request identifier associated with each of the one or more Web service requests.

14. The method as recited in claim 13, further comprising the latency measurement service generating one or more Web service latency reports from the stored data, wherein at least one of the reports indicates the Web service request latency, the Web service request processing time, and a network latency for at least a subset of the Web service requests, wherein the network latency for a Web service request is the Web service request latency minus the Web service request processing time.

15. The method as recited in claim 13, wherein the Web service request processing time for a Web service request indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending a Web service response to the Web service client via the network.

16. The method as recited in claim 13, wherein the Web service request latency data collected from the Web service client for a Web service request indicates elapsed time between the Web service client sending the Web service request to the Web service via the network and the Web service client receiving a Web service response from the Web service via the network.

17. The method as recited in claim 13,

wherein said collecting Web service request processing time data from a Web service for Web service requests received by the Web service from a Web service client via a network comprises receiving Web service request messages each including the Web service request processing time data for a particular Web service request from the Web service;

and

wherein said collecting Web service request latency data from the Web service client for the Web service requests sent to the Web service by the Web service client via the network comprises receiving Web service request messages each including the Web service request latency data for a particular Web service request from the Web service client.

18. A computer-accessible storage medium storing program instructions, wherein the program instructions are computer-executable to implement a latency measurement service configured to:

collect Web service request processing time data sent to the latency measurement service from a Web service, the processing time data including both a respective unique request identifier and a server-side processing time for each of one or more Web service requests received by the Web service from a Web service client via a network, wherein the latency measurement service is a separate service from the Web service, and wherein the request processing time data is sent separate from the one or more Web service requests;

collect Web service request latency data sent to the latency measurement service from the Web service client, the service request latency data including both a respective client-side request latency and unique request identifier for each of the Web service requests sent to the Web service by the Web service client via the network, wherein the service request latency data is sent separate from the one or more Web service requests; and

correlate and store the Web service request processing time data and the Web service request latency data for each Web service request according to the respective unique request identifier associated with each of the one or more Web service requests.

19. The computer-accessible storage medium as recited in claim 18, wherein the latency measurement service is further configured to generate one or more Web service latency reports from the stored data, wherein at least one of the reports indicates the Web service request latency, the Web service request processing time, and a network latency for at least a subset of the Web service requests, wherein the network latency for a Web service request is the Web service request latency minus the Web service request processing time.

20. The computer-accessible storage medium as recited in claim 18, wherein the Web service request processing time for a Web service request indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending a Web service response to the Web service client via the network.

21. The computer-accessible storage medium as recited in claim 18, wherein Web service request latency data collected from the Web service client for a Web service request indicates elapsed time between the Web service client sending the Web service request to the Web service via the network and the Web service client receiving a Web service response from the Web service via the network.

22. The computer-accessible storage medium as recited in claim 18,

wherein, in said collecting Web service request processing time data from a Web service for Web service requests received by the Web service from a Web service client via a network, the latency measurement service is further configured to receive Web service request messages each including the Web service request processing time data for a particular Web service request from the Web service;

and

wherein, in said collecting Web service request latency data from the Web service client for the Web service requests sent to the Web service by the Web service client via the network, the latency measurement service is further configured to receive Web service request messages each including the Web service request latency data for a particular Web service request from the Web service client.

23. A system, comprising:

a processor; and

a memory comprising program instructions, wherein the program instructions are executable by the processor to implement a Web service configured to;

receive a Web service request from a Web service client via a network;

process the Web service request to generate response data for the Web service request;

record a Web service processing time indicating elapsed time for said processing the Web service request;

generate a request identifier that uniquely identifies the Web service request;

send a Web service response comprising the response data and the request identifier to the Web service client via the network; and

separate from the Web service request and the Web service response, send the Web service processing time and the request identifier to a latency measurement service, wherein the Web service is a separate service from the latency measurement service, wherein the latency measurement service is configured to correlate and store a client processing time for the Web service request and the Web service processing time for the Web service request according to the request identifier.

24. The system as recited in claim 23, wherein the Web service is further configured to include the request identifier and the Web service processing time for the Web service request in the Web service response.

25. The system as recited in claim 23, wherein the Web service processing time indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending the Web service response to the Web service client via the network.

26. The system as recited in claim 23, wherein the client processing time for the Web service request indicates elapsed time between the Web service client sending the Web service request to the Web service via the network and the Web service client receiving the Web service response from the Web service via the network.

27. The system as recited in claim 23, wherein the Web service is further configured to receive input from the latency measurement service comprising one or more Web service latency reports generated from the correlated and stored client processing times and Web service processing times for the Web service and the Web service client.

28. A method, comprising:

receiving, by a Web service, a Web service request from a Web service client via a network;

processing, by the Web service, the Web service request to generate response data for the Web service request;

recording, by the Web service, a Web service processing time indicating elapsed time for said processing the Web service request;

generating, by the Web service, a request identifier that uniquely identifies the Web service request;

sending, by the Web service, a Web service response comprising the response data and the request identifier to the Web service client via the network; and

separate from the Web service request and the Web service response, sending, by the Web service, the Web service processing time and the request identifier to a latency measurement service, wherein the Web service is a separate service from the latency measurement service, wherein the latency measurement service is configured to correlate and store a client processing time for the Web service request and the Web service processing time for the Web service request according to the request identifier.

29. The method as recited in claim 28, further comprising including the request identifier and the Web service processing time for the Web service request in the Web service response.

30. The method as recited in claim 28, wherein the Web service processing time indicates elapsed time between receiving the Web service request from the Web service client via the network and sending the Web service response to the Web service client via the network, and wherein the client processing time for the Web service request indicates elapsed time between the Web service client sending the Web service request via the network and the Web service client receiving the Web service response via the network.

31. The method as recited in claim 28, further comprising receiving input from the latency measurement service comprising one or more Web service latency reports generated from the correlated and stored client processing times and Web service processing times for the Web service client.

32. A computer-accessible storage medium storing program instructions, wherein the program instructions are computer-executable to implement a Web service configured to:

receive a Web service request from a Web service client via a network;

process the Web service request to generate response data for the Web service request;

record a Web service processing time indicating elapsed time for said processing the Web service request;

generate a request identifier that uniquely identifies the Web service request;

send a Web service response comprising the response data and the request identifier to the Web service client via the network; and

separate from the Web service request and the Web service response, send the Web service processing time and the request identifier to a latency measurement service, wherein the Web service is a separate service from the latency measurement service, wherein the latency measurement service is configured to correlate and store a client processing time for the Web service request and the Web service processing time for the Web service request according to the request identifier.

33. The computer-accessible storage medium as recited in claim 32, wherein the Web service is further configured to include the request identifier and the Web service processing time for the Web service request in the Web service response.

34. The computer-accessible storage medium as recited in claim 32, wherein the Web service processing time indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending the Web service response to the Web service client via the network, and wherein the client processing time for the Web service request indicates elapsed time between the Web service client sending the Web service request to the Web service via the network and the Web service client receiving the Web service response from the Web service via the network.

35. The computer-accessible storage medium as recited in claim 32, wherein the Web service is further configured to receive input from the latency measurement service comprising one or more Web service latency reports generated from the correlated and stored client processing times and Web service processing times for the Web service and the Web service client.

36. A system, comprising:

a processor; and

a memory comprising program instructions, wherein the program instructions are executable by the processor to implement a Web service client configured to;

send a Web service request to a Web service on a network;

receive a Web service response from the Web service including response data for the Web service request and a request identifier that uniquely identifies the Web service request;

record a client processing time for the Web service request; and

separate from the Web service request and the Web service response, send the client processing time for the Web service request and the request identifier to a latency measurement service, wherein the latency measurement service is a separate service from the Web service, and wherein the latency measurement service is configured to correlate and store the client processing time for the Web service request and a Web service processing time for the Web service request according to the request identifier.

37. The system as recited in claim 36, wherein the client processing time for the Web service request indicates elapsed time between said sending the Web service request and said receiving the Web service response.

38. The system as recited in claim 36, wherein the Web service response further includes the Web service processing time for the Web service request.

39. The system as recited in claim 36, wherein the Web service processing time indicates elapsed time between the Web service receiving the Web service request from the Web service client via the network and the Web service sending the Web service response to the Web service client via the network.

40. The system as recited in claim 36, wherein the Web service client is further configured to receive input from the latency measurement service comprising one or more Web service latency reports generated from the correlated and stored client processing times and Web service processing times for the Web service and the Web service client.

41. A method, comprising:

sending, by a Web service client, a Web service request to a Web service on a network;

receiving, by the Web service client, a Web service response from the Web service including response data for the Web service request and a request identifier that uniquely identifies the Web service request;

recording, by the Web service client, a client processing time for the Web service request; and

separate from the Web service request and the Web service response, sending, by the Web service client, the client processing time for the Web service request and the request identifier to a latency measurement service, wherein the latency measurement service is a separate service from the Web service, and wherein the latency measurement service is configured to correlate and store the client processing time for the Web service request and a Web service processing time for the Web service request according to the request identifier.

42. The method as recited in claim 41, wherein the client processing time for the Web service request indicates elapsed time between said sending the Web service request and said receiving the Web service response.

43. The method as recited in claim 41, wherein the Web service response further includes the Web service processing time for the Web service request.

44. The method as recited in claim 41, wherein the Web service processing time indicates elapsed time between the Web service receiving the Web service request via the network and the Web service sending the Web service response via the network.

45. The method as recited in claim 41, further comprising receiving input from the latency measurement service comprising one or more Web service latency reports generated from correlated and stored client processing times and Web service processing times for the Web service.

46. A computer-accessible storage medium storing program instructions, wherein the program instructions are computer-executable to implement a Web service client configured to:

send a Web service request to a Web service on a network;

receive a Web service response from the Web service including response data for the Web service request and a request identifier that uniquely identifies the Web service request;

record a client processing time for the Web service request; and

separate from the Web service request and the Web service response, send the client processing time for the Web service request and the request identifier to a latency measurement service, wherein the latency measurement service is a separate service from the Web service, and wherein the latency measurement service is configured to correlate and store the client processing time for the Web service request and a Web service processing time for the Web service request according to the request identifier.

47. The computer-accessible storage medium as recited in claim 46, wherein the client processing time for the Web service request indicates elapsed time between said sending the Web service request and said receiving the Web service response.

48. The computer-accessible storage medium as recited in claim 46, wherein the Web service response further includes the Web service processing time for the Web service request.

49. The computer-accessible storage medium as recited in claim 46, wherein the Web service processing time indicates elapsed time between the Web service receiving the Web service request via the network and the Web service sending the Web service response via the network.

50. The computer-accessible storage medium as recited in claim 46, wherein the Web service client is further configured to receive input from the latency measurement service comprising one or more Web service latency reports generated from correlated and stored client processing times and Web service processing times for the Web service and the Web service client.

51. A latency measurement system, comprising one or more hardware servers coupled to a network and configured to:

host a Web service, wherein the Web service is configured to, for each Web service request received from a Web service client on the network, record a Web service processing time for the Web service to process the Web service request;

host a latency measurement service, wherein the latency measurement service is a separate service from the Web service, wherein the latency measurement service is configured to, for and Web service request sent to the Web service by the Web service client, collect, correlate and store the Web service processing time and a client processing time according to a request identifier that uniquely identifies the Web service request, wherein the service processing time is sent to the latency measurement service by the Web service along with the request identifier and wherein the client processing time is sent to the latency measurement service by the Web services client along with the request identifier;

wherein the client processing time indicates elapsed time between the Web service client sending the Web service request to the Web service and the Web service client receiving the Web service response to the Web service request from the Web service, and wherein the Web service processing time and the client processing time are sent to the latency measurement service separate from the Web service request and the Web service response.

52. The system as recited in claim 51, wherein the latency measurement service is further configured to generate one or more Web service latency reports from the stored client processing times and Web service processing times.

53. The system as recited in claim 51, wherein the latency measurement service is implemented as a Web service.

54. A method, comprising:

for each Web service request received from a Web service client on a network, a Web service recording a Web service processing time indicating elapsed time for the Web service to process the Web service request;

for each Web service request sent to the Web service by the Web service client, a latency measurement service collecting, correlating and storing the Web service processing time and a client processing time according to a request identifier that uniquely identifies the Web service request, wherein the service processing time is sent to the latency measurement service by the Web service along with the request identifier and wherein the client processing time is sent to the latency measurement service by the Web services client along with the request identifier;

wherein the latency measurement service is a separate service from the Web service, wherein the client processing time indicates elapsed time between the Web service client sending the Web service request to the Web service and the Web service client receiving the Web service response to the Web service request from the Web service, and wherein the Web service processing time and the client processing time are sent to the latency measurement service separate from the Web service request and the Web service response.