Automatic web service monitoring

• US 9,191,285 B1
• Filed: 02/22/2013
• Issued: 11/17/2015
• Est. Priority Date: 03/14/2012
• Status: Active Grant

• Current Assignee: Domo, Inc.
• Original Assignee: Domo, Inc.
• Inventors: Isaacson, Derrick
• Primary Examiner(s): Serrao, Ranodhi
• Application Number:
  US13/774,113
• Time in Patent Office: 998 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5096 wherein the managed service relates to distributed or central networked applications
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks

Abstract

According to various embodiments, the present invention provides live site performance monitoring in the context of logical resource hierarchies found in cloud computing systems. In particular, in one embodiment, the system of the present invention facilitates monitoring of new computing services without necessarily requiring explicit knowledge about such new services. Such services can include, for example and without limitation, those provided using a Representational State Transfer (REST) architecture, although the techniques described herein can also be used with other architectures. In one embodiment, the system of the present invention operates by automatically creating groups of items for performance monitoring, based on the hierarchy of a URI. Heuristics can be used to determine aggregatable hierarchy nodes.

Claims

1. A computer-implemented method for automatically monitoring a web service, comprising:

intercepting an incoming request directed to the web service at a computing device comprising a processor and memory, the intercepted request comprising a uniform resource identifier (URI) that comprises one or more static path elements and one or more dynamic elements;

associating the intercepted request with one of a plurality of monitoring groups, wherein the monitoring groups are embodied as entries in a data structure maintained on a non-transitory storage device such that each entry corresponds to a resource of the web service, wherein the entries comprise respective counters comprising performance statistics pertaining to requests directed to the resource of the entry, and wherein associating the intercepted request with a hierarchical monitoring group comprises;

forming a modified URI for the incoming request in the memory of the computing device using the processor, wherein forming the modified URI comprises replacing a dynamic element of the incoming request URI with a common value using the processor,

matching the modified URI in the memory of the computing device to a web resource corresponding to one of the entries of the data structure using the processor, and

associating the incoming request with the hierarchical monitoring group of the matching web resource,

gathering performance statistics pertaining to a response generated by the web service to the incoming request at the computing device; and

updating the counter of the entry associated with the incoming request in response to gathering the performance statistics.

2. The computer-implemented method of claim 1, wherein the web service provides resources at respective resource paths, and wherein each entry in the data structure comprises a path identifier that corresponds to one or more of the resource paths, the method further comprising:

establishing a path identifier for a particular entry corresponding to a resource path of the web service by including a common value in the resource path to represent a dynamic element in a URI of an incoming request; and

matching the particular entry to the modified URI of the incoming request by comparing the modified URI to the path identifier of the particular entry.

3. The computer-implemented method of claim 2, wherein establishing the path identifier for the particular entry comprises inserting the common value between two or more static path elements of the resource path.

4. The computer-implemented method of claim 1, wherein the resources of the web service comprise respective hierarchical resource paths, the method further comprising:

maintaining the entries of the respective monitoring groups in a hierarchical tree data structure on the non-transitory storage device, such that each entry corresponds to a respective hierarchical resource path of the web service; and

establishing hierarchical relationships between the entries in the hierarchical tree data structure that correspond to the hierarchical resource paths of the respective entries.

5. The computer-implemented method of claim 4, further comprising:

forming a path identifier for a particular entry in the hierarchical tree data structure, the particular entry corresponding to a hierarchical resource path that comprises a plurality of static path elements by inserting a common value into the path identifier to represent dynamic elements of a URI directed to the corresponding resource of the web service; and

matching the particular entry to the modified URI of the incoming request by comparing the modified URI of the incoming request to the path identifier of the particular entry.

6. The computer-implemented method of claim 4, further comprising aggregating performance statistics of a plurality of entries based on hierarchical relationships between the entries in the hierarchical tree data structure.

7. The computer-implemented method of claim 6, wherein aggregating the performance statistics comprises combining performance statistics of a selected entry of the hierarchical tree data structure and one or more child entries of the selected entry in the hierarchical tree data structure.

8. The computer-implemented method of claim 1, further comprising:

creating a new entry on the non-volatile storage medium in response to determining that the modified URI of the incoming request does not match web resources of any of the entries in the data structure; and

associating the incoming request with the new entry.

9. The computer-implemented method of claim 1, further comprising establishing entries of a monitoring tree data structure on the non-transitory storage device in response to intercepting requests directed to the web service such that the entries correspond to respective hierarchical resource paths, and hierarchical relationships between the entries correspond to hierarchical relationships of the respective resource paths.

10. A system, comprising:

a monitoring data store comprising a non-transitory storage medium storing a tree data structure comprising a plurality of tree nodes, wherein each of the tree nodes comprises a) a resource path identifier corresponding to a resource of a web service, b) monitoring statistics pertaining to the respective resource;

a computing device comprising a processor and network interface, computing device comprising a request filter configured for execution on the processor, wherein the request filter is configured to track monitoring statistics pertaining to the respective resources of the web service by use of the tree data structure, and wherein the request filter is further configured to;

monitor incoming requests directed to the web service received through a network,

generate modified resource identifiers corresponding to the incoming requests, wherein the modified resource identifier of an incoming request replaces a variable embedded within a resource identifier of the incoming request with a common value,

correlate the incoming requests with respective tree nodes by comparing the modified resource identifiers of the incoming requests to the resource path identifiers of the tree nodes,

capture metrics pertaining to the respective incoming requests, wherein the captured metrics of an incoming request comprise metrics related to a response generated by the web service to the incoming request, and

update the monitoring statistics of the tree nodes that are correlated to the respective incoming requests on the monitoring data store based on the captured metrics pertaining to the respective incoming requests.

11. The system of claim 10, wherein the resource path identifier for the resource of a particular tree node comprises a plurality of static path elements and a common value to represent a variable, and wherein the request filter is configured to correlate an incoming request to the particular tree node by matching the static path elements and the common value of the resource path identifier to static path elements and a common value of the modified resource identifier of the incoming request.

12. The system of claim 11, wherein the tree nodes are arranged in the tree data structure according to hierarchical relationships between resource path identifiers of the respective tree nodes.

13. The system of claim 12, wherein the resource path identifiers of the tree nodes comprise a hierarchy of one or more path elements, and wherein the hierarchical relationships between the tree nodes corresponds to the hierarchy of the one or more path elements of the respective tree nodes.

14. The system of claim 13, wherein hierarchy of one or more path elements of a particular tree node comprises one or more static path elements and one or more common values, wherein the common values are configured to represent variables embedded within resource identifiers of requests directed to the web service resource corresponding to the particular tree node.

15. The system of claim 13, wherein the request filter is configured to automatically identify hierarchical relationships between the tree nodes by parsing resource identifiers of the incoming requests, wherein the request filter is configured to add a new tree node to the tree data structure in response to monitoring an incoming request having a modified resource identifier that does not correlate to a resource path of any of the tree nodes.

16. A non-transitory computer-readable storage medium comprising executable instructions configured to cause a computing device to perform operations, the operations comprising:

filtering incoming Hypertext Transfer Protocol (HTTP) requests directed to a web server, each incoming HTTP request comprising a uniform resource locator (URL) to reference a resource of the web server;

accessing a monitoring tree data structure stored on non-volatile storage medium, the monitoring tree comprising nodes that correspond to respective resources of the web server, each node comprising a resource path identifier corresponding to a resource of the web server and a set of monitoring statistics;

aggregating monitoring statistics pertaining to the incoming HTTP requests within the monitoring tree data structure, wherein aggregating monitoring statistics for an incoming HTTP request comprises;

deriving a modified resource identifier from the URL of the incoming HTTP request by replacing dynamic path elements of the URL with a common value,

searching the monitoring tree to identify a node that comprises a resource path identifier that matches the modified resource identifier of the incoming HTTP request,

acquiring monitoring statistics corresponding to a response generated by the web server servicing to the incoming HTTP request, and

updating monitoring statistics of the identified node with the acquired monitoring statistics.

17. The non-transitory computer-readable storage medium of claim 16, wherein the monitoring statistics of a node comprise one or more of a request count, an average latency, a 99th percentile latency, and a response status code count.

18. The non-transitory computer-readable storage medium of claim 16, further comprising inferring hierarchical relationships between nodes in the tree data structure based on a resource hierarchy of the respective resource path identifiers of the nodes.

19. The non-transitory computer-readable storage medium of claim 17, further comprising combining monitoring statistics of a plurality of nodes based on the inferred hierarchical relationships between the plurality of nodes.

20. The non-transitory computer-readable storage medium of claim 17, wherein deriving the modified resource identifier further comprises parsing the URL of the incoming HTTP request to distinguish the static path elements of the URL from the dynamic path element.