Interaction reconstruction in a service-oriented system

• US 9,928,517 B1
• Filed: 11/17/2014
• Issued: 03/27/2018
• Est. Priority Date: 11/17/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hitchcock, Daniel Wade, Goldberg, Anton Vladilenovich, Longley, James Daniel, Porter, Brandon William
• Primary Examiner(s): Park, Jeong S
• Application Number:
  US14/543,822
• Time in Patent Office: 1,226 Days
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 41/14 Network analysis or design
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data

Abstract

Methods, systems, and computer-readable media for implementing interaction reconstruction in a service-oriented system are disclosed. Routes are stored for a first set of service interactions between individual services in a service-oriented system. The routes comprise data indicative of call paths for node identifiers in the first set of service interactions. A sparse trace is generated for a second set of service interactions between individual ones of the services. The sparse trace comprises a plurality of node identifiers in the second set of service interactions. Based on the node identifiers in the sparse trace and the node identifiers in the plurality of routes, the sparse trace is matched to one or more of the routes.

Claims

1. A system, comprising:

a plurality of computing devices configured to implement a service-oriented system and a trace analysis system, wherein the service-oriented system comprises a plurality of services, and wherein the trace analysis system is configured to;

generate a plurality of routes for a first set of service interactions between individual ones of the plurality of services, wherein the plurality of routes are generated based on a first set of trace data received from the service-oriented system, and wherein the plurality of routes comprise data indicative of call paths for a plurality of node identifiers in the first set of service interactions; and

store data indicative of the plurality of routes;

wherein the service-oriented system is configured to;

generate a sparse trace for a second set of service interactions between individual ones of the plurality of services, wherein the sparse trace comprises a trace identifier and a plurality of node identifiers, and wherein the sparse trace excludes data indicative of call paths for at least some of the second set of service interactions; and

wherein the trace analysis system is configured to;

retrieve the sparse trace from the service-oriented system; and

match the sparse trace to one or more of the plurality of routes, wherein the sparse trace is matched based on the plurality of node identifiers in the sparse trace and the plurality of node identifiers in the plurality of routes.

2. The system as recited in claim 1, wherein, in generating the sparse trace, the service-oriented system is configured to:

activate, at a root node, a sparse tracing mode for the sparse trace; and

send, from the root node to one or more downstream nodes, trace metadata activating the sparse tracing mode for the sparse trace at the one or more downstream nodes.

3. The system as recited in claim 1, wherein, in generating the sparse trace, the service-oriented system is configured to:

activate a sparse tracing mode for the sparse trace, wherein the sparse tracing mode is activated based on analysis of the trace identifier at the individual ones of the plurality of services in the second set of service interactions, wherein the analysis is performed according to a policy shared by the individual ones of the plurality of services in the second set of service interactions.

4. The system as recited in claim 1, wherein the plurality of services are implemented by a plurality of hosts, and wherein the service-oriented system is configured to:

store the first set of trace data using local storage for the plurality of hosts.

5. A computer-implemented method, comprising:

storing data indicative of a plurality of routes for a first set of interactions between individual ones of a plurality of nodes, wherein the plurality of routes comprise data indicative of call paths for a plurality of node identifiers in the first set of interactions;

generating a sparse trace for a second set of interactions between individual ones of the plurality of nodes, wherein the sparse trace comprises a plurality of node identifiers in the second set of interactions; and

matching the sparse trace to one or more of the plurality of routes, wherein the sparse trace is matched based on the plurality of node identifiers in the sparse trace and the plurality of node identifiers in the plurality of routes.

6. The method as recited in claim 5, wherein the sparse trace excludes data indicative of call paths for at least some of the second set of interactions.

7. The method as recited in claim 5, wherein the sparse trace comprises a trace identifier, and wherein the plurality of node identifiers in the sparse trace are determined using the trace identifier.

8. The method as recited in claim 5, wherein generating the sparse trace comprises:

activating, at a root node, a sparse tracing mode for the sparse trace; and

sending, from the root node to one or more downstream nodes, trace metadata activating the sparse tracing mode for the sparse trace at the one or more downstream nodes.

9. The method as recited in claim 5, wherein the sparse trace comprises a trace identifier, and wherein generating the sparse trace comprises:

activating a sparse tracing mode for the sparse trace, wherein the sparse tracing mode is activated based on analysis of the trace identifier at the individual ones of the plurality of nodes in the second set of interactions, wherein the analysis is performed according to a policy shared by the individual ones of the plurality of nodes in the second set of interactions.

10. The method as recited in claim 5, wherein the plurality of routes are determined based on a set of trace events generated for the first set of interactions, and wherein the method further comprises:

storing the set of trace events using persistent storage at individual ones of the nodes at which the respective trace events were generated.

11. The method as recited in claim 5, wherein the method further comprises:

storing data indicative of the sparse trace using persistent storage at individual ones of the nodes in the second set of interactions.

12. The method as recited in claim 5, further comprising:

receiving user input representing a request to perform interaction reconstruction for the sparse trace, wherein the sparse trace is matched to one or more of the plurality of routes in response to the request.

13. A non-transitory computer-readable storage medium storing program instructions computer-executable to perform:

generating data indicative of a plurality of routes for a first set of service interactions between individual ones of a plurality of services in a service-oriented system, wherein the plurality of routes comprise data indicative of call paths for a plurality of node identifiers in the first set of service interactions;

generating a sparse trace for a second set of service interactions between individual ones of the plurality of services, wherein the sparse trace comprises a plurality of node identifiers in the second set of service interactions; and

matching the sparse trace to one or more of the plurality of routes, wherein the sparse trace is matched based on the plurality of node identifiers in the sparse trace and the plurality of node identifiers in the plurality of routes.

14. The non-transitory computer-readable storage medium as recited in claim 13, wherein the sparse trace excludes data indicative of call paths for at least some of the second set of service interactions.

15. The non-transitory computer-readable storage medium as recited in claim 13, wherein the sparse trace comprises a trace identifier, and wherein the plurality of node identifiers in the sparse trace are determined using the trace identifier.

16. The non-transitory computer-readable storage medium as recited in claim 13, wherein generating the sparse trace comprises:

activating, at a root node, a sparse tracing mode for the sparse trace; and

sending, from the root node to one or more downstream nodes, trace metadata activating the sparse tracing mode for the sparse trace at the one or more downstream nodes.

17. The non-transitory computer-readable storage medium as recited in claim 13, wherein the sparse trace comprises a trace identifier, and wherein generating the sparse trace comprises:

activating a sparse tracing mode for the sparse trace, wherein the sparse tracing mode is activated based on analysis of the trace identifier at the individual ones of the plurality of services in the second set of service interactions, wherein the analysis is performed according to a policy shared by the individual ones of the plurality of services in the second set of service interactions.

18. The non-transitory computer-readable storage medium as recited in claim 13, wherein the plurality of services are implemented by a plurality of hosts, wherein the plurality of routes are determined based on a set of trace events generated by the service-oriented system for the first set of service interactions, and wherein the program instructions are further computer-executable to perform:

storing the set of trace events using persistent storage at individual ones of the hosts at which the respective trace events were generated.

19. The non-transitory computer-readable storage medium as recited in claim 13, wherein the plurality of services are implemented by a plurality of hosts, and wherein the program instructions are further computer-executable to perform:

storing data indicative of the sparse trace using persistent storage at individual ones of the hosts in the second set of service interactions.

20. The non-transitory computer-readable storage medium as recited in claim 13, wherein the program instructions are further computer-executable to perform:

receiving user input representing a request to perform interaction reconstruction for the sparse trace, wherein the sparse trace is matched to one or more of the plurality of routes in response to the request.