Request routing based on server software versions

• US 10,560,372 B1
• Filed: 08/28/2017
• Issued: 02/11/2020
• Est. Priority Date: 08/28/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Lucas, Tai Geoffrey, Kowalski, Marcin Piotr
• Primary Examiner(s): Christensen, Scott B
• Application Number:
  US15/688,674
• Time in Patent Office: 897 Days
• Field of Search: None
• CPC Class Codes: H04L 45/121 by minimising delays
• CPC Class Codes: H04L 45/123 Evaluation of link metrics techniques for monitoring network metrics H04L43/08
• CPC Class Codes: H04L 45/306 Route determination based on the nature of the carried application
• CPC Class Codes: H04L 45/34 Source routing
• CPC Class Codes: H04L 45/70 Routing based on monitoring results
• CPC Class Codes: H04L 45/72 Routing based on the source address
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Systems and methods are disclosed to implement a request routing layer of a service provider system, where the request routers are aware of different versions of a service implemented by the servers of the system. In embodiments, the request routers may discover the different versions of the service via periodic health messages or request responses from the servers. Requests are routed to the servers according to a routing policy that specifies the relative proportions of requests that are to be routed to each version. Version metrics generated from the requests may be captured and provided to a version change manager, which may update the routing policy based on the metrics, either increasing or decreasing the proportion of requests for the different versions. In embodiments, the request router may implement additional rules that control their routing behavior in response to detected error conditions.

Claims

1. A system, comprising:

one or more processors with memory implementing a request router configured to, automatically and without user intervention;

discover, based on messages received from a plurality of backend nodes, different versions of a resource that are deployed on individual ones of the plurality of backend nodes;

update a version map based at least in part on the discovered versions of the resource;

receive and route a plurality of requests to the plurality of backend nodes according to a routing policy and the version map, wherein the routing policy specifies relative proportions of requests to be routed to the different versions of the resource;

monitor metrics resulting from the routing of the plurality of requests to the different versions of the resource;

analyze the metrics to determine an update to the routing policy to change the relative proportions of requests to be routed to the different versions; and

apply the determined update to the routing policy.

2. The system of claim 1, wherein, to discover the different versions of the resource deployed on the backend nodes, the request router is configured to:

receive periodic health data from a backend node of the plurality of backend nodes; and

determine a specific version of the resource deployed on the backend node based on the health data.

3. The system of claim 1, wherein, to discover the different versions the resource deployed on the backend nodes, the individual request router is configured to:

receive a message from a backend node; and

determine a specific version of the resource deployed on the backend node based on version data included as metadata with the message.

4. The system of claim 1, wherein the request router maintains a routing policy that is updated independently from other routing polices for different request routers.

5. The system of claim 1, further comprising:

a version change manager node implemented by one or more processors and associated memory to;

receive the metrics from a plurality of request routers including the request router;

analyze the metrics to determine another update to the routing policy to change the relative proportions of requests to be routed to the different versions; and

send the other update to the request routers.

6. A method, comprising:

performing, by a request router implemented by one or more processors and associated memory automatically and without user intervention;

discovering, based on messages received from a plurality of backend nodes, different versions of a resource that are deployed on individual ones of the plurality of backend nodes;

updating a version map based at least in part on the discovered versions of the resource;

receiving and routing a plurality of requests to the plurality of backend nodes according to a routing policy and the version map, wherein the routing policy specifies relative proportions of requests to be routed to the different versions of the resource;

monitoring metrics resulting from the routing of the plurality of requests to the different versions of the resource;

analyzing the metrics to determine an update to the routing policy to change the relative proportions of requests to be routed to the different versions; and

applying the determined update to the routing policy.

7. The method of claim 6, wherein discovering the different versions of the resource deployed on a backend node comprises determining the different versions based on a heartbeat message received from the backend node.

8. The method of claim 6, wherein discovering the different versions of the resource deployed on a backend node comprises determining the different versions based on metadata in a response generated by the backend node in response to a request routed to the backend node.

9. The method of claim 6,

wherein the different versions are assigned to different respective ports on a backend node hosting the resource;

and

discovering the different versions comprises receiving an indication that a particular version is listening on a particular port of the backend node.

10. The method of claim 6, wherein analyzing the metrics comprises:

determining a diversity metric for a particular version of the resource from the metrics, wherein the diversity metric is based on amounts of different request types that have been handled by the particular version of the resource.

11. The method of claim 6, wherein analyzing the metrics comprises:

determining a performance difference between the metrics for a newer version of the resource and an older version of the resource.

12. The method of claim 6, wherein analyzing the metrics comprises:

determining response codes or error codes associated with requests routed to the particular version.

13. The method of claim 6, wherein:

said analyzing the metrics comprises determining from the metrics an error condition associated with a particular version of the different versions; and

said applying the update to the routing policy causes routing of additional request to the particular version to cease.

14. The method of claim 6, wherein:

said analyzing the metrics comprises determining from the metrics that a first version of the different versions is in an error condition;

and further comprising;

determining a pending request that is routed to the first version; and

rerouting the pending request to a second version of the different versions.

15. The method of claim 6, further comprising:

sending the metrics, by a request router of the plurality of request routers to a version change manager node that is remote from the request router;

performing, by the version change manager node;

aggregating metrics from the plurality of request routers;

analyzing the aggregated metrics to determine another update to the routing policy; and

sending the other update to the routing policy to the plurality of request routers.

16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors of a request router, cause the request router to, automatically and without user intervention:

discover, based on messages received from a plurality of backend nodes, different versions of a resource that are deployed on individual ones of the plurality of backend nodes;

update a version map based at least in part on the discovered versions of the resource;

receive and routing a plurality of requests to the plurality of backend nodes according to a routing policy and the version map, wherein the routing policy specifies relative proportions of requests to be routed to the different versions of the resource;

monitor metrics resulting from the routing of the plurality of requests to the different versions of the resource;

analyze the metrics to determine an update to the routing policy to change the relative proportions of requests to be routed to the different versions; and

apply the determined update to the routing policy.

17. The non-transitory computer-accessible storage medium of claim 16, wherein to discover the different versions of the resource deployed on the backend nodes, the program instructions when executed on the one or more processors cause the request router to discover the different versions based on periodic health data received from the backend nodes.

18. The non-transitory computer-accessible storage medium of claim 16, wherein to discover the different versions of the resource deployed on the backend nodes, the program instructions when executed on the one or more processors cause the request router to discover the different versions based on metadata of responses generated in response to requests routed to the backend nodes.

19. The non-transitory computer-accessible storage medium of claim 16, wherein to discover the different versions of the resource deployed on the backend nodes, the program instructions when executed on the one or more processors of the request router cause the request router to discover a composite version from a backend node, wherein the composite version includes component versions of a plurality of components of the resource used by the backend node to service a particular request.

20. The non-transitory computer-accessible storage medium of claim 16, wherein the program instructions when executed on the one or more processors of the request router cause the request router to:

determine, from the analyzing of the metrics, that a first version of the resource is in an error condition;

determine a pending request that is routed to the first version; and

reroute the pending request to a second version of the resource.