Prioritizing service requests

• US 8,972,551 B1
• Filed: 04/27/2010
• Issued: 03/03/2015
• Est. Priority Date: 04/27/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yanacek, David C., Coleman, Paul Lee IV, Atkin, Drew Evan
• Primary Examiner(s): Mejia, Anthony
• Application Number:
  US12/768,297
• Time in Patent Office: 1,771 Days
• Field of Search: 709/200, 709/223, 709/240, 709/206
• US Class Current: 709/223
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

Disclosed are various embodiments for prioritizing service requests. A service request destined for processing by a service provider is obtained in at least one computing device. The service provider is configured to provide a service. The service request includes at least one client-specified parameter indicating a priority of the service request. A load associated with the service is determined. The service request is prioritized for an initial processing stage of the service provider when the load meets at least one threshold. The prioritizing for the initial processing stage is based at least in part on the at least one client-specified parameter. The service request is prioritized for an additional processing stage of the service provider when the load meets the at least one threshold. The prioritizing for the additional processing stage is based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

Claims

1. A non-transitory computer-readable medium embodying a program executable in at least one computing device, the program comprising:

code that determines a load associated with a service according to data describing an aggregate load status across a plurality of instances of a service provider application that is configured to provide the service;

code that determines at least one threshold relative to a load associated with a brownout condition producing degraded performance across all service requests for the service;

code that, in response to receiving a service request destined for processing by the service provider application, prioritizes the service request when the load associated with the service meets the at least one threshold, the service request being initially prioritized for an initial processing stage of the service provider application based at least in part on a plurality of client-specified parameters included in the service request and indicating a priority of the service request, the service request being subsequently prioritized for an additional processing stage of the service provider application based at least in part on the plurality of client-specified parameters and a result of the initial processing stage; and

wherein the plurality of client-specified parameters include a droppability parameter indicating whether the service request is droppable, a deadline parameter indicating a deadline for completion of processing of the service request, and a latency sensitivity parameter indicating a latency sensitivity of the service request.

2. The computer-readable medium of claim 1, wherein the initial processing stage determines an identity of a service client that originated the service request, and the subsequent prioritization is based at least in part on a service-level agreement associated with the service client.

3. A system, comprising:

at least one computing device;

a service provider application executable in the at least one computing device, the service provider application being configured to provide a service; and

a request prioritization application executable in the at least one computing device, the request prioritization application comprising;

logic that, in response to receiving a service request destined for processing by the service provider application, prioritizes the service request when a load associated with the service meets at least one threshold, the service request including at least one client-specified parameter indicating a priority of the service request, the service request being initially prioritized for an initial processing stage of the service provider application based at least in part on the at least one client-specified parameter, the service request being subsequently prioritized for an additional processing stage of the service provider application based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

4. The system of claim 3, wherein the at least one client-specified parameter includes a droppability parameter, and the logic that prioritizes further comprises logic that drops the service request when the droppability parameter indicates that the service request is droppable.

5. The system of claim 3, wherein the at least one client-specified parameter includes a deadline parameter, and the logic that prioritizes further comprises logic that drops the service request when processing of the service request cannot be completed by a deadline indicated by the deadline parameter.

6. The system of claim 3, wherein the at least one client-specified parameter includes a latency sensitivity parameter, and the logic that prioritizes further comprises logic that sequences the service request for processing according to the latency sensitivity parameter.

7. The system of claim 6, wherein the logic that sequences the service request for processing is configured to sequence the service request within a queue containing a plurality of service requests.

8. The system of claim 3, wherein the logic that prioritizes further comprises logic that degrades a processing performance associated with the service provider application processing the service request according to at least one rule.

9. The system of claim 3, wherein the service provider application and the request prioritization application are executable in different ones of the at least one computing device.

10. The system of claim 3, wherein the initial processing stage includes logic that determines an identity of a service client that originated the service request.

11. The system of claim 10, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on at least one rule.

12. The system of claim 10, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on the identity of the service client.

13. The system of claim 11, wherein the at least one rule comprises logic that drops the service request when a current resource usage of the service client meets at least one threshold of a service level agreement associated with the service client.

14. The system of claim 11, wherein the at least one rule comprises logic that enforces a restriction on the at least one client-specified parameter based at least in part on the identity of the service client.

15. The system of claim 3, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on at least one rule, and the at least one rule comprises logic that prioritizes the service request based at least in part on a type of service request.

16. The system of claim 3, wherein the at least one threshold is determined relative to a load associated with a brownout condition producing degraded performance across all service requests.

17. The system of claim 3, wherein the request prioritization application further comprises logic that determines the load associated with the service according to data describing an aggregate load status across a plurality of instances of the service provider application, the data being obtained from at least one other computing device.

18. The system of claim 3, wherein the at least one client-specified parameter comprises a plurality of client-specified parameters including at least one of:

a droppability parameter indicating whether the service request is droppable, a deadline parameter indicating a deadline for completion of processing of the service request, or a latency sensitivity parameter indicating a latency sensitivity of the service request.

19. A method, comprising:

receiving, via at least one of one or more computing devices, a service request destined for processing by a service provider configured to provide a service, the service request including at least one client-specified parameter indicating a priority of the service request;

determining, via at least one of the one or more computing devices, a load associated with the service;

prioritizing, via at least one of the one or more computing devices, the service request for an initial processing stage of the service provider when the load meets at least one threshold, the prioritizing being based at least in part on the at least one client-specified parameter; and

prioritizing, via at least one of the one or more computing devices, the service request for at least one additional processing stage of the service provider when the load meets the at least one threshold, the prioritizing being based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

20. The method of claim 19, wherein determining the load associated with the service further comprises:

receiving, via at least one of the one or more computing devices, corresponding load statuses associated with individual ones of a plurality of instances of the service provider; and

determining, via at least one of the one or more computing devices, the load according to the corresponding load statuses.

21. The method of claim 19, wherein the at least one client-specified parameter includes a droppability parameter, a deadline parameter, and a latency sensitivity parameter, and prioritizing the service request for the initial processing stage or the at least one additional processing stage further comprises:

dropping, via at least one of the one or more computing devices, the service request when the droppability parameter indicates that the service request is droppable;

dropping, via at least one of the one or more computing devices, the service request when processing of the service request cannot be completed by a deadline indicated by the deadline parameter; and

sequencing, via at least one of the one or more computing devices, the service request for processing according to the latency sensitivity parameter.

22. The method of claim 19, wherein prioritizing the service request for the at least one additional processing stage is repeated for a plurality of additional processing stages of the service provider.

23. The method of claim 19, wherein prioritizing the service request for the at least one additional processing stage of the service provider further comprises prioritizing, via at least one of the one or more computing devices, the service request for the at least one additional processing stage of the service provider based at least in part on at least one rule and an identity of a service client that originated the service request.

24. The method of claim 23, wherein the at least one rule enforces a restriction on at least one of the at least one client-specified parameter based at least in part on the identity of the service client.