PRIORITIZING SERVICE REQUESTS

• US 20150172134A1
• Filed: 02/27/2015
• Published: 06/18/2015
• Est. Priority Date: 04/27/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yanacek, David C., Atkin, Drew Evan, Coleman, Paul Lee IV
• Granted Patent: US 9,258,197 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

Disclosed are various embodiments for prioritizing service requests. A service request destined for processing by a service provider is obtained in at least one computing device. The service provider is configured to provide a service. The service request includes at least one client-specified parameter indicating a priority of the service request. A load associated with the service is determined. The service request is prioritized for an initial processing stage of the service provider when the load meets at least one threshold. The prioritizing for the initial processing stage is based at least in part on the at least one client-specified parameter. The service request is prioritized for an additional processing stage of the service provider when the load meets the at least one threshold. The prioritizing for the additional processing stage is based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

Claims

1. A non-transitory computer-readable medium embodying a program executable in at least one computing device, the program comprising:

code that obtains a service request destined for processing by a service provider application, the service provider application being configured to provide a service, the service request including a plurality of client-specified parameters indicating a priority of the service request, the client-specified parameters including;

a droppability parameter indicating whether the service request is droppable,

a deadline parameter indicating a deadline for completion of processing of the service request, and

a latency sensitivity parameter indicating a latency sensitivity of the service request;

code that determines a load associated with the service according to data describing an aggregate load status across a plurality of instances of the service provider application;

code that determines at least one threshold relative to a load associated with a brownout condition producing degraded performance across all service requests for the service; and

code that prioritizes the service request when the load associated with the service meets the at least one threshold, the service request being initially prioritized for an initial processing stage of the service provider application based at least in part on the client-specified parameters, the service request being subsequently prioritized for an additional processing stage of the service provider application based at least in part on the client-specified parameters and a result of the initial processing stage.

2. The computer-readable medium of claim 1, wherein the initial processing stage determines an identity of a service client that originated the service request, and the subsequent prioritization is based at least in part on a service-level agreement associated with the service client.

3. A system, comprising:

at least one computing device;

a service provider application executable in the at least one computing device, the service provider application being configured to provide a service; and

a request prioritization application executable in the at least one computing device, the request prioritization application comprising;

logic that obtains a service request destined for processing by the service provider application, the service request including at least one client-specified parameter indicating a priority of the service request; and

logic that prioritizes the service request when a load associated with the service meets at least one threshold, the service request being initially prioritized for an initial processing stage of the service provider application based at least in part on the at least one client-specified parameter, the service request being subsequently prioritized for an additional processing stage of the service provider application based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

4. The system of claim 3, wherein the at least one client-specified parameter includes a droppability parameter, and the logic that prioritizes further comprises logic that drops the service request when the droppability parameter indicates that the service request is droppable.

5. The system of claim 3, wherein the at least one client-specified parameter includes a deadline parameter, and the logic that prioritizes further comprises logic that drops the service request when processing of the service request cannot be completed by a deadline indicated by the deadline parameter.

6. The system of claim 3, wherein the at least one client-specified parameter includes a latency sensitivity parameter, and the logic that prioritizes further comprises logic that sequences the service request for processing according to the latency sensitivity parameter.

7. The system of claim 6, wherein the logic that sequences the service request for processing is configured to sequence the service request within a queue containing a plurality of service requests.

8. The system of claim 3, wherein the logic that prioritizes further comprises logic that degrades a processing performance associated with the service provider application processing the service request according to at least one rule.

9. The system of claim 3, wherein the service provider application and the request prioritization application are executable in different ones of the at least one computing device.

10. The system of claim 3, wherein the initial processing stage includes logic that determines an identity of a service client that originated the service request.

11. The system of claim 10, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on at least one rule.

12. The system of claim 11, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on the identity of the service client.

13. The system of claim 11, wherein the at least one rule comprises logic that drops the service request when a current resource usage of the service client meets at least one threshold of a service level agreement associated with the service client.

14. The system of claim 11, wherein the at least one rule comprises logic that enforces a restriction on the at least one client-specified parameter based at least in part on the identity of the service client.

15. The system of claim 3, wherein the logic that prioritizes is configured to prioritize the service request for the additional processing stage based at least in part on at least one rule, and the at least one rule comprises logic that prioritizes the service request based at least in part on a type of service request.

16. The system of claim 3, wherein the at least one threshold is determined relative to a load associated with a brownout condition producing degraded performance across all service requests.

17. The system of claim 3, wherein the request prioritization application further comprises logic that determines the load associated with the service according to data describing an aggregate load status across a plurality of instances of the service provider application, the data being obtained from at least one other computing device.

18. A method, comprising the steps of:

obtaining, in at least one computing device, a service request destined for processing by a service provider configured to provide a service, the service request including at least one client-specified parameter indicating a priority of the service request;

determining, in the at least one computing device, a load associated with the service;

prioritizing, in the at least one computing device, the service request for an initial processing stage of the service provider when the load meets at least one threshold, the prioritizing being based at least in part on the at least one client-specified parameter; and

prioritizing, in the at least one computing device, the service request for at least one additional processing stage of the service provider when the load meets the at least one threshold, the prioritizing being based at least in part on the at least one client-specified parameter and a result of the initial processing stage.

19. The method of claim 18, wherein the determining step further comprises the steps of:

obtaining, in the at least one computing device, a corresponding load status associated with each one of a plurality of instances of the service provider; and

determining, in the at least one computing device, the load according to the load statuses.

20. The method of claim 18, wherein the at least one client-specified parameter includes a droppability parameter, a deadline parameter, and a latency sensitivity parameter, and the prioritizing steps each further comprise the steps of:

dropping, in the at least one computing device, the service request when the droppability parameter indicates that the service request is droppable;

dropping, in the at least one computing device, the service request when processing of the service request cannot be completed by a deadline indicated by the deadline parameter; and

sequencing, in the at least one computing device, the service request for processing according to the latency sensitivity parameter.