Prioritizing service requests

• US 9,258,197 B2
• Filed: 02/27/2015
• Issued: 02/09/2016
• Est. Priority Date: 04/27/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yanacek, David C., Atkin, Drew Evan, Coleman, Paul Lee IV
• Primary Examiner(s): Mejia, Anthony
• Application Number:
  US14/633,632
• Publication Number:
  US 20150172134A1
• Time in Patent Office: 347 Days
• Field of Search: 709/200, 709/223, 709/240
• US Class Current: 1/1
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

Disclosed are various embodiments for prioritizing service requests. A load associated with a service is determined. A request destined for processing by the service is received. One or more client-specified parameters are determined from the request. The client-specified parameters include one or more of: a droppability parameter indicating whether the request is droppable, or a deadline parameter indicating a deadline for completion of processing of the request. When the load meets a threshold, a prioritization is assigned to the request based at least in part on the client-specified parameter.

Claims

1. A non-transitory computer-readable medium embodying a program executable in at least one computing device, comprising:

code that determines a load associated with a service;

code that, in response to receiving a request destined for processing by the service, determines a client-specified parameter from the request, the client-specified parameter comprising at least one of;

a droppability parameter indicating whether the request is droppable, or a deadline parameter indicating a deadline for completion of processing of the request; and

code that, in response to determining that the load meets a threshold, assigns a prioritization to the request based at least in part on the client-specified parameter.

2. The non-transitory computer-readable medium of claim 1, wherein the service is configured to prioritize the request for an initial processing stage of the service based at least in part on the prioritization.

3. The non-transitory computer-readable medium of claim 2, wherein the initial processing stage determines an identity of a client that originated the request.

4. The non-transitory computer-readable medium of claim 2, further comprising code that assigns a subsequent prioritization to the request based at least in part on the client-specified parameter and a result of the initial processing stage.

5. The non-transitory computer-readable medium of claim 4, wherein the service is configured to prioritize the request for a subsequent processing stage of the service based at least in part on the subsequent prioritization.

6. The non-transitory computer-readable medium of claim 4, wherein the subsequent prioritization is based at least in part on a service-level agreement.

7. The non-transitory computer-readable medium of claim 1, wherein the threshold is associated with a brownout condition for the service.

8. The non-transitory computer-readable medium of claim 1, wherein the client-specified parameter further comprises a latency sensitivity parameter indicating a latency sensitivity of the request.

9. A system, comprising:

at least one computing device; and

at least one application executed in the at least one computing device, the at least one application comprising;

logic that, in response to receiving a request destined for processing by a service that includes an initial processing stage and a subsequent processing stage, determines a client-specified parameter from the request, the client-specified parameter comprising at least one of;

a droppability parameter indicating whether the request is droppable, or a deadline parameter indicating a deadline for completion of processing of the request; and

logic that assigns a prioritization to the request for the subsequent processing stage based at least in part on the client-specified parameter and a result of the initial processing stage.

10. The system of claim 9, wherein the at least one application further comprises logic that determines a load associated with the service, and the prioritization is assigned in response to determining that the load meets a brownout condition threshold.

11. The system of claim 10, wherein the logic that determines the load is configured to determine the load based at least in part on data describing an aggregate load status across a plurality of instances of the service.

12. The system of claim 9, wherein the at least one application further comprises logic that assigns an initial prioritization to the request for the initial processing stage based at least in part on the client-specified parameter.

13. The system of claim 9, wherein the client-specified parameter further comprises a latency sensitivity parameter indicating a latency sensitivity of the request.

14. A method, comprising:

determining, via at least one of one or more computing devices, that a load associated with a service meets a brownout condition threshold;

receiving, via at least one of the one or more computing device, a request destined for processing by the service;

extracting, via at least one of the one or more computing devices, a client-specified parameter from the request; and

prioritizing, via at least one of the one or more computing devices, the request for processing by the service based at least in part on at least one of;

whether the client-specified parameter indicates that the request is droppable, or whether the client-specified parameter indicates a deadline for completion of processing of the request.

15. The method of claim 14, wherein prioritizing the request further comprises dropping, via at least one of the one or more computing devices, the request when the client-specified parameter indicates that the request is droppable.

16. The method of claim 14, wherein prioritizing the request further comprises dropping, via at least one of the one or more computing devices, the request when processing of the request cannot be completed by the deadline.

17. The method of claim 14, wherein the brownout condition threshold is associated with a degraded performance for all requests destined for the service.

18. The method of claim 14, further comprising subsequently prioritizing, via at least one of the one or more computing devices, the request for processing by a subsequent processing stage of the service based at least in part on a result of an initial processing stage of the service.

19. The method of claim 14, further comprising enforcing, via at least one of the one or more computing devices, a restriction on the client-specified parameter based at least in part on an identity of a client that originated the request.

20. The method of claim 14, further comprising sequencing, via at least one of the one or more computing devices, the request for processing according to a latency sensitivity indicated by the client-specified parameter.