System and method for service availability management

• US 7,904,759 B2
• Filed: 01/11/2006
• Issued: 03/08/2011
• Est. Priority Date: 01/11/2006
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Nordstrom, Paul G.
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US11/329,665
• Publication Number:
  US 20070162260A1
• Time in Patent Office: 1,882 Days
• Field of Search: 714/25, 714/47, 709/224, 709/232, 709/219, 709/24, 715/736, 715/742, 715/738, 715/749
• US Class Current: 714/47.1
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 9/505 considering the load

Abstract

A service framework uses importance ranking information in making call decisions and/or wait-time decisions in connection with service requests. Each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed. The health of a service may be controlled by permitting fewer (or more) service requests to be made to the service when the service is less (or more) healthy. The likelihood of a service request being made to the service may depend on the importance ranking of the service request. When waiting for a response, a determination whether to continue waiting may be made based on the likelihood that the service has failed and based on the importance of the information to the task being performed.

Claims

1. A computer-implemented method for use in a service framework, in a computer executing code for:

receiving a request for a target service,

receiving a request for at least one additional service;

receiving a first assessment of the target service, the first assessment comprising an assessment of a relative state of health of the target service;

receiving a second assessment of the target service, the second assessment comprising an assessment of a relative importance of the information to be provided responsive to a request for the target service; and

determining access to the target service relative to the request for the at least one additional service, according to the first and second assessments for the target service.

2. A method as defined in claim 1, wherein determining access to the target service comprises determining whether to continue to await a response from the service to a request sent to the service.

3. A method as defined in claim 1, wherein determining access comprises permitting a first level of requests to be made to the service from the task when the relative state of health of the service is assessed as low and the relative importance of the service requests is assessed as low;

a second level of requests to be made to the service when the relative state of health of the service is assessed as low and the relative importance of the service requests is assessed as high; and

a third level of requests to be made to the service when the relative state of health of the service is assessed as high;

the first level being less than or equal to the second level and the second level being less than or equal to the third level.

4. A computer-implemented method for use in a service framework, in a computer executing code for:

sending a service request to a computer-implemented service;

waiting for a response from the service, including determining whether to continue waiting for the response based on the likelihood that the service has failed, and an importance ranking of the information to a task being performed relative to an importance ranking of the information to be provided responsive to another service request, wherein the service request and another service request are associated with each other.

5. A method as defined in claim 4, wherein the likelihood that the service has failed is determined based on statistical health information regarding performance of the service, and wherein the importance ranking reflects the importance of the information provided by the service to the task being performed.

6. A method as defined in claim 5, wherein the health information regarding the performance of the service is determined based on response times taken by the service to respond to prior service requests.

7. A method as defined in claim 5, wherein the health information regarding the performance of the service is determined based on a number of service requests unanswered by the service during a preceding time period.

8. A method as defined in claim 5, wherein the importance ranking is manually specified.

9. A method as defined in claim 4, wherein determining whether to continue waiting further comprises:

determining that another service request having a higher importance ranking is still being waited upon;

waiting for the other service request with the higher importance ranking; and

determining to stop waiting for the service request once a response for the other service request with the higher importance ranking has been received.

10. A method as defined in claim 4, wherein determining whether to continue waiting comprises calculating a resource allocation and a failure expectation time.

11. A method as defined in claim 10, wherein the resource allocation time is calculated based on the information regarding the importance of the information and based on an amount of time remaining before expiration of a deadline for completing the task.

12. A method as defined in claim 11, wherein the failure expectation time is calculated based on the health of the service and is configured to provide a metric for determining when the service has likely failed.

13. A method as defined in claim 4, wherein determining whether to continue waiting is further performed based on a deadline for completing the task.

14. A method as defined in claim 4, wherein the task is constructing a web page for publication to a visitor of a website.

15. A method of constructing a web page for publication to a user, comprising:

sending a plurality of service requests to a plurality of computer-implemented services, the service requests being requests for information to be used to populate different sections of the web page, each of the plurality of service requests having an importance ranking which reflects the importance of the information to be provided responsive to the service request in the context of the web page; and

waiting for responses from the plurality of services, including determining whether to continue waiting for the responses based on (1) the likelihood that the service request has failed, and (2) the importance of the information to the web page;

wherein the waiting step is performed such that the service requests with a lower importance ranking continue to be waited upon as long as service requests with a higher importance ranking are waited upon.

16. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until a response to the service request is received.

17. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until it is determined that the service that is to provide the response has likely failed.

18. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until a deadline for producing completing the web page expires.

19. A method as defined in claim 15, further comprising rendering different fields associated with the different sections of the web page, wherein the different fields are rendered in an order which is dynamically determined.

20. A method as defined in claim 19, wherein the order is dynamically determined based on the order in which the responses from the plurality of services are received.

21. A method as defined in claim 19, wherein the order is dynamically determined based on the amount of time required to render the different fields.

22. A method as defined in claim 21, wherein the rendering of fields that require more time to render is initiated before the rendering of fields that require less time to render.

23. A computer-implemented method for use in a service framework, in a computer executing code for:

controlling the health of a service, including monitoring the health of the service, permitting fewer service requests to be made to the service when the service is less healthy, and permitting more service requests to be made to the service when the service is more healthy; and

wherein each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed; and

wherein, during the controlling step, a service request will be waited upon as long as another service request with a higher importance ranking is being waited on, wherein the service request and another service request are associated with each other.

24. A method as defined in claim 23, wherein the controlling step is performed using a negative feedback control loop which uses information regarding the health of the service as feedback.

25. A method as defined in claim 24, wherein the information regarding the health of the service is performance information concerning the response time of the service.

26. A method as defined in claim 23, wherein the task is constructing a web page for publication to a visitor of a website.

27. A method as defined in claim 23, wherein the controlling step is performed during a start-up mode of the service, wherein the health of the service is determined to be initially unhealthy, wherein the health of the services improves as the service comes online, and wherein the service requests with the highest importance ranking are responded to first, with additional service requests being responded to as the health of the service improves.

28. A system including a memory and a processor providing services comprising:

a plurality of services, executed on the number of computing devices, to receive service requests and to provide responses thereto;

wherein the system is configured to provide each of the plurality of services with a plurality of modes of operation, including;

a first mode of operation, the system being placed into the first mode of operation when the service is relatively healthy,

a second mode of operation, the system being placed into the second mode of operation when the service is between being relatively healthy and relatively unhealthy, and

a third mode of operation, the system being placed into the third mode of operation when the service is relatively unhealthy; and

wherein, in the second mode of operation, which of the service requests that are permitted to be made to the service is determined based on an importance ranking of the service request, the importance ranking reflecting the importance of the information provided by the service to a task being performed relative to the importance of the information provided by another service request, wherein the service request will be waited upon as long as another service request with a higher importance ranking is being waited upon.

29. A system as defined in claim 28, wherein the importance ranking is specified as a continuous value and the number of the plurality of modes of operation is potentially infinite.

30. A system as defined in claim 28, wherein the task is constructing a web page for publication to a visitor of a website.

31. An article of manufacture comprising a machine-readable media whose contents direct a computing system to;

send a service request to a computer-implemented service operating in a service framework; and

determine to wait for a response from the service based on an importance ranking of the information to be provided responsive to the service request in relation to the importance ranking of the information to be provided responsive to another service request, wherein the service request will be waited upon as long as the other service request with a higher importance ranking is being waited on.

32. An article of manufacture as defined in claim 31, wherein the contents direct the computing system determine to wait for the response based on the likelihood that the service has failed and based on the importance of the information to be provided responsive to the service request in relation to the importance ranking of the information to be provided responsive to another service request.

33. An article of manufacture as defined in claim 32, wherein the likelihood that the service has failed is determined based on statistical health information regarding performance of the service.

34. An article of manufacture as defined in claim 33, wherein the health information regarding the performance of the service is based on response times taken by the service to respond to prior service requests.

35. An article of manufacture as defined in claim 33, wherein the health information regarding the performance of the service is based on a number of service requests unanswered by the service during a preceding time period.

36. An article of manufacture as defined in claim 33, wherein the importance ranking is manually specified.

37. An article of manufacture as defined in claim 32, wherein the contents direct the computing system to determine whether to continue waiting by:

determining that another service request having a higher importance ranking is still being waited upon;

waiting for the other service request with the higher importance ranking; and

determining to stop waiting for the service request once a response for the other service request with the higher importance ranking has been received.

38. An article of manufacture as defined in claim 32, wherein the contents direct the computing system to determine whether to continue waiting by calculating a resource allocation and a failure expectation time, wherein the resource allocation time is calculated based on the information regarding the importance of the information and based on an amount of time remaining before expiration of a deadline for completing the task, and wherein the failure expectation time is calculated based on the health of the service and is configured to provide a metric for determining when the service has likely failed.

39. An article of manufacture as defined in claim 32, wherein the contents direct the computing system to determine whether to continue waiting based on a deadline for completing the task.

40. An article of manufacture as defined in claim 32, wherein the task is constructing a web page for publication to a visitor of a website.

41. An article of manufacture comprising a machine-readable media whose contents direct a computing system to:

control the health of a service, including monitor the health of the service, permit fewer service requests to be made to the service when the service is less healthy, and permit more service requests to be made to the service when the service is more healthy; and

wherein each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed; and

wherein, to control the health of the service, a service request is waited upon as long as another service request with a higher importance ranking is being waited on, wherein the service request and another service request are associated with each other.

42. An article of manufacture as defined in claim 41, wherein the contents further direct the computer system to control the health of the service using a negative feedback control loop which uses information regarding the health of the service as feedback.

43. An article of manufacture as defined in claim 42, wherein the information regarding the health of the service is performance information concerning the response time of the service.

44. An article of manufacture as defined in claim 41, wherein the task is constructing a web page for publication to a visitor of a website.

45. An article of manufacture as defined in claim 41, wherein the contents direct the computer system to control the health of the service during a start-up mode of the service, wherein the health of the service is determined to be initially unhealthy, wherein the health of the services improves as the service comes on-line, and wherein the service requests with the highest importance ranking are responded to first, with additional service requests being responded to as the health of the service improves.

46. An article of manufacture as defined in claim 45, wherein the start-up mode is a cold cache mode of operation.