System and method for service availability management

• US 20070162260A1
• Filed: 01/11/2006
• Published: 07/12/2007
• Est. Priority Date: 01/11/2006
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Nordstrom, Paul
• Granted Patent: US 7,904,759 B2
• Time in Patent Office: Days
• US Class Current: 702/186
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 9/505 considering the load

Abstract

A service framework uses importance ranking information in making call decisions and/or wait-time decisions in connection with service requests. Each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed. The health of a service may be controlled by permitting fewer (or more) service requests to be made to the service when the service is less (or more) healthy. The likelihood of a service request being made to the service may depend on the importance ranking of the service request. When waiting for a response, a determination whether to continue waiting may be made based on the likelihood that the service has failed and based on the importance of the information to the task being performed.

Claims

1. A data processing method for use in a service framework, comprising:

receiving a first assessment of a service, the first assessment comprising an assessment of a relative state of health of the service;

receiving a second assessment of the service, the second assessment comprising an assessment of a relative importance of the service to a task being performed; and

mediating access to the service according to the first and second assessments.

2. A method as defined in claim 1, wherein mediating access to the service comprises determining whether to continue to await a response from the service to a request sent to the service.

3. A method as defined in claim 1, wherein access to the service is mediated for multiple service requests, and wherein mediating access comprises permitting a first level of requests to be made to the service from the task when the relative state of health of the service is assessed as low and the relative importance of the service requests is assessed as low;

a second level of requests to be made to the service when the relative state of health of the service is assessed as low and the relative importance of the service requests is assessed as high; and

a third level of requests to be made to the service when the relative state of health of the service is assessed as high;

the first level being less than or equal to the second level and the second level being less than or equal to the third level.

4. A data processing method for use in a service framework, comprising:

sending a service request to a computer-implemented service;

waiting for a response from the service, including determining whether to continue waiting for the response based on the likelihood that the service has failed and based on the importance of the information to a task being performed.

5. A method as defined in claim 4, wherein the likelihood that the service has failed is determined based on statistical health information regarding performance of the service, and wherein the importance of the information to the task being performed is determined based on an importance ranking which reflects the importance of the information provided by the service to the task being performed.

6. A method as defined in claim 5, wherein the health information regarding the performance of the service is determined based on response times taken by the service to respond to prior service requests.

7. A method as defined in claim 5, wherein the health information regarding the performance of the service is determined based on a number of service requests unanswered by the service during a preceding time period.

8. A method as defined in claim 5, wherein the importance ranking is manually specified.

9. A method as defined in claim 4, wherein determining whether to continue waiting further comprises

determining that another service request having a higher importance ranking is still being waited upon;

waiting for the other service request with the higher importance ranking; and

determining to stop waiting for the service request once a response for the other service request with the higher importance ranking has been received.

10. A method as defined in claim 4, wherein determining whether to continue waiting comprises calculating a resource allocation and a failure expectation time.

11. A method as defined in claim 10, wherein the resource allocation time is calculated based on the information regarding the importance of the information and based on an amount of time remaining before expiration of a deadline for completing the task.

12. A method as defined in claim 11, wherein the failure expectation time is calculated based on the health of the service and is configured to provide a metric for determining when the service has likely failed.

13. A method as defined in claim 4, wherein determining whether to continue waiting is further performed based on a deadline for completing the task.

14. A method as defined in claim 4, wherein the task is constructing a web page for publication to a visitor of a website.

15. A method of constructing a web page for publication to a user, comprising:

sending a plurality of service requests to a plurality of computer-implemented services, the service requests being requests for information to be used to populate different sections of the web page, each of the plurality of service requests having an importance ranking which reflects the importance of the information to be provided responsive to the service request in the context of the web page; and

waiting for responses from the plurality of services, including determining whether to continue waiting for the responses based on (1) the likelihood that the service request has failed, and (2) the importance of the information to the web page;

wherein the waiting step is performed such that the service requests with a lower importance ranking continue to be waited upon as long as service requests with a higher importance ranking are waited upon.

16. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until a response to the service request is received.

17. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until it is determined that the service that is to provide the response has likely failed.

18. A method as defined in claim 15, wherein the service requests with a higher importance ranking are waited upon until a deadline for producing completing the web page expires.

19. A method as defined in claim 15, further comprising rendering different fields associated with the different sections of the web page, wherein the different fields are rendered in an order which is dynamically determined.

20. A method as defined in claim 19, wherein the order is dynamically determined based on the order in which the responses from the plurality of services are received.

21. A method as defined in claim 19, wherein the order is dynamically determined based on the amount of time required to render the different fields.

22. A method as defined in claim 21, wherein the rendering of fields that require more time to render is initiated before the rendering of fields that require less time to render.

23. A data processing method for use in a service framework, comprising:

controlling the health of a service, including monitoring the health of the service, permitting fewer service requests to be made to the service when the service is less healthy, and permitting more service requests to be made to the service when the service is more healthy; and

wherein each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed; and

wherein, during the controlling step, the service requests with a higher importance ranking are more likely to be made to the service and the service requests with a lower importance ranking are less likely to be made to the service.

24. A method as defined in claim 23, wherein the controlling step is performed using a negative feedback control loop which uses information regarding the health of the service as feedback.

25. A method as defined in claim 24, wherein the information regarding the health of the service is performance information concerning the response time of the service.

26. A method as defined in claim 23 wherein, during the controlling step, an importance ranking threshold is implemented such that service requests with an importance ranking above the threshold are permitted to be made and service requests with an importance ranking below the threshold are not permitted to be made.

27. A method as defined in claim 26, wherein the importance threshold is determined as a function of the health of the service.

28. A method as defined in claim 23, wherein the task is constructing a web page for publication to a visitor of a website.

29. A method as defined in claim 23, wherein the controlling step is performed during a start-up mode of the service, wherein the health of the service is determined to be initially unhealthy, wherein the health of the services improves as the service comes on-line, and wherein the service requests with the highest importance ranking are responded to first, with additional service requests being responded to as the health of the service improves.

30. A service framework comprising:

a plurality of services configured to receive service requests and to provide responses thereto;

wherein the service framework is configured to provide each of the plurality of services with a plurality of modes of operation, including

a first mode of operation, the service being placed into the first mode of operation when the service is relatively healthy,

a second mode of operation, the service being placed into the second mode of operation when the service is between being relatively healthy and relatively unhealthy, and

a third mode of operation, the service being placed into the third mode of operation when the service is relatively unhealthy; and

wherein, in the second mode of operation, which of the service requests that are permitted to be made to the service is determined based on an importance ranking of the service request, the importance ranking reflecting the importance of the information provided by the service to a task being performed.

31. A method as defined in claim 30, wherein the importance ranking is specified as a continuous value and the number of the plurality of modes of operation is potentially infinite.

32. A method as defined in claim 30, wherein the task is constructing a web page for publication to a visitor of a website.

33. An article of manufacture comprising a computer accessible medium whose contents direct a computing system to:

send a service request to a computer-implemented service operating in a service framework; and

wait for a response from the service.

34. An article of manufacture as defined in claim 33, wherein the contents direct the computing system to wait for the response by determining whether to continue to wait for the response based on the likelihood that the service has failed and based on the importance of the information to a task being performed.

35. An article of manufacture as defined in claim 34, wherein the contents direct the computer system to determine the likelihood that the service has failed based on statistical health information regarding performance of the service.

36. An article of manufacture as defined in claim 34, wherein the contents direct the computer system to determine the importance of the information to the task being performed based on an importance ranking which reflects the importance of the information provided by the service to the task being performed.

37. An article of manufacture as defined in claim 36, wherein the contents direct the computer system to determine the health information regarding the performance of the service based on response times taken by the service to respond to prior service requests.

38. An article of manufacture as defined in claim 36, wherein the contents direct the computer system to determine the health information regarding the performance of the service based on a number of service requests unanswered by the service during a preceding time period.

39. An article of manufacture as defined in claim 36, wherein the importance ranking is manually specified.

40. An article of manufacture as defined in claim 34, wherein the contents direct the computer system to determine whether to continue waiting by

determining that another service request having a higher importance ranking is still being waited upon;

waiting for the other service request with the higher importance ranking; and

determining to stop waiting for the service request once a response for the other service request with the higher importance ranking has been received.

41. An article of manufacture as defined in claim 34, wherein the contents direct the computer system to determine whether to continue waiting by calculating a resource allocation and a failure expectation time, wherein the resource allocation time is calculated based on the information regarding the importance of the information and based on an amount of time remaining before expiration of a deadline for completing the task, and wherein the failure expectation time is calculated based on the health of the service and is configured to provide a metric for determining when the service has likely failed.

42. An article of manufacture as defined in claim 34, wherein the contents direct the computer system to determine whether to continue waiting based on a deadline for completing the task.

43. An article of manufacture as defined in claim 34, wherein the task is constructing a web page for publication to a visitor of a website.

44. An article of manufacture comprising a computer accessible medium whose contents direct a computing system to:

control the health of a service, including monitor the health of the service, permit fewer service requests to be made to the service when the service is less healthy, and permit more service requests to be made to the service when the service is more healthy; and

wherein each of the service requests has an importance ranking which reflects the importance of the information provided by the service to a task being performed; and

wherein, to control the health of the service, the service requests with a higher importance ranking are more likely to be made to the service and the service requests with a lower importance ranking are less likely to be made to the service.

45. An article of manufacture as defined in claim 44, wherein the contents direct the computer system to control the health of the service using a negative feedback control loop which uses information regarding the health of the service as feedback.

46. An article of manufacture as defined in claim 45, wherein the information regarding the health of the service is performance information concerning the response time of the service.

47. An article of manufacture as defined in claim 44, wherein the contents direct the computer system to control the health of the service by implementing an importance ranking threshold such that service requests with an importance ranking above the threshold are permitted to be made and service requests with an importance ranking below the threshold are not permitted to be made.

48. An article of manufacture as defined in claim 47, wherein the importance threshold is determined as a function of the health of the service.

49. An article of manufacture as defined in claim 44, wherein the task is constructing a web page for publication to a visitor of a website.

50. An article of manufacture as defined in claim 44, wherein the contents direct the computer system to control the health of the service during a start-up mode of the service, wherein the health of the service is determined to be initially unhealthy, wherein the health of the services improves as the service comes on-line, and wherein the service requests with the highest importance ranking are responded to first, with additional service requests being responded to as the health of the service improves.

51. An article of manufacture as defined in claim 50, wherein the start-up mode is a cold cache mode of operation.

52. A data processing method for use in a service framework comprising a plurality of computer-implemented services, comprising:

causing the services to operate at or near a maximum service request processing capacity of the services, including controlling a flow of service requests to the services;

wherein controlling the flow of service requests to the services comprises, for each service, monitoring the health of the service, and permitting fewer service requests to be made to the service when the service is less healthy.

53. A data processing method as defined in claim 52, wherein control the flow of service requests further comprises permitting more service requests to be made to the service when the service is more healthy.