Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment

• US 10,452,997 B2
• Filed: 11/30/2015
• Issued: 10/22/2019
• Est. Priority Date: 12/10/2010
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Yang, Ronald, Devadhar, Vijayanth, Cheenath, Manoj
• Primary Examiner(s): Alriyashi, Abdulkader M
• Application Number:
  US14/953,672
• Publication Number:
  US 20160080273A1
• Time in Patent Office: 1,422 Days
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment. Requests directed to an application server are received into one or more queues in front of the application server. An acknowledgement of the requests is provided in response to the requests being received by the one or more queues. Metadata associated with the requests is utilized to manage the one or more queues. The requests from the one or more queues are processed based on the metadata.

Claims

1. A method for servicing requests in an environment having multiple application servers each having multiple queues to store at least indications of requests to be serviced by the application servers, wherein the queues have different priorities, the method comprising:

determining, with one or more processors corresponding to the multiple application servers, a first utilization level for a first application server of the multiple application servers, wherein the first utilization level indicates utilization of one or more resources of the first application server;

maintaining, with the one or more processors, a directory of utilization information received from other application servers within the environment, wherein the directory includes a list of healthy application servers of the multiple application servers to which request can be sent when a selected server has reached a workload threshold, wherein the list of healthy application servers is based on utilization information from the multiple application servers;

pushing, by the first application server, information related to the first utilization level at random intervals to one or more randomly chosen application servers of the multiple application servers to at least update the list of healthy application servers;

receiving, from at least one of the multiple application servers, utilization information for one or more application servers of the multiple application servers to at least update the list of healthy application servers;

determining, with the one or more processors, the first application server is on the healthy list and should service one or more requests allocated to other application servers of the multiple application servers based on the first utilization level and utilization information received from the other application servers based, at least in part, on the directory of utilization information in the first application server;

pulling, based on the determining, by the first application server a request allocated to a second application server of the multiple application servers, wherein said pulling comprises identifying the pulled request by walking a queue of the multiple queues of the second application server in accordance with a modified first-in-first-out (FIFO) strategy, including skipping one or more requests in the queue having an associated predicted cost deemed too expensive to be processed by the first application server based on current conditions of the first application server; and

servicing, by the first application server, the pulled request.

2. The method of claim 1 wherein the utilization level is based, at least in part on one or more of queue depth, queue processing time, bandwidth utilization, processor utilization, and memory utilization.

3. The method of claim 1 further comprising:

analyzing, with the one or more processors, one or more queue parameters for the one or more application servers; and

selectively servicing, with the one or more processors, requests allocated to the one or more application servers based on the analysis.

4. The method of claim 1 wherein the application servers are part of a multitenant database environment that stores data for multiple client entities each client entity identified by a tenant identifier (ID) having one of one or more users associated with the tenant ID, wherein users of each of multiple client entities can only access data identified by a tenant ID associated with a respective client entity, and wherein the multitenant database is a hosted database provided by an entity separate from the client entities, and provides on-demand database service to the client entities.

5. The method of claim 1, wherein the multiple application servers are coupled in communication via a network and wherein the requests are stored in a network layer of the network while they wait to be serviced.

6. The method of claim 1, wherein requests associated with a particular application server of the multiple application servers are queued on an appropriate queue of the multiple queues based on a supplied comparator object, wherein the supplied comparator object facilitates determination of a priority of the different priorities of a particular request of the requests.

7. An article comprising a non-transitory computer-readable medium having stored thereon instructions that when executed by one or more processors are configurable to service requests in an environment having multiple application servers each having one or more queues to store at least indications of requests to be serviced by the application servers by:

determining, with one or more processors corresponding to the multiple application servers, a first utilization level for a first application server of the multiple application servers, wherein the first utilization level indicates utilization of one or more resources of the first application server;

maintaining, with the one or more processors, a directory of utilization information received from other application servers within the environment, wherein the directory includes a list of healthy application servers of the multiple application servers to which request can be sent when a selected server has reached a workload threshold, wherein the list of healthy application servers is based on utilization information from the multiple application servers;

pushing, by the first application server, information related to the first utilization level at random intervals to one or more randomly chosen application servers of the multiple application servers to at least update the list of healthy application servers;

receiving, from at least one of the multiple application servers, utilization information for one or more application servers of the multiple application servers to at least update the list of healthy application servers;

determining, with the one or more processors, the first application server is on the healthy list and should service one or more requests allocated to other application servers of the multiple application servers based on the first utilization level and utilization information received from the other application servers based, at least in part, on the directory of utilization information in the first application server;

pulling, based on the determining, by the first application server a request allocated to a second application server of the multiple application servers, wherein said pulling comprises identifying the pulled request by walking a queue of the multiple queues of the second application server in accordance with a modified first-in-first-out (FIFO) strategy, including skipping one or more requests in the queue having an associated predicted cost deemed too expensive to be processed by the first application server based on current conditions of the first application server; and

servicing, by the first application server, the pulled request.

8. The article of claim 7 wherein the utilization level is based, at least in part on one or more of queue depth, queue processing time, bandwidth utilization, processor utilization, and memory utilization.

9. The article of claim 7 further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

analyze one or more queue parameters for the one or more application servers; and

selectively service requests allocated to the one or more application servers based on the analysis.

10. The article of claim 7, wherein the multiple application servers are coupled in communication via a network and wherein the requests are stored in a network layer of the network while they wait to be serviced.

11. The article of claim 7, wherein requests associated with a particular application server of the multiple application servers are queued on an appropriate queue of the multiple queues based on a supplied comparator object, wherein the supplied comparator object facilitates determination of a priority of the different priorities of a particular request of the requests.

12. A system comprising:

at least one memory device;

one or more processors coupled with the at least one memory device, the one or more processors to determine a utilization level for a first application server of multiple application servers, to

determine, with one or more processors corresponding to the multiple application servers, a first utilization level for a first application server of the multiple application servers, wherein the first utilization level indicates utilization of one or more resources of the first application server;

maintain, with the one or more processors, a directory of utilization information received from other application servers of the multiple application servers within the environment, wherein the directory includes a list of healthy application servers of the multiple application servers to which request can be sent when a selected server has reached a workload threshold, wherein the list of healthy application servers is based on utilization information from the multiple application servers;

push, by the first application server, information related to the first utilization level at random intervals to one or more randomly chosen application servers of the multiple application servers to at least update the list of healthy application servers;

receive, from at least one of the multiple application servers, utilization information for one or more application servers of the multiple application servers to at least update the list of healthy application servers;

determine, with the one or more processors, the first application server is on the healthy list and should service one or more requests allocated to other application servers of the multiple application servers based on the first utilization level and utilization information received from the other application servers based, at least in part, on the directory of utilization information in the first application server;

pull, based on the determining, by the first application server a request allocated to a second application server of the multiple application servers, wherein the pulled request is identified by walking a queue of the multiple queues of the second application server in accordance with a modified first-in-first-out (FIFO) strategy, including skipping one or more requests in the queue having an associated predicted cost deemed too expensive to be processed by the first application server based on current conditions of the first application server; and

service, by the first application server, the pulled request.

13. The system of claim 12 wherein the utilization level is based, at least in part on one or more of queue depth, queue processing time, bandwidth utilization, processor utilization, and memory utilization.

14. The system of claim 12 further configurable to analyze one or more queue parameters for the second application server, and to selectively service requests allocated to the second application server based on the analysis.

15. The system of claim 12 wherein the application servers are part of a multitenant database environment that stores data for multiple client entities each client entity identified by a tenant identifier (ID) having one of one or more users associated with the tenant ID, wherein users of each of multiple client entities can only access data identified by a tenant ID associated with a respective client entity, and wherein the multitenant database is a hosted database provided by an entity separate from the client entities, and provides on-demand database service to the client entities.

16. The system of claim 12, wherein the multiple application servers are coupled in communication via a network and wherein the requests are stored in a network layer of the network while they wait to be serviced.

17. The system of claim 12, wherein requests associated with a particular application server of the multiple application servers are queued on an appropriate queue of the multiple queues based on a supplied comparator object, wherein the supplied comparator object facilitates determination of a priority of the different priorities of a particular request of the requests.