Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment

• US 9,201,696 B2
• Filed: 11/14/2011
• Issued: 12/01/2015
• Est. Priority Date: 12/10/2010
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Yang, Ronald, Devadhar, Vijayanth, Cheenath, Manoj
• Primary Examiner(s): Hwang, Joon H
• Assistant Examiner(s): ALRIYASHI, ABDULKADER MOHAMED
• Application Number:
  US13/295,644
• Publication Number:
  US 20120151063A1
• Time in Patent Office: 1,478 Days
• Field of Search: 709223-226, 709238-244
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment. Requests directed to an application server are received into one or more queues in front of the application server. An acknowledgement of the requests is provided in response to the requests being received by the one or more queues. Metadata associated with the requests is utilized to manage the one or more queues. The requests from the one or more queues are processed based on the metadata.

Claims

1. A method for managing incoming requests for an application server coupled to provide responses to the requests within a multi-organization computing environment, the method comprising:

receiving the requests directed to the application server, from a network socket and placing the requests into one or more of at least three queues maintained in front of the application server so that the requests are maintained in the network layer and not in application server memory or disk, and wherein the at least three queues each have an associated priority, wherein the queues each have an associated sensitivity mechanism that respond to shared resources globally and application server specific resources locally;

utilizing metadata associated with the requests to manage the one or more queues by analyzing the metadata with a peeker thread and creating an object with the peeker thread to encapsulate the metadata comprising organization information related to the requests, request information, priority information, and resource information, wherein the organization information corresponds to an organization that is one of multiple organizations in the multi-organization computing environment;

maintaining, with a plurality of application servers a list of healthy application servers to which requests are sent when a target application server has reached a workload threshold;

monitoring the at least three queues with a pinger thread to maintain a map of application server specific resources;

pushing a request from an unhealthy application server to a healthy server from the list in response to an application server being designated unhealthy by exceeding the workload threshold;

processing the requests from the one or more queues based on the metadata, wherein the requests are processed by worker threads that are sensitive to availability of one or more server resources.

2. The method of claim 1 wherein the processing is performed in a database environment that comprises a multitenant database environment, wherein the multitenant database environment stores data for multiple client entities and provides on-demand database service to the client entities.

3. The method of claim 1 wherein the metadata comprises one or more of:

a portion of a browser header and a portion of a packet header.

4. The method of claim 1 wherein utilizing the metadata associated with the requests to manage the one or more queues comprises:

determining an associated priority for the respective requests based on the metadata; and

assigning the requests to the queues based on the associated priority.

5. An article comprising a non-transitory computer-readable medium having stored thereon instructions that when executed by one or more processors to manage incoming requests for an application server within a multi-organization computing environment coupled to provide responses to the requests by:

receive the requests directed to the application server, from a network socket and placing the requests into one or more of at least three queues maintained in front of the application server so that the requests are maintained in the network layer and not in application server memory or disk, and wherein the at least three queues each have an associated priority, wherein the queues each have an associated sensitivity mechanism that respond to shared resources globally and application server specific resources locally;

utilizing metadata associated with the requests to manage the one or more queues by analyzing the metadata with a peeker thread and creating an object with the peeker thread to encapsulate the metadata comprising organization information related to the requests, request information, priority information, and resource information, wherein the organization information corresponds to an organization that is one of multiple organizations in the multi-organization computing environment;

maintaining, with a plurality of application servers a list of healthy application servers to which requests are sent when a target application server has reached a workload threshold;

monitoring the at least three queues with a pinger thread to maintain a map of application server specific resources;

pushing a request from an unhealthy application server to a healthy server from the list in response to an application server being designated unhealthy by exceeding the workload threshold;

processing the requests from the one or more queues based on the metadata, wherein the requests are processed by worker threads that are sensitive to availability of one or more server resources.

6. The article of claim 5 wherein the processing is performed in a database environment that comprises a multitenant database environment, wherein the multitenant database environment stores data for multiple client entities and provides on-demand database service to the client entities.

7. The article of claim 5 wherein the metadata comprises one or more of:

a portion of a browser header and a portion of a packet header.

8. The article of claim 5 wherein the instructions that cause the one or more processors to utilize the metadata associated with the requests to manage the one or more queues comprise instructions that, when executed by the one or more processors:

determine an associated priority for the respective requests based on the metadata; and

assign the requests to the queues based on the associated priority.