Mechanism for facilitating sliding window resource tracking in message queues for fair management of resources for application servers in an on-demand services environment

• US 9,268,605 B2
• Filed: 03/15/2013
• Issued: 02/23/2016
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Wang, Xiaodan, Devadhar, Vijayanth, Murugesan, Praveen
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): Zhao, Bing
• Application Number:
  US13/841,713
• Publication Number:
  US 20140075446A1
• Time in Patent Office: 1,075 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: H04L 41/30 Decision processes by autonomous network management units using voting and bidding
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/83 based on usage prediction

Abstract

In accordance with embodiments, there are provided mechanisms and methods for facilitating sliding window resource tracking in message queues for fair management of resources for application servers in an on-demand services environment. In one embodiment and by way of example, a method includes monitoring, in real-time, in-flight jobs in message queues for incoming jobs from organizations in a distributed environment having application servers in communication over a network, applying local sliding windows to the message queues to estimate wait time associated with each incoming job in a message queue. A local sliding window may include segment of time being monitored in each message queue for estimating the wait time. The method may further include allocating, in real-time, based on the estimated wait time, thread resources to one or more of the incoming jobs associated with the one or more of the organizations.

Claims

1. A database system-implemented method comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

2. The method of claim 1, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

3. The method of claim 1, further comprising denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

4. The method of claim 1, further comprising communicating the expected wait time or the expected execution time to a resource aggregator at a history cache.

5. The method of claim 4, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the method further comprises aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

6. The method of claim 5, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

7. A system comprising:

a computing device having a memory to store instructions, and a processing device to execute the instructions, the computing device further having a mechanism to;

monitor, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predict, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocate, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

delay, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

8. The system of claim 7, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

9. The system of claim 7, wherein the mechanism is further to deny, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

10. The system of claim 7, wherein the mechanism is further to communicate the expected wait time or the expected execution time to a resource aggregator at a history cache.

11. The system of claim 10, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the mechanism is further to aggregate historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

12. The system of claim 11, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

13. A non-transitory machine-readable medium having stored thereon instructions which, when executed by a machine, cause the machine to perform one or more operations comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

14. The non-transitory machine-readable medium of claim 13, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

15. The non-transitory machine-readable medium of claim 13, wherein the one or more operations further comprise denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

16. The non-transitory machine-readable medium of claim 13, wherein the one or more operations further comprise communicating the expected wait time or the expected execution time to a resource aggregator at a history cache.

17. The non-transitory machine-readable medium of claim 16, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the one or more operations further comprise aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

18. The non-transitory machine-readable medium of claim 17, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

19. A database system-implemented method comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant; and

denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

20. The method of claim 19, further comprising delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

21. The method of claim 19, further comprising communicating the expected wait time or the expected execution time to a resource aggregator at a history cache.

22. The method of claim 21, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the method further comprises aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

23. The method of claim 22, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

24. A system comprising:

a computing device having a memory to store instructions, and a processing device to execute the instructions, the computing device further having a mechanism to;

monitor, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predict, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocate, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant; and

deny, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

25. The system of claim 24, wherein the mechanism is further to delay, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

26. The system of claim 24, wherein the mechanism is further to communicate the expected wait time or the expected execution time to a resource aggregator at a history cache.

27. The system of claim 26, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the mechanism is further to aggregate historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

28. The system of claim 27, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

29. A non-transitory machine-readable medium having stored thereon instructions which, when executed by a machine, cause the machine to perform one or more operations comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant; and

denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

30. The non-transitory machine-readable medium of claim 29, wherein the one or more operations further comprise delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

31. The non-transitory machine-readable medium of claim 29, wherein the one or more operations further comprise communicating the expected wait time or the expected execution time to a resource aggregator at a history cache.

32. The non-transitory machine-readable medium of claim 31, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types, and

wherein the one or more operations further comprise aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

33. The non-transitory machine-readable medium of claim 32, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

34. A database system-implemented method comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

communicating the expected wait time or the expected execution time to a resource aggregator at a history cache, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types based on information from history cache.

35. The method of claim 34, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

36. The method of claim 34, further comprising denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

37. The method of claim 34, further comprising delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

38. The method of claim 37, wherein the method further comprises aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

39. The method of claim 38, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

40. A system comprising:

a computing device having a memory to store instructions, and a processing device to execute the instructions, the computing device further having a mechanism to;

monitor, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predict, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocate, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

communicating the expected wait time or the expected execution time to a resource aggregator at a history cache, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types based on information from history cache.

41. The system of claim 40, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

42. The system of claim 40, wherein the mechanism is further to deny, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

43. The system of claim 40, wherein the mechanism is further to delay, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

44. The system of claim 43, wherein the mechanism is further to aggregate historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

45. The system of claim 44, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.

46. A non-transitory machine-readable medium having stored thereon instructions which, when executed by a machine, cause the machine to perform one or more operations comprising:

monitoring, in real-time, by the database system, time usage associated with a plurality of job types in message queues, wherein the plurality of job types are requested by a plurality of tenants in a multi-tenant environment;

predicting, in real-time, by applying a sliding time scale, a segment of time of the time usage relating to a job type in a message queue, wherein the segment of time includes an expected wait time for the job type in the message queue and an expected execution time to complete the job type;

allocating, in real-time, based on the segment of time, a set of thread resources to the job type associated with a tenant, wherein allocating further includes enforcing equitable distribution of the thread resources to other job types of the plurality of job types associated with other tenants of the plurality of tenants; and

communicating the expected wait time or the expected execution time to a resource aggregator at a history cache, wherein the sliding time scale is further to determine conflicts between the job type and the in-flight job types based on information from history cache.

47. The non-transitory machine-readable medium of claim 46, wherein allocating comprises relinquishing one or more of the set of thread resources from the job type, and offering the relinquished one or more of the set of thread resources to another job type associated with another tenant.

48. The non-transitory machine-readable medium of claim 46, wherein the one or more operations further comprise denying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type.

49. The non-transitory machine-readable medium of claim 46, wherein the one or more operations further comprise delaying, in real-time, based on the segment of time, the allocation of the set of thread resources to the job type to allow the set of thread resources to continue to be used by an existing job type or preserving the set of thread resources for a later use by the job type or one or more of the plurality of job types.

50. The non-transitory machine-readable medium of claim 49, wherein the one or more operations further comprise aggregating historical consumption of the thread resources into groups including one or more of a first group of resources historically consumed by the job type, a second group of resources historically consumed by the tenant, and a third group of resources historically consumed by long-running job types of the in-flight job types.

51. The non-transitory machine-readable medium of claim 50, wherein if a conflict exists, the history cache is updated accordingly to facilitate continuous fair allocation of the thread resources.