System and method for equitable processing of asynchronous messages in a multi-tenant platform

• US 10,148,738 B2
• Filed: 11/09/2015
• Issued: 12/04/2018
• Est. Priority Date: 11/12/2014
• Status: Active Grant

• Current Assignee: Zuora Incorporated
• Original Assignee: Zuora Incorporated
• Inventors: Gagliardi, Marco, Aronson, Marc, Pattabhiraman, Pushkala, Medvedev, Dmitri, Stepanian, Levon
• Primary Examiner(s): Hussain, Imad
• Application Number:
  US14/936,516
• Publication Number:
  US 20160134573A1
• Time in Patent Office: 1,121 Days
• Field of Search: 709204
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems, methods and media are shown for equitable job processing of asynchronous messages for multiple tenants in a multi-tenant platform that involve receiving messages of a given message type in an input buffer from at least one front end tier (FET) device, providing messages of the given message type from an output buffer to at least one back end tier (BET) device, determining a priority weight W for each message in the input buffer based on a tenant identifier and message type for the message, and evaluating the priority weight W for the message and delaying the message if the priority weight W is below a priority threshold and moving the message to the output buffer if the priority weight W is not below the priority threshold.

Claims

1. A job processing system for processing asynchronous messages for multiple tenants in a multi-tenant platform, the system comprising:

at least one hardware processor;

memory storing computer instructions, the computer instructions when executed by the at least one hardware processor configured to cause the at least one hardware processor to;

receive by an input buffer a plurality of asynchronous messages from at least one front end tier device, the plurality of asynchronous messages being associated with one or more of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages having a message type of multiple message types and being associated with a particular one of the one or more of the multiple tenants;

determine an execution demand of the plurality of asynchronous messages of each message type for each of the one or more of the multiple tenants;

determine a priority weight for each asynchronous message of the plurality of asynchronous messages received based on a service level associated with a tenant associated with the asynchronous message and based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message;

generate a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the priority weight determined for the asynchronous message;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed by the at least one hardware processor; and

adjust the priority weight of each asynchronous message in the delay buffer based on delay information.

2. The job processing system of claim 1, wherein the computer instructions are further configured to cause the at least one hardware processor to check a current backlog status, determine a priority threshold based on the current backlog status, and move the particular asynchronous message from the delay buffer to the output buffer if the priority weight of the particular asynchronous message is not less than the priority threshold.

3. The job processing system of claim 2, wherein the computer instructions are further configured to cause the at least one hardware processor to:

move the particular asynchronous message to a rejected buffer if the priority weight of the particular asynchronous message is less than the priority threshold by a rejection threshold value;

check an amount of message traffic for the tenant and message type of the particular asynchronous message;

update the priority weight of the particular asynchronous message if the amount of message traffic has changed; and

move the particular asynchronous message from the rejected buffer to the delay buffer if the updated priority weight has increased for the particular asynchronous message.

4. The job processing system of claim 1, wherein the computer instructions are further configured to cause the at least one hardware processor to:

increment a re-evaluation value for the particular asynchronous message when the particular asynchronous message is delayed; and

set the priority weight to a value that is equal to or greater than a priority threshold if the re-evaluation value exceeds a maximum delay threshold.

5. The job processing system of claim 1, wherein the input buffer, the delay buffer and the output buffer are parts of the memory.

6. The job processing system of claim 1, wherein the execution demand is calculated as a count of the plurality of asynchronous messages of a particular message type for a particular one of the one or more of the multiple tenants.

7. The job processing system of claim 1, wherein the execution demand is calculated as an execution time.

8. The job processing system of claim 1, wherein the execution demand is calculated as a frequency.

9. The job processing system of claim 1, wherein the execution demand is calculated as an interval rate.

10. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages received based on the service level associated with the tenant associated with the asynchronous message and based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message include computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages received by reducing the service level based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message.

11. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages received based on the service level associated with the tenant associated with the asynchronous message and based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message include computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages received by reducing the service level, but no less than a low-end boundary, based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message.

12. The job processing system of claim 1, wherein the asynchronous messages of a particular message type of at least one particular tenant of the multiple tenants retain a static priority value.

13. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages received include computer instructions configured to cause the at least one hardware processor to determine the priority weight for each asynchronous message of the plurality of asynchronous messages of a first type for a first particular tenant according to a first rule, and to determine the priority weight for each asynchronous message of the plurality of asynchronous messages of the first type for a second particular tenant according to a second rule.

14. A method of job processing asynchronous messages for multiple tenants in a multi-tenant platform, the method comprising the steps of:

receiving by an input buffer a plurality of asynchronous messages from at least one front end tier device, the plurality of asynchronous messages being associated with one or more of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages having a message type of multiple message types and being associated with a particular one of the one or more of the multiple tenants;

determining an execution demand of the plurality of asynchronous messages of each message type for each of the one or more of the multiple tenants;

determining a priority weight for each asynchronous message of the plurality of asynchronous messages received based on a service level associated with a tenant associated with the asynchronous message and based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message;

generating a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the priority weight determined for the asynchronous message;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed; and

adjusting the priority weight of each asynchronous message in the delay buffer based on delay information.

15. The method of claim 14, further comprising the steps of checking a current backlog status, determining a priority threshold based on the current backlog status, and moving the particular asynchronous message from the delay buffer to the output buffer if the priority weight of the particular asynchronous message is not less than the priority threshold.

16. The method of claim 15, further comprising the steps of:

moving the particular asynchronous message to a rejected buffer if the priority weight of the particular asynchronous message is less than the priority threshold by a rejection threshold value;

checking an amount of message traffic for the tenant and message type of the particular asynchronous message;

updating the priority weight of the particular asynchronous message if the amount of message traffic has changed; and

moving the particular asynchronous message from the rejected buffer to the delay buffer if the updated priority weight has increased for the particular asynchronous message.

17. The method of claim 14, wherein:

the step of delaying the particular asynchronous message further includes incrementing a re-evaluation value for the particular asynchronous message when the particular asynchronous message is delayed; and

the step of determining the priority weight for each asynchronous message further includes setting the priority weight to a value that is equal to or greater than a priority threshold if the re-evaluation value exceeds a maximum delay threshold.

18. A non-transitory computer readable medium storing computer code having instructions stored therein that configure a processing device to operate to perform job processing of asynchronous messages as follows:

receive by an input buffer a plurality of asynchronous messages from at least one front end tier device, the plurality of asynchronous messages being associated with one or more of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages having a message type of multiple message types and being associated with a particular one of the one or more of the multiple tenants;

determine an execution demand of the plurality of asynchronous messages of each message type for each of the one or more of the multiple tenants;

determine a priority weight for each asynchronous message of the plurality of asynchronous messages received based on a tenant associated with the asynchronous message and based on the execution demand of the plurality of asynchronous messages for the tenant of the message type associated with the asynchronous message;

generate a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the priority weight determined for the asynchronous message;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed; and

adjust the priority weight of each asynchronous message in the delay buffer based on delay information.

19. The non-transitory computer readable medium of claim 18, wherein the computer code further includes instructions for checking a current backlog status, determining a priority threshold based on the current backlog status, and moving the particular asynchronous message from the delayed buffer to the output buffer if the priority weight of the particular asynchronous message is not less than the priority threshold.

20. The non-transitory computer readable medium of claim 19, wherein the computer code further includes instructions for:

moving the particular asynchronous message to a rejected buffer if the priority weight of the particular asynchronous message is less than the priority threshold by a rejection threshold value;

checking an amount of message traffic for the tenant and message type of the particular asynchronous message;

updating the priority weigh of the particular asynchronous message if the amount of message traffic has changed; and

moving the particular asynchronous message from the rejected buffer to the delay buffer if the updated priority weight has increased for the particular asynchronous message.