System and method for equitable processing of asynchronous messages in a multi-tenant platform

• US 10,506,024 B2
• Filed: 12/04/2018
• Issued: 12/10/2019
• Est. Priority Date: 11/12/2014
• Status: Active Grant

• Current Assignee: Zuora Incorporated
• Original Assignee: Zuora Incorporated
• Inventors: Gagliardi, Marco, Aronson, Marc, Pattabhiraman, Pushkala, Medvedev, Dmitri, Stepanian, Levon
• Primary Examiner(s): Hussain, Imad
• Application Number:
  US16/209,753
• Publication Number:
  US 20190109893A1
• Time in Patent Office: 371 Days
• Field of Search: 709204
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems, methods and media are shown for equitable job processing of asynchronous messages for multiple tenants in a multi-tenant platform that involve receiving messages of a given message type in an input buffer from at least one front end tier (FET) device, providing messages of the given message type from an output buffer to at least one back end tier (BET) device, determining a priority weight W for each message in the input buffer based on a tenant identifier and message type for the message, and evaluating the priority weight W for the message and delaying the message if the priority weight W is below a priority threshold and moving the message to the output buffer if the priority weight W is not below the priority threshold.

Claims

1. A job processing system for processing asynchronous messages for multiple tenants in a multi-tenant platform, the system comprising:

at least one hardware processor;

memory storing computer instructions, the computer instructions when executed by the at least one hardware processor configured to cause the at least one hardware processor to;

receive by an input buffer a plurality of asynchronous messages, the plurality of asynchronous messages being associated with one or more tenants of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages being associated with a particular tenant of the one or more tenants of the multiple tenants, each tenant of the multiple tenants having a tenant priority level;

determine a current execution demand for each tenant of the one or more tenants of the multiple tenants;

determine a message priority for each asynchronous message of the plurality of asynchronous messages based on the tenant priority level associated with the tenant associated with the asynchronous message and based on the current execution demand for the tenant associated with the asynchronous message;

generate a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the message priority determined for the asynchronous message and on a current backlog;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, deliver the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed by the at least one hardware processor; and

adjust the message priority of each asynchronous message in the delay buffer based on delay information.

2. The job processing system of claim 1, wherein the computer instructions are further configured to cause the at least one hardware processor to:

move the particular asynchronous message from the delay buffer to the output buffer if the message priority of the particular asynchronous message meets an output buffer upgrade threshold condition.

3. The job processing system of claim 2, wherein the computer instructions are further configured to cause the at least one hardware processor to:

move the particular asynchronous message to a rejected buffer if the message priority of the particular asynchronous message meets a rejected buffer downgrade threshold condition;

check the current execution demand for the tenant;

update the message priority of the particular asynchronous message if the current execution demand has changed; and

move the particular asynchronous message from the rejected buffer to the delay buffer if the updated message priority for the particular asynchronous message meets a delay buffer upgrade threshold condition.

4. The job processing system of claim 1, wherein the computer instructions are further configured to cause the at least one hardware processor to:

modify a re-evaluation value for the particular asynchronous message when the particular asynchronous message is delayed; and

set the message priority to a value that satisfies an output buffer upgrade threshold condition if the re-evaluation value exceeds a maximum delay threshold.

5. The job processing system of claim 1, wherein the input buffer, the delay buffer and the output buffer are parts of the memory.

6. The job processing system of claim 1, wherein the current execution demand is calculated as a count of asynchronous messages.

7. The job processing system of claim 1, wherein the current execution demand for the tenant is calculated as an execution time.

8. The job processing system of claim 1, wherein the current execution demand for the tenant is calculated as a frequency.

9. The job processing system of claim 1, wherein the current execution demand for the tenant is calculated as an interval rate.

10. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages include computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages by adjusting the tenant priority level based on the current execution demand for the tenant associated with the asynchronous message.

11. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages include computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages by adjusting the tenant priority level, but no less than a low-end boundary, based on the current execution demand for the tenant associated with the asynchronous message.

12. The job processing system of claim 1, wherein asynchronous messages of a particular message type of at least one particular tenant of the multiple tenants retain a static priority value.

13. The job processing system of claim 1, wherein the computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages include computer instructions configured to cause the at least one hardware processor to determine the message priority for each asynchronous message of the plurality of asynchronous messages for a first particular tenant according to a first rule, and to determine the message priority for each asynchronous message of the plurality of asynchronous messages for a second particular tenant according to a second rule.

14. The job processing system of claim 1, wherein the plurality of asynchronous messages belong to a particular message type.

15. A method of job processing asynchronous messages for multiple tenants in a multi-tenant platform, the method comprising the steps of:

receiving by an input buffer a plurality of asynchronous messages, the plurality of asynchronous messages being associated with one or more tenants of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages being associated with a particular tenant of the one or more tenants of the multiple tenants, each tenant of the multiple tenants having a tenant priority level;

determining a current execution demand for each tenant of the one or more tenants of the multiple tenants;

determining a message priority for each asynchronous message of the plurality of asynchronous messages based on the tenant priority level associated with the tenant associated with the asynchronous message and based on the current execution demand for the tenant associated with the asynchronous message;

generating a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the message priority determined for the asynchronous message and on a current backlog;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed; and

adjusting the message priority of each asynchronous message in the delay buffer based on delay information.

16. The method of claim 15, further comprising:

moving the particular asynchronous message from the delay buffer to the output buffer if the message priority of the particular asynchronous message meets an output buffer upgrade threshold condition.

17. The method of claim 16, further comprising:

moving the particular asynchronous message to a rejected buffer if the message priority of the particular asynchronous message meets a rejected buffer downgrade threshold condition;

checking the current execution demand for the tenant;

updating the message priority of the particular asynchronous message if the current execution demand has changed; and

moving the particular asynchronous message from the rejected buffer to the delay buffer if the updated message priority for the particular asynchronous message meets a delay buffer upgrade threshold condition.

18. The method of claim 15, further comprising:

modifying a re-evaluation value for the particular asynchronous message when the particular asynchronous message is delayed; and

setting the message priority to a value that satisfies an output buffer upgrade threshold condition if the re-evaluation value exceeds a maximum delay threshold.

19. The method of claim 15, further comprising determining that the plurality of asynchronous messages belong to a particular message type.

20. A non-transitory computer readable medium storing computer code having instructions stored therein that configure a processing device to perform the following steps:

receiving by an input buffer a plurality of asynchronous messages, the plurality of asynchronous messages being associated with one or more tenants of multiple tenants in a multi-tenant platform, each asynchronous message of the plurality of asynchronous messages being associated with a particular tenant of the one or more tenants of the multiple tenants, each tenant of the multiple tenants having a tenant priority level;

determining a current execution demand for each tenant of the one or more tenants of the multiple tenants;

determining a message priority for each asynchronous message of the plurality of asynchronous messages received based on the tenant priority level associated with the tenant associated with the asynchronous message and based on the current execution demand for the tenant associated with the asynchronous message;

generating a decision whether to allow or to delay each asynchronous message of the plurality of asynchronous messages based on the message priority determined for the asynchronous message and on a current backlog;

if the decision indicates to delay a particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to a delay buffer;

if the decision indicates to allow the particular asynchronous message of the plurality of asynchronous messages, delivering the particular asynchronous message to an output buffer for the particular asynchronous message to wait its turn to be processed by the processing device; and

adjusting the message priority of each asynchronous message in the delay buffer based on delay information.