Methods and apparatus for load balancing processing of management information

• US 7,600,229 B1
• Filed: 09/30/2003
• Issued: 10/06/2009
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Young, Eric, Sun, Dongjun, Shmuylovich, Samuil, Farizon, Boris, Liubovich, Ilya
• Primary Examiner(s): Nguyen; Van H
• Assistant Examiner(s): WAI, ERIC CHARLES
• Application Number:
  US10/675,220
• Time in Patent Office: 2,198 Days
• Field of Search: 718/102, 718/105, 718/104, 709/225, 709/203
• US Class Current: 718/105
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Mechanisms and techniques for processing information in a management application, such as a storage area network management application, include a load manager, a set of distributed agents that collect information, and a set of store process that store the agent-collected information. The load manager receives load information from a plurality of store processes. The load information indicates a relative processing load for respective store processes. The load manager determines store process availability based on the received load information. The load manager receives a store assignment request from an agent that has an agent transaction to perform with a store process and assigns a store process of the plurality of store processes for the agent to use to perform the agent transaction based on the determined store process availability. In addition, other embodiments include the use of agent transaction weights to allow the store process and/or the load manager to further load-balance agent transactions between store processes.

Claims

1. A computer-implemented method for processing information in a management application in which at least one computer system accesses instructions from computer storage and executes the instructions to perform steps of:

receiving at least one first store assignment request from at least one first agent that has an agent transaction to perform with a store process from a plurality of store processes, the agent transaction for processing management data via a store process assigned to the agent transaction in order to convert the management data into a managed data object, the management data collected by the first agent;

determining an existence of a non-available store condition in which received load information from each store process in the plurality of store processes is not within an acceptable threshold load factor range, the load information received from each store process indicating a relative processing load for the store process;

in response to determining the existence of the non-available store condition, maintaining an agent wait table containing an agent entry for each agent sending store assignment requests during the non-available store condition, each agent entry identifying a wait threshold time specific to a corresponding agent and further identifying the store assignment request associated with the corresponding agent;

assigning a first store process from the plurality of store processes for the first agent to use to perform the agent transaction based on a store process availability of the first store process;

after assignment of the first store process for the first agent, establishing a recently assigned agent condition associated with the first store process, the recently assigned agent condition defining an acceptable number of agents assigned to the first store process during a predetermined agent assignment interval, whereby;

if the recently assigned agent condition associated with first store process equals the acceptable number of agents during the predetermined agent assignment interval, selecting a store process from the plurality of store processes for processing a second agent transaction other than the first store process to which the recently assigned agent condition applies; and

clearing the recently assigned agent condition after the predetermined agent assignment interval has elapsed.

2. The computer-implemented method of claim 1, comprising:

for each store process of the plurality of store processes, determining store process availability based on the received load information, comprising;

i) if the load information for the store process is within the acceptable threshold load factor range, identifying the store process as an available store process within the plurality of store processes; and

ii) if the load information for the store process is not within the acceptable threshold load factor range, identifying the store process as an unavailable store process within the plurality of store processes.

3. The computer-implemented method of claim 1, wherein maintaining the agent wait table comprises:

if a wait time for an agent identified in an agent entry in the agent wait table exceeds the agent'"'"'s specific wait threshold time, identifying the agent entry in the agent wait table as a starving agent entry.

4. The computer-implemented method of claim 3, wherein assigning the first store process comprises:

if there is at least one starving agent entry identified in the agent wait table, and if the store assignment request is received from an agent associated with a starving agent entry, and if there is at least one store process of the plurality of store processes that is identified as an available store process, then;

i) assigning an available store process of the plurality of store processes that has the most favorable load information as a selected store process for use in processing the agent transaction for the agent identified in the starving agent entry in the agent wait table; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request corresponding to the starving agent entry in the agent wait table; and

iii) removing the starving agent entry from the agent wait table.

5. The computer-implemented method of claim 1, wherein assigning the first store process comprises:

if there is at least one starving agent entry identified in the agent wait table and the store assignment request is received from an agent that is not associated with a starving agent entry, then;

i) updating the agent entry associated with the agent that provided the store assignment request in the agent wait table to indicate receipt of the store assignment request; and

ii) skipping assignment of an available store process to the agent that provided the store assignment request in order to wait for receipt of a store assignment request from an agent associated with a starving agent entry in the agent wait table.

6. The computer-implemented method of claim 1, wherein maintaining the agent wait table comprises:

identifying when an agent entry in the agent wait table has received no store assignment requests for a predetermined agent timeout period and in response, identifying the agent entry associated with the agent in the agent wait table as a non-responding agent.

7. The computer-implemented method of claim 1, wherein assigning the first store process comprises:

determining if there is at least one store process of the plurality of store processes that is identified as an available store process, and if so;

i) assigning a store process of the plurality of store processes that has the most favorable load information as a selected store process for use in processing the agent transaction for the agent; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request.

8. The computer-implemented method of claim 1 comprising:

repeating the steps of receiving, determining, maintaining, assigning and establishing such that, over time, assignment of store processes to handle processing of agent transactions associated with a plurality of agents is load balanced across the plurality of store processes based on the load information received from the store processes.

9. The computer-implemented method of claim 1, wherein:

the load information received from the plurality of store process includes a current collective transaction weight of all currently assigned transactions for each store process; and

the first store assignment request received from the first agent has an associated transaction weight of the agent transaction to be performed with a store process;

and wherein assigning the first store process comprises;

for each available store process, calculating a new collective transaction weight as a sum of the current collective transaction weight and the transaction weight of the agent transaction to be performed with the first store process;

determining if there is at least one store process of the plurality of store processes that has a new collective transaction weight that is within an acceptable collective transaction weight, and if so;

i) assigning a store process of the plurality of store processes that has the new collective transaction weight that is within an acceptable collective transaction weight as a selected store process for use in processing the agent transaction for the agent; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request.

10. The computer-implemented method of claim 9, wherein:

the management application is a storage area network management application;

the store assignment requests are received from agent processes operating on host computer systems in the storage area network that collect management data on behalf of managed entities associated with the agent processes, the agent processes transferring the management data within agent transactions to store processes to which they are assigned; and

the plurality of store processes operate to process the agent transactions to store the management data into a management database on behalf of the plurality of agent processes.

11. A computer system comprising:

a memory;

a processor;

a communications interface;

an interconnection mechanism coupling the memory, the processor and the communications interface;

wherein the memory is encoded with a load manager application that when performed on the processor, provides a load manager process for processing information in a management application, the load manager process causing the computer system to perform the operations of;

receiving at least one first store assignment request from at least one first agent that has an agent transaction to perform with a store process from a plurality of store processes, the agent transaction for processing management data via a store process assigned to the agent transaction in order to convert the management data into a managed data object, the management data collected by the first agent;

determining an existence of a non-available store condition in which received load information from each store process in a plurality of store processes is not within an acceptable threshold load factor range, the load information received from each store process indicating a relative processing load for the store process;

in response to determining the existence of the non-available store condition, maintaining an agent wait table containing an agent entry for each agent sending store assignment requests during the non-available store condition, each agent entry identifying a wait threshold time specific to a corresponding agent and further identifying the store assignment request associated with the corresponding agent;

assigning a first store process from the plurality of store processes for the first agent to use to perform the agent transaction based on a store process availability of the first store process;

after assignment of the first store process for the first agent, establishing a recently assigned agent condition associated with the first store process, the recently assigned agent condition defining an acceptable number of agents assigned to the first store process during a predetermined agent assignment interval, whereby;

if the recently assigned agent condition associated with first store process equals the acceptable number of agents during the predetermined agent assignment interval, selecting a store process from the plurality of store processes for processing a second agent transaction other than the first store process to which the recently assigned agent condition applies; and

clearing the recently assigned agent condition after the predetermined agent assignment interval has elapsed.

12. The computer system of claim 11, wherein when the load manager process causes the computer system to perform the operation of comprising:

for each store process of the plurality of store processes, determining store process availability based on the received load information, comprising;

i) if the load information for the store process is within the acceptable threshold load factor range, identifying the store process as an available store process within the plurality of store processes; and

ii) if the load information for the store process is not within the acceptable threshold load factor range, identifying the store process as an unavailable store process within the plurality of store processes.

13. The computer system of claim 11, wherein when the load manager process causes the computer system to perform the operation of maintaining the agent wait table comprising:

if a wait time for an agent identified in an agent entry in the agent wait table exceeds the agent'"'"'s specific wait threshold time, identifying the agent entry in the agent wait table as a starving agent entry.

14. The computer system of claim 13, wherein when the load manager process causes the computer system to perform the operation of assigning the first store process comprising:

if there is at least one starving agent entry identified in the agent wait table, and if the store assignment request is received from an agent associated with a starving agent entry, and if there is at least one store process of the plurality of store processes that is identified as an available store process, then;

i) assigning an available store process of the plurality of store processes that has the most favorable load information as a selected store process for use in processing the agent transaction for the agent identified in the starving agent entry in the agent wait table; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request corresponding to the starving agent entry in the agent wait table; and

iii) removing the starving agent entry from the agent wait table.

15. The computer system of claim 11, wherein when the load manager process causes the computer system to perform the operation of assigning the first store process comprising:

if there is at least one starving agent entry identified in the agent wait table and the store assignment request is received from an agent that is not associated with a starving agent entry, then;

i) updating the agent entry associated with the agent that provided the store assignment request in the agent wait table to indicate receipt of the store assignment request; and

ii) skipping assignment of an available store process to the agent that provided the store assignment request in order to wait for receipt of a store assignment request from an agent associated with a starving agent entry in the agent wait table.

16. The computer system of claim 11, wherein when the load manager process causes the computer system to perform the operation of maintaining the agent wait table comprising:

identifying when an agent entry in the agent wait table has received no store assignment requests for a predetermined agent timeout period and in response, identifying the agent entry associated with the agent in the agent wait table as a non-responding agent.

17. The computer system of claim 11, wherein when the load manager process causes the computer system to perform the operation of assigning the first store process comprising:

determining if there is at least one store process of the plurality of store processes that is identified as an available store process, and if so;

i) assigning a store process of the plurality of store processes that has the most favorable load information as a selected store process for use in processing the agent transaction for the agent; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request.

18. The computer system of claim 11 wherein when the load manager process causes the computer system to perform the operations of:

repeating the steps of receiving, determining, maintaining, assigning and establishing such that, over time, assignment of store processes to handle processing of agent transactions associated with a plurality of agents is load balanced across the plurality of store processes based on the load information received from the store processes.

19. The computer system of claim 11, wherein:

the load information received from the plurality of store process includes a current collective transaction weight of all currently assigned transactions for each store process; and

the first store assignment request received from the first agent has an associated transaction weight of the agent transaction to be performed with a store process;

and wherein assigning the first store process comprises;

for each available store process, calculating a new collective transaction weight as a sum of the current collective transaction weight and the transaction weight of the agent transaction to be performed with the first store process;

determining if there is at least one store process of the plurality of store processes that has a new collective transaction weight that is within an acceptable collective transaction weight, and if so;

i) assigning a store process of the plurality of store processes that has the new collective transaction weight that is within an acceptable collective transaction weight as a selected store process for use in processing the agent transaction for the agent; and

ii) forwarding a store assignment response identifying the selected store process to the agent providing the store assignment request.

20. The computer system of claim 19, wherein:

the management application is a storage area network management application;

the store assignment requests are received from agent processes operating on host computer systems in the storage area network that collect management data on behalf of managed entities associated with the agent processes, the agent processes transferring the management data within agent transactions to store processes to which they are assigned; and

the plurality of store processes operate to process the agent transactions to store the management data into a management database on behalf of the plurality of agent processes.

21. A computer program product having a computer readable medium including computer program logic encoded thereon that, when executed on a computer system provides a method for processing information in a management application by causing the computer system to perform the operations of:

receiving at least one first store assignment request from at least one first agent that has an agent transaction to perform with a store process from a plurality of store processes, the agent transaction for processing management data via a store process assigned to the agent transaction in order to convert the management data into a managed data object, the management data collected by the first agent;

determining an existence of a non-available store condition in which received load information from each store process in a plurality of store processes is not within an acceptable threshold load factor range, the load information received from each store process indicating a relative processing load for the store process;

in response to determining the existence of the non-available store condition, maintaining an agent wait table containing an agent entry for each agent sending store assignment requests during the non-available store condition, each agent entry identifying a wait threshold time specific to a corresponding agent and further identifying the store assignment request associated with the corresponding agent;

assigning a first store process from the plurality of store processes for the first agent to use to perform the agent transaction based on a store process availability of the first store process;

after assignment of the first store process for the first agent, establishing a recently assigned agent condition associated with the first store process, the recently assigned agent condition defining an acceptable number of agents assigned to the first store process during a predetermined agent assignment interval, whereby;

if the recently assigned agent condition associated with first store process equals the acceptable number of agents during the predetermined agent assignment interval, selecting a store process from the plurality of store processes for processing a second agent transaction other than the first store process to which the recently assigned agent condition applies; and

clearing the recently assigned agent condition after the predetermined agent assignment interval has elapsed.