Automatic reconnection of partner software processes in a fault-tolerant computer system

• US 6,629,260 B1
• Filed: 08/25/2000
• Issued: 09/30/2003
• Est. Priority Date: 02/16/2000
• Status: Expired due to Term

• Current Assignee: Metaswitch Networks Limited (Microsoft Corporation)
• Original Assignee: Data Connection Ltd. (Microsoft Corporation)
• Inventors: Shepherd, Adam Paul, Dancer, Colin Michael
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): Bonzo, Bryce P.
• Application Number:
  US09/645,535
• Time in Patent Office: 1,131 Days
• Field of Search: 714/3, 714/15, 714/38, 714/48, 714/13
• US Class Current: 714/3
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

In a fault-tolerant system employing software fault-tolerance techniques, automatic reconnection of partner processes after a failure of one or more of the partner processes requires that the partner processes are aware of the redundancy strategy employed by their partners. The present invention insulates the partner processes from the need for this knowledge, and also enables automatic reconnection between partner processes running in different parts of a heterogeneous distributed system.

Claims

1. A method for automatically reconnecting software processes that share an interface after one or more said software processes, known as partner processes, fails over to a redundant backup copy, said method comprising of

a. first means for registering each copy of said partner processes with a join manager component when said copy of said partner processes finishes initializing

b. second means for controlling the primary or backup status of each said copy of said partner processes that executes on one or a plurality of processing units in a computer system

c. third means for said partner processes requesting that said join manager create or release an association between two or more said partner processes, said association being known as a join

d. fourth means for detecting the failure of the primary or backup copy of said partner process associated with said join

e. fifth means for notifying the remaining partner processes associated with said join of the failure of said primary copy of said partner process associated with said join

f. sixth means for said join manager notifying said remaining partner processes when a replacement primary copy of said failed partner process is restored to service

whereby said partner processes do not have to be aware of the redundancy or replication strategy used by other said partner processes associated with said join, which may be different for each said partner process associated with said join, and whereby said automatic reconnection of said partner processes is independent of the hardware or operating system support for such reconnection.

2. The method as set forth in claim 1, wherein said join manager is a software process and a copy of said join manager process executes on one or a plurality of said processing units with one said copy of said join manager process chosen to be the primary copy of said join manager process, said primary copy of said join manager process being responsible for all interaction between said join manager process and said partner processes and for replication of any state information required by the backup copies of said join manager process if said primary copy of said join manager process fails and a new primary copy is chosen from said backup copies of said join manager process.

3. The method as set forth in claim 1, wherein said first means is achieved by said join manager having responsibility for starting and stopping said copy of said partner processes.

4. The method as set forth in claim 1, wherein said first means is achieved by said copy of said partner process informing said join manager when it is initialized via use of a message-based or function call-based interface provided by said join manager.

5. The method as set forth in claim 1, wherein said second means is achieved by said join manager sending messages to a said copy of said partner process when said join manager requires said copy to change to primary or backup state.

6. The method as set forth in claim 1, wherein said third means is achieved by said partner processes communicating with said join manager of said joins via a message-based interface provided by said join manager.

7. The method as set forth in claim 1, wherein said third means is achieved by said partner processes communicating with said join manager of said joins via a function call-based interface provided by said join manager.

8. The method as set forth in claim 1, wherein said fourth means is achieved via use of facilities provided by the operating system to detect the failure of said primary or backup partner process copy.

9. The method as set forth in claim 1, wherein said fourth means is achieved by requiring said copies of said partner processes each to send a watchdog message to said join manager on a regular basis to confirm the continued correct operation of said copies of said partner processes.

10. The method as set forth in claim 1, wherein said fourth means is achieved by said join manager using seventh means to monitor for the failure of said processing units on which one or more of said partner processes are executing.

11. The method as set forth in claim 10, wherein said seventh means is achieved via use of facilities provided by the operating system to detect the failure of said processing units.

12. The method as set forth in claim 10, wherein said seventh means is achieved by requiring said operating system executing on each of said processing units to send a watchdog message to said join manager on a regular basis to confirm the continued correct operation of said processing units.

13. The method as set forth in claim 1, wherein said fifth means is achieved via use of a message-based interface provided by said join manager.

14. The method as set forth in claim 1, wherein said fifth means is achieved via use of a function call-based interface provided by said join manager.

15. The method as set forth in claim 1, wherein said fifth means further includes eighth means for said remaining partner processes to determine whether said primary copy of said failed partner process is likely to be restored to service fast enough for said remaining partner processes to consider said join to persist across the failure and recovery of said failed partner process, the timing requirements for said join to persist being determined independently by said remaining partner processes.

16. The method as set forth in claim 15, wherein said eighth means is achieved by said system manger identifying on said notification to said remaining partner processes whether a backup copy of said failed partner process is already registered with said join manager and will be instructed by said join manager to become said replacement primary copy of said failed partner process.

17. The method as set forth in claim 1, wherein said sixth means is achieved via use of a message-based interface provided by said join manager.

18. The method as set forth in claim 1, wherein said sixth means is achieved via use of a function call-based interface provided by said join manager.

19. The method as set forth in claim 1, wherein said sixth means further includes an ninth means for the remaining partner processes to determine whether said replacement primary copy of said failed partner process has been restored to service fast enough for said remaining partner processes to consider said join to persist across the failure and recovery of said failed partner process, the timing requirements for said join to persist being determined independently by said remaining said partner processes.

20. The method as set forth in claim 19, wherein said ninth means is achieved by said system manger identifying on said notification to the remaining said partner processes the time interval between the failure of the previous said primary copy of said failed partner process and the activation of said replacement primary copy thereof.

21. The method as set forth in claim 19, wherein said ninth means is achieved by said system manger identifying on said notification to the remaining said partner processes the time of the failure of the previous said primary copy of said failed partner process.

22. The method as set forth in claim 1, wherein said third means further includes tenth means for said join manager to identify the partner processes associated with said request to create or release said join.

23. The method set forth in claim 22, wherein said tenth means is achieved by means of static or dynamic configuration information available to said join manager.

24. The method set forth in claim 22, wherein said tenth means is achieved by said partner process specifying an identifier for each said partner process associated with said join, said identifier being registered with said join manager by said first means.

25. The method as set forth in claim 1, wherein said first means is achieved by said join manager using eleventh means to discover the capabilities of each said processing unit to run said partner processes and said join manager starting or stopping said partner processes on said processing units in accordance to the load presented to the system and the capabilities of said processing units.

26. The method as set forth in claim 25, wherein said eleventh means is achieved by said join manager using operating system facilities to discover when said processing units are added to or removed from the system and to learn the type of each said processing unit, said type being matched by said join manager against static or dynamic configuration information available to said join manager to determine the capability of said processing units to run said partner processes.