High availability and failover

• US 10,061,664 B2
• Filed: 01/15/2015
• Issued: 08/28/2018
• Est. Priority Date: 01/15/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Verkaik, Patrick Douglas, Shanks, Robert Tristan
• Primary Examiner(s): Bonzo, Bryce P
• Assistant Examiner(s): Arcos, Jeison C
• Application Number:
  US14/598,107
• Publication Number:
  US 20160210209A1
• Time in Patent Office: 1,321 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/2005 using redundant communication controllers
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/2033 switching over of hardware resources
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling

Abstract

Systems, methods, and computer-readable storage media for high availability and failover. A device obtains an external identity designated for a set of devices on a network, the set of devices comprising the device and a second device, and the external identity comprising public address settings which the set of devices can use when in live mode to communicate with devices outside of the network. While the device is in failover mode and the second device is in live mode, the device listens for heartbeat messages transmitted from the second device. Next, the device detects a failover event when a predetermined number of heartbeat messages have not been received by the device. In response to the failover event, the device then changes from failover mode to live mode and assumes the external identity.

Claims

1. A method comprising:

listening, via a first device on a network, for heartbeat messages transmitted from one or more second devices on the network, the first device operating in a failover mode and one of the one or more second devices operating in a live mode, each of the heartbeat messages transmitted by the one or more second devices as well as corresponding heartbeat messages transmitted by the first device having a respective priority associated therewith;

detecting a failover event when a number of heartbeat messages are not received from the one of the one or more second devices within a period of time;

immediately after detecting the failover event, determining whether to switch the first device from the failover mode to the live mode by comparing the respective priority associated with heartbeat messages transmitted by the first device and the respective priority associated with the heartbeats transmitted by all of the one or more second devices except the one of the one or more second devices operating in the live mode; and

obtaining, via the first device, an external identity for communicating with devices outside of the network upon determining to switch the first device from the failover mode to the live mode.

2. The method of claim 1, wherein the external identity is a virtual identity generated by a cloud system for shared use by the first device and the one or more second devices depending on which device is operating in live mode, the virtual identity comprising a public internet protocol (IP) address and a virtual media access control (MAC) address, wherein the first device is also assigned a private identity for communicating with devices within the network, the private identity comprising a private IP address and a MAC address.

3. The method of claim 1, wherein the listening for the heartbeat messages comprises receiving, from the one or more second devices, the heartbeat messages at identified intervals of time, the method further comprising sending, via the first device, the corresponding heartbeat messages when operating in live mode.

4. The method of claim 1, further comprising synchronizing data between the first device and the one of the one or more second devices, the data comprising at least one of dynamic host configuration protocol (DHCP) settings, network address translation (NAT) state, or a port number.

5. The method of claim 4, wherein the port number comprises a VPN port number associated with a VPN tunnel connecting the network to a cloud system, the VPN tunnel being shared by the first device and the one of the one or more second devices to communicate with the cloud system, the data synchronized between the first device and the one of the one or more second devices allowing for a handoff of the VPN tunnel between the first device and the one of the one or more second devices that is triggered by at least one of a failover event or a change in an operating mode of one of the first device or the one of the one or more second devices.

6. The method of claim 1, further comprising sending status information to a cloud system via a tunnel between the cloud system and the first device, wherein the status information comprises at least one of a failover state, an operating mode, or a health status, and wherein the status information is associated with at least one of the first device or the one of the one or more second devices.

7. The method of claim 1, wherein the first device listens to the heartbeat messages over one or more virtual local area networks (VLANs) configured on one or more ports on the first device, and wherein the first device receives, from the one of the one or more second devices, synchronization information for establishing a connection with a cloud system through a tunnel previously used by the one of the one or more second devices to communicate with the cloud system.

8. The method of claim 1, wherein the first device actively provisions services in the network while operating in live mode, and wherein the first device remains on but does not actively provision the services on the network while operating in failover mode, and wherein the first device reports data to a cloud system via a VPN tunnel between the cloud system and the network while operating in live mode, wherein the first device establishes the VPN tunnel to the cloud system using a VPN port number received from the one of the one or more second devices and at least one of the external identity and a private identity associated with the first device.

20. The method of claim 1, further comprising:

receiving an indication that the one of the one or more second devices is ready to resume operation in the live mode;

upon receiving the indication, switching the one of the one or more second devices from operating in the failover mode to operating in the live mode if the priority indicated in the heartbeat messages of the one of the one or more second devices is greater than the priority indicated by the heartbeat messages of first device.

9. A system operating in live mode on a computer network, the system comprising:

a processor; and

a computer-readable storage medium having stored therein instructions which, when executed by the processor, cause the processor to perform operations comprising;

associating an external identity generated by a cloud system with the system, the external identity comprising a public address for communicating with devices outside of the computer network;

sending heartbeat messages to at least one failover device operating in failover mode in the computer network;

sending the external identity to one of the at least one the failover device for use by the one of the at least one failover device to communicate with devices outside of the computer network when operating in live mode, the one of the at least one failover device being selected to operate in the live mode upon determining that a priority associated with heartbeat messages transmitted by the one of the at least one failover device is higher than a priority associated with heartbeat messages transmitted by each additional failover device available in the computer network; and

in response to a failover event;

switching the system from live mode to at least one of a failover mode, a standby mode, an inactive mode, or a shutoff mode; and

disassociating the external identity with the system.

10. The system of claim 9, wherein the external identity comprises a virtual internet protocol (IP) address and a virtual media access control (MAC) address, wherein the system is assigned a private identity for communicating within the network, the private identity comprising a private IP address and a MAC address.

11. The system of claim 9, the computer-readable storage medium having stored therein instructions which, when executed by the processor, perform operations comprising:

synchronizing data between the system and the one of the at least one failover device, the data comprising settings associated with a tunnel between the system and the cloud system, wherein the data is synchronized between the system and the one of the at least one failover device via a link between the system and the one of the at least one failover device established based on settings generated by the cloud system.

12. The system of claim 9, the computer-readable storage medium having stored therein instructions which, when executed by the processor, perform operations comprising sending the external identity to the one of the at least one failover device in response to at least one of a request or the failover event.

13. A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform operations comprising:

receiving, via a first device on a network, heartbeat messages transmitted from one or more second devices on the network, the first device operating in a failover mode and one of the one or more second devices operating in live mode, each of the heartbeat messages transmitted by the one or more second devices as well as corresponding heartbeat messages transmitted by the first device having a respective priority associated therewith;

synchronizing data between the first device and the one of the one or more second devices, the data comprising settings associated with a tunnel between a cloud system and the one of the one or more second devices;

detecting a failover event when a number of heartbeat messages are not received by the first device from the one of the one or more second devices within a period of time; and

immediately after detecting the failover event, determining whether to switch the first device from the failover mode to the live mode by comparing the respective priority associated with heartbeat messages transmitted by the first device and the respective priority associated with the heartbeats transmitted by all of the one or more second devices except the one of the one or more second devices operating in the live mode;

associating an external identity generated by the cloud system with the first device upon determining to switch the first device from the failover mode to the live mode, the external identity comprising a public address for communicating with devices outside of the network; and

based on the data synchronized between the first device and the one of the one or more second devices, establishing a connection with the cloud system via the tunnel to yield a tunnel handoff between the one of the one or more second devices and the first device.

14. The non-transitory computer-readable storage medium of claim 13, wherein the external identity comprises a virtual internet protocol (IP) address and a virtual hardware address, wherein the first device is assigned a private identity for communicating within the network, the private identity comprising a private IP address and a hardware address.

15. The non-transitory computer-readable storage medium of claim 13, wherein the data is synchronize between the first device and the one of the one or more second devices via a link between the first device and the one of the one or more second devices, the link being established based on settings received from the cloud system.

16. The non-transitory computer-readable storage medium of claim 13, wherein the tunnel comprises a VPN tunnel and wherein the data comprises VPN port settings, wherein the first device uses the VPN port settings to establish the VPN tunnel with the cloud system.

17. A system comprising:

a processor; and

a computer-readable storage medium having stored therein instructions which, when executed by the processor, cause the processor to perform operations comprising;

receiving heartbeat messages transmitted from a live device on a network, the heartbeat messages comprising a priority associated with the live device, the system operating in failover mode on the network and the live device operating in live mode on the network;

detecting a failover event when a number of the heartbeat messages are not received from the live device within a period of time;

in response to the failover event, switching the system from failover mode to the live mode;

obtaining an external identity for communicating with devices outside of the network, the external identify being previously used by the live device when operating in live mode;

upon the live device becoming active again after the failover event, comparing the priority of the live device with a priority of the system; and

switching the system from the live mode to the failover mode if the priority of the live device is greater than the priority of the system, the live device resuming operation in the live mode after the switching of the system to the failover mode.

18. The system of claim 17, wherein the external identity is a virtual identity generated by a cloud system, the virtual identity comprising a public internet protocol (IP) address and a virtual media access control (MAC) address, wherein the system is also assigned a private identity for communicating with devices within the network, the private identity comprising a private IP address and a MAC address.

19. The system of claim 17, wherein the system and the live device synchronize data via a link between the system and the live device, wherein the link is established based on settings received from a cloud system, and wherein the computer-readable storage medium stores additional instructions which, when executed by the processor, cause the processor to perform an operation comprising:

in response to the failure event, performing a VPN tunnel handoff between the system and the live device to establish a connection between the system and the cloud system based on a VPN tunnel previously established between the cloud system and the live device, the VPN tunnel being established based on the data synchronized via the link between the system and the live device, the data comprising at least one of a VPN port number or NAT state.