Verifying controller actions in software-defined networks with controller clusters

• US 9,942,348 B2
• Filed: 08/07/2015
• Issued: 04/10/2018
• Est. Priority Date: 08/07/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dhawan, Mohan, Mahajan, Kshiteej Sharad, Mann, Vijay, Poddar, Rishabh
• Primary Examiner(s): Joo, Joshua
• Application Number:
  US14/821,089
• Publication Number:
  US 20170041427A1
• Time in Patent Office: 977 Days
• Field of Search: None
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/0695 the faulty arrangement being the maintenance, administration or management system
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/645 Splitting route computation layer and forwarding layer, e.g. routing according to path computational element [PCE] or based on OpenFlow functionality
• CPC Class Codes: H04L 63/00 Network architectures or network communication protocols for network security cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00; network architectures or network communication protocols for wireless network security H04W12/00; security arrangements for protecting computers or computer systems against unauthorised activity G06F21/00
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

Methods and arrangements for verifying controller actions in a clustered software defined network. A contemplated method includes: intercepting at least one message at the primary controller; wherein the at least one intercepted message comprises a message that induces one or more changes in the network; intercepting at least one cache update in the network; replicating the at least one intercepted message and the at least one cache update; directing the at least one replicated message and the at least one replicated cache update to a verification subsystem; and verifying the at least one replicated message and the at least one replicated cache update in the verification subsystem. Other variants and embodiments are broadly contemplated herein.

Claims

1. A method of verifying controller actions in a clustered software defined network comprising a primary controller and a plurality of secondary controllers, said method comprising:

utilizing one or more processors to execute computer code that performs the steps of;

intercepting at least one message at the primary controller wherein the at least one message is associated with an external trigger comprising a communication event arising from outside the primary controller;

wherein the at least one intercepted message comprises a message that induces one or more changes in the network;

intercepting at least one cache update in the network, wherein the at least one cache update is responsive to an internal trigger comprising a communication event arising within the primary controller;

replicating, to at least one randomly selected secondary controller from the plurality of secondary controllers, the at least one intercepted message and the at least one intercepted cache update;

directing a response to the at least one message from the primary controller, a response to the at least one cache update, a response to the at least one replicated message, and a response to the replicated at least one cache update to a verification subsystem; and

verifying the at least one replicated message and the at least one replicated cache update in the verification subsystem, wherein the verifying comprises comparing the responses from the primary controller with the responses from the at least one randomly selected secondary controller.

2. The method according to claim 1 wherein said verifying comprises verifying that the response of the at least one randomly selected secondary controller matches the response of the primary controller.

3. The method according to claim 2, wherein said verifying comprises propagating one or more tainted messages.

4. The method according to claim 1, wherein the verification subsystem comprises a data store that receives the at least one replicated cache update.

5. The method according to claim 1, wherein the verification subsystem resides on a network separate from the clustered software defined network.

6. The method according to claim 1, comprising:

detecting a trigger at one or more of;

the primary controller, and at least one of the secondary controllers; and

propagating at least one taint within the verification subsystem to map one or more responses to the detected trigger.

7. The method according to claim 1, wherein the verification subsystem comprises a subsystem having access to more than one controller.

8. The method according to claim 1, wherein said verifying comprises verifying responses to the at least one message via a majority check of the plurality of secondary controllers.

9. The method according to claim 1, wherein said intercepting said at least one message comprises intercepting at least one message at a hypervisor.

10. A system for verifying controller actions in a clustered software defined network comprising a primary controller and a plurality of secondary controllers, said system comprising:

a plurality of processors; and

one or more computer readable storage medium having computer readable program code embodied therewith and executable by the plurality of processors, the computer readable program code comprising;

computer readable program code that intercepts at least one message at the primary controller, wherein the at least one message is associated with an external trigger comprising a communication event arising from outside the primary controller;

wherein the at least one intercepted message comprises a message that induces one or more changes in the network;

computer readable program code that intercepts at least one cache update in the network, wherein the at least one cache update is responsive to an internal trigger comprising a communication event arising within the primary controller;

computer readable program code that replicates, to at least one randomly selected secondary controller from the plurality of secondary controllers, the at least one intercepted message and the at least one intercepted cache update;

computer readable program code that directs a response to the at least one message from the primary controller, a response to the at least one cache update, a response to the at least one replicated messages and a response to the at least one replicated cache update to a verification subsystem; and

computer readable program code that verifies the at least one replicated message and the at least one replicated cache update in the verification subsystem, wherein the verifying comprises comparing the responses from the primary controller with the responses from the at least one randomly selected secondary controller.

11. A computer program product for verifying controller actions in a clustered software defined network comprising a primary controller and a plurality of secondary controllers, said computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code that intercepts at least one message at the primary controller, wherein the at least one message is associated with an external trigger comprising a communication event arising from outside the primary controller;

wherein the at least one intercepted message comprises a message that induces one or more changes in the network;

computer readable program code that intercepts at least one cache update in the network, wherein the at least one cache update is responsive to an internal trigger comprising a communication event arising within the primary controller;

computer readable program code that replicates, to at least one randomly selected secondary controller from the plurality of secondary controllers, the at least one intercepted message and the at least one intercepted cache update;

computer readable program code that directs a response to the at least one message from the primary controller, a response to the at least one cache update, a response to the at least one replicated message, and a response to the at least one replicated cache update to a verification subsystem; and

computer readable program code that verifies the at least one replicated message and the at least one replicated cache update in the verification subsystem, wherein the verifying comprises comparing the responses from the primary controller with the responses from the at least one randomly selected secondary controller.

12. The computer program product according to claim 11, wherein the verification subsystem comprises a data store that receives the at least one replicated cache update.

13. The computer program product according to claim 11, wherein the verification subsystem resides on a network separate from the clustered software defined network.

14. The computer program product according to claim 11, comprising:

computer readable program code that detects a trigger at one or more of;

the primary controller, and at least one of the secondary controllers; and

computer readable program code that propagates at least one taint within the verification subsystem to map one or more responses to the detected trigger.

15. The computer program product according to claim 11, wherein the verification subsystem comprises a subsystem having access to more than one controller.

16. The computer program product according to claim 11, wherein the verifying comprises verifying responses to the at least one message via a majority check of the plurality of secondary controllers.

17. The computer program product according to claim 11, wherein the intercepting said at least one message comprises intercepting at least one message at a hypervisor.

18. A method comprising:

intercepting messages, at a hypervisor, to and from a controller in a clustered software defined network;

the intercepted messages including at least one network update corresponding to an external trigger comprising a communication event arising from outside the controller, and at least one cache update for a distributed data store in communication with the network, wherein the at least one cache update is responsive to an internal trigger comprising a communication event arising within the controller;

replicating all the intercepted messages to a verification subsystem comprising at least one randomly selected secondary controller;

the verification subsystem comprising a distributed subsystem which includes at least one randomly selected controller node in the network and an out-of-band verifier; and

utilizing the verification subsystem to verify responses to the messages from a suspect controller by comparing the responses, to the external trigger and the internal triggers, from the controller, with responses, to the replicated intercepted messages, from the at least one randomly selected secondary controller.