Method of certifying resiliency and recoverability level of services based on gaming mode chaosing

• US 10,574,544 B2
• Filed: 01/04/2017
• Issued: 02/25/2020
• Est. Priority Date: 01/04/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gururao, Indumathy, Kairali, Sudheesh Sivadasan, Shetty, Rohit
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US15/398,563
• Publication Number:
  US 20180191579A1
• Time in Patent Office: 1,147 Days
• Field of Search: 709223, 709224, 717103
• CPC Class Codes: G06F 11/3668 Software testing software testing in telephone exchanges H04M3/242, testing of hardware G06F11/22
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 43/55 Testing of service level quality, e.g. simulating service usage

Abstract

Embodiments of the present disclosure provide a method, system, and computer program product for certifying resiliency and recoverability level of services. The method, system and computer program product include receiving a level of testing for a target service used by an application in an ordinary course of operation. The monitoring system disrupts expected behavior of the application by adjusting one or more parameters of the application. The monitoring system determines whether the service is able to recover based on the application behavior. Upon determining that the service is able to recover based on the application behavior, reporting the level of testing as the level of testing received.

Claims

1. A method, comprising:

receiving a level of testing for a service used by an application in an ordinary course of operation;

disrupting expected behavior of the application by adjusting one or more parameters of the application;

monitoring first application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the first application behavior; and

upon determining that the service is able to recover based on the first application behavior, reporting the level of testing as the level of testing received.

2. The method of claim 1, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

disrupting expected behavior of the application by adjusting one or more parameters of the application;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the level of testing received.

3. The method of claim 1, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the starting level.

4. The method of claim 1, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is equal to the second level of testing received, reporting the level of testing as the second level of testing received.

5. The method of claim 1, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is not equal to the second level of testing received;

increasing the current level of testing to an increased level of testing,

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the increased level of testing, and

determining whether the service is able to recover based on the adjusted one or more parameters associated with increased level of testing.

6. The method of claim 1, wherein disrupting expected behavior of the application by adjusting one or more parameters of the application, comprises:

defining a first environment in which the one or more parameters will be adjusted;

defining one or more processes to be terminated based on a policy; and

terminating the defined one or more processes.

7. The method of claim 1, wherein reporting the level of testing as the level of testing received comprises:

certifying to an end customer that the application has passed a level of testing equal to the received level.

8. A system, comprising:

a computer processor; and

a memory containing a program that, when executed on the computer processor, performs an operation comprising;

receiving a level of testing for a service used by an application in an ordinary course of operation;

disrupting expected behavior of the application by adjusting one or more parameters of the application;

monitoring first application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the first application behavior; and

upon determining that the service is able to recover based on the first application behavior, reporting the level of testing as the level of testing received.

9. The system of claim 8, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

disrupting expected behavior of the application by adjusting one or more parameters of the application;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the level of testing received.

10. The system of claim 8, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the starting level.

11. The system of claim 8, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is equal to the second level of testing received, reporting the level of testing as the second level of testing received.

12. The system of claim 8, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is not equal to the second level of testing received;

increasing the current level of testing to an increased level of testing,

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the increased level of testing, and

determining whether the service is able to recover based on the adjusted one or more parameters associated with increased level of testing.

13. The system of claim 8, wherein disrupting expected behavior of the application by adjusting one or more parameters of the application, comprises:

defining a first environment in which the one or more parameters will be adjusted;

defining one or more processes to be terminated based on a policy; and

terminating the defined one or more processes.

14. The system of claim 8, wherein reporting the level of testing as the level of testing received comprises:

certifying to an end customer that the application has passed a level of testing equal to the received level.

15. A computer program product, comprising:

a computer-readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

receiving a level of testing for a service used by an application in an ordinary course of operation;

computer readable program code to disrupt expected behavior of the application by adjusting one or more parameters of the application;

computer readable program code to monitor first application behavior during execution of the service used by the application in the ordinary course of operation;

computer readable program code to determine whether the service is able to recover based on the first application behavior; and

computer readable program code to report the level of testing as the level of testing received, upon determining that the service is able to recover based on the first application behavior.

16. The computer program product of claim 15, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

disrupting expected behavior of the application by adjusting one or more parameters of the application;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the level of testing received.

17. The computer program product of claim 15, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring second application behavior during execution of the service used by the application in the ordinary course of operation;

determining whether the service is able to recover based on the second application behavior; and

upon determining that the service is not able to recover, reporting the level of testing as one less than the starting level.

18. The computer program product of claim 15, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is equal to the second level of testing received, reporting the level of testing as the second level of testing received.

19. The computer program product of claim 15, further comprising:

receiving a second level of testing for the service used by the application in ordinary course of operation;

setting a starting level at a first level of testing;

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the first level of testing;

monitoring execution of the service used by the application in the ordinary course of operation;

upon determining that the service is able to recover, determining whether a current level of testing is equal to the second level of testing received; and

upon determining that the current level of testing is not equal to the second level of testing received;

increasing the current level of testing to an increased level of testing,

disrupting expected behavior of the application by adjusting one or more parameters of the application based on the increased level of testing, and

determining whether the service is able to recover based on the adjusted one or more parameters associated with increased level of testing.

20. The computer program product of claim 15, wherein disrupting expected behavior of the application by adjusting one or more parameters of the application, comprises:

defining a first environment in which the one or more parameters will be adjusted;

defining one or more processes to be terminated based on a policy; and

terminating the defined one or more processes.