Generating a fingerprint representing a response of an application to a simulation of a fault of an external service

• US 9,811,447 B2
• Filed: 06/20/2013
• Issued: 11/07/2017
• Est. Priority Date: 06/20/2013
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: entIT Software LLC (Open Text Corporation)
• Inventors: Horovitz, Yair, Cohen, Ira, Mordechai, Eli
• Primary Examiner(s): Ehne, Charles
• Application Number:
  US14/787,489
• Publication Number:
  US 20160085664A1
• Time in Patent Office: 1,601 Days
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Examples disclosed herein relate to generating a fingerprint representing a response of an application to a simulation of a fault of an external service. Examples include causing simulation of a fault of an external service in a simulation of the external service, and generating a testing application fingerprint representing a response of an application to the simulation of the fault of the external service during the testing of the application.

Claims

1. A non-transitory machine-readable storage medium comprising instructions executable by a processing resource to:

instruct an external service virtualization system to simulate a fault of an external service in a simulation of the external service, wherein the simulation of the external service is accessed by an application during testing of the application;

acquire a plurality of testing metrics representing a response of the application to the simulation of the fault of the external service during the testing of the application;

generate a testing application fingerprint representing the response of the application to the simulation of the fault of the external service during the testing of the application, wherein the testing application fingerprint indicates, for each of the testing metrics, whether the testing metric became abnormal in response to the simulation of the fault; and

associate the testing application fingerprint with the external service and the fault in a repository storing the testing application fingerprint.

2. The storage medium of claim 1, wherein the instructions to generate further comprise instructions to:

generate a production application fingerprint representing a state of the application in response to detection of a problem during non-testing execution of the application, wherein the application is to utilize the external service during the non-testing execution.

3. The storage medium of claim 2, further comprising instructions executable by the processing resource to:

compare the testing and production application fingerprints; and

identify the external service and the fault associated with the testing application fingerprint as sources of the detected problem based on the result of the comparison.

4. The storage medium of claim 3, wherein:

the instructions to compare comprise instructions to determine that, among a plurality of fingerprints stored in the repository, the production application fingerprint is most similar to the testing application fingerprint included in the plurality, and that the production application fingerprint satisfies a similarity threshold relative to the testing application fingerprint; and

the instructions to identify comprise instructions to identify the external service and the fault associated with the testing application fingerprint as sources of the detected problem in response to the determination.

5. The storage medium of claim 2, further comprising instructions executable by the processing resource to:

acquire a plurality of production metrics representing performance of the application during the non-testing execution of the application; and

compare at least one of the production metrics to a respective problem threshold; and

detect the problem during the non-testing execution of the application in response to at least one result of the comparison.

6. The storage medium of claim 1, further comprising instructions executable by the processing resource to:

generate the testing application fingerprint as a binary value of “

1”

or “

0”

to indicate that a respective one the testing metrics became abnormal in response to the simulation of the fault.

7. A system comprising:

a processor; and

a memory storing instructions that when executed by the processor cause the processor to;

instruct an external service virtualization system to simulate a target fault of an external service in a given simulation of the external service;

instruct a test generator to cause the application to access the given simulation of the external service such that the application is exposed to the simulation of the target fault of the external service during testing of the application and acquire a plurality of testing metrics representing a response of the application to the simulation of the target fault of the external service during the testing of the application;

generate a given testing application fingerprint representing the response of the application to the simulation of the target fault of the external service during the testing of the application, wherein the testing application fingerprint indicates, for each of the testing metrics, whether the testing metric became abnormal in response to the simulation of the target fault; and

store the given testing application fingerprint in a repository with a description of the external service and the target fault of the external service.

8. The system of claim 7, wherein the instructions are to cause the processor to:

cause a testing system to initiate the testing of the application.

9. The system of claim 8, wherein the instructions are to cause the processor to:

instruct the external service virtualization system to simulate the target fault of the external service multiple times such that each simulation of the target fault coincides with different testing conditions for the testing of the application;

for each respective simulation of the target fault, acquire a plurality of production metrics representing a response of the application to the respective simulation of the target fault during the testing of the application; and

combine the plurality of production metrics for each simulation of the target fault to generate the testing application fingerprint.

10. The system of claim 9, wherein the instructions are to cause the processor to:

for each of a plurality of simulations of different external services, instruct the external service virtualization system to simulate each of a plurality of different faults in the external service; and

for each of the faults simulated in each of the simulations, generate an associated testing application fingerprint representing a response of the application to the fault simulated during the testing of the application,

wherein the plurality of simulations includes the given simulation, and the plurality of faults for the given simulation includes the target fault, and the testing application fingerprint associated with the target fault is the given testing application fingerprint.

11. The system of claim 10, wherein the instructions are to cause the processor to:

for each testing application fingerprint, store the testing application fingerprint in the repository with a description of the external service and the fault of the external service associated with the testing application fingerprint.

12. The system of claim 11, wherein the instructions are to cause the processor to:

generate a production application fingerprint representing a state of the application in response to detection of a problem during non-testing execution of the application, wherein the application is to utilize each of the external services in the non-testing execution.

13. The system of claim 12, wherein the instructions are to cause the processor to:

determine a selected testing application fingerprint, among the testing application fingerprints stored in the repository, that is most similar to the production application fingerprint and that satisfies a similarity threshold relative to the production application fingerprint; and

identify, in response to the determination, the external service and the fault associated with the selected testing application fingerprint as sources of the detected problem.

14. The system of claim 7, wherein to generate the testing application fingerprint, the instructions are to cause the processor to:

generate the testing application fingerprint as a binary value of “

1”

or “

0”

to indicate that a respective one of the testing metrics became abnormal in response to the simulation of the fault.

15. A method comprising:

causing an external service simulation, utilized by an application during testing of the application, to simulate a fault of the external service;

acquiring a plurality of testing metrics representing a response of the application to the simulation of the fault of the external service during the testing of the application;

generating, with a processing resource of a computing device, a testing application fingerprint representing the response of the application to the simulation of the fault of external service based on the acquired testing metrics, wherein the testing application fingerprint indicates, for each of the testing metrics, whether the testing metric became abnormal in response to the simulation of the fault;

storing the testing application fingerprint in a repository with an associated description of the external service and the fault of the external service;

in response to detection of a problem during non-testing execution of the application in which the application is to utilize the external service, generating a production application fingerprint representing a state of the application;

comparing the testing and production application fingerprints; and

identifying the external service and the fault associated with the testing application fingerprint as sources of the detected problem based on a result of the comparison.

16. The method of claim 15, wherein the comparing comprises:

determining that, among a plurality of application fingerprints of the repository, the production application fingerprint is most similar to the testing application fingerprint; and

determining that the testing application fingerprint satisfies a similarity threshold relative to the production application fingerprint;

wherein the identifying comprises identifying the external service and the fault associated with the testing application fingerprint as sources of the detected problem in response to the determinations.

17. The method of claim 15, wherein generating the testing application fingerprint includes:

generating the testing application fingerprint as a binary value of “

1”

or “

0”

to indicate that a respective one of the testing metrics became abnormal in response to the simulation of the fault.

18. The method of claim 15, further comprising:

instructing an external service virtualization system to simulate the target fault of the external service multiple times such that each simulation of the target fault coincides with different testing conditions for the testing of the application;

for each respective simulation of the target fault, acquiring a plurality of production metrics representing a response of the application to the respective simulation of the target fault during the testing of the application; and

combining the plurality of production metrics for each simulation of the target fault to generate the testing application fingerprint.

19. The method of claim 15, further comprising:

for each of a plurality of simulations of different external services, instructing the external service virtualization system to simulate each of a plurality of different faults in the external service; and

for each of the faults simulated in each of the simulations, generating an associated testing application fingerprint representing a response of the application to the fault simulated during the testing of the application,

wherein the plurality of simulations includes the given simulation, and the plurality of faults for the given simulation includes the target fault, and the testing application fingerprint associated with the target fault is the given testing application fingerprint.

20. A non-transitory machine-readable storage medium comprising instructions executable by a processing resource to:

instruct an external service virtualization system to simulate a fault of an external service in a simulation of the external service, wherein the simulation of the external service is accessed by an application during testing of the application;

generate a testing application fingerprint representing a response of the application to the simulation of the fault of the external service during the testing of the application;

associate the testing application fingerprint with the external service and the fault in a repository storing the testing application fingerprint;

generate a production application fingerprint representing a state of the application in response to detection of a problem during non-testing execution of the application, wherein the application is to utilize the external service during the non-testing execution;

compare the testing application fingerprint and the production application fingerprint, including determining that, among a plurality of fingerprints stored in the repository, the production application fingerprint is most similar to the testing application fingerprint included in the plurality, and that the production application fingerprint satisfies a similarity threshold relative to the testing application fingerprint; and

identify the external service and the fault associated with the testing application fingerprint as sources of the detected problem in response to the determination.