System for metamorphic relationship based code testing using mutant generators

• US 10,642,723 B1
• Filed: 02/05/2019
• Issued: 05/05/2020
• Est. Priority Date: 02/05/2019
• Status: Active Grant

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Krishnamoorthy, Madhusudhanan
• Primary Examiner(s): Bui, Hanh Thi-Minh
• Application Number:
  US16/267,658
• Time in Patent Office: 455 Days
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 8/30 Creation or generation of source code
• CPC Class Codes: G06F 8/40 Transformation of program code
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Embodiments of the invention are directed to metamorphic relationship based code testing using mutant generators. The system is configured for identifying and remediating defects in an original program based on constructing at least one mutated program by distorting the original program, and analyzing expression of mutants in test results. In particular, the system receives a request to perform defect analysis of an original program. In response, the system constructs a first mutated program by embedding one or more mutants in the original program code. Moreover, the system typically maps the one or more mutants to one or more metamorphic relationships of process functions of the original program. The system may then implement tests of the original program and the first mutated program by providing one or more predetermined test cases as input to both, to determine whether the original program comprises at least one defect.

Claims

1. A system for metamorphic relationship based code testing using mutant generators, wherein the system is configured for identifying and remediating defects in an original program based on constructing at least one mutated program by distorting the original program, and analyzing expression of mutants in test results, the system comprising:

at least one memory device with computer-readable program code stored thereon;

at least one communication device;

at least one processing device operatively coupled to the at least one memory device and the at least one communication device, wherein executing the computer-readable code is configured to cause the at least one processing device to;

receive, via an operative communication channel with a user device, a user request to perform defect analysis of an original program;

retrieve original program code associated with the original program;

determine one or more metamorphic relationships associated with one or more process functions of the original program based on analyzing the original program;

construct a first mutated program, wherein constructing the mutated program comprises;

constructing one or more mutants, wherein each mutant of the one or more mutants is constructed by distorting a portion of the original program code; and

constructing the first mutant program based on embedding the one or more mutants in the original program code;

map the one or more mutants to the one or more metamorphic relationships;

implement a first test by providing one or more predetermined test cases as input to the original program, wherein the original program provides an original test output in response to the first test;

implement a second test by providing the one or more predetermined test cases as input to the first mutated program, wherein the first mutated program provides a first mutated test output in response to the second test;

determine whether the original program comprises at least one defect based on at least comparing the original test output to the first mutated test output;

in response to determining that the original program comprises the at least one defect, determine a first mutant of the one or more mutants associated with the at least one defect;

identify a first metamorphic relationship of the one or more metamorphic relationships mapped to the first mutant;

transmit, via the operative communication channel with the user device, a notification to the user indicating the at least one defect and the first metamorphic relationship; and

in response to a defect correction input from the user, initiate correction of the original program code by at least modifying code associated with a first process function of the one or more process functions of the original program, thereby constructing a corrected original program code.

2. The system of claim 1, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the first mutated test output; and

determining that the original program comprises the at least one defect based on at least determining that the original test output matches the first mutated test output.

3. The system of claim 1, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the first mutated test output; and

determining that the original program does not comprise the at least one defect based on at least determining that the original test output does not match the first mutated test output.

4. The system of claim 1, wherein the original program is a machine learning model program or a deep learning model program.

5. The system of claim 1, wherein executing the computer-readable code is configured to cause the at least one processing device to

perform defect analysis of the corrected original program code based on constructing a second mutated program.

6. The system of claim 1, wherein constructing the one or more mutants by distorting the portion of the original program code further comprises:

determining a first process function of the one or more process functions associated with the portion of the original program code;

determining a first type of process function associated with the first process function;

determining a second type of process function that is (i) associated with the first process function and (ii) structured to return a different output than the first type of process function; and

constructing a first mutant of the one or more mutants such that the first mutant is associated with the second type of process function.

7. The system of claim 1, wherein constructing the one or more mutants by distorting the portion of the original program code further comprises:

determining a first process function of the one or more process functions associated with the portion of the original program code;

analyzing first code associated with the first process function; and

constructing a first mutant of the one or more mutants by introducing a predetermined error in the first code associated with the first process function, such that the first mutant is structured to return a different output than the first code associated with the first process function.

8. The system of claim 1, wherein constructing the first mutant program based on embedding the one or more mutants in the original program code further comprises:

constructing a replica of the original program code; and

embedding the one or more mutants in the original program code by overwriting a portion of the replica of the original program code with at least one of the one or more mutants.

9. The system of claim 1, wherein constructing the first mutant program based on embedding the one or more mutants in the original program code further comprises:

constructing a replica of the original program code; and

embedding the one or more mutants in the original program code by inserting at least one of the one or more mutants at a predetermined location.

10. The system of claim 1, wherein determining whether the original program comprises the at least one defect, further comprises determining a first divergence parameter between the original test output and the first mutated test output.

11. The system of claim 10, wherein executing the computer-readable code is configured to cause the at least one processing device to:

in response to comparing the original test output to the first mutated test output, construct a second mutated program;

implement a third test on the original program, wherein the original program provides an updated original test output in response to the third test;

implement a fourth test on the second mutated program, wherein the second mutated program provides a second mutated test output in response to the fourth test;

determine a second divergence parameter between the updated original test output and the mutated test output; and

in response to determining that a difference between the first divergence parameter and the second divergence parameter is below a predetermined threshold;

terminate the defect analysis of the original program; and

transmit, via the operative communication channel with the user device, a notification to the user indicating completion of the defect analysis of the original program.

12. The system of claim 10, wherein the first divergence parameter is associated with a difference in accuracy between the original test output and the first mutated test output.

13. A computer program product for metamorphic relationship based code testing using mutant generators, wherein the computer program product is configured for identifying and remediating defects in an original program based on constructing at least one mutated program by distorting the original program, and analyzing expression of mutants in test results, the computer program product comprising a non-transitory computer-readable storage medium having computer-executable instructions to:

receive, via an operative communication channel with a user device, a user request to perform defect analysis of an original program;

retrieve original program code associated with the original program;

determine one or more metamorphic relationships associated with one or more process functions of the original program based on analyzing the original program;

construct a first mutated program, wherein constructing the mutated program comprises;

constructing one or more mutants, wherein each mutant of the one or more mutants is constructed by distorting a portion of the original program code; and

constructing the first mutant program based on embedding the one or more mutants in the original program code;

map the one or more mutants to the one or more metamorphic relationships;

implement a first test by providing one or more predetermined test cases as input to the original program, wherein the original program provides an original test output in response to the first test;

implement a second test by providing the one or more predetermined test cases as input to the first mutated program, wherein the first mutated program provides a first mutated test output in response to the second test;

determine whether the original program comprises at least one defect based on at least comparing the original test output to the first mutated test output;

in response to determining that the original program comprises the at least one defect, determine a first mutant of the one or more mutants associated with the at least one defect;

identify a first metamorphic relationship of the one or more metamorphic relationships mapped to the first mutant;

transmit, via the operative communication channel with the user device, a notification to the user indicating the at least one defect and the first metamorphic relationship; and

in response to a defect correction input from the user, initiate correction of the original program code by at least modifying code associated with a first process function of the one or more process functions of the original program, thereby constructing a corrected original program code.

14. The computer program product of claim 13, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the mutated test output; and

determining that the original program comprises the at least one defect based on at least determining that the original test output matches the mutated test output.

15. The computer program product of claim 13, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the first mutated test output; and

determining that the original program does not comprise the at least one defect based on at least determining that the original test output does not match the first mutated test output.

16. The computer program product of claim 13,

wherein the non-transitory computer-readable storage further comprises computer-executable instructions to

perform defect analysis of the corrected original program code based on constructing a second mutated program.

17. A method for metamorphic relationship based code testing using mutant generators, wherein the method is configured for identifying and remediating defects in an original program based on constructing at least one mutated program by distorting the original program, and analyzing expression of mutants in test results, the method comprising:

receiving, via an operative communication channel with a user device, a user request to perform defect analysis of an original program;

retrieving original program code associated with the original program;

determining one or more metamorphic relationships associated with one or more process functions of the original program based on analyzing the original program;

constructing a first mutated program, wherein constructing the mutated program comprises;

constructing one or more mutants, wherein each mutant of the one or more mutants is constructed by distorting a portion of the original program code; and

constructing the first mutant program based on embedding the one or more mutants in the original program code;

map the one or more mutants to the one or more metamorphic relationships;

implementing a first test by providing one or more predetermined test cases as input to the original program, wherein the original program provides an original test output in response to the first test;

implementing a second test by providing the one or more predetermined test cases as input to the first mutated program, wherein the first mutated program provides a first mutated test output in response to the second test;

determining whether the original program comprises at least one defect based on at least comparing the original test output to the mutated test output;

in response to determining that the original program comprises the at least one defect, determine a first mutant of the one or more mutants associated with the at least one defect;

identifying a first metamorphic relationship of the one or more metamorphic relationships mapped to the first mutant;

transmitting, via the operative communication channel with the user device, a notification to the user indicating the at least one defect and the first metamorphic relationship; and

in response to a defect correction input from the user, initiate correction of the original program code by at least modifying code associated with a first process function of the one or more process functions of the original program, thereby constructing a corrected original program code.

18. The method of claim 17, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the mutated test output; and

determining that the original program comprises the at least one defect based on at least determining that the original test output matches the mutated test output.

19. The method of claim 17, wherein determining whether the original program comprises the at least one defect, further comprises:

determining whether the original test output matches the first mutated test output; and

determining that the original program does not comprise the at least one defect based on at least determining that the original test output does not match the first mutated test output.

20. The method of claim 17, wherein the method further comprises

performing defect analysis of the corrected original program code based on constructing a second mutated program.