Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems

• US 7,490,319 B2
• Filed: 11/03/2004
• Issued: 02/10/2009
• Est. Priority Date: 11/04/2003
• Status: Expired due to Fees

• Current Assignee: Kimberly-Clark Worldwide Incorporated (Kimberly-Clark Corporation)
• Original Assignee: Kimberly-Clark Worldwide Incorporated (Kimberly-Clark Corporation)
• Inventors: Kokke, Barbara Marguerite, Lindsay, Jeffrey D., Collen, John Paul, Blackwell, Barry Mark, Irwin, Anthony G., Guzman, Jr., Luis Reinaldo
• Primary Examiner(s): Zhen, Wei Y
• Assistant Examiner(s): CHOW, CHIH CHING
• Application Number:
  US10/980,920
• Publication Number:
  US 20050166094A1
• Time in Patent Office: 1,560 Days
• Field of Search: 717/124
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

The present invention relates to a testing tool for testing of complex software systems having a plurality of interrelated system components and a suite of test scenarios for testing the performance of the plurality of interrelated system components. The testing tool includes an Automated Multidimensional Traceability Matrix system for determining linkages between interrelated system components, a means for identifying a change in one or more of the interrelated system components, a means for applying the Automated Multidimensional Traceability Matrix, a means for executing all of or a subset of the test scenarios associated with the interrelated system components that may be affected by the change and a means for evaluating the results of the executed test scenarios.

Claims

1. A method for testing a complex software system, wherein the complex software system has interrelated system programs, the method comprising the steps of:

determining linkages between interrelated system programs using an automated multidimensional traceability matrix;

identifying a change in one or more of the interrelated system programs;

applying the automated multidimensional traceability matrix by using a graphical user interface that graphically displays information pertaining to a selected test scenario pertaining to the identified change, wherein the graphical user interface is adapted to display other test scenarios linked by two or more levels through at least one of dependencies and tributaries and wherein the graphical user interface is adapted to display linkages across three or more levels of dependency;

executing one or more test scenarios associated with the interrelated system programs that may be affected by the change; and

evaluating the results of the executed test scenarios, wherein the results include at least one of a list of dependencies and tributaries for the identified interrelated system programs and a list of test scenarios that invoke the identified interrelated system programs.

2. The method of claim 1, wherein the step of identifying a change in one or more of the interrelated system programs comprises an electronic tool that detects a change in an executable file.

3. The method of claim 1, wherein the step of identifying a change in one or more of the interrelated system programs comprises-manually entering information about the change.

4. The method of claim 1, wherein the step of evaluating the results of the executed test scenarios comprises manually comparing output information to previously generated output information from the test scenarios prior to the change.

5. The method of claim 1, wherein the step of evaluating the results of the executed test scenarios comprises comparing output information to previously generated output information from the test scenarios prior to the change using a computer.

6. The method of claim 1, wherein the method is applied for implementation of SAP R/3, and the method further comprises a step of using a computer assisted test tool to automatically execute selected scripts.

7. The method of claim 1, wherein the complex software system comprises software from at least one vendor interfaced with legacy software written by at least one other party.

8. The method of claim 1, wherein the complex software system is an ERP system.

9. The method of claim 1, further comprising a step of auditing changes in the complex software system and documentation associated with the complex software system.

10. The method of claim 1, wherein the graphical user interface comprises a Web-based display featuring hyperlinks for the displayed test scenarios.

11. A method of selecting a subset of test cases from a body of test cases for testing the performance of a complex software systems, each complex software system comprising a plurality of software components, each test case being associated with at least one software component in the complex software systems and each test case being associated with a script providing a series of instructions to be executed by the complex software systems for the test case, the method comprising the steps of:

a) displaying a graphical user interface to receive input information about dependencies and tributaries for the test cases and wherein the graphical user interface is adapted to display the dependencies and tributaries linked by two or more levels;

b) storing the input information in a database associated with an automated multidimensional traceability matrix;

c) receiving a change notice indicating that one or more of the software components have been changed;

d) applying the automated multidimensional traceability matrix to determine which of the test cases are linked via previously defined linkages to the one or more of the software components that have been changed;

e) displaying information using a graphical user interface that graphically displays which test cases are linked to the one or more of software components that have been changed and wherein the graphical user interface is adapted to display the linkages by two or more levels;

f) selecting all or a subset of the linked test cases;

g) executing the scripts associated with the selected linked test cases; and

h) storing actual results for the executed scripts for later comparison with expected results for the executed scripts.

12. The method of claim 11, wherein the step of displaying information to indicate which test cases are linked to the one or more of software components that have been changed further comprises the step of displaying information about linkages across three or more levels.

13. The method of claim 11, wherein the step of displaying information to indicate which test cases are linked to the one or more of software components that have been changed further comprises the step of displaying information about a number of dependency levels and a number of tributary levels, the number of tributary levels being greater than the number of dependency levels.

14. A method for user testing of updated software modules for complex software systems comprising a plurality of integrated software modules provided by one or more vendors, the method comprising the steps of:

a) receiving a software update for a software module from a vendor;

b) installing the software update on a server to provide an updated software module in the complex software system;

c) applying an automated multidimensional traceability matrix to create a list of integrated software modules whose performance is linked to the updated software module, and then to create a list of the test cases associated with the integrated software modules whose performance is linked to the updated software module by using a graphical user interface that is adapted to graphically display linkages between the test cases and linkages between the integrated software modules by two or more levels;

d) selecting a subset of the test cases list in step (c) and executing associated test scripts; and

e) evaluating the results of the executed associated test scripts, wherein the results include a list of dependencies and tributaries for the integrated software modules.

15. A method for modifying a complex software system comprising a plurality of interrelated software modules, each module being associated with one or more interrelated test cases for testing the performance of the software modules, the method comprising the steps of:

a) creating an automated multidimensional traceability matrix to electronically identify the relationships between the interrelated test cases, including dependency and tributary relationships, each test case in the automated multidimensional traceability matrix being linked to information about the associated interrelated software modules;

b) creating a baseline of performance results for all or a subset of the interrelated test cases associated with the complex software system according to one or more testing scenarios comprising predetermined input data and one or more scripts directing the execution of a plurality of tests cases;

c) creating a change in one or more of the interrelated software modules;

d) selecting a subset of the test cases for testing of the complex software system, wherein the electing is done with the aid of an interactive analysis roadmap based on information from the automated multidimensional traceability matrix, the analysis roadmap using a graphical user interface that is adapted to display the selected subset of test cases and to display the relationships between the test cases by two or more levels and wherein the graphical user interface is adapted to display the interrelated software modules that may be affected by the change;

e) executing all of or a subset of one or more test cases associated with the interrelated software modules that may be affected by the change to provide a set of post-change results; and

f) comparing the post-change results with corresponding portions of the a baseline of performance.

16. The method of claim 15, wherein a plurality of servers exist that independently run complex software systems comprising original versions of the one or more integrated software modules that were changed, further comprising the step of updating the original versions with changed versions of the one or more integrated software modules that were changed.

17. The method of claim 16, wherein the step of updating further comprises the steps of creating a patch and executing the patch on the plurality of servers.

18. The method of claim 15, wherein the step of executing all of or a subset of one or more test scenarios further comprises the step of iteratively carrying out (1) execution of a subset of the test cases in a scenario, and (2) step (e), until the one or more test scenarios are completed.

19. The method of claim 15, wherein information pertaining to the testing and change in the software is archived.

20. The method of claim 15, further comprising the steps of determining if, in response to the change in the software, the Automated Traceability Matrix needs to be updated, and if so, updating the Automated Traceability Matrix prior to selecting a subset of the test cases for testing of the complex software system.

21. The method of claim 15, wherein testing of at least a portion of the selected test cases is done with automated testing tools.

22. The method of claim 21, wherein at least one parameter in input data for use with the selected test cases is automatically varied between a plurality of values during execution of the testing.