Method and apparatus for testing a software component using an abstraction matrix

• US 6,941,546 B2
• Filed: 08/01/2001
• Issued: 09/06/2005
• Est. Priority Date: 08/01/2001
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hoskins, Curtis L., Apuzzo, Joseph T., Suri, Hemant R., Marino, John P., Race, Timothy L.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): VU, TUAN A
• Application Number:
  US09/919,753
• Publication Number:
  US 20030028856A1
• Time in Patent Office: 1,497 Days
• Field of Search: 717103-105, 717/108, 717/143, 717120-130, 714 38- 39, 714/738, 714/26, 714/46, 716/4, 716/5, 702/120, 702/185
• US Class Current: 717/135
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3696 Methods or tools to render software testable

Abstract

A functional testing technique is provided employing an abstraction matrix that describes a complex software component to be tested. The abstraction matrix includes state and event information. The technique is an automated process which parses the abstraction matrix to generate test cases and mapped expected results therefore. The test cases are separated based on layers of the software component and data structures are associated with the separated test cases of the layers. The data structures allow the test cases of the various layers to be uncorrelated. The software component executable is employed to generate test case execution threads from the test cases and mapped expected results for a particular layer. These execution threads can then be executed in parallel, thereby testing the software component.

Claims

1. A method of testing a software component comprising multiple layers of software, the method comprising:

creating an abstraction matrix in mathematical abstract form by automatically partitioning the software component into multiple layers, the abstraction matrix comprising state and event information taking into account relationships that exist between the multiple software layers;

parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

separating the test cases based on the software layers of the software component, and associating data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

2. The method of claim 1, wherein the creating of the abstraction matrix comprises creating the abstraction matrix from a functional specification of the software component.

3. The method of claim 1, wherein the parsing, the separating, the employing and the executing comprise automated processes.

4. The method of claim 1, wherein the executing comprises executing in parallel at least some of the test case execution threads for multiple layers of the software component.

5. The method of claim 4, wherein the executing comprises executing in parallel at least some test case execution threads of each layer of the software component.

6. The method of claim 1, wherein the separating the test cases based on layers of the software component results in each layer of the software component having multiple test cases associated therewith.

7. The method of claim 1, wherein the creating comprises creating the abstraction matrix so that the abstraction matrix describes each layer of the software component.

8. The method of claim 7, wherein the creating comprises creating a separate abstraction file for each layer of the layers of the software component.

9. The system of claim 1, further comprising means for providing attributes for use in generating the test cases and mapped expected results therefor.

10. The method of claim 1, wherein the creating comprises creating the abstraction matrix to comprise state information and events that are associated with each state of the software component.

11. The method of claim 10, wherein the creating comprises creating the abstraction matrix to include current state information and next state information, and wherein for at least one current state the event information comprises an event which leads from that current state to at least one next state.

12. The method of claim 1, wherein the software component comprises an operating system component.

13. The method of claim 1, wherein the creating comprises creating the abstraction matrix to identify a minimal number of states required to model the software component.

14. The method of claim 1, wherein the associating data structures for the separated test cases of the layers comprises associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.

15. The method of claim 1, wherein the data structures for at least some software layers simulate information passed between layers during normal operation of the software component.

16. The method of claim 15, wherein the abstraction matrix comprises an abstraction file for each software layer, and wherein the parsing comprises generating a mapped expected result for each line of each abstraction file based on associated state information.

17. The method of claim 16, wherein the separating comprises eliminating dependencies between at least one input of a software layer based on other inputs of other software layers of the software component, thereby uncorrelating test cases of the various software layers.

18. The method of claim 17, wherein the software component comprises a cluster operating system component.

19. A computer-implemented method of testing a software component, the software component comprising multiple layers of software, said method comprising:

ascertaining a functional specification of the software component;

creating an abstraction matrix in mathematical abstract form that describes the software component by automatically partitioning the software component into multiple software layers using the functional specification, the abstraction matrix comprising state and event information taking into account relationships that exist between the multiple software layers;

parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

separating the test cases based on the software layers of the software component, and associating data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

20. The method of claim 19, wherein the parsing and the separating comprise automated processes.

21. The method of claim 19, wherein the creating comprises creating a separate abstraction file for each layer of the layers of the software component.

22. The method of claim 19, wherein the creating comprises creating the abstraction matrix to include current state information and next state information, and wherein for at least one current state the event information comprises an event which leads from that current state to at least one next state.

23. The method of claim 19, wherein the associating data structures for the separated test cases of the layers comprises associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.

24. A system for testing a software component comprising multiple layers of software, the system comprising:

an abstraction matrix in mathematical abstract form that describes the software component, the abstraction matrix partitioning the software component into multiple software layers and comprising state and event information taking into account relationships that exist between the multiple software layers;

means for parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

means for separating the test cases based on the software layers of the software component, and for associating data structures with the separated test cases of the software layers, which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

means for employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

means for executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

25. The system of claim 24, wherein the abstraction matrix is created from a functional specification of the software component.

26. The system of claim 24, wherein for parsing, the means for separating, the means for employing and the means for executing comprise automated means.

27. The system of claim 24, wherein the means for executing comprises means for executing in parallel at least some of the test case execution threads for multiple layers of the software component.

28. The system of claim 27, wherein the means for executing comprises means for executing in parallel at least some test case execution threads of each layer of the software component.

29. The system of claim 24, wherein the means for separating the test cases based on layers of the software component results in each layer of the software component having multiple test cases associated therewith.

30. The system of claim 24, wherein the abstraction matrix describes each layer of the software component.

31. The system of claim 30, wherein the abstraction matrix comprises a separate abstraction file for each layer of the layers of the software component.

32. The system of claim 24, further comprising means for providing attributes for use in generating the test cases and mapped expected results therefor.

33. The system of claim 24, wherein the abstraction matrix comprises state information and events that are associated with each state software component.

34. The system of claim 33, wherein the abstraction matrix includes current state information and next state information, and wherein for ate least one current state the event information comprises an event which leads from that current state to at least one next state.

35. The system of claim 24, wherein the software component comprises an operating system component.

36. The system of claim 24, wherein the abstraction matrix identifies an minimal number of states required to model the software component.

37. The system of claim 24, wherein the means for associating data structures for the separated test cases of the layers comprises means for associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.

38. A computer-implemented system for testing a software component comprising multiple layers of software, the system comprising:

an abstraction matrix in mathematical abstract form that describes the software component, the abstraction matrix partitioning the software component into multiple software layers and comprising state and event information taking into account relationships that exist between the multiple software layers;

a computer-implemented abstraction engine for parsing the abstraction matrix and automatically generating and factoring out test cases and mapped expected results therefor;

wherein the abstraction engine separates the test cases based on the software layers of the software component, and associates data structures with the separate test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

a computer-implemented functional verification test engine adapted to take the software component in executable form and generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer, wherein the functional test verification engine outputs in parallel at least some test case execution threads for at least one software layer of the software component, thereby allowing testing of the software component; and

means for executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

39. A system for testing a software component, the software component comprising multiple layers of software, said system comprising:

a functional specification of the software component;

an abstraction matrix in mathematical abstract form that describes the software component, the abstraction matrix partitioning the software component into multiple software layers and comprising state and event information taking into account relationships that exist between the multiple software layers, and having been created from the functional specification;

means for parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

means for separating the test cases based on the software layers of the software component, and for associating data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

means for employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

means for executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

40. The system of claim 39, wherein the means for parsing and the means for separating comprise automated means.

41. The system of claim 39, wherein the abstraction matrix comprises a separate abstraction file for each layer of the layers of the software component.

42. The system of claim 39, wherein the abstraction matrix includes current state information and next state information, and wherein for at least one current state the event information comprises an event which leads from that current state to at least one next state.

43. The system of claim 39, wherein the means for associating data structures for the separated test cases of the layers comprises means for associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.

44. A computer-implemented system for testing a software component comprising multiple layers of software, the system comprising:

a storage medium for storing an abstraction matrix in mathematical abstract form that describes the software component by partitioning the software component into multiple software layers, the abstraction matrix, which comprises state and event information taking into account relationships that exist between the multiple software layers, having been created from an automatic partitioning of a functional specification of the software component;

a computer-implemented abstraction engine for automatically retrieving and parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

wherein the abstraction engine separates the test cases based on the software layers of the software component and associates data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

executing in parallel at least some of the test case execution threads for at least one software layer of the software component thereby testing the software component.

45. At least one program storage device readable by machine, tangibly embodying at least one program of instructions executable by the machine to form a method for testing a software component comprising multiple layers of software, the method comprising:

storing an abstraction matrix in mathematical abstract form that describes the software component by partitioning the software component into multiple software layers, the abstraction matrix comprising state and event information taking into account relationships that exist between the multiple software layers;

parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

separating the test cases based on the software layers of the software component, and associating data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

46. The at least one program storage device of claim 45, wherein the executing comprises executing in parallel at least some of the test case execution threads for multiple layers of the software component.

47. The at least one program storage device of claim 46, wherein the executing comprises executing in parallel at least some test case execution threads of each layer of the software component.

48. The at least one program storage device of claim 45, wherein the separating the test cases based on layers of the software component results in each layer of the software component having multiple test cases associated therewith.

49. The at least one program storage device of claim 45, wherein abstraction matrix describes each layer of the software component.

50. The at least one program storage device of claim 49, wherein the abstraction matrix comprises a separate abstraction file for each layer of the layers of the software component.

51. The at least one program storage device of claim 45, wherein the abstraction matrix comprises state information and events that are associated with each state of the software component.

52. The at least one program storage device of claim 51, wherein the abstraction matrix includes current state information and next state information, and wherein for at least one current state the event information comprises an event which leads from that current state to at least one next state.

53. The at least one program storage device of claim 45, wherein the software component comprises an operating system component.

54. The at least one program storage device of claim 45, wherein the associating data structures for the separated test cases of the layers comprises associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.

55. At least one program storage device readable by a machine, tangibly embodying at least one program of instructions executable by the machine to perform a method for testing a software component comprising multiple layers of software, the method comprising:

storing an abstraction matrix of the software component in mathematical abstract form that describes the software component by partitioning the software component into multiple software layers, the abstraction matrix comprising state and event information taking into account relationships that exist between the multiple software layers;

parsing the abstraction matrix to automatically generate and factor out test cases and mapped expected results therefor;

separating the test cases based on the software layers of the software component, and associating data structures with the separated test cases of the software layers which allow the test cases of the various software layers to be uncorrelated, wherein for at least one layer, the data structures simulate inputs to the layer from at least one other layer of the multiple software layers;

employing the software component in executable form to generate for each software layer of the software component test case execution threads from the test cases and mapped expected results for that software layer; and

executing in parallel at least some of the test case execution threads for at least one software layer of the software component, thereby testing the software component.

56. The at least one program storage device of claim 55, wherein the abstraction matrix comprises a separate abstraction file for each layer of layers of the software component.

57. The at least one program storage device of claim 55, wherein the abstraction matrix includes current state information and next state information, and wherein for at least one current state the event information comprises an event which leads from that current state to the at least one next state.

58. The at least one program storage device of claim 55, wherein the associating data structures for the separated test cases of the layers comprises associating information on the relationships between layers, thus allowing test cases of the various layers to be uncorrelated.