Automatic model-based testing

• US 7,979,849 B2
• Filed: 10/15/2004
• Issued: 07/12/2011
• Est. Priority Date: 10/15/2004
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Feldstein, Jeffrey, Bosch, Robert
• Primary Examiner(s): Dao; Thuy
• Assistant Examiner(s): Lee; Marina
• Application Number:
  US10/966,962
• Publication Number:
  US 20060085681A1
• Time in Patent Office: 2,461 Days
• Field of Search: 717/124, 717/125
• US Class Current: 717/128
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Automatic model-based testing is disclosed, including receiving a set of objects associated with a computer program application under test; automatically classifying the objects as state objects representing states of the application, or transition objects representing transitions in the application from one state to another state, or static information objects; linking the state objects with the transition objects to result in creating a model in memory of all states and transitions provided by the application; associating, in the model, one or more tests with one or more of the state objects; creating and storing, based on the states and transitions, a finite state machine that represents execution behavior of the application according to the model; and testing the application by (a) automatically navigating the finite state machine and (b) providing state and transition data from the finite state machine to a graphical user interface functional test application that executes the one or more tests upon reaching the states represented by the state objects.

Claims

1. A method, comprising the computer-implemented steps of:

receiving a set of objects associated with a computer program application under test;

prior to testing the application, automatically classifying the objects as state objects representing states of the application, or transition objects representing transitions in the application from one state to another state, or static information objects;

linking the state objects with the transition objects to result in creating a model in memory of all states and transitions provided by the application;

associating, in the model, one or more tests with one or more of the state objects;

creating and storing, based on the states and transitions, a finite state machine that represents execution behavior of the application according to the model;

selecting a set of states or transitions associated with application behavior that is unwanted for testing;

testing the application by (a) automatically navigating the states and transitions of the finite state machine which are not in the set of states or transitions that are unwanted for testing and (b) providing state and transition data from the finite state machine to a graphical user interface functional test application that executes the one or more tests upon reaching the states represented by the state objects;

wherein the set of objects are graphical user interface (GUI) objects and the automatically classifying comprises analyzing a GUI map;

wherein the GUI map stores information about elements generated by the application based at least in part on information about states of the application, transitions in the application from one state to another state, or static information objects;

wherein the method is performed by one or more computing devices.

2. A method as recited in claim 1, wherein linking comprises:

displaying a graphical representation of the state objects;

receiving user input that selects one of the state objects;

displaying transition objects associated with the selected state object;

receiving user input specifying graphically linking one or more of the transition objects associated with the selected state object to one or more other state objects.

3. A method as recited in claim 1, wherein linking comprises:

displaying, in a graphical panel of a user interface, a first state icon representing a particular state object;

displaying, in a state tree panel of the user interface, one or more transition names representing all transition objects of the application;

receiving user input in the graphic user interface that selects one of the transition names and associates the selected transition name with the first state icon and a second state object; and

linking, in the model, the first state object, the transition object corresponding to the selected name, and the second state object, based on the user input.

4. A method as recited in claim 3, further comprising receiving user input selecting addition or modification of one or more states or transitions associated with application behavior that is unwanted for purposes of testing.

5. A method as recited in claim 1, wherein associating one or more tests comprises defining one or more verification points, and associating the verification points with states of the application under test.

6. A method as recited in claim 5, further comprising automatically creating a verification point for each of the static information objects.

7. A method as recited in claim 1, wherein the computer program application under test is a hyper text mark-up language-based Web application.

8. A method as recited in claim 1, wherein the computer program application under test is an operating system application, and wherein the testing step comprises testing the application by (a) automatically navigating the finite state machine and (b) calling operating system application program interface functions using the state and transition data from the finite state machine to execute the one or more tests upon reaching the states represented by the state objects.

9. A method as recited in claim 1, further comprising creating and storing one or more data stores, wherein each of the data stores comprises a stored association of (a) a set of objects associated with of a computer program application under test, (b) a computer program application under test, and (c) computer program code for the one or more tests.

10. A method, comprising the computer-implemented steps of:

receiving a set of objects associated with a computer program application under test;

prior to testing the application, automatically classifying the objects as state objects representing states of the application, or transition objects representing transitions in the application from one state to another state, or static information objects;

creating and displaying a graphical user interface of a model-based test application, wherein the graphical user interface comprises a state information panel and a graphical workspace panel;

displaying, in the state information panel, a hierarchical representation of the state objects and the transition objects, wherein the hierarchical representation corresponds to an organization of the states and transitions of the application;

displaying, in the graphical workspace panel, state icons representing the state objects;

linking the state objects with the transition objects in response to user input in the graphical workspace panel;

creating and storing, based on the states and transitions, a finite state machine that represents execution behavior of the application according to the model;

wherein the set of objects are graphical user interface (GUI) objects and the automatically classifying comprises analyzing a GUI map;

wherein the GUI map stores information about elements generated by the application based at least in part on information about states of the application, transitions in the application from one state to another state, or static information objects;

associating, in the model, one or more tests with one or more of the state objects;

selecting a set of states or transitions associated with application behavior that is unwanted for testing;

testing the application by (a) automatically navigating the states and transitions of the finite state machine which are not in the set of states or transitions that are unwanted for testing and (b) providing state and transition data from the finite state machine to a graphical user interface functional test application that executes the one or more tests upon reaching the states represented by the state objects;

wherein the method is performed by one or more computing devices.

11. A computer-readable volatile or non-volatile, non-transitory medium comprising one or more stored sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

receiving a set of objects associated with a computer program application under test;

prior to testing the application, automatically classifying the objects as state objects representing states of the application, or transition objects representing transitions in the application from one state to another state, or static information objects;

linking the state objects with the transition objects to result in creating a model in memory of all states and transitions provided by the application;

associating, in the model, one or more tests with one or more of the state objects;

creating and storing, based on the states and transitions, a finite state machine that represents execution behavior of the application according to the model;

selecting a set of states or transitions associated with application behavior that is unwanted for testing;

testing the application by (a) automatically navigating the states and transitions of the finite state machine which are not in the set of states or transitions that are unwanted for testing and (b) providing state and transition data from the finite state machine to a graphical user interface functional test application that executes the one or more tests upon reaching the states represented by the state objects;

wherein the set of objects are graphical user interface (GUI) objects and the automatically classifying comprises analyzing a GUI map;

wherein the GUI map stores information about elements generated by the application based at least in part on information about states of the application, transitions in the application from one state to another state, or static information objects;

wherein the method is performed by one or more computing devices.

12. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, wherein the instructions for performing linking comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

displaying a graphical representation of the state objects;

receiving user input that selects one of the state objects;

displaying transition objects associated with the selected state object;

receiving user input specifying graphically linking one or more of the transition objects associated with the selected state object to one or more other state objects.

13. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, wherein the instructions for performing linking comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

displaying, in a graphical panel of a user interface, a first state icon representing a particular state object;

displaying, in a state tree panel of the user interface, one or more transition names representing all transition objects of the application;

receiving user input in the graphic user interface that selects one of the transition names and associates the selected transition name with the first state icon and a second state object; and

linking, in the model, the first state object, the transition object corresponding to the selected name, and the second state object, based on the user input.

14. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 13, further comprising instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of receiving user input selecting addition or modification of one or more states or transitions associated with application behavior that is unwanted for purposes of testing.

15. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, wherein the instructions for associating one or more tests comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of defining one or more verification points, and associating the verification points with states of the application under test.

16. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 15, further comprising instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of automatically creating a verification point for each of the static information objects.

17. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, wherein the computer program application under test is a hyper text mark-up language-based Web application.

18. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, wherein the computer program application under test is a an operating system application, and wherein the instructions for testing further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of testing the application by (a) automatically navigating the finite state machine and (b) calling operating system application program interface functions using the state and transition data from the finite state machine to execute the one or more tests upon reaching the states represented by the state objects.

19. A computer-readable volatile or non-volatile, non-transitory medium as recited in claim 11, further comprising instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of creating and storing one or more data stores, wherein each of the data stores comprises a stored association of (a) a set of objects associated with of a computer program application under test, (b) a computer program application under test, and (c) computer program code for the one or more tests.

20. An apparatus, comprising:

one or more processors;

means for receiving a set of objects associated with a computer

program application under test;

means for automatically classifying, prior to testing the application, the objects as state objects representing states of the application, or transition objects representing transitions in the application from one state to another state, or static information objects;

means for linking the state objects with the transition objects to result in creating a model in memory of all states and transitions provided by the application;

means for associating, in the model, one or more tests with one or more of the state objects;

means for creating and storing, based on the states and transitions, a finite state machine that represents execution behavior of the application according to the model;

means for selecting a set of states or transitions associated with application behavior that is unwanted for testing;

means for testing the application by (a) automatically navigating the states and transitions of the finite state machine which are not in the set of states or transitions that are unwanted for testing and (b) providing state and transition data from the finite state machine to a graphical user interface functional test application that executes the one or more tests upon reaching the states represented by the state objects;

wherein the set of objects are graphical user interface (GUI) objects and the automatically classifying comprises analyzing a GUI map;

wherein the GUI map stores information about elements generated by the application based at least in part on information about states of the application, transitions in the application from one state to another state, or static information objects.

21. An apparatus as recited in claim 20, wherein the means for linking comprises:

means for displaying a graphical representation of the state objects;

means for receiving user input that selects one of the state objects;

means for displaying transition objects associated with the selected state object;

means for receiving user input specifying graphically linking one or more of the transition objects associated with the selected state object to one or more other state objects.

22. An apparatus as recited in claim 20, wherein the means for linking comprises:

means for displaying, in a graphical panel of a user interface, a first state icon representing a particular state object;

means for displaying, in a state tree panel of the user interface, one or more transition names representing all transition objects of the application;

means for receiving user input in the graphic user interface that selects one of the transition names and associates the selected transition name with the first state icon and a second state object; and

means for linking, in the model, the first state object, the transition object corresponding to the selected name, and the second state object, based on the user input.

23. An apparatus as recited in claim 22, further comprising means for receiving user input selecting addition or modification of one or more states or transitions associated with application behavior that is unwanted for purposes of testing.

24. An apparatus as recited in claim 20, wherein the means for associating one or more tests comprises means for defining one or more verification points, and means for associating the verification points with states of the application under test.

25. An apparatus as recited in claim 24, further comprising means for automatically creating a verification point for each of the static information objects.

26. An apparatus as recited in claim 20, wherein the computer program application under test is a hyper text mark-up language-based Web application.

27. An apparatus as recited in claim 20, wherein the computer program application under test is an operating system application, and wherein the means for testing comprises means for testing the application by (a) automatically navigating the finite state machine and (b) calling operating system application program interface functions using the state and transition data from the finite state machine to execute the one or more tests upon reaching the states represented by the state objects.

28. An apparatus as recited in claim 20, further comprising means for creating and storing one or more data stores, wherein each of the data stores comprises a stored association of (a) a set of objects associated with of a computer program application under test, (b) a computer program application under test, and (c) computer program code for the one or more tests.