Automated test case verification that is loosely coupled with respect to automated test case execution

• US 7,823,132 B2
• Filed: 09/29/2004
• Issued: 10/26/2010
• Est. Priority Date: 09/29/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hunter, Michael J., Gallacher, Michael D., Ulrich, Adam M.
• Primary Examiner(s): Yigdall; Michael J
• Assistant Examiner(s): BODDEN, EVRAL E
• Application Number:
  US10/953,437
• Publication Number:
  US 20060075303A1
• Time in Patent Office: 2,218 Days
• Field of Search: None
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3672 Test management

Abstract

A system and method for verifying the systemic results of an action applied to an application and for providing the expected state of the application at any time or on demand, in which a verification manager determines an expected application state and a current application state of the application, a test case in communication with the verification manager executes the action, and the verification manager compares the expected application state and the current application state.

Claims

1. A system for verifying a plurality of results of an action applied to an application according to a test case, the system comprising:

a plurality of expected state generators operating independent of one another and independent of the test case for calculating a plurality of expected results of applying the action to the application and for updating a plurality of expected application states; and

a verification manager for maintaining a plurality of current application states resulting from the applied action and comparing the plurality of expected application states with the plurality of current application states, wherein verification of the plurality of expected application states is separated from the test case.

2. The system of claim 1, wherein the plurality of expected state generators determines the plurality of expected application states prior to execution of the action.

3. The system of claim 1, wherein the plurality of expected state generators determines the plurality of expected application states on demand.

4. The system of claim 1, further comprising a plurality of expected state generators for communicating to the verification manager a plurality of expected component states for a plurality of components of the application.

5. The system of claim 4, wherein the plurality of expected state generators are external to the verification manager.

6. The system of claim 4, further comprising at least one component database in communication with the plurality of expected state generators, wherein the at least one component database facilitates the plurality of expected state generators in determining the plurality of expected component states of the plurality of components.

7. The system of claim 1, further comprising a test case in communication with and independent of the verification manager for executing the action.

8. The system of claim 1, wherein the action comprises:

a stimulus; and

a parameter.

9. The system of claim 1, wherein the action is at least one of a functional test and an integration test.

10. The system of claim 1, further comprising an expected application state data structure and a current application state data structure.

11. The system of claim 10, wherein the expected application state data structure comprises information received from the plurality of expected state generators.

12. The system of claim 10, wherein the current application state data structure comprises information received from the verification manager.

13. The system of claim 1, wherein the verification manager compares the plurality of expected application states with the plurality of current application states offline.

14. The system of claim 1, wherein the verification manager compares the plurality of expected application states with the plurality of current application states online.

15. The system of claim 1, wherein each of the plurality of expected state generators are loaded from at least one of a database and a network location.

16. The system of claim 1, wherein the verification manager provides a notification of the comparison of the plurality of expected application states with the plurality of current application states.

17. The system of claim 16, wherein the notification is completed offline.

18. A computer-readable storage medium having stored thereon a data structure, comprising:

a first data field stored in a verification manager containing data representing a plurality of expected application states of an application based on an action to be implemented on the application; and

a second data field stored in the verification manager containing data representing a plurality of current application states of the application resulting from applying the action,

wherein the action is defined and applied by a test case that is independent of the verification manager and the plurality of expected application states are calculated by a plurality of expected state generators independent of one another and independent of the test case such that verification of the plurality of expected application states is separated from the test case.

19. The computer-readable medium of claim 18, wherein the plurality of expected application states and plurality of current application states pertain to states of components of the application.

20. A method for verifying results of an action applied to an application according to a test case, comprising:

saving a plurality of current application states of the application;

calculating a plurality of expected application states from an action to be applied to the application according to a test case;

saving the plurality of expected application states;

executing the action;

updating the plurality of current application states of the application resulting from executing the action; and

comparing the plurality of expected application states of the application and the plurality of current application states of the application,

wherein the plurality of expected application states are calculated by a plurality of expected state generators operating independent of one another and independent of the test case and wherein verification of the plurality of expected application states is separated from the test case.

21. The method of claim 20, further comprising creating a copy of the plurality of initial current states.

22. The method of claim 20, further comprising receiving a plurality of expected component states for a plurality of components of the application.

23. The method of claim 22, wherein the plurality of expected component states are stored in an expected application state data structure.

24. The method of claim 22, wherein the plurality of current application states are determined by a verification manager.

25. The method of claim 24, wherein the plurality of expected state generators are located external to the verification manager.

26. The method of claim 20, further comprising:

notifying the test case if the plurality of expected application states and plurality of current application states are not substantially equal.

27. The method of claim 20, wherein the comparing step is completed using XML.

28. The method of claim 20, wherein the step of saving a plurality of current application states of the application is completed after the action is applied.

29. The method of claim 20, wherein the step of saving the plurality of expected application states is completed before the action is executed.

30. A computer-readable storage medium having computer-executable instructions for performing steps comprising:

calculating a plurality of expected application states from an action to be applied to the application according to a test case;

executing the action;

determining a plurality of current application states of the application resulting from executing the action; and

comparing the plurality of expected application states of the application to the plurality of current application state of the application,

wherein the plurality of expected application states are calculated by a plurality of expected state generators operating independent of one another and independent of the test case and wherein verification of the plurality of expected application states is separated from the test case.

31. The computer-readable medium of claim 30, having further computer-executable instructions for performing the step of receiving a plurality of expected component states for a plurality of components of the application.

32. The computer-readable medium of claim 31, having further computer-executable instructions for performing the step of storing the plurality of expected component states in an expected application state data structure.

33. The computer-readable medium of claim 31, having further computer-executable instructions for performing the step of providing notification if the plurality of expected application states are not substantially equal to the plurality of current application states.