Method and framework for test case management

• US 7,797,680 B2
• Filed: 06/17/2004
• Issued: 09/14/2010
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Haas, Rolf W.
• Primary Examiner(s): Bullock, Jr.; Lewis A
• Assistant Examiner(s): WANG, JUE S
• Application Number:
  US10/868,896
• Publication Number:
  US 20050283761A1
• Time in Patent Office: 2,280 Days
• Field of Search: 717/124
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

In accordance with an embodiment of the present invention, a method may include obtaining a list of active life cycle test objects in a test run from a life cycle test object controller, and obtaining a list of active central activity test objects in the test run from a central activity test object controller. While a test period remains in the test run, the method may continue selecting a next test period, requesting the test step initialization controller initialize the next test period, requesting all central activity test objects associated with the next test period to execute their beginning central activities, requesting all life cycle test objects associated with the next test period to execute their test activities, and requesting all central activity test objects associated with the next test period to execute their ending central activities.

Claims

1. A computer-implemented method for testing software applications using a plurality of test cases representing different practical scenarios, comprising:

reading, by the computer, object data defining a plurality of test cases, each test case identifying unique object(s) corresponding to the test case, each test case further identifying central test activities to be performed by a test program upon all objects of the test cases and individual test activities to be performed by the test program upon the objects corresponding to the respective test case, execution of each individual test activity within a specific test case being independent of execution of individual test activities of any other test cases so that an addition of a new test case does not affect any existing test cases;

scheduling, by the computer, execution of the central test activities and the individual test activities by;

scheduling execution of central test activities of a first set of test cases;

scheduling execution of the individual test activities of the first set of test cases according to their positions with respect to the central test activities within the respective the first set of test cases, wherein execution of an individual test activity of a first test case within the first set of test cases occurs concurrently with execution of another individual test activity of a second test case within the first set of test cases;

scheduling execution of central test activities of a second set of test cases;

scheduling execution of the individual test activities of the second set of test cases according to their positions with respect to the central test activities within the respective the second set of test cases, wherein execution of an individual test activity of a first test case within the second set of test cases occurs concurrently with execution of another individual test activity of a second test case within the second set of test cases; and

executing the central test activities and the individual test activities of the first set of test cases as scheduled; and

executing the central test activities and the individual test activities of the second set of test cases as scheduled,

wherein each central test activity is only executed once at a synchronization point for all of concurrently executing test cases.

2. The method of claim 1 wherein the reading the object data comprises:

obtaining a listing of all of the plurality of independent test cases; and

obtaining a listing of all beginning and ending central test activities.

3. The method of claim 2 wherein the obtaining the listing of all of the plurality of independent test cases comprises:

obtaining a listing of all life cycle test objects for each of the plurality of independent test cases.

4. The method of claim 3 wherein the obtaining the listing of all life cycle test objects for each of the plurality of independent test cases comprises:

obtaining a listing of actions associated with each of the life cycle test objects for each test period for each of the plurality of independent test cases.

5. The method of claim 4 wherein at least one of the test periods has actions associated with at least one life cycle test object.

6. The method of claim 5 wherein the reading the object data comprises:

determining a first test period with the beginning central test activity.

7. The method of claim 5 wherein the reading the object data comprises:

determining a first test period with an activity associated with the at least one of the plurality of independent test cases.

8. The method of claim 2 wherein the obtaining the listing of all beginning and ending central test activities comprises:

obtaining a listing of all beginning central test activities associated with each test period; and

obtaining a listing of all ending central test activities associated with each test period.

9. The method of claim 8 wherein at least one test period is associated with beginning central test activities.

10. The method of claim 8 wherein at least one test period is associated with ending test activities.

11. The method of claim 1 wherein the executing the beginning comprises:

determining execution of all other test activities has stopped; and

executing the beginning central test activity.

12. The method of claim 1 wherein the executing individual test activities for at least one of a plurality of independent test cases associated with the first test period comprises:

determining execution of the beginning one of the plurality of central test activities has stopped; and

executing the individual test activities for the at least one plurality of independent test cases associated with the first test period.

13. The method of claim 1 wherein the executing an ending one of the plurality of central test activities comprises:

determining execution of all test activities associated with the plurality of independent test cases has stopped; and

executing the ending central test activity.

14. The method of claim 1 further comprising:

storing results of all of the execution from all test periods,

wherein the results are stored in a single data object.

15. The method of claim 14 wherein the storing the results in a single data object comprises:

storing error messages in the single data object; and

storing results from correctly executing test activities.

16. The method of claim 1 wherein a first test period represents a first specific time period within each of the test cases.

17. The method of claim 1 wherein the independent test cases run in parallel.

18. The method of claim 1 wherein the central test activity exerts an influence on all of the test cases.

19. The method of claim 1 wherein a number count of the individual test activities associated with the first test case within a same set of test cases is different from a number count of the individual test activities associated with the second test case within the same set of test cases.

20. The method of claim 1 wherein the individual test activities associated with the first test case within a same set of test cases are same as the individual test activities associated with the second test case within the same set of test cases.

21. The method of claim 1 wherein the individual test activities associated with the first test case within a same set of test cases are different from the individual test activities associated with the second test case within the same set of test cases.

22. The method of claim 1 wherein a first time duration starts according to business days.

23. The method of claim 1 wherein a first time duration starts monthly.

24. The method of claim 1 wherein the beginning one of the central test activities is executed according to a predetermined schedule.

25. The computer-implemented method of claim 1, wherein the testing is executed through a plurality of synchronization points, and wherein each of the central test activities is executed at a synchronization point.

26. A non-transitory machine-readable medium having stored thereon a plurality of executable instructions to perform a method for testing software applications using a plurality of test cases representing different practice scenarios, comprising:

reading object data defining a plurality of test cases, each test case identifying unique object(s) corresponding to the test case, each test case further identifying central test activities to be performed by a test program upon all objects of the test cases and individual test activities to be performed by the test program upon the objects corresponding to the respective test case, execution of each individual test activity within a specific test case being independent of execution of individual test activities of any other test cases so that an addition of a new test case does not affect any existing test cases;

scheduling, by the computer, execution of the central test activities and the individual test activities by;

scheduling execution of central test activities of a first set of test cases;

scheduling execution of the individual test activities for the first set of test cases according to their positions with respect to the central test activities within the respective the first set of test cases, wherein execution of an individual test activity of a first test case within the first set of test cases occurs concurrently with execution of another individual test activity of a second test case within the first set of test cases;

scheduling execution of central test activities of a second set of test cases;

scheduling execution of the individual test activities of the second set of test cases according to their positions with respect to the central test activities within the respective the second set of test cases, wherein execution of an individual test activity of a first test case within the second set of test cases occurs concurrently with execution of another individual test activity of a second test case within the second set of test cases; and

executing the central test activities and the individual test activities of the first set of test cases as scheduled; and

executing the central test activities and the individual test activities of the second set of test cases as scheduled,

wherein each central test activity is only executed once at a synchronization point for all of the concurrently executing test cases.

27. The non-transitory machine-readable medium of claim 26 wherein the reading the object data comprises:

obtaining a listing of all of the plurality of independent test cases; and

obtaining a listing of all beginning and ending central test activities.

28. The non-transitory machine-readable medium of claim 27 wherein the obtaining the listing of all of the plurality of independent test cases comprises:

obtaining a listing of all life cycle test objects for each of the plurality of independent test cases.

29. The non-transitory machine-readable medium of claim 28 wherein the obtaining the listing of all life cycle test objects for each of the plurality of independent test cases comprises:

obtaining a listing of actions associated with each of the life cycle test objects for each test period for each of the plurality of independent test cases.

30. The non-transitory machine-readable medium of claim 29 wherein at least one of the test periods has actions associated with at least one life cycle test object.

31. The non-transitory machine-readable medium of claim 30 wherein at least one test period is associated with the beginning test activities.

32. The non-transitory machine-readable medium of claim 30 wherein at least one test period is associated with the ending test activities.

33. The non-transitory machine-readable medium of claim 27 wherein the obtaining the listing of all beginning and ending central test activities comprises:

obtaining a listing of all beginning central test activities associated with each test period; and

obtaining a listing of all ending central test activities associated with each test period.

34. The non-transitory machine-readable medium of claim 33 wherein the reading the object data comprises:

determining a first test period with the beginning central test activity.

35. The non-transitory machine-readable medium of claim 33 wherein the reading the object data comprises:

determining a first test period with an activity associated with the at least one of the plurality of independent test cases.

36. The non-transitory machine-readable medium of claim 26 wherein the executing the beginning central test activity comprises:

determining execution of all other test activities has stopped; and

executing the beginning central test activity.

37. The non-transitory machine-readable medium of claim 26 wherein the executing individual test activities for at least one of a plurality of independent test cases associated with the first test period comprises:

determining execution of the beginning one of the plurality of central test activities has stopped; and

executing the individual test activities for the at least one plurality of independent test cases associated with the first test period.

38. The non-transitory machine-readable medium of claim 26 wherein the executing the ending central test activity comprises:

determining execution of all test activities associated with the plurality of independent test cases has stopped; and

executing the ending central test activity.

39. The non-transitory machine-readable medium of claim 26 further comprising:

storing results of all of the execution from all test periods,

wherein the results are stored in a single data object.

40. The non-transitory machine-readable medium of claim 39 wherein the storing the results in a single data object comprises:

storing error messages in the single data object; and

storing results from correctly executing test activities.

41. A non-transitory machine-readable medium having stored thereon a plurality of executable instructions encoding a test framework, the test framework comprising:

a test run controller;

at least one life cycle test object registered to the test run controller, wherein each of the at least one life cycle test object is independent of any other, each life cycle test object including at least one life cycle test object activity to be executed during a predetermined test period;

a life cycle test object controller registered to the test run controller and the at least one life cycle test object, the life cycle test object controller to maintain a list of the at least one life cycle test object;

at least one central activity test object registered to the test run controller, each central activity test object to be executed during a test period without any life cycle test object activities being executed;

a central activity test object controller registered to the test run controller and the at least one central activity test object, the central activity test object controller to maintain a list of the at least one central activity test object;

a test period initialization controller registered to the test run controller, the test period initialization controller to read object data defining a plurality of test cases, each test case identifying unique object(s) corresponding to the test case, each test case further identifying the at least one life cycle object corresponding individual test activities and the at least one central activity test object corresponding central test activities to be performed by the test run controller upon all objects of the test cases, and to schedule execution of the central test activities and the individual test activities, wherein execution of an individual test activity of a first test case occurs concurrently with execution of another individual test activity of a second test case;

an application to be tested, the application connected to the test period initialization controller, the at least one life cycle test object and the at least one central activity test object; and

a test result controller to receive results from the test run controller, the life cycle test object controller, the central activity test object controller, the test period initialization controller, the at least one life cycle test object, and the at least one central activity test object,

wherein execution of each individual test activity within a specific test case is independent of execution of individual test activities of any other test cases so that an addition of a new test case does not affect any existing test cases, and

wherein the each central test activity is only executed once at a synchronization point for all concurrently executing test cases.

42. The non-transitory machine-readable medium of claim 41 wherein each life cycle test object controls the life cycle of an associated single application object.

43. The non-transitory machine-readable medium of claim 42 wherein execution of each life cycle test object only effects the single application object associated with the life cycle test object.

44. The non-transitory machine-readable medium of claim 41 wherein the test run controller comprises:

a master test object to control execution of a test run including a plurality of test periods.

45. The non-transitory machine-readable medium of claim 44 wherein the master test object is to instruct life cycle test objects associated with a test period to execute during the test period and to instruct central activity test objects associated with the test period to execute during the test period.

46. The non-transitory machine-readable medium of claim 45 wherein the master test object is to determine which life cycle test objects are associated with a given test period by requesting the list of the at least one life cycle test object from the life cycle test object controller.

47. The non-transitory machine-readable medium of claim 45 wherein the master test object is to determine which central activity test objects are associated with a given test period by requesting the list of the at least one central activity test object from the central activity test object controller.

48. The non-transitory machine-readable medium of claim 44 wherein the master test object is to:

obtain the list of active life cycle test objects in the test run from the life cycle test object controller;

obtain the list of active central activity test objects in the test run from the central activity test object controller; and

while a test period remains in the test run select a time duration of a next test period,

request the test step initialization controller initialize the next test period,

request all central activity test objects associated with the next test period to execute their beginning central test activities,

request all life cycle test objects associated with the next test period to execute their test activities, and

request all central activity test objects associated with the next test period to execute their ending central test activities.

49. The non-transitory machine-readable medium of claim 48 wherein the test run is to terminate upon an error occurring during execution of one of the central activity test objects.

50. The non-transitory machine-readable medium of claim 48 wherein the test run is to pause execution after a predetermined test period, receive an upgrade of the application, and resume execution of the test run using the upgraded application.

51. The non-transitory machine-readable medium of claim 48 wherein the test run controller is to determine which central activity test objects are associated with each test period from the life cycle test objects associated with the test period.

52. The non-transitory machine-readable medium of claim 48 wherein each life cycle test object is to determine which central activity test objects are associated with the test period associated with the life cycle test object.

53. A non-transitory computer-readable medium having stored thereon a plurality of executable instructions to perform a method of managing operations of a test program to test functionality of an application program, comprising:

reading, by the computer, object data defining a plurality of test cases, wherein the object data is a data structure comprising fields for;

a plurality of test cases, each test case identifying object(s) managed by the computer application to which the respective test case relates, each test case storing;

first fields for storage of identifiers representing central test activities common to all the test cases, the central test activity identifiers identifying respective test operations to be performed by the test program upon all objects of the test cases, the first fields identifying a time when the respective central test operation is to be performed during execution of the test cases;

second fields for storage of identifiers representing individual test activities unique to the respective test case, the individual test activity identifiers identifying respective test operations to be performed by the test program upon the objects of the respective test case, the second fields identifying a time when the respective individual test activity is to be performed during execution of the respective test cases,

scheduling, by the computer, execution of the central test activities and the individual test activities by;

scheduling execution of the central test activities upon the objects of all the test cases to occur simultaneously,

scheduling execution of the individual test activities of the respective test cases according to their position with respect to the central test activities within the respective test cases, wherein execution of an individual test activity of a first test case occurs concurrently with execution of another individual test activity of a second test case, wherein execution of each individual test activity within a specific test case is independent of execution of individual test activities of any other test cases so that an addition of a new test case does not affect any existing test cases, and

executing the central test activities and the individual test activities as scheduled,

wherein each central test activity is only executed once at a synchronization point for all of the concurrently executing test cases.

54. A computer-implemented method of managing operations of a test program to test functionality of an application program, comprising:

reading, by the computer, object data defining a plurality of test cases, each test case identifying unique object(s) corresponding to the test case, each test case further identifying central test activities to be performed by the test program upon all objects of the test cases and individual test activities to be performed by the test program upon the objects corresponding to the respective test case, execution of each individual test activity within a specific test case being independent of execution of individual test activities of any other test cases so that an addition of a new test case does not affect any existing test cases;

scheduling, by the computer, execution of the central test activities and the individual test activities by;

scheduling execution of the central test activities upon the objects of all the test cases to occur simultaneously,

scheduling execution of the individual test activities of the respective test cases according to their position with respect to the central test activities within the respective test cases, wherein execution of an individual test activity of a first test case occurs concurrently with execution of another individual test activity of a second test case; and

executing the central test activities and the individual test activities as scheduled,

wherein each central test activity is only executed once at a synchronization point for all of the concurrently executing test cases.

55. The computer-implemented method of claim 54, wherein the test program is executed through a plurality of synchronization points, and wherein each of the central test activities is executed at a synchronization point.

56. The computer-implemented method of claim 55, further comprising:

registering, by the computer, each of the objects with the test program to inform the test program about the central test activities at the sync points and the individual test activities of the test case identified by the object.

57. The computer-implemented method of claim 54, further comprising:

updating the object data with a new plurality of test cases.

58. The computer-implemented method of claim 57, wherein the update includes adding at least one new test case to the existing plurality of test cases.

59. The computer-implemented method of claim 54, wherein the object data is stored as a data structure, the data structure including:

the plurality of test cases, each test case identifying the object(s) to which the respective test case relates, the each test case storing;

first fields for storage of identifiers representing the central test activities, the first fields identifying synchronization points at which the central test activities are to be performed;

second fields for storage of identifiers representing the individual test activities unique to the respective test case, the first fields identifying a time during execution of the respective test case when the respective test activity is to be performed.