Methods and apparatus for adaptive problem determination in distributed service-based applications

• US 20050154939A1
• Filed: 11/26/2003
• Published: 07/14/2005
• Est. Priority Date: 11/26/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Williams, Clay E., De Pauw, Wim
• Granted Patent: US 7,360,120 B2
• Time in Patent Office: Days
• US Class Current: 714/25
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

A technique for problem determination in a distributed application is provided. Testing results of the application are first obtained through execution of test cases of a test group in the application. The testing of the application is then adaptively refined when the testing results have one or more failures, to expose problems that caused the one or more failures.

Claims

1. A method of problem determination in a distributed application, comprising the steps of:

obtaining at least one testing result of the application through execution of at least one test case of a test group in the application; and

adaptively refining the testing of the application when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure.

2. The method of claim 1, further comprising the step of fixing at least one problem that caused at least one failure, when the cause of the at least one problem has been localized.

3. The method of claim 1, wherein the step of obtaining testing results comprises the steps of:

generating a test group having at least one test case, from a general model of the application;

executing the at least one test case of the test group in the application;

passing the at least one result of the test group to an outcome analyzer; and

verifying the at least one result against expected output at the outcome analyzer.

4. The method of claim 3, wherein the step of verifying the at least one result comprises the step of marking each test case of the test group with a failure or a success.

5. The method of claim 3, further comprising the step of enabling one or more probes that return one or more testing results to the outcome analyzer.

6. The method of claim 5, wherein the step of adaptively refining the testing of the application comprises the steps of:

enabling one or more additional probes that return one or more results relating to the at least one problem; and

repeating the method of problem determination in a distributed application.

7. The method of claim 5, wherein the step of adaptively refining the testing of the application comprises the steps of:

disabling one or more probes that returned one or more results that did not relate to the at least one problem; and

repeating the method of problem determination in a distributed application.

8. The method of claim 5, wherein the one or more probes collect one or more intermediary results from the application.

9. The method of claim 5, wherein the one or more probes return one or more results relating to the functioning of the application.

10. The method of claim 5, wherein the step of adaptively refining the testing of the application comprises the steps of:

enabling one or more additional probes that return one or more results relating to the at least one problem;

disabling one or more probes that returned one or more results that did not relate to the at least one problem;

adapting the test group to comprise at least one test case focused on the at least one problem; and

repeating the method of problem determination in a distributed application.

11. The method of claim 1, wherein the step of adaptively refining the testing of the application comprises the steps of:

adapting the test group to comprise at least one test case focused on the at least one problem; and

repeating the method of problem determination in a distributed application.

12. The method of claim 11, wherein the step of adapting the test group comprises the step of representing at least one action that correlates to the at least one failure in a model for generating the test group.

13. The method of claim 11, wherein the step of adapting the test group comprises the step of increasing coverage requirements for at least one state that correlates to the at least one failure in a model for generating the test group.

14. The method of claim 1, wherein the test group provides thorough coverage across the application.

15. Apparatus for problem determination in a distributed application, the apparatus comprising:

a memory; and

at least one processor, coupled to the memory, operative to;

(i) obtain at least one testing result of the application through execution of at least one test case of a test group in the application; and

(ii) adaptively refine the testing of the application when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure.

16. The apparatus of claim 15, wherein the at least one processor is further operative to fix at least one problem that caused at least one failure, when the cause of the at least one problem has been localized.

17. The apparatus of claim 15, wherein the operation of obtaining testing results comprises the operations of:

generating a test group having at least one test case, from a general model of the application;

executing the at least one test case of the test group in the application;

passing the at least one result of the test group to an outcome analyzer; and

verifying the at least one result against expected output at the outcome analyzer.

18. The apparatus of claim 17, wherein the operation of verifying the at least one result comprises the operation of marking each test case of the test group with a failure or a success.

19. The apparatus of claim 17, wherein the processor is further operative to enable one or more probes that return one or more testing results to the outcome analyzer.

20. The apparatus of claim 19, wherein the operation of adaptively refining the testing of the application comprises the operations of:

enabling one or more additional probes that return one or more results relating to the at least one problem; and

repeating the method of problem determination in a distributed application.

21. The apparatus of claim 19, wherein the operation of adaptively refining the testing of the application comprises the operations of:

disabling one or more probes that returned one or more results that did not relate to the at least one problem; and

repeating the method of problem determination in a distributed application.

22. The apparatus of claim 19, wherein the one or more probes collect one or more intermediary results from the application.

23. The apparatus of claim 19, wherein the one or more probes return one or more results relating to the functioning of the application.

24. The apparatus of claim 19, wherein the operation of adaptively refining the testing of the application comprises the operations of:

enabling one or more additional probes that return one or more results relating to the at least one problem;

disabling one or more probes that returned one or more results that did not relate to the at least one problem;

adapting the test group to comprise at least one test case focused on the at least one problem; and

repeating the method of problem determination in a distributed application.

25. The apparatus of claim 15, wherein the operation of adaptively refining the testing of the application comprises the operations of:

adapting the test group to comprise at least one test case focused on the at least one problem; and

repeating the method of problem determination in a distributed application.

26. The apparatus of claim 25, wherein the operation of adapting the test group comprises the operation of representing at least one action that correlates to the at least one failure in a model for generating the test group.

27. The apparatus of claim 25, wherein the operation of adapting the test group comprises the operation of increasing coverage requirements for at least one state that correlates to the at least one failure in a model for generating the test group.

28. The apparatus of claim 15, wherein the test group provides thorough coverage across the application.

29. An article of manufacture for problem determination in a distributed application, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

obtaining at least one testing result of the application through execution of at least one test case of a test group in the application; and

adaptively refining the testing of the application when the at least one testing result comprises at least one failure, to expose at least one problem that caused at the least one failure.