Method and apparatus for validating system properties exhibited in execution traces

• US 7,930,597 B2
• Filed: 09/18/2008
• Issued: 04/19/2011
• Est. Priority Date: 09/18/2008
• Status: Expired due to Fees

• Current Assignee: Alcatel-Lucent USA, Inc. (Nokia Corporation)
• Original Assignee: Alcatel-Lucent USA, Inc. (Nokia Corporation)
• Inventors: Chang, Fangzhe, Wood, Thomas L., Ren, Yangsong
• Primary Examiner(s): Chu; Gabriel L
• Application Number:
  US12/212,758
• Publication Number:
  US 20100070805A1
• Time in Patent Office: 943 Days
• Field of Search: 714/39
• US Class Current: 714/39
• CPC Class Codes: G06F 11/3636 by tracing the execution of the program
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

The invention includes a method and apparatus for validating system properties exhibited in execution traces. In one embodiment, a method for testing a system under test (SUT) includes determining a system testing result for the SUT using at least one structured term generated by monitoring an execution trace of the SUT using at least one parameterized pattern. A test procedure is executed for the SUT. The test procedure has at least one parameterized pattern associated therewith. An execution trace generated during execution of the test procedure is parsed, where the execution trace includes unstructured information and the execution trace is parsed using the at least one parameterized pattern to identify at least one matching pattern. A system testing result for the SUT is determined using at least one structured term that is generated using the at least one matching pattern. In this manner, behavior of the system under test during the test procedure may be validated.

Claims

1. A method for testing a system under test, comprising:

executing a test procedure for the system under test, wherein the test procedure has at least one parameterized pattern associated therewith;

monitoring an execution trace generated during execution of the test procedure, wherein the execution trace comprises unstructured information, wherein the execution trace is monitored using the at least one parameterized pattern to identify at least one matching pattern;

generating at least one structured term using the at least one matching pattern; and

determining a system testing result for the system under test using the at least one structured term.

2. The method of claim 1, wherein each of the at least one parameterized pattern is defined using a pattern template and at least one pattern parameter.

3. The method of claim 2, wherein, for each of the at least one parameterized pattern, the pattern template defines an expected data format and each of the at least one pattern parameter defines an expected data type.

4. The method of claim 1, wherein monitoring the execution trace comprises:

monitoring the execution trace on at least one interface of the system under test, wherein the at least one interface comprises at least one of an input interface, an output interface, a log file, a database, a terminal, a secure shell, and a command-line-interface.

5. The method of claim 1, wherein monitoring the execution trace comprises:

parsing the execution trace after completion of execution of the test procedure on the system under test and transfer of the execution trace from the system under test to another system.

6. The method of claim 1, wherein monitoring the execution trace using the at least one parameterized pattern comprises:

parsing the unstructured information of the execution trace to identify information matching at least one pattern template of the at least one parameterized pattern.

7. The method of claim 1, wherein the execution trace is monitored using at least one of a static pattern interval and a dynamic pattern interval.

8. The method of claim 7, wherein:

a static pattern interval specifies a beginning boundary condition (B) and an ending boundary condition (E) between which a pattern (P) is expected to occur; and

a dynamic pattern interval indicates that eventually there is an interval, beginning with a beginning boundary condition (B) and ending with an ending boundary condition (E), during which a pattern (P) holds.

9. The method of claim 1, wherein the execution trace is monitored using at least one of a tight sequence operator and a loose sequence operator.

10. The method of claim 9, wherein:

a tight sequence operator is adapted for identifying a sequence of data in which two patterns must be adjacent to each other; and

a loose sequence operator is adapted for identifying a sequence of data in which a first pattern must occur before a second pattern.

11. The method of claim 1, wherein generating a structured term using a matching pattern identified from a parameterized pattern comprises:

identifying, for each pattern parameter defined by the parameterized pattern, a corresponding data value in the unstructured information of the execution trace;

extracting each identified data value from the execution trace; and

organizing the extracted data values using a structured format.

12. The method of claim 11, further comprising:

determining a location within the execution trace at which the matching pattern is identified; and

associating the location of the matching pattern with the structured term.

13. The method of claim 1, wherein determining a system test result using the at least one structured term comprises:

initiating at least one additional test procedure on the system under test using the at least one structured term.

14. The method of claim 1, wherein determining a system test result using the at least one structured term comprises:

parsing the unstructured information of the execution trace using the at least one structured term to identify thereby a portion of the unstructured information of the execution trace.

15. The method of claim 14, wherein parsing the unstructured information of the execution trace using the at least one structured term to identify thereby a portion of the unstructured information of the execution trace comprises:

parsing the execution trace to identify a beginning boundary condition of the parameterized pattern and an ending boundary condition of the parameterized pattern.

16. The method of claim 14, further comprising:

determining whether the identified portion of the unstructured information of the execution trace satisfies at least one condition.

17. The method of claim 16, wherein determining whether the identified portion of the unstructured information of the execution trace satisfies at least one condition comprises:

parsing the identified portion of the unstructured information of the execution trace using at least one other parameterized pattern.

18. The method of claim 1, further comprising at least one of:

storing the at least one structured term in a structured terms library; and

storing the system testing result.

19. An apparatus for testing a system under test, comprising:

means for executing a test procedure for the system under test, wherein the test procedure has at least one parameterized pattern associated therewith;

means for monitoring an execution trace generated during execution of the test procedure, wherein the execution trace comprises unstructured information, wherein the execution trace is monitored using the at least one parameterized pattern to identify at least one matching pattern;

means for generating at least one structured term using the at least one matching pattern; and

means for determining a system testing result for the system under test using the at least one structured term.

20. A computer-readable medium storing a software program which, when executed by a computer, causes the computer to perform a method for testing a system under test, the method comprising:

executing a test procedure for the system under test, wherein the test procedure has at least one parameterized pattern associated therewith;

monitor an execution trace generated during execution of the test procedure, wherein the execution trace comprises unstructured information, wherein the execution trace is monitored using the at least one parameterized pattern to identify at least one matching pattern;

generating at least one structured term using the at least one matching pattern; and

determining a system testing result for the system under test using the at least one structured term.

21. A method for testing a system under test, comprising:

parsing an execution trace generated during execution of a test procedure, wherein the test procedure has at least one parameterized pattern associated therewith, wherein the execution trace comprises unstructured information, wherein the execution trace is parsed using the at least one parameterized pattern to identify at least one matching pattern;

generating at least one structured term using the at least one matching pattern; and

determining a system testing result for the system under test using the at least one structured term.