Test engine and method for verifying conformance for server applications
First Claim
Patent Images
1. A method to verify standards conformance for a server application, comprising the steps of:
- providing configuration information and a sequence of test commands to the server application with response requirements for expected responses associated with each of the commands;
issuing at least one of the commands from the sequence to the server application;
receiving at least one response from the server application;
permitting non-deterministic ordering of responses when a plurality of responses are received; and
comparing the at least one response with expected responses for performing compliance testing based on the response requirements.
1 Assignment
0 Petitions
Accused Products
Abstract
A method to verify standards conformance for a server application, in accordance with the present invention includes the steps of providing configuration information and a sequence of test commands to the server application with response requirements for expected responses associated with each of the commands, issuing at least one of the commands from the sequence to the server application, receiving at least one response from the server application and comparing the at least one response with expected responses for performing compliance testing based on the response requirements. Systems for conformance testing are also included.
-
Citations
36 Claims
-
1. A method to verify standards conformance for a server application, comprising the steps of:
-
providing configuration information and a sequence of test commands to the server application with response requirements for expected responses associated with each of the commands;
issuing at least one of the commands from the sequence to the server application;
receiving at least one response from the server application;
permitting non-deterministic ordering of responses when a plurality of responses are received; and
comparing the at least one response with expected responses for performing compliance testing based on the response requirements. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
permitting ordering of components within each response.
-
-
3. The method as recited in claim 1, further comprising the step of performing a stress test with stopping criteria to test durability of the server application.
-
4. The method as recited in claim 1, further comprising the step of specifying responses associated with the at least one command to be saved and used for subsequent commands and comparisons to the expected responses.
-
5. The method as recited in claim 1, further comprising the step of grouping commands such that a failure of compliance of any command in the group results in exiting the group and continuing in subsequent commands outside the group.
-
6. The method as recited in claim 5, wherein the grouping is hierarchical.
-
7. The method as recited claim 1, further comprising the step of generating a test sequence from a state machine.
-
8. The method as recited in claim 7, wherein the test sequence is based on probabilistic transitions.
-
9. The method as recited in claim 1, wherein the expected responses are marked as one of a mandatory response, an optional response, a forbidden response, several mandatory responses and several optional responses for permitting testing of types of responses, types of test data and implementations of the server application.
-
10. The method as recited in claim 1, wherein the expected responses include pattern symbols.
-
11. The method as recited in claim 1, further comprising the step of performing a scalability test.
-
12. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for verifying standards conformance for a server application, the method steps comprising:
-
providing configuration information and a sequence of test commands from the server application with response requirements for expected responses associated with each of the commands;
issuing at least one of the commands from the sequence to the server application;
receiving at least one response from the server application;
permitting non-detenninistic ordering of responses when a plurality of responses are received; and
comparing the at least one response with expected responses for performing compliance testing based on the response requirements. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21, 22)
permitting ordering of components within each response.
-
-
14. The program storage device as recited in claim 12, further comprising the step of performing a stress test with stopping criteria to test durability of the server application.
-
15. The program storage device as recited in claim 12, further comprising the step of specifying responses associated with the at least one command to be saved and used for subsequent commands and comparisons to the expected responses.
-
16. The program storage device as recited in claim 12, further comprising the step of grouping commands such that a failure of compliance of any command in the group results in exiting the group and continuing in subsequent commands outside the group.
-
17. The program storage device as recited in claim 16, wherein the grouping is hierarchical.
-
18. The program storage device as recited claim 12, further comprising the step of generating a test sequence from a state machine.
-
19. The program storage device as recited in claim 18, wherein the test sequence is based on probabilistic transitions.
-
20. The program storage device as recited in claim 12, wherein the expected responses are marked as one of a mandatory response, an optional response, a forbidden response, several mandatory responses and several optional responses for permitting testing of types of responses, types of test data and implementations of the server application.
-
21. The program storage device as recited in claim 12, wherein the expected responses include pattern symbols.
-
22. The program storage device as recited in claim 12, further comprising the step of performing a scalability test.
-
23. A test engine for verifying standard conformance for server applications comprising:
-
a test processor and a protocol component for communicating with a server having the server application;
the test processor including;
a parser which parses and checks test instructions input to the test processor, the parser converting parsed test instructions into objects; and
an execution engine which executes the objects output from the parser and analyzes the results of the execution of the instructions such that responses of the server application upon execution of the instructions are compared to expected responses included in the test instructions to determine conformance;
the execution engine including a response analyzer which analyzes the responses and provides for non-deterministically ordering of the responses. - View Dependent Claims (24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34)
-
-
35. A method to verify standards conformance for a server application, comprising the steps of:
-
providing configuration information and a sequence of test commands to the server application with response requirements for expected responses associated with each of the commands;
issuing at least one of the commands from the sequence to the server application;
receiving at least one response from the server application;
comparing the at least one response with expected responses for performing compliance testing based on the response requirements; and
grouping commands such that a failure of compliance of any command in the group results in exiting the group and continuing in subsequent commands outside the group. - View Dependent Claims (36)
-
Specification