Data driven test automation of web sites and web services

• US 7,334,220 B2
• Filed: 03/11/2004
• Issued: 02/19/2008
• Est. Priority Date: 03/11/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chang, Ivy A., Sannakkayala, Sudhakar S.
• Primary Examiner(s): Dam; Tuan
• Assistant Examiner(s): RUTTEN, JAMES D
• Application Number:
  US10/798,658
• Publication Number:
  US 20050267976A1
• Time in Patent Office: 1,440 Days
• Field of Search: 717/131, 717/126
• US Class Current: 717/126
• CPC Class Codes: H04L 63/0815 providing single-sign-on or federations
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network

Abstract

A system, method and data structure for testing a web location including a web site or web service. A test generator or user generates an XML test case and a driver interprets the XML test case into an http request to be sent to the web location as a test.

Claims

1. A system for testing a web location including a web site or web service comprising a processor executing:

a test generator for generating an XML test case; and

a driver for interpreting the XML test case into an http request to be sent to the web location, said driver comprising;

a parsing engine for parsing the XML test case into a sequence of requests/response pairs; and

a runtime engine for providing to the web location the http request corresponding to the sequence of requests/response pairs wherein the runtime engine receives requests/response pairs from the parsing engine, and includes instructions for identifying dynamic data in the request/response pairs received from the parsing engine;

instructions for evaluating the identified dynamic data for a result;

instructions for replacing the evaluated result to generate a replaced request, wherein a tag value of the identified dynamic data is randomly replaced with at least one or more of the possible values listed under a node of an XML data file corresponding to the identified dynamic data.

2. The system of claim 1 wherein said runtime engine receives from the web location an http response corresponding to the http request and retrieves, validates, and stores data in the http response, said retrieved, validated, and stored data being indicative of an operational status of the web location.

3. The system of claim 2 wherein the parsing engine comprises:

instructions for identifying components in the XML test case wherein the XML test case is received from the test generator;

instructions for identifying configuration data of the components; and

instructions for replacing the identified configuration data in the request to be sent to the web location to be tested.

4. The system of claim 2 wherein the parsing engine comprises:

instructions for identifying any sequence of request/response pairs in the XML test case received from the test generator or user input;

instructions for determining if each identified request is a simple request or a composite of other requests and for reducing a composite request to a plurality of simple requests;

for each simple request/response pair, instructions for identifying a data structure of the simple request/response pair;

instructions for identifying aspects of the request data structure which include configuration data; and

instructions for replacing the identified configuration data in the request to be sent to the web location to be tested and outputting the resulting sequence of request/response pairs.

5. The system of claim 4 wherein the data structure includes the elements of an http request including the following:

a URL, a path, query string parameters, post data, and a verb.

6. The system of claim 4 wherein the runtime engine comprises:

instructions for identifying dynamic data in the request/response pairs received from the parsing engine;

instructions for replacing the identified dynamic data to generate a replaced request;

instructions for creating an http request from the data in the replaced request;

instructions for instantiating an http engine; and

instructions for sending the http request to the web location.

7. The system of claim 6 further comprising:

instructions for receiving from the web location an http response to the corresponding http request;

instructions for identifying the location and data to be retrieved from the http response regarding the operability of the web location; and

instructions for retrieving, validating, and storing the identified location and data.

8. The system of claim 1 wherein the runtime engine further comprises:

instructions for creating an http request from the data in the replaced request;

instructions for instantiating an http engine; and

instructions for sending the http request to the web location.

9. The system of claim 8 further comprising:

instructions for receiving from the web location an http response to the corresponding http request;

instructions for identifying the location and data to be retrieved from the http response regarding the operability of the web location; and

instructions for retrieving, validating, and storing the identified location and data.

10. The system of claim 1 wherein the test case includes at least one of the following tags:

dynamic user names, parameters specific to the test case and retrieved from a configuration file corresponding to the web location, and optional tags.

11. The system of claim 1 further comprising instructions for at least one of the following:

handling cookies, manipulating data, supporting XML parameters, sending hidden fields, inserting random parameters, and using use case attributes.

12. The system of claim 1 wherein the XML test case has a data structure comprising elements of an http request including the following:

a URL, a path, query string parameters, post data, and a verb.

13. A method of testing a web location including a web site or web service comprising:

generating a test case in XML format;

interpreting the XML test case into a http request to be sent to the web location;

parsing the XML test case into a sequence of request/response pairs;

identifying dynamic data in the request/response pairs;

evaluating the identified dynamic data for a result;

replacing the evaluated result to generate a replaced request, wherein a tag value of the identified dynamic data is randomly replaced with at least one or more of the possible values listed under a node of an XML data file corresponding to the identified dynamic data; and

providing to the web location the http request corresponding to the sequence of request/response pairs.

14. The method of claim 13 wherein the parsing comprises:

identifying components in the XML test case received from the test generator;

identifying configuration data of the components; and

replacing the identified configuration data in the request to be sent to the web location to be tested.

15. The method of claim 13 wherein the parsing comprises:

identifying any sequence of request/response pairs in the XML test case received from the test generator or user input;

determining if each identified request is a simple request or a composite of other requests and for reducing a composite request to a plurality of simple requests;

for each simple request/response pair, identifying a data structure of the simple request/response pair;

identifying aspects of the request data structure which include configuration data; and

replacing the identified configuration data in the request to be sent to the web location to be tested and outputting the resulting sequence of request/response pairs.

16. The method of claim 15 wherein the data structure includes the elements of an http request including the following:

a URL, a path, query string parameters, post data, and a verb.

17. The method of claim 16 wherein providing to the web location the http request comprises:

identifying dynamic data in the request/response pairs received from the parsing engine;

replacing the identified dynamic data to generate a replaced request;

creating an http request from the data in the replaced request;

instantiating an http engine; and

sending the http request to the web location.

18. The method of claim 17 further comprising:

receiving from the web location an http response to the corresponding http request;

identifying the location and data to be retrieved from the http response regarding the operability of the web location; and

retrieving, validating, and storing the identified location and data.

19. The method of claim 13 wherein providing to the web location the http request comprises:

creating an http request from the data in the replaced request;

instantiating an http engine; and

sending the http request to the web location.

20. The method of claim 19 further comprising:

receiving from the web location an http response to the corresponding http request;

identifying the location and data to be retrieved from the http response regarding the operability of the web location; and

retrieving, validating, and storing the identified location and data.

21. The method of claim 13 further comprising receiving from the web location an http response corresponding to the http request and retrieving, validating and storing data from the http response wherein said retrieved, validated and stored data is indicative of an operational status of the web location.

22. The method of claim 13 wherein the test case includes at least one of the following tags:

dynamic user names, parameters specific to the test case and retrieved from a configuration file corresponding to the test case, and optional tags.

23. The method of claim 13 further comprising at least of the following in conjunction with the generating of the XML test case:

handling cookies, manipulating data, supporting XML parameter, sending hidden fields, inserting random parameters, and using use case attributes.

24. A method of testing a plurality of web locations connected by a multi-site user authentication system according to an XML test case comprising:

generating a specific XML test case associated with the web locations;

parsing the XML test case into a sequence of request/response pairs with a parsing engine; and

sending each of the http requests to the corresponding web locations, said sending comprising;

receiving requests/response pairs from the parsing engine;

identifying dynamic data in the request/response pairs received from the parsing engine;

evaluating the identified dynamic data for a result; and

replacing the evaluated result to generate a replaced request, wherein a tag value of the identified dynamic data is randomly replaced with at least one or more of the possible values listed under a node of an XML data file corresponding to the identified dynamic data.

25. The method of claim 24 further comprising receiving from each of the web locations an http response corresponding to the http request sent to the location and retrieving, validating, and storing data from the http response wherein said retrieved, validated, and stored data is indicative of an operational status of the web location.

26. The method of claim 24 wherein parsing the XML test case comprises:

identifying components in the XML test case received from the test generator;

identifying configuration data of the components; and

replacing the identified configuration data in the request to be sent to the web location to be tested.

27. The method of claim 24 wherein parsing the XML test case comprises:

identifying any sequence of request/response pairs in the XML test case received from the test generator or user input;

determining if each identified request is a simple request or a composite of other requests and for reducing a composite request to a plurality of simple requests;

for each simple request/response pair, identifying a data structure of the simple request/response pair;

identifying aspects of the request data structure which include configuration data; and

replacing the identified configuration data in the request to be sent to the web location to be tested and outputting the resulting sequence of request/response pairs.

28. The method of claim 27 wherein the data structure includes the elements of an http request including the following:

a URL, a path, query string parameters, post data, and a verb.

29. The method of claim 24 wherein sending each of the http requests comprises:

creating an http request from the data in the replaced request;

instantiating an http engine; and

sending the http request to the web location.

30. The method of claim 29 further comprising:

receiving from the web location an http response to the corresponding http request;

identifying the location and data to be retrieved from the http response regarding the operability of the web location; and

retrieving, validating, and storing the identified location and data.

31. A computer readable medium (CRM) having instructions for testing a web location including a web site or web service comprising:

instructions for generating an XML test case; and

instructions for interpreting the XML test case into an http request to be sent to the web location, said instructions for interpreting comprising;

instructions for parsing the XML test case into a sequence of request/response pairs with a parsing engine;

instructions for receiving the requests/response pairs from the parsing engine;

instructions for identifying dynamic data in the request/response pairs received from the parsing engine;

instructions for evaluating the identified dynamic data for a result; and

instructions for replacing the evaluated result to generate a replaced request,

wherein a tag value of the identified dynamic data is randomly replaced with at least one or more of the possible values listed under a node of an XML data file corresponding to the identified dynamic data.

32. The CRM of claim 31 further comprising:

instructions for providing to the web location the http request corresponding to the sequence of requests/response pairs.

33. The CRM of claim 31 further comprising instructions for receiving from the web location an http response corresponding to the http request and instructions for retrieving, validating, and storing data from the http response wherein said retrieved, validated, and stored data is indicative of an operational status of the web location.

34. The CRM of claim 31 wherein the test case includes at least one of the following tags:

dynamic user names, parameters specific to the test case and retrieved from a configuration file corresponding to the test case, and optional tags.

35. The CRM of claim 31 further comprising at least one of the following in conjunction with the instructions for generating of the XML test case:

instructions for handling cookies, instructions for manipulating data, instructions for supporting XML parameter, instructions for sending hidden fields, instructions for inserting random parameters, and instructions for using use case attributes.