Template identification for control of testing

• US 9,720,814 B2
• Filed: 05/22/2015
• Issued: 08/01/2017
• Est. Priority Date: 05/22/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sol, Alisson A. S., Bobukh, Eugene V., Boia, Dragos D., Qiu, Jiong, Ankney, Donald J., Fish, Robert D., Ramdatmisier, Viresh, Markey, Barry
• Primary Examiner(s): DUNCAN, MARC M
• Application Number:
  US14/719,834
• Publication Number:
  US 20160342500A1
• Time in Patent Office: 802 Days
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2119 Authenticating web pages, e.g. with suspicious links

Abstract

Template identification techniques for control of testing are described. In one or more implementations, a method is described to control testing of one or more services by one or more computing devices using inferred template identification. Templates are inferred, by the one or more computing devices, that are likely used for documents for respective services of a service provider that are available via corresponding universal resource locators (URLs) to form an inferred dataset. Overlaps are identified by the one or computing devices in the inferred dataset to cluster services together that have likely used corresponding templates. Testing is controlled by the one or more computing devices of the one or more services based at least in part on the clusters.

Claims

1. A method to control testing of one or more services by one or more computing devices using inferred template identification, the method comprising:

inferring templates, by the one or more computing devices, that are likely used for documents for respective services of a service provider that are available via corresponding universal resource locators (URLs) to form an inferred dataset;

identifying an overlap, by the one or more computing devices, in the inferred dataset to cluster services together that have likely used corresponding templates;

removing one or more duplicates, by the one or more computing devices, in the inferred dataset based on the identified overlap to form a de-duplicated dataset to be used to test the clustered services; and

controlling testing, by the one or more computing devices, of the one or more services based at least in part on the clustered services and the de-duplicated dataset.

2. The method of claim 1, wherein the inferring includes applying template inference techniques to symbolically represent the likely templates used by the respective services.

3. The method of claim 2, wherein the applying is performed using a set of known templated documents and the documents obtained from the clustered services, an output of which is the symbolic representations.

4. The method of claim 1, wherein the identifying of overlaps in the inferred dataset to cluster services together that have likely used corresponding templates is performed using machine learning.

5. The method of claim 4, wherein the machine learning uses known template documents as a ground truth to process the documents obtained from the clustered services.

6. The method of claim 1, wherein the inferring uses a meta-domain descriptor that is generated to describe structural components of the documents obtained from the clustered services independent of content included in the documents.

7. The method as described in claim 1, wherein the documents are webpages or web documents.

8. The method as described in claim 1, wherein the removing includes at least some duplicate templates that are used to validate quality.

9. A service testing system comprising:

a template inference module implemented at least partially in hardware, the template inference module configured to;

infer templates that are likely used for documents for respective services of a service provider that are available via corresponding universal resource locators (URLs) to form an inferred dataset; and

identify overlaps by the one or more computing devices in the inferred dataset to cluster services together that have likely used corresponding templates, wherein the identifying includes machine learning; and

a synthetic test generator configured to control testing of the one or more services based at least in part on the clustered services.

10. The system of claim 9, wherein the templates are inferred by applying template inference techniques to symbolically represent the likely templates used by the respective services.

11. The system of claim 10, wherein the applying is performed using a set of known templated documents and the documents obtained from the clustered services, an output of which is the symbolic representations.

12. The system of claim 9, wherein the machine learning uses known template documents as a ground truth to process the documents obtained from the clustered services.

13. The system of claim 9, wherein the templates are inferred through use of a meta-domain descriptor that is generated to describe structural components of the documents obtained from the clustered services independent of content included in the documents.

14. A computing device to infer template usage by services for use in testing, the computing device comprising:

one or more processors; and

one or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by the one or more processors, causes the one or more processors to perform operations comprising;

applying template inference techniques to symbolically represent likely templates used by respective services of a service provider to infer templates that are likely used for documents for the respective services that are available via corresponding universal resource locators (URLs) to form an inferred dataset;

identifying overlaps in the inferred dataset to cluster services together that have likely used corresponding templates;

removing one or more duplicates from the inferred dataset based on the identified overlaps to form a de-duplicated dataset; and

controlling testing of the one or more services based at least in part on the clustered services and the de-duplicated dataset.

15. The computing device of claim 14, wherein the applying is performed using a set of known templated documents and the documents obtained from the clustered services, an output of which is the symbolic representations.

16. The computing device of claim 14, wherein the identifying of overlaps in the inferred dataset to cluster URLs together that have likely used corresponding templates is performed using machine learning.

17. The computing device of claim 16, wherein the machine learning uses known template documents as a ground truth to process the documents obtained from the clustered services.

18. The computing device of claim 14, wherein the applying uses a meta-domain descriptor that is generated to describe structural components of the documents obtained from the clustered services independent of content included in the documents.