Identifying user interface elements using element signatures

• US 10,474,564 B1
• Filed: 01/25/2019
• Issued: 11/12/2019
• Est. Priority Date: 01/25/2019
• Status: Active Grant

• Current Assignee: Softesis Inc.
• Original Assignee: Softesis Inc.
• Inventors: Konyshev, Ivan, Shamatrin, Dmytro, Grebeniuk, Pavlo
• Primary Examiner(s): Aponte, Francisco J
• Application Number:
  US16/258,249
• Time in Patent Office: 291 Days
• Field of Search: 717124
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology

Abstract

Systems and methods for performing automated software testing on user interface elements are disclosed. For instance, a first element of an application can be identified. A signature can be generated for the first element. The signature for the first element can include one or more attributes descriptive of the first element. After an update of the application, a candidate element signature can be generated for one or more candidate elements in the updated application. The signature for the first element can be compared to each candidate element signature to determine whether the first element matches each candidate element, thereby enabling the first element to be located after the update of the application.

Claims

1. A computer-implemented method of identifying user interface elements, the method comprising:

(i) identifying, by one or more computing devices, a first element of an application;

(ii) generating, by one or more computing devices, a signature for the first element, the signature for the first element comprising one or more attributes descriptive of the first element;

(iii) executing, by one or more computing devices, a software test script to validate proper operation of the application, the executing (iii) comprising performing one or more testing operations on the first element;

(iv) after the executing (ii), generating, by one or more computing devices, a candidate element signature for one or more candidate elements in the application, each candidate element signature comprising one or more attributes descriptive of the candidate element;

(v) comparing, by one or more computing devices, the signature for the first element to each candidate element signature to determine whether the first element matches each candidate element, thereby enabling the first element to be located after the update of the application; and

(vi) when a first candidate element is determined to match the test element, automatically executing, by one or more computing devices, the software test script to validate proper operation of the updated application, wherein the executing (vi) comprises performing one or more testing operations on the first candidate element;

wherein the comparing (v) comprises;

providing as input to a neural network data indicative of the signature for the first element and data indicative of each candidate element signature; and

receiving as output of the neural network a match result indicating whether the first element corresponds to each candidate element.

2. The computer-implemented method of claim 1, wherein the one or more attributes descriptive of the first element comprise one or more of a location attribute, address attribute, style attribute, size attribute, name attribute, or ID attribute.

3. The computer-implemented method of claim 1, further comprising determining, by one or more computing devices, that the first candidate element matches the first element based at least in part on the comparing.

4. The computer-implemented method of claim 1, wherein the first element is a test element specified in the software test script.

5. The computer-implemented method of claim 1, further comprising:

generating, by one or more computing devices, an internal HTML representation associated with the application; and

wherein

generating, by one or more computing devices, the signature for the first element comprises generating the signature for the first element based at least in part on the internal HTML representation; and

generating, by one or more computing devices, the candidate element signature comprises generating the candidate element signature based at least in part on the internal HTML representation.

6. A computing system comprising:

one or more processors;

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors to perform operations, the operations comprising;

(i) identifying a first element of an application;

(ii) generating a signature for the first element, the signature for the first element comprising one or more attributes descriptive of the first element;

(iii) executing a software test script to validate proper operation of the application, the executing (iii) comprising performing one or more testing operations on the first element;

(iv) after the executing (ii), generating a candidate element signature for one or more candidate elements in the application, each candidate element signature comprising one or more attributes descriptive of the candidate element;

(v) comparing the signature for the first element to each candidate element signature to determine whether the first element matches each candidate element, thereby enabling the first element to be located after the update of the application; and

(vi) when a first candidate element is determined to match the test element, automatically executing the software test script to validate proper operation of the updated application, wherein the executing (vi) comprises performing one or more testing operations on the first candidate element;

wherein the comparing (v) comprises;

providing as input to a neural network data indicative of the signature for the first element and data indicative of each candidate element signature; and

receiving as output of the neural network a match result indicating whether the first element corresponds to each candidate element.

7. The computing system of claim 6, wherein the one or more attributes descriptive of the first element comprise one or more of a location attribute, address attribute, style attribute, size attribute, name attribute, or ID attribute.

8. The computing system of claim 6, further comprising determining that the first candidate element matches the first element based at least in part on the comparing.

9. The computing system of claim 6, wherein the first element is a test element specified in the software test script.

10. The computing system of claim 6, further comprising:

generating an internal HTML representation associated with the application; and

wherein

generating the signature for the first element comprises generating the signature for the first element based at least in part on the internal HTML representation; and

generating the candidate element signature comprises generating the candidate element signature based at least in part on the internal HTML representation.

11. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

(i) identifying a first element of an application;

(ii) generating a signature for the first element, the signature for the first element comprising one or more attributes descriptive of the first element;

(iii) executing a software test script to validate proper operation of the application, the executing (iii) comprising performing one or more testing operations on the first element;

(iv) after the executing (ii), generating a candidate element signature for one or more candidate elements in the application, each candidate element signature comprising one or more attributes descriptive of the candidate element;

(v) comparing the signature for the first element to each candidate element signature to determine whether the first element matches each candidate element, thereby enabling the first element to be located after the update of the application; and

(vi) when a first candidate element is determined to match the test element, automatically executing the software test script to validate proper operation of the updated application, wherein the executing (vi) comprises performing one or more testing operations on the first candidate element;

wherein the comparing (v) comprises;

providing as input to a neural network data indicative of the signature for the first element and data indicative of each candidate element signature; and

receiving as output of the neural network a match result indicating whether the first element corresponds to each candidate element.

12. The one or more tangible, non-transitory computer-readable media of claim 11, wherein the one or more attributes descriptive of the first element comprise one or more of a location attribute, address attribute, style attribute, size attribute, name attribute, or ID attribute.

13. The one or more tangible, non-transitory computer-readable media of claim 11, further comprising determining that the first candidate element matches the first element based at least in part on the comparing.

14. The one or more tangible, non-transitory computer-readable media of claim 11, wherein the first element is a test element specified in the software test script.

15. The one or more tangible, non-transitory computer-readable media of claim 11, further comprising:

generating an internal HTML representation associated with the application; and

wherein

generating the signature for the first element comprises generating the signature for the first element based at least in part on the internal HTML representation; and

generating the candidate element signature comprises generating the candidate element signature based at least in part on the internal HTML representation.