AUTOMATED VALIDATION OF THE APPEARANCE OF GRAPHICAL USER INTERFACES

• US 20150269721A1
• Filed: 03/19/2014
• Published: 09/24/2015
• Est. Priority Date: 03/19/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bartley, Timothy S., Bray, Gavin G., Hughes, Elizabeth M., Singh, Kalvinder P.
• Granted Patent: US 9,703,770 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/221 Parsing markup language streams streaming G06F40/149
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach

Abstract

According to embodiments of the present invention, a first metadata defining how a user interface associated with an application under test should appear is generated using one or more computer processors. An image of the user interface associated with the application under test that is invoked during a test run is captured using one or more one or more computer processors. The captured image is converted, using one or more computer processors, into one or more of a second metadata and text. The first metadata is compared, using one or more computer processors, to the second metadata and the text. In response to the comparison resulting in a difference between the first metadata and the second metadata, the image is stored, using one or more computer processors, and an assessment request is generated using one or more computer processors.

Claims

1. A method comprising:

generating, using one or more computer processors, a first metadata defining how a user interface associated with an application should appear;

capturing, using one or more computer processors, an image of the user interface associated with the application during a test run;

converting, using one or more computer processors, the captured image into one or more of a second metadata and text; and

comparing, using one or more computer processors, the first metadata to the second metadata and the text;

in response to the comparison resulting in a difference between the first metadata and the second metadata, storing, using one or more computer processors, the image and generating, using one or more computer processors, an assessment request.

2. The method of claim 1, wherein metadata that is associated with one or more of a window, page, or dialog box comprises a caption, software widget, and associated layout.

3. The method of claim 1, wherein metadata that is associated with one or more of a software widget comprises widget type, size, and relative position.

4. The method of claim 1, wherein metadata that is associated with text widgets comprise a globalization identifier.

5. The method of claim 1, wherein one or more of the first metadata and the second metadata are generated using a markup language.

6. The method of claim 1, wherein the step of converting, using the one or more computer processors, the captured image comprises using image segmentation to identify and separate elements included in the captured image.

7. A computer program product comprising:

one or more computer readable tangible storage media and program instructions stored on the one or more computer readable tangible storage media, the program instructions executable by one or more processors to;

generate a first metadata defining how a user interface associated with an application should appear;

capture an image of the user interface associated with the application during a test run;

convert the captured image into one or more of a second metadata and text; and

compare the first metadata to the second metadata and the text;

in response to the comparison resulting in a difference between the first metadata and the second metadata, store the image and generate an assessment request.

8. The computer program product of claim 7, wherein metadata that is associated with one or more of a window, page, or dialog box comprises a caption, software widget, and associated layout.

9. The computer program product of claim 7, wherein metadata that is associated with one or more of a software widget comprises widget type, size, and relative position.

10. The computer program product of claim 7, wherein metadata that is associated with text widgets comprise a globalization identifier.

11. The computer program product of claim 7, wherein one or more of the first metadata and the second metadata are generated using a markup language.

12. The computer program product of claim 7, wherein the computer instructions to convert the captured image comprise computer instructions to use image segmentation to identify and separate elements included in the captured image.

13. A computer system comprises:

one or more computer processors;

one or more computer readable tangible storage media;

program instructions stored on the one or more computer readable tangible storage media for execution by at least one of the one or more computer processors, the program instructions comprising program instructions to;

generate a first metadata defining how a user interface associated with an application should appear;

capture an image of the user interface associated with the application during a test run;

convert the captured image into one or more of a second metadata and text; and

compare the first metadata to the second metadata and the text;

in response to the comparison resulting in a difference between the first metadata and the second metadata, store the image and generate an assessment request.

14. The computer system of claim 13, wherein metadata that is associated with one or more of a window, page, or dialog box comprises a caption, software widget, and associated layout.

15. The computer system of claim 13, wherein metadata that is associated with one or more of a software widget comprises widget type, size, and relative position.

16. The computer system of claim 13, wherein metadata that is associated with text widgets comprise a globalization identifier.

17. The computer system of claim 13, wherein one or more of the first metadata and the second metadata are generated using a markup language.

18. The computer system of claim 13, wherein the program instructions to convert the captured image comprise program instructions to use image segmentation to identify and separate elements included in the captured image.