Graphical User Interface (Gui) Noise Reduction in a Cognitive Control Framework

• US 20080294985A1
• Filed: 11/11/2005
• Published: 11/27/2008
• Est. Priority Date: 11/11/2005
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Milov, Denis Sergeevich
• Granted Patent: US 8,429,612 B2
• Time in Patent Office: Days
• US Class Current: 715/704
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Reducing graphical user interface (GUI) noise maybe achieved by recording a first execution scenario for control of operation of an application program having a GUI during a recording phase of operation of a cognitive control framework system, setting soft conditions for a search for the application program for the first execution scenario, playing back the application program according to the first execution scenario during a playback phase of operation of the cognitive control framework system, updating the first execution scenario to form a second execution scenario to reduce GUI noise conditions observed during playback, including updating recorded images originally generated by the GUI during the recording phase and updating coordinates for user input data, setting stronger conditions for the search for use in subsequent playbacks; and playing back the application program according to the second execution scenario with the stronger conditions for search.

Claims

1. A computer-implemented method of reducing graphical user interface (GUI) noise comprising:

recording a first execution scenario for control of operation of an application program having a GUI during a recording phase of operation of a cognitive control framework system;

setting soft conditions for a search for the application program for the first execution scenario;

playing back the application program according to the first execution scenario during a playback phase of operation of the cognitive control framework system;

updating the first execution scenario to form a second execution scenario to reduce GUI noise conditions observed during playback, including updating recorded images originally generated by the GUI during the recording phase and updating coordinates for user input data;

setting stronger conditions for the search for use in subsequent playbacks; and

playing back the application program according to the second execution scenario with the stronger conditions for search.

2. The method of claim 1, wherein the soft conditions comprise a first set of bounds for differences in shapes of contours, text and image content, or layout.

3. The method of claim 2, where the stronger conditions comprise a second set of bounds for differences in shapes of contours, text and image content, or layout, the second set being different than the first set.

4. The method of claim 1, wherein the user input data comprises mouse selections.

5. The method of claim 1, wherein GUI noise conditions comprise at least one of changeable color schemes, highlighting of items, noise from video sources, and anti-aliasing effects.

6. The method of claim 1, wherein updating recorded images comprises using playback images as recorded images for subsequent playbacks.

7. An article comprising:

a machine accessible medium containing instructions, which when executed, result in reducing graphical user interface (GUI) noise by

recording a first execution scenario for control of operation of an application program having a GUI during a recording phase of operation of a cognitive control framework system;

setting soft conditions for a search for the application program for the first execution scenario;

playing back the application program according to the first execution scenario during a playback phase of operation of the cognitive control framework system;

updating the first execution scenario to form a second execution scenario to reduce GUI noise conditions observed during playback, including updating recorded images originally generated by the GUI during the recording phase and updating coordinates for user input data;

setting stronger conditions for the search for use in subsequent playbacks; and

playing back the application program according to the second execution scenario with the stronger conditions for search.

8. The article of claim 7, wherein the soft conditions comprise a first set of bounds for differences in shapes of contours, text and image content, or layout.

9. The article of claim 8, where the stronger conditions comprise second set of bounds for differences in shapes of contours, text and image content, or layout, the second set being different than the first set.

10. The article of claim 7, wherein the user input data comprises mouse selections.

11. The article of claim 7, wherein GUI noise conditions comprise at least one of changeable color schemes, highlighting of items, noise from video sources, and anti-aliasing effects.

12. The article of claim 7, wherein instructions to update recorded images comprise instructions to use playback images as recorded images for subsequent playbacks.

13. A method of automatically controlling execution of an application program having a GUI to reduce GUI noise comprising:

capturing user input data and images displayed by the GUI during a recording phase of execution of the application program;

analyzing the captured user input data and recorded images to generate a first execution scenario during the recording phase;

setting soft conditions for a search for the application program for the first execution scenario;

generating simulated user input data based on the first execution scenario during a playback phase of execution of the application program and inputting the simulated user input data to the application program;

performing image analysis on playback images displayed by the GUI as a result of processing the simulated user input data during the playback phase and the recorded images;

updating the first execution scenario to form a second execution scenario to reduce GUI noise conditions observed during playback, including updating the recorded images originally generated by the GUI during the recording phase and updating coordinates for user input data;

setting stronger conditions for the search for use in subsequent playbacks; and

playing back the application program according to the second execution scenario with the stronger conditions for search.

14. The method of claim 13, wherein the soft conditions comprise a first set of bounds for differences in shapes of contours, text and image content, or layout.

15. The method of claim 14, where the stronger conditions comprise a second set of bounds for differences in shapes of contours, text and image content, or layout, the second set being different than the first set.

16. The method of claim 14, wherein GUI noise conditions comprise at least one of changeable color schemes, highlighting of items, noise from video sources, and anti-aliasing effects.

17. The method of claim 14, wherein updating recorded images comprises using playback images as recorded images for subsequent playbacks.

18. An article comprising:

a machine accessible medium containing instructions, which when executed, result in automatically controlling execution of an application program having a GUI to reduce GUI noise by

capturing user input data and images displayed by the GUI during a recording phase of execution of the application program;

analyzing the captured user input data and recorded images to generate a first execution scenario during the recording phase;

setting soft conditions for a search for the application program for the first execution scenario;

generating simulated user input data based on the first execution scenario during a playback phase of execution of the application program and inputting the simulated user input data to the application program;

performing image analysis on playback images displayed by the GUI as a result of processing the simulated user input data during the playback phase and the recorded images;

updating the first execution scenario to form a second execution scenario to reduce GUI noise conditions observed during playback, including updating the recorded images originally generated by the GUI during the recording phase and updating coordinates for user input data;

setting stronger conditions for the search for use in subsequent playbacks; and

playing back the application program according to the second execution scenario with the stronger conditions for search.

19. The article of claim 18, wherein the soft conditions comprise a first set of bounds for differences in shapes of contours, text and image content, or layout.

20. The article of claim 19, where the stronger conditions comprise a second set of bounds for differences in shapes of contours, text and image content, or layout, the second set being different than the first set.

21. The article of claim 18, wherein GUI noise conditions comprise at least one of changeable color schemes, highlighting of items, noise from video sources, and anti-aliasing effects.

22. The article of claim 18, wherein instructions to update recorded images comprise instructions to use playback images as recorded images for subsequent playbacks.