Display-independent recognition of graphical user interface control
First Claim
Patent Images
1. A method for automatic extraction of an image of a control in a graphical user interface (GUI), the method comprising:
- capturing a first screenshot of a GUI of a computerized application, and automatically analyzing said first screenshot to identify a scroll bar by extracting at least one parameter of the scroll bar from within the first screenshot;
automatically operating said scroll bar to reveal a previously-invisible region of said GUI, and capturing a second screenshot of said previously-invisible region;
receiving coordinates of at least one position from a user in at least one of said first screenshot and said second screenshot;
analyzing an area surrounding said at least one position, to identify at least one GUI control; and
storing an image of said at least one GUI control separate from the first screenshot of the GUI or the second screenshot of the GUI from which the image of the GUI control is taken.
3 Assignments
0 Petitions
Accused Products
Abstract
A method for display-independent recognition of a graphical user interface (GUI) control, the method comprising capturing a screenshot of a GUI of a computerized application, determining a cursor position in the screenshot, analyzing an area surrounding the cursor position, to identify a GUI control, and storing an image of the GUI control.
108 Citations
19 Claims
-
1. A method for automatic extraction of an image of a control in a graphical user interface (GUI), the method comprising:
-
capturing a first screenshot of a GUI of a computerized application, and automatically analyzing said first screenshot to identify a scroll bar by extracting at least one parameter of the scroll bar from within the first screenshot; automatically operating said scroll bar to reveal a previously-invisible region of said GUI, and capturing a second screenshot of said previously-invisible region; receiving coordinates of at least one position from a user in at least one of said first screenshot and said second screenshot; analyzing an area surrounding said at least one position, to identify at least one GUI control; and storing an image of said at least one GUI control separate from the first screenshot of the GUI or the second screenshot of the GUI from which the image of the GUI control is taken. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A non-transitory computer-readable medium having stored therein instructions for automatic extraction of an image of a control in a graphical user interface (GUI), wherein the instructions, when executed by the computer, cause the computer to:
-
capture a first image of a GUI of a computerized application, and automatically analyze said first image, to identify a scroll bar; automatically operate said scroll bar to reveal a previously-invisible region of said GUI, and capture a second image of said previously-invisible region; identify a cursor position in at least one of said first image and said second image; and analyze, in at least one of said first image and said second image, an area surrounding said position, to identify at least one GUI control. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. A method for automatic extraction of an image of a control in a graphical user interface (GUI) for operating a computerized application, the method comprising:
-
capturing a first screenshot of a GUI of a computerized application, and automatically analyzing said first screenshot to identify a scroll bar by extracting at least one parameter of the scroll bar from within the first screenshot; automatically operating said scroll bar to reveal a previously-invisible region of said GUI, and capturing a second screenshot of said previously-invisible region; receiving coordinates of at least one position in at least one of said first screenshot and said second screenshot from a user; analyzing an area surrounding said at least one position, to identify at least one GUI control; storing an image of said at least one GUI control separate from the first screenshot of the GUI or the second screenshot of the GUI from which the image of the GUI control is taken; creating an operation sequence of the computerized application; and associating the image of said at least one GUI control, with a step of an operation sequence, to produce a computerized guidance script adapted to provide guidance to a user including moving a cursor associated with an input device, to imitate user-controlled movement of said cursor via said input device. - View Dependent Claims (19)
-
Specification