Method for monitoring a graphical user interface on a second computer display from a first computer
First Claim
1. A method for using a first computer system to remotely monitor and interact with the operation of a second computer system through a graphical user interface of said second computer system, comprising the steps of:
- receiving a bitmap image of said second computer system graphical user interface at said first computer system;
searching said bitmap image of said second computer system graphical user interface for a first graphical element contained within and comprising less than said bitmap image through an automated execution of said first computer system commands;
responsive to said receiving step and results of said searching step, generating a user peripheral input device input action within said second computer system graphical user interface as interpreted by said second computer by automatically creating and passing a signal through a communications channel from said first computer system to said second computer system graphical user interface;
monitoring said bitmap image of said second computer system graphical user interface automatically from said first computer system for an expected second graphical element contained within and comprising less than said bitmap image within a predetermined time interval; and
signaling a failure at said first computer system if said predetermined time interval elapses without detecting said expected second graphical element.
6 Assignments
0 Petitions
Accused Products
Abstract
A high level interface between a remote computer and local computer operator permits the remote computer to be controlled via a sequence of interactions. The remote computer may be monitored for display information which is expected, and also controlled in the event the expected information either is or is not obtained. Command language extensions are provided which extend and complement a basic scripting language. Scripts with embedded command language extensions may be executed through a remote interface, permitting remote testing, operation and evaluation. The development of the scripts, including embedded command language extensions, may be prepared through execution of commands at the local computer on a depiction of the GUI received from the remote computer, and subsequently saved as a script.
-
Citations
68 Claims
-
1. A method for using a first computer system to remotely monitor and interact with the operation of a second computer system through a graphical user interface of said second computer system, comprising the steps of:
-
receiving a bitmap image of said second computer system graphical user interface at said first computer system; searching said bitmap image of said second computer system graphical user interface for a first graphical element contained within and comprising less than said bitmap image through an automated execution of said first computer system commands; responsive to said receiving step and results of said searching step, generating a user peripheral input device input action within said second computer system graphical user interface as interpreted by said second computer by automatically creating and passing a signal through a communications channel from said first computer system to said second computer system graphical user interface; monitoring said bitmap image of said second computer system graphical user interface automatically from said first computer system for an expected second graphical element contained within and comprising less than said bitmap image within a predetermined time interval; and signaling a failure at said first computer system if said predetermined time interval elapses without detecting said expected second graphical element. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method for enabling a local system to automatically remotely operate a remote computer system through a graphical user interface on said remote computer system by using local scripts that selectively respond to changes in graphical displays upon said graphical user interface of said remote computer system, comprising the steps of:
-
displaying a depiction of said remote system graphical user interface display on said local system; capturing user input effected in said depiction of said remote system graphical user interface display; implementing automatically through a local system command language set user input emulations that are representative of said captured user input when reproduced at said remote computer system graphical user interface through a peripheral input device channel; image processing said remote computer system graphical displays automatically using a local system script that searches for and detects a first entity contained within and comprising less than said graphical display upon said graphical user interface of said remote computer system; controlling a flow of execution of said local system automatically through a scripting language having scripting commands in combination with said command language set, said flow which varies responsive to a result of detection of said first entity during said image processing step; and communicating said user input emulations between said local system and said remote computer system graphical user interface through a communication interface responsive to said flow controlling step. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A method for using a first computer system to remotely monitor and interact with the operation of a second computer system through a graphical user interface of said second computer system, comprising the steps of:
-
receiving a representation of said second computer system graphical user interface at said first computer system;
searching said representation of said second computer system graphical user interface for a first graphical entity contained within and comprising less than said representation of said second computer system graphical user interface through an automated execution of said first computer system commands; andresponsive to said receiving step and results of said searching step, generating a user peripheral input device input action within said second computer system graphical user interface as interpreted by a second computer by automatically creating and passing a signal through a communications channel from said first computer system to said second computer system graphical user interface. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34)
-
-
35. A method for enabling a first computing machine to remotely operate a second computing machine through a graphical user interface on said second computing machine by using computing scripts that selectively respond to changes within graphical displays upon said graphical user interface of said second computing machine, comprising the steps of:
-
displaying a depiction of said second computing machine graphical user interface display on a graphical user interface of said first computing machine; capturing user input effected in said depiction of said second computing machine graphical user interface display; image processing said second computing machine graphical displays using a first computing machine script that searches for and detects the presence of a first entity contained within and comprising less than said graphical display upon said graphical user interface of said second computing machine; controlling a flow of execution of said first computing machine through a scripting language having scripting commands in combination with said a command language set, said flow which varies responsive to a result of detection of said first entity during said image processing step; and implementing user input commands at said second computing machine graphical user interface emulating said captured user input by communicating between said first computing machine and said second computing machine graphical user interface through a communication interface responsive to said flow controlling step, a first user input command implemented responsive to a detection of said first entity during said image processing step and a second user input command different from said first user input command implemented responsive to a non-detection of said first entity during said image processing step. - View Dependent Claims (36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48)
-
-
49. A method for using a first computer system to remotely monitor and interact with the operation of a second computer system through a graphical user interface of said second computer system, comprising the steps of:
-
receiving a bitmap image of said second computer system graphical user interface at said first computer system; searching said bitmap image of said second computer system graphical user interface for a first graphical element contained within and comprising less than said bitmap image through an automated execution of said first computer system commands; responsive to said receiving step and a not-found result of said searching step, generating a first user peripheral input device input action within said second computer system graphical user interface by automatically creating and passing a signal through a communications channel from said first computer system to said second computer system graphical user interface; and responsive to said receiving step and a found result of said searching step, generating a second user peripheral input device input action within said second computer system graphical user interface different from said first user peripheral input device input action by automatically creating and passing a signal through a communications channel from said first computer system to said second computer system graphical user interface. - View Dependent Claims (50, 51, 52, 53, 54, 55)
-
-
56. A method for enabling a first computing machine to remotely operate a second computing machine through a graphical user interface on said second computing machine by using computing scripts that selectively respond to changes within graphical displays upon said graphical user interface of said second computing machine, comprising the steps of:
-
displaying a depiction of said second computing machine graphical user interface display on a graphical user interface of said first computing machine; image processing said second computing machine graphical displays using a first computing machine search command set that searches for and detects the presence of a first entity contained within and comprising less than said graphical display upon said graphical user interface of said second computing machine; controlling a flow of execution of said first computing machine through a scripting language having scripting commands in combination with a command language set, said flow which varies responsive to a result of detection of said first entity during said image processing step; and communicating between said first computing machine and said second computing machine graphical user interface through a communication interface responsive to said flow controlling step, a first user input command implemented at said second computing machine responsive to a detection of said first entity during said image processing step and a second user input command different from said first user input command implemented at said second computing machine responsive to a non-detection of said first entity during said image processing step. - View Dependent Claims (57, 58, 59, 60, 61, 62)
-
-
63. A method for enabling a first computing machine to remotely operate a second computing machine through a graphical user interface on said second computing machine by using computing scripts that selectively respond to changes within graphical displays upon said graphical user interface of said second computing machine, comprising the steps of:
-
displaying a depiction of said second computing machine graphical user interface display on a graphical user interface of said first computing machine; capturing user input effected in said depiction of said second computing machine graphical user interface display; developing at least one computing script incorporating user input into said first computing machine selected from ones of various scripting functions and commands, said at least one computing script further incorporating said captured user input; image processing said second computing machine graphical displays using a first computing machine search command set that searches for and detects the presence of a first entity contained within and comprising less than said graphical display upon said graphical user interface of said second computing machine; controlling a flow of execution of said first computing machine through said at least one computing script, said flow which varies responsive to a result of detection of said first entity during said image processing step; and implementing user input commands at said second computing machine graphical user interface emulating said captured user input by communicating between said first computing machine and said second computing machine graphical user interface through a communication interface responsive to said flow controlling step, a first user input command implemented at said second computing machine graphical user interface responsive to a detection of said first entity during said image processing step and a second user input command different from said first user input command implemented at said second computing machine graphical user interface responsive to a non-detection of said first entity during said image processing step. - View Dependent Claims (64, 65, 66, 67, 68)
-
Specification