Systems and Methods for Isolating On-Screen Textual Data
First Claim
1. A method of determining a user interface is displaying a textual element identifying contact information and automatically providing in response to the determination a selectable user interface element near the textual element to initiate a telecommunication session based on the contact information, the method comprising the steps of:
- (a) capturing, by a client agent, an image of a portion of a screen of a client, the portion of the screen displaying a textual element identifying contact information;
(b) recognizing, by the client agent, via optical character recognition text of the textual element in the captured image;
(c) determining, by the client agent, the recognized text comprises contact information; and
(d) displaying, by the client agent in response to the determination, a user interface element near the textual element on the screen selectable to initiate a telecommunication session based on the contact information.
1 Assignment
0 Petitions
Accused Products
Abstract
The systems and methods of the client agent describe herein provides a solution to obtaining, recognizing and taking an action on text displayed by an application that is performed in a non-intrusive and application agnostic manner. In response to detecting idle activity of a cursor on the screen, the client agent captures a portion of the screen relative to the position of the cursor. The portion of the screen may include a textual element having text, such as a telephone number or other contact information. The client agent calculates a desired or predetermined scanning area based on the default fonts and screen resolution as well as the cursor position. The client agent performs optical character recognition on the captured image to determine any recognized text. By performing pattern matching on the recognized text, the client agent determines if the text has a format or content matching a desired pattern, such as phone number. In response to determining the recognized text corresponds to a desired pattern, the client agent displays a user interface element on the screen near the recognized text. The user interface element may be displayed as an overlay or superimposed to the textual element such that it seamlessly appears integrated with the application. The user interface element is selectable to take an action associated with the recognized text.
268 Citations
44 Claims
-
1. A method of determining a user interface is displaying a textual element identifying contact information and automatically providing in response to the determination a selectable user interface element near the textual element to initiate a telecommunication session based on the contact information, the method comprising the steps of:
-
(a) capturing, by a client agent, an image of a portion of a screen of a client, the portion of the screen displaying a textual element identifying contact information; (b) recognizing, by the client agent, via optical character recognition text of the textual element in the captured image; (c) determining, by the client agent, the recognized text comprises contact information; and (d) displaying, by the client agent in response to the determination, a user interface element near the textual element on the screen selectable to initiate a telecommunication session based on the contact information. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A system for determining a user interface is displaying a textual element identifying contact information and automatically providing in response to the determination a selectable user interface element near the textual element to initiate a telecommunication session based on the contact information, the system comprising:
-
a client agent executing on a client, the client agent comprising a cursor activity detector to detect activity of a cursor on a screen; a screen capture mechanism capturing, in response to the cursor activity detector, an image of a portion of the screen displaying a textual element identifying contact information; an optical character recognizer recognizing text of the textual element in the captured image; a pattern matching engine determining the recognized text comprises contact information; and wherein the client agent displays in response to the determination a user interface element near the textual element on the screen selectable to initiate a telecommunication session based on the contact information. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32)
-
-
33. A method of automatically recognizing text of a textual element displayed by an application on a screen of a client and in response to the recognition displaying a selectable user interface element to take an action based on the text, the method comprising:
-
(a) detecting, by a client agent, a cursor on a screen of a client is idle for a predetermined length of time; (b) capturing, by the client agent in response to the detection, an image of a portion of a screen of a client, the portion of the screen displaying a textual element; (c) recognizing, by the client agent, via optical character recognition text of the textual element in the captured image; (d) determining, by the client agent, the recognized text corresponds to a predetermined pattern; and (e) displaying, by the client agent, near the textual element on the screen a selectable user interface element to take an action based on the recognized text in response to the determination. - View Dependent Claims (34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44)
-
Specification