Custom widgets based on graphical user interfaces of applications

• US 10,394,437 B2
• Filed: 07/19/2016
• Issued: 08/27/2019
• Est. Priority Date: 07/19/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Jambou, Emmanuel
• Primary Examiner(s): Tran, Tan H
• Application Number:
  US15/213,518
• Publication Number:
  US 20180024730A1
• Time in Patent Office: 1,134 Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Systems, methods, and computer program products to perform an operation comprising receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a display device, rendering the GUI of the first application on a first virtual display of the first computing device, and not on the first display device, and generating a first widget which displays the first portion of the GUI of the first application on the display device, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display.

Claims

1. A method, comprising:

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion excludes at least one component of the GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device;

generating a first widget which displays the first portion of the GUI of the first application on the first display device;

associating a unique identifier of the first widget with a unique identifier of the first virtual display;

upon receiving user input on the first widget displayed on the first display device;

determining coordinates on the first widget where the user input was received;

transmitting, to the first virtual display, an indication of;

(i) the coordinates on the first widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping the coordinates on the first widget to corresponding coordinates on the GUI rendered on the first virtual display; and

generating an input at the corresponding coordinates on the GUI rendered on the first virtual display, based on the user input;

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

determining that a portion of the output is not visible on the first display device due to a size of the first widget;

increasing the size of the first widget to display, on the first display device, the output including the portion of the output that was not visible; and

refreshing the first widget to display the output on the first display device.

2. The method of claim 1, further comprising:

receiving user input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

3. The method of claim 1, further comprising:

determining that the received user input is associated with a first rule, of a plurality of rules for the first widget;

determining that the first rule is associated with a first response, wherein the first response specifies to;

minimize the first widget such that the first widget is not displayed on the first display device; and

display the GUI of the first application on the first display device.

4. The method of claim 1, further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device.

5. A system, comprising:

one or more computer processors;

a memory containing a program which when executed by the processors performs an operation comprising;

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device of the system and wherein the specified first portion excludes at least one component of the GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device;

generating a first widget displays the first portion of the GUI of the first application on the first display device;

associating a unique identifier of the first widget with a unique identifier of the first virtual display;

upon receiving user input on the first widget displayed on the first display device;

determining coordinates on the first widget where the user input was received;

transmitting, to the first virtual display, an indication of;

(i) the coordinates on the first widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping the coordinates on the first widget to corresponding coordinates on the GUI rendered on the first virtual display; and

generating an input at the corresponding coordinates on the GUI rendered on the first virtual display, based on the user input;

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

determining that a portion of the output is not visible on the first display device due to a size of the first widget;

increasing the size of the first widget to display, on the first display device, the output including the portion of the output that was not visible; and

refreshing the first widget to display the output on the first display device.

6. The system of claim 5, the operation further comprising:

receiving user input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

7. The system of claim 5, the operation further comprising:

determining that the received user input is associated with a first rule, of a plurality of rules for the first widget;

determining that the first rule is associated with a first response, wherein the first response specifies to;

minimize the first widget such that the first widget is not displayed on the first display device; and

display the GUI of the first application on the first display device.

8. The system of claim 5, the operation further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the second display device is of a second system, communicably coupled to the system.

9. A computer program product, comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising;

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion excludes at least one component of the GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device; and

generating a first widget which displays the first portion of the GUI of the first application on the first display device;

associating a unique identifier of the first widget with a unique identifier of the first virtual display;

upon receiving user input on the first widget displayed on the first display device;

determining coordinates on the first widget where the user input was received;

transmitting, to the first virtual display, an indication of;

(i) the coordinates on the first widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping the coordinates on the first widget to corresponding coordinates on the GUI rendered on the first virtual display; and

generating an input at the corresponding coordinates on the GUI rendered on the first virtual display, based on the user input;

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

determining that a portion of the output is not visible on the first display device due to a size of the first widget;

increasing the size of the first widget to display, on the first display device, the output including the portion of the output that was not visible; and

refreshing the first widget to display the output on the first display device.

10. The computer program product of claim 9, the operation further comprising:

receiving user input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

11. The computer program product of claim 9, the operation further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device.