CUSTOM WIDGETS BASED ON GRAPHICAL USER INTERFACES OF APPLICATIONS

• US 20180024730A1
• Filed: 07/19/2016
• Published: 01/25/2018
• Est. Priority Date: 07/19/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: JAMBOU, Emmanuel
• Granted Patent: US 10,394,437 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Systems, methods, and computer program products to perform an operation comprising receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a display device, rendering the GUI of the first application on a first virtual display of the first computing device, and not on the first display device, and generating a first widget which displays the first portion of the GUI of the first application on the display device, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display.

Claims

1. A method, comprising:

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion is less than the entire GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device; and

generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI.

2. The method of claim 1, further comprising:

receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

3. The method of claim 1, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the method further comprising:

receiving user input by the first widget displayed on the first display device;

transmitting an indication of;

(i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and

providing the user input to the GUI of the first application on the first virtual display.

4. The method of claim 3, further comprising:

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

refreshing the first widget to display the received output on the first display device.

5. The method of claim 4, wherein a portion of the received output is not visible due to a size of the first widget, the method further comprising:

increasing the size of the first widget to display the received output including the portion of the output that was not visible.

6. The method of claim 3, further comprising:

determining that the received user input is associated with a first rule, of a plurality of rules for the first widget;

determining that the first rule is associated with a first response, wherein the first response specifies to;

minimize the first widget such that the first widget is not displayed on the first display device; and

display the GUI of the first application on the first display device.

7. The method of claim 1, further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device.

8. A system, comprising:

one or more computer processors;

a memory containing a program which when executed by the processors performs an operation comprising;

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device of the system and wherein the specified first portion is less than the entire GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device; and

generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI.

9. The system of claim 8, the operation further comprising:

receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

10. The system of claim 8, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the operation further comprising:

receiving user input by the first widget displayed on the first display device;

transmitting an indication of;

(i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and

providing the user input to the GUI of the first application on the first virtual display.

11. The system of claim 10, the operation further comprising:

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

refreshing the first widget to display the received output on the first display device.

12. The system of claim 11, wherein a portion of the received output is not visible due to a size of the first widget, the operation further comprising:

increasing the size of the first widget to display the received output including the portion of the output that was not visible.

13. The system of claim 10, the operation further comprising:

determining that the received user input is associated with a first rule, of a plurality of rules for the first widget;

determining that the first rule is associated with a first response, wherein the first response specifies to;

minimize the first widget such that the first widget is not displayed on the first display device; and

display the GUI of the first application on the first display device.

14. The system of claim 8, the operation further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the display device is of a second system, communicably coupled to the first system.

15. A computer program product, comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising;

receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion is less than the entire GUI;

rendering the GUI of the first application on a first virtual display of the first computing device;

removing, based on the rendering, the GUI of the first application from the first display device; and

generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI.

16. The computer program product of claim 15, the operation further comprising:

receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and

providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI.

17. The computer program product of claim 15, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the operation further comprising:

receiving user input by the first widget displayed on the first display device;

transmitting an indication of;

(i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget;

mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and

providing the user input to the GUI of the first application on the first virtual display.

18. The computer program product of claim 17, the operation further comprising:

generating, by the first application, an output responsive to the received user input;

transmitting, by the first virtual display to the first widget, the output; and

refreshing the first widget to display the received output on the first display device.

19. The computer program product of claim 18, wherein a portion of the received output is not visible due to a size of the first widget, the operation further comprising:

increasing the size of the first widget to display the received output including the portion of the output that was not visible.

20. The computer program product of claim 15, the operation further comprising:

receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device;

rendering the GUI of the second application on a second virtual display, and not on the first display device;

generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and

outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device.