AUTOMATIC EXTRACTION, MODELING, AND CODE MAPPING OF APPLICATION USER INTERFACE DISPLAY SCREENS AND COMPONENTS

• US 20140215344A1
• Filed: 08/17/2013
• Published: 07/31/2014
• Est. Priority Date: 01/29/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ligman, Joseph W., Pistoia, Marco, Ponzo, John J., Thomas, Gegi
• Granted Patent: US 9,531,600 B2
• Time in Patent Office: Days
• US Class Current: 715/736
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: H04L 41/0226 Mapping or translating multiple network management protocols
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Managing user applications is provided. A visual model of user interface display screens of a user application is generated by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application. A runtime policy for each component in a selected set of components within the user interface display screens of the user application is generated based on user interaction with the generated visual model of the user interface display screens. The generated runtime policy for each component in the selected set of components is sent to a client device via a network. Activity of each component in the selected set of components in the client device is tracked using data that is tracked and logged on the client device based the generated runtime policy for each selected component.

Claims

1. A computer system for managing user applications, the computer system comprising:

a bus system;

a storage device connected to the bus system, wherein the storage device stores computer readable program code; and

a processor unit connected to the bus system, wherein the processor unit executes the computer readable program code to generate a visual model of user interface display screens of a user application by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application without accessing the original source code of the user application;

generate a runtime policy for each component in a selected set of components within the user interface display screens of the user application based on user interaction with the generated visual model of the user interface display screens;

send the generated runtime policy for each component in the selected set of components within the user interface display screens of the user application to a client device via a network; and

track activity of each component in the selected set of components within the user interface display screens of the user application running on the client device using data that is tracked and logged on the client device based on the generated runtime policy for each component in the selected set of components.

2. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to associate labels with the components displayed within the user interface display screens of the user application in the generated visual model.

3. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to receive a display screen data package that includes images of the user interface display screens of the user application located on the client device and screen position coordinates corresponding to the components displayed within the user interface display screens from the client device via the network;

and parse the display screen data package that includes the images of the user interface display screens of the user application on the client device and the screen position coordinates corresponding to the components within the user interface display screens, wherein the computer parses the display screen data package to identify each component within each of the user interface display screens.

4. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to display a rendering of a user interface display screen that is currently being displayed on the client device on a display device coupled to the computer system.

5. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to map the screen position coordinates corresponding to the components displayed within the user interface display screens to defined areas within the user interface display screens.

6. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to determine whether a runtime policy corresponding to a particular component within a user interface display screen of the user application located on the client device has been modified;

and send the modified runtime policy corresponding to the particular component within the user interface display screen of the user application to registered client devices that include the user application via the network in response to determining that the runtime policy corresponding to the particular component within the user interface display screen of the user application located on the client device has been modified.

7. A computer program product stored on a computer readable storage device having computer readable program code encoded thereon that is executable by a computer for managing user applications, the computer program product comprising:

computer readable program code for generating a visual model of user interface display screens of a user application by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application without accessing the original source code of the user application;

computer readable program code for generating a runtime policy for each component in a selected set of components within the user interface display screens of the user application based on user interaction with the generated visual model of the user interface display screens;

computer readable program code for sending the generated runtime policy for each component in the selected set of components within the user interface display screens of the user application to a client device via a network; and

computer readable program code for tracking activity of each component in the selected set of components within the user interface display screens of the user application running on the client device using data that is tracked and logged on the client device based on the generated runtime policy for each component in the selected set of components.

8. The computer program product of claim 7 further comprising:

computer readable program code for associating labels with the components displayed within the user interface display screens of the user application in the generated visual model.

9. The computer program product of claim 7 further comprising:

computer readable program code for receiving a display screen data package that includes images of the user interface display screens of the user application located on the client device and screen position coordinates corresponding to the components displayed within the user interface display screens from the client device via the network; and

computer readable program code for parsing the display screen data package that includes the images of the user interface display screens of the user application on the client device and the screen position coordinates corresponding to the components within the user interface display screens, wherein the computer parses the display screen data package to identify each component within each of the user interface display screens.

10. The computer program product of claim 7 further comprising:

computer readable program code for displaying a rendering of a user interface display screen that is currently being displayed on the client device on a display device coupled to the computer.

11. The computer program product of claim 7 further comprising:

computer readable program code for mapping the screen position coordinates corresponding to the components displayed within the user interface display screens to defined areas within the user interface display screens.

12. The computer program product of claim 7 further comprising:

computer readable program code for determining whether a runtime policy corresponding to a particular component within a user interface display screen of the user application located on the client device has been modified; and

computer readable program code for sending the modified runtime policy corresponding to the particular component within the user interface display screen of the user application to registered client devices that include the user application via the network in response to determining that the runtime policy corresponding to the particular component within the user interface display screen of the user application located on the client device has been modified.

13. A data processing system for managing user applications, the data processing system comprising:

a bus system;

a storage device connected to the bus system, wherein the storage device stores computer readable program code; and

a processor unit connected to the bus system, wherein the processor unit executes the computer readable program code to capture images of displayed user interface display screens of a running user application on the data processing system, wherein the data processing system uses an embedded instrumentation library in the running user application to capture the images of the displayed user interface display screens;

calculate screen position coordinates corresponding to components within the displayed user interface display screens of the running user application; and

generate a display screen data package that includes the captured images of the displayed user interface display screens of the running user application and the calculated screen position coordinates corresponding to the components within the displayed user interface display screens.

14. The data processing system of claim 13, wherein the processor unit further executes the computer readable program code to send the display screen data package that includes the captured images of the displayed user interface display screens of the running user application and the calculated screen position coordinates corresponding to the components within the displayed user interface display screens to an application manager server via a network.

15. The data processing system of claim 13, wherein the processor unit further executes the computer readable program code to receive a set of runtime policies corresponding to a set of components within the displayed user interface display screens of the running user application from an application manager server via a network;

and update the embedded instrumentation library in the running user application with the set of runtime policies corresponding to the set of components within the displayed user interface display screens of the running user application.

16. The data processing system of claim 13, wherein the processor unit further executes the computer readable program code to retrieve a runtime policy for each component within the displayed user interface display screens that has a corresponding runtime policy from the embedded instrumentation library using labels associated with the components;

and enforce the runtime policy for each component within the displayed user interface display screens that has the corresponding runtime policy to at least one of track activity of corresponding components and control functionality of the corresponding components.

17. A computer program product stored on a computer readable storage device having computer readable program code encoded thereon that is executable by a data processing system for managing user applications, the computer program product comprising:

computer readable program code for capturing images of displayed user interface display screens of a running user application on the data processing system, wherein the data processing system uses an embedded instrumentation library in the running user application to capture the images of the displayed user interface display screens;

computer readable program code for calculating screen position coordinates corresponding to components within the displayed user interface display screens of the running user application; and

computer readable program code for generating a display screen data package that includes the captured images of the displayed user interface display screens of the running user application and the calculated screen position coordinates corresponding to the components within the displayed user interface display screens.

18. The computer program product of claim 17 further comprising:

computer readable program code for sending the display screen data package that includes the captured images of the displayed user interface display screens of the running user application and the calculated screen position coordinates corresponding to the components within the displayed user interface display screens to an application manager server via a network.

19. The computer program product of claim 17 further comprising:

computer readable program code for receiving a set of runtime policies corresponding to a set of components within the displayed user interface display screens of the running user application from an application manager server via a network; and

computer readable program code for updating the embedded instrumentation library in the running user application with the set of runtime policies corresponding to the set of components within the displayed user interface display screens of the running user application.

20. The computer program product of claim 17 further comprising:

computer readable program code for retrieving, by the data processing system, a runtime policy for each component within the displayed user interface display screens that has a corresponding runtime policy from the embedded instrumentation library using labels associated with the components; and

computer readable program code for enforcing, by the data processing system, the runtime policy for each component within the displayed user interface display screens that has the corresponding runtime policy to at least one of track activity of corresponding components and control functionality of the corresponding components.