Automatic extraction, modeling, and code mapping of application user interface display screens and components

• US 9,531,600 B2
• Filed: 08/17/2013
• Issued: 12/27/2016
• Est. Priority Date: 01/29/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ligman, Joseph W., Pistoia, Marco, Ponzo, John J., Thomas, Gegi
• Primary Examiner(s): DAO, THUY CHAN
• Application Number:
  US13/969,524
• Publication Number:
  US 20140215344A1
• Time in Patent Office: 1,228 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: H04L 41/0226 Mapping or translating multiple network management protocols
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Managing user applications is provided. A visual model of user interface display screens of a user application is generated by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application. A runtime policy for each component in a selected set of components within the user interface display screens of the user application is generated based on user interaction with the generated visual model of the user interface display screens. The generated runtime policy for each component in the selected set of components is sent to a client device via a network. Activity of each component in the selected set of components in the client device is tracked using data that is tracked and logged on the client device based the generated runtime policy for each selected component.

Claims

1. A computer system for managing user applications, the computer system comprising:

a bus system;

a storage device connected to the bus system, wherein the storage device stores computer readable program code; and

a processor unit connected to the bus system, wherein the processor unit executes the computer readable program code to;

generate a visual model of user interface display screens of a user application by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application;

associate labels with the components displayed within the user interface display screens of the user application in the generated visual model;

generate a runtime policy for each component in a selected set of components within the user interface display screens of the user application based on user interaction with the generated visual model of the user interface display screens;

send the generated runtime policy for each component in the selected set of components within the user interface display screens of the user application to a client device via a network; and

track activity of each component in the selected set of components within the user interface display screens of the user application running on the client device using data that is tracked and logged on the client device based on the generated runtime policy for each component in the selected set of components.

2. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to:

receive a display screen data package that includes images of the user interface display screens of the user application located on the client device and screen position coordinates corresponding to the components displayed within the user interface display screens from the client device via the network; and

parse the display screen data package that includes the images of the user interface display screens of the user application on the client device and the screen position coordinates corresponding to the components within the user interface display screens, wherein the computer parses the display screen data package to identify each component within each of the user interface display screens.

3. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to display a rendering of a user interface display screen that is currently being displayed on the client device on a display device coupled to the computer system.

4. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to map the screen position coordinates corresponding to the components displayed within the user interface display screens to defined areas within the user interface display screens.

5. The computer system of claim 1, wherein the processor unit further executes the computer readable program code to:

determine whether a runtime policy corresponding to a particular component within a user interface display screen of the user application located on the client device has been modified; and

send the modified runtime policy corresponding to the particular component within the user interface display screen of the user application to registered client devices that include the user application via the network in response to determining that the runtime policy corresponding to the particular component within the user interface display screen of the user application located on the client device has been modified.

6. A computer program product stored on a computer readable storage device having computer readable program code encoded thereon that is executable by a computer for managing user applications, the computer program product comprising:

computer readable program code for generating a visual model of user interface display screens of a user application by mapping components displayed within the user interface display screens to screen position coordinates and by mapping the screen position coordinates corresponding to the components to original source code of the user application;

computer readable program code for associating labels with the components displayed within the user interface display screens of the user application in the generated visual model;

computer readable program code for generating a runtime policy for each component in a selected set of components within the user interface display screens of the user application based on user interaction with the generated visual model of the user interface display screens;

computer readable program code for sending the generated runtime policy for each component in the selected set of components within the user interface display screens of the user application to a client device via a network; and

computer readable program code for tracking activity of each component in the selected set of components within the user interface display screens of the user application running on the client device using data that is tracked and logged on the client device based on the generated runtime policy for each component in the selected set of components.

7. The computer program product of claim 6 further comprising:

computer readable program code for receiving a display screen data package that includes images of the user interface display screens of the user application located on the client device and screen position coordinates corresponding to the components displayed within the user interface display screens from the client device via the network; and

computer readable program code for parsing the display screen data package that includes the images of the user interface display screens of the user application on the client device and the screen position coordinates corresponding to the components within the user interface display screens, wherein the computer parses the display screen data package to identify each component within each of the user interface display screens.

8. The computer program product of claim 6 further comprising:

computer readable program code for displaying a rendering of a user interface display screen that is currently being displayed on the client device on a display device coupled to the computer.

9. The computer program product of claim 6 further comprising:

computer readable program code for mapping the screen position coordinates corresponding to the components displayed within the user interface display screens to defined areas within the user interface display screens.

10. The computer program product of claim 6 further comprising:

computer readable program code for determining whether a runtime policy corresponding to a particular component within a user interface display screen of the user application located on the client device has been modified; and

computer readable program code for sending the modified runtime policy corresponding to the particular component within the user interface display screen of the user application to registered client devices that include the user application via the network in response to determining that the runtime policy corresponding to the particular component within the user interface display screen of the user application located on the client device has been modified.