Analyzing software users with instrumentation data and user group modeling and analysis

• US 7,739,666 B2
• Filed: 06/15/2007
• Issued: 06/15/2010
• Est. Priority Date: 06/15/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Yantao, Sun, Bing, Wang, Qiang, Zhang, Haidong, Wang, Min, Wang, Jian, Ye, Shuguang, Zhu, Wenli
• Primary Examiner(s): Rodriguez; Paul L
• Assistant Examiner(s): Chad; Aniss
• Application Number:
  US11/818,610
• Publication Number:
  US 20080313617A1
• Time in Patent Office: 1,096 Days
• Field of Search: 717/127, 714/38, 714/39
• US Class Current: 717/127
• CPC Class Codes: G06F 11/321 Display for diagnostics, e.g. diagnostic result display, self-test user interface
• CPC Class Codes: G06F 11/328 Computer systems status display G06F11/327 takes precedence
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

Described is a technology by which software instrumentation data collected from user program sessions are analyzed, including by determining program usage metrics and/or command usage metrics. Information representative of the program usage metrics and/or the command usage metrics is output, such as in the form of a report. The software instrumentation data may be further analyzed, such as to determine at least one usage trend over time, and to determine user groups. For example, a usage subset of sessions that meet specified session usage criteria based on a set of session data may be located, along with a subset of users based on users whose sessions meet specified user criteria. The usage and user subsets may be combined via Boolean logic to produce a result set.

Claims

1. In a computing environment, a method comprising:

analyzing software instrumentation data collected from user sessions corresponding to one or more programs and one or more commands associated with the user sessions, wherein the analyzing the software instrumentation data includes determining program usage metrics, command usage metrics and at least one usage trend over time; and

upon determining the program usage metrics, command usage metrics and at least one usage trend over time, outputting information representative of at least one of the program usage metrics, the command usage metrics, and at least one usage trend over time.

2. The method of claim 1 further comprising, collecting the instrumentation data during actual user sessions.

3. The method of claim 1 wherein determining the program usage metrics comprises determining for a set of users, session length information based on session time and session count, depth of usage information based on a percentage of commands used, and at least one of:

session count information based on a number of application sessions, session frequency information based on a time measurement between sessions, running time information based on session time, session length information based on session time and session count.

4. The method of claim 1 wherein determining the command usage metrics comprises determining for a set of users and a selected command, session count information based on a number of sessions in which the selected command occurred, percentage of users information corresponding to a percentage of users of the set who use the selected command, and at least one of:

user count information based on a number of users of the set who use the selected command, percentage of session information corresponding to a percentage of application sessions in which the selected command was used, click count information corresponding to a number of clicks corresponding to the selected command, percentage of click count information corresponding to a percentage of program clicks corresponding to the selected command, click count per user information based on click count and user count of the selected command, and click count per session information corresponding to a click count per session.

5. The method of claim 1 wherein analyzing the software instrumentation data to determine at least one usage trend over time comprises determining at least one of:

user count information corresponding to a number of users that used a program during a reporting interval, session count information corresponding to a number of sessions of a program during a reporting interval, session count per user information based on a number of sessions and a total number of users that used a program during a reporting interval, running time per user information corresponding to a session length of users and a number of users that used a program during a reporting interval, cumulative session count per user information corresponding to a total number of sessions and a number of users from a start time of a period to analyze to an end of each reporting interval, and cumulative running time per user information corresponding to a session length of users that used a program and a number of users from a start time of a period to analyze to an end of each reporting interval.

6. The method of claim 1 further comprising, analyzing the software instrumentation data to determine at least one type of user.

7. The method of claim 6 wherein analyzing the software instrumentation data to determine at least one type of user comprises categorizing users by at least one of depth of usage and categorizing users by the types of activities in which they engage.

8. The method of claim 1 further comprising, analyzing the software instrumentation data to determine at least one potential outlier corresponding to command usage that appears different from command usage of other users.

9. The method of claim 1 wherein analyzing the software instrumentation data to determine at least one potential outlier comprises computing an entropy value corresponding to an occurrence distribution of a command, computing an average occurrence per session of the command usage by a potential outlier, and using the entropy value or the average occurrence per session, or both, as outlier criterion or criteria.

10. The method of claim 1 further comprising, locating a subset of the user sessions based on at least one session criterion, wherein each session criteria comprises a dimension and a value for that dimension, and each dimension comprises a variable recorded in a session, a feature, or a set of one or more variables computed from a plurality of variables recorded in a session, and wherein analyzing the information corresponding to the software instrumentation data comprises analyzing the subset.

11. The method of claim 10 wherein locating the subset comprises mathematically combining each of at least two session criteria.

12. The method of claim 1 further comprising, providing a mechanism for modeling a user group, including providing an interface for receiving one or more measures that specify one or more conditions that any user needs to meet in order to belong to the user group, and locating the users from the software instrumentation data based on the one or more measures.

13. The method of claim 12 wherein locating the users comprises determining at least one of:

session length information corresponding to a session length of the sessions of a user, crash information corresponding to a number of sessions of a user that crashed, depth of usage information corresponding to which commands were used by a user, enrollment length information corresponding to a time between an earliest session and a latest session of a user, failure information corresponding to a number of sessions that failed of a user, mean time to crash information corresponding to session length and sessions of a user that crashed, mean time to failure information corresponding to session length and number of sessions of a user that failed, session count information corresponding to a total number of sessions of a user, session frequency information corresponding to time between consecutive sessions of a user, and total running time information corresponding to total session time of sessions of a user.

14. A computer-readable storage medium having computer executable instructions, which when executed perform steps comprising,

analyzing software instrumentation data collected from user sessions corresponding to one or more programs and one or more commands associated with the user sessions, wherein the analyzing the software instrumentation data includes determining program usage metrics and command usage metrics and at least one usage trend over time;

and

upon determining the program usage metrics and, command usage metrics and at least one usage trend over time, outputting information representative of at least one of the program usage metrics, the command usage metrics, and at least one usage trend over time.

15. In a computing environment, a system comprising:

an analyzer that processes information corresponding to software instrumentation data recorded from user software program usage sessions to produce a first subset comprising software usage data;

wherein the analyzer comprises means for performing a command usage analysis, an application usage analysis, and a trend analysis;

a group modeling mechanism that processes the information corresponding to the software instrumentation data and a second subset comprising user data; and

means for combining the first subset with the second subset to provide an output that corresponds to a selected group of users and their software program usage.

16. The system of claim 15 further comprising a user interface for facilitating selection of one or more criteria by which the first subset and second subset are located from the information.

17. The system of claim 15 further comprising means for recording the software instrumentation data.