Software feature usage analysis and reporting

• US 20080313633A1
• Filed: 06/15/2007
• Published: 12/18/2008
• Est. Priority Date: 06/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Yantao, Sun, Bing, Wang, Qiang, Zhang, Haidong, Liu, Guowei, Wang, Min, Wang, Jian, Ye, Shuguang, Zhu, Wenli, Mahmud, Adnan Azfar
• Granted Patent: US 7,747,988 B2
• Time in Patent Office: Days
• US Class Current: 718/100
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

Described is a technology for analyzing usage of a software program'"'"'s features. Software instrumentation data is during actual user program usage sessions. The collected data is then processed to determine various feature usage counts and other information, cross-feature usage (e.g., among users who use a feature, how many use another feature or program), and characteristics of feature users, e.g., how long, how much, how often and how extensive feature users use a program. Session analysis may be performed to provide information about the number of sessions in which a set of features occur. Feature usage trends over time may also be determined via analysis. A user interface is described for facilitating selection of one or more features to analyze, for facilitating selection of a group of users, and/or for outputting results corresponding to the analysis.

Claims

1. In a computing environment, a method comprising:

analyzing information corresponding to software instrumentation data collected from user sessions corresponding to one or more programs, including determining feature usage metrics; and

outputting information representative of at least one of the feature usage metrics.

2. The method of claim 1 further comprising, collecting the instrumentation data during actual user sessions.

3. The method of claim 1 wherein determining the feature usage metrics comprises determining for a set of users at least one of:

percentage of feature users information corresponding to a percentage of program users who use the feature, percentage of feature sessions information corresponding to a percentage of program sessions in which the feature occurred, feature occurrence information corresponding to a number of times the feature occurred, feature controls count information corresponding to a number of uses of necessary commands, noise count information corresponding to a number of uses of unnecessary commands, feature length information corresponding to the a number of feature control clicks and noise clicks per feature occurrence, feature ratio information corresponding to ratio of the sum of feature controls count and noise count to total click count, feature details information corresponding to a count of each command in the feature, command details information corresponding to a count of each command broken down by command properties, noise details information corresponding to exact match occurrence and noise click count, feature path details information corresponding to a number of paths of the feature, including total occurrence and commands that make up the path, or feature length distribution information, or any combination of percentage of feature users information, percentage of feature sessions information, feature occurrence information, feature controls count information, noise count information, feature length information, feature ratio information, feature details information, command details information, noise details information, feature path details information, or feature length distribution information.

4. The method of claim 1 wherein determining the feature usage metrics comprises determining cross-feature usage for a set of users and a specified set of one or more features, or a specified set of one or more programs, or a specified set of one or more features and a specified set of one or more programs.

5. The method of claim 4 wherein determining the cross-feature usage comprises determining information corresponding to at least one of:

user count information corresponding to a number of users who use all or a specified set of the specified features, program usage information corresponding to a users who use all the specified programs, feature usage corresponding to occurrence of the feature by users who use all or a specified set of the specified features, feature user count corresponding to a number of users who use the feature, program user count corresponding to a number of users who use the application, or occurrence information corresponding to a count of occurrence of the feature by feature users, or any combination of user count information, program usage information, feature usage, feature user count, program user count, or occurrence information.

6. The method of claim 1 wherein determining the feature usage metrics comprises determining feature user characteristics for a set of users, including determining at least one of:

session count information corresponding to a number of application sessions by the users, session frequency information corresponding to a time between consecutive sessions by users, running time information corresponding to a session time by users, session length information corresponding to session time by users, depth of usage information corresponding to a percentage of total commands of the application used by users.

7. The method of claim 1 wherein determining the feature usage metrics comprises performing a session analysis for a specified set of one or more features, comprising determining at least one of:

session count information corresponding to a number of program sessions by users, occurrence information corresponding to a number of sessions in which the specified feature or features occurred in the session, percentage of sessions information corresponding to a percentage of application sessions in which the specified feature or features occurred, user count information corresponding to a total number of program users, user same session information corresponding to a number of users who used the specified feature or features in the same session, percentage of users information corresponding to a percentage of program users who use the specified feature or features in the same session, frequency of sessions by users information, or distribution information corresponding to a number and percentage of users who use specified feature or features in the same session whose frequency of such sessions falls within each interval, or any combination of session count information, occurrence information, percentage of sessions information, user count information, user same session information, percentage of users information, frequency of sessions by users information, or distribution information.

8. The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least one feature usage trend over time.

9. The method of claim 8 wherein analyzing the information corresponding to the software instrumentation data to determine at least one feature usage trend over time comprises determining information corresponding to at least one of:

feature user count information corresponding to a number of users using a feature during a reporting interval, feature session count information corresponding to a number of sessions in which a feature occurred during a reporting interval, feature occurrence information corresponding to a number of times a feature occurred during a reporting interval, feature occurrence and user information corresponding to the feature occurrence divided by the feature user count during a reporting interval, feature occurrence and session information corresponding to the feature occurrence divided by the feature session count during a reporting interval, program count information corresponding to a number of users using a program with the feature during a reporting interval, percentage of users information corresponding to the feature user count divided by the program count during a reporting interval, program session count information corresponding to a number of sessions of a program with the feature during a reporting interval, percentage of sessions information corresponding to the feature session count divided by the program session count during a reporting interval, cumulative feature occurrence information corresponding to a number of times a feature occurred from a start time of a period to analyze to an end of each reporting interval, cumulative percentage of users information corresponding to a percentage of feature users from a start time of a period to analyze to an end of each reporting interval, or cumulative percentage of sessions information corresponding to a percentage of feature sessions from a start time of a period to analyze to an end of each reporting interval, or any combination of feature user count information, feature session count information, feature occurrence information, feature occurrence and user information, feature occurrence and session information, program count information, percentage of users information, program session count information, percentage of sessions information, cumulative feature occurrence information, cumulative percentage of users information, or cumulative percentage of sessions information.

10. The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least potential outlier corresponding to feature usage that appears different from feature usage by other users.

11. A computer-readable medium having computer executable instructions, which when executed perform steps comprising, processing information related to software instrumentation data collected during program usage sessions to analyze feature usage by at least some users during at least some of the program usage sessions;

and

outputting results of the analysis.

12. The computer-readable medium of claim 11 wherein processing the information comprises performing one or more of:

a feature usage analysis, a cross-feature usage analysis, an analysis of characteristics of feature users, a session analysis, or an analysis of feature usage trends.

13. The computer-readable medium of claim 11 wherein outputting the results of the analysis comprises outputting a report to a user interface.

14. The computer-readable medium of claim 11 having further computer-executable instructions comprising, receiving analysis parameters via a user interface, including receiving a selection of at least one feature or receiving a selection of a user source, or receiving a selection of at least one feature and receiving a selection of a user source.

15. In a computing environment, a system comprising, an analyzer that processes information corresponding to software instrumentation data recorded from user software program usage sessions to produce an output, the analyzer including means for performing a feature usage analysis, means for performing a cross-feature usage analysis, means for performing an analysis of characteristics of feature users, means for performing a session analysis, or means for performing an analysis of feature usage trends, or any combination of means for performing a feature usage analysis, means for performing a cross-feature usage analysis, means for performing an analysis of characteristics of feature users, means for performing a session analysis, or means for performing an analysis of feature usage trends.

16. The system of claim 15 further comprising a user interface coupled to the analyzer for facilitating selection of one or more features.

17. The system of claim 16 wherein the user interface further facilitates selection of a group of users.

18. The system of claim 15 further comprising a user interface coupled to the analyzer for outputting results corresponding to the analysis.

19. The system of claim 15 further comprising means for filtering at least some of the information corresponding to the software instrumentation data.

20. The system of claim 15 wherein the analyzer is coupled to a recognition process that recognizes features from the information corresponding to the software instrumentation data.