Software feature usage analysis and reporting

• US 7,747,988 B2
• Filed: 06/15/2007
• Issued: 06/29/2010
• Est. Priority Date: 06/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mahmud, Adan Azfar, Ye, Shuguang, Li, Yantao, Sun, Bing, Wang, Qiang, Zhang, Haidong, Liu, Guowei, Wang, Min, Wang, Jian, Zhu, Wenli
• Primary Examiner(s): Rodriguez; Paul L
• Assistant Examiner(s): Chad; Aniss
• Application Number:
  US11/818,600
• Publication Number:
  US 20080313633A1
• Time in Patent Office: 1,110 Days
• Field of Search: 717/713, 717/131, 717/130, 718/100
• US Class Current: 717/131
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

Described is a technology for analyzing usage of a software program'"'"'s features. Software instrumentation data is collected during actual user program usage sessions. The collected data is then processed to determine various feature usage counts and other information, cross-feature usage (e.g., among users who use a feature, how many use another feature or program), and characteristics of feature users, e.g., how long, how much, how often and how extensive feature users use a program. Session analysis may be performed to provide information about the number of sessions in which a set of features occur. Feature usage trends over time may also be determined via analysis. A user interface is described for facilitating selection of one or more features to analyze, for facilitating selection of a group of users, and/or for outputting results corresponding to the analysis.

Claims

1. In a computing environment, a method comprising:

analyzing software instrumentation data collected from user sessions corresponding to one or more programs, wherein analyzing the software instrumentation data comprises determining feature usage metrics associated with a feature of the one or more programs, wherein the feature usage metrics include feature usage, cross-feature usage, and feature usage trends; and

upon determining the feature usage metrics, outputting information representative of at least one of the determined feature usage metrics associated with a feature of the one or more programs.

2. The method of claim 1 further comprising, collecting the instrumentation data during actual user sessions.

3. The method of claim 1 wherein determining the feature usage metrics comprises determining for a set of users a percentage of feature sessions information corresponding to a percentage of program sessions in which the feature occurred and at least one of:

percentage of feature users information corresponding to a percentage of program users who use the feature, feature occurrence information corresponding to a number of times the feature occurred, feature controls count information corresponding to a number of uses of necessary commands, noise count information corresponding to a number of uses of unnecessary commands, feature length information corresponding to a number of feature control clicks and noise clicks per feature occurrence, feature ratio information corresponding to ratio of the sum of feature controls count and noise count to total click count, feature details information corresponding to a count of each command in the feature, command details information corresponding to a count of each command broken down by command properties, noise details information corresponding to exact match occurrence and noise click count, feature path details information corresponding to a number of paths of the feature, including total occurrence and commands that make up the path, and feature length distribution information.

4. The method of claim 1 wherein determining the feature usage metrics comprises determining cross-feature usage for a set of users, a specified set of one or more features, and a specified set of one or more programs.

5. The method of claim 4 wherein determining the cross-feature usage comprises determining information corresponding to at least one of:

user count information corresponding to a number of users who use all or a specified set of the specified features, program usage information corresponding to a number of users who use all the specified programs, feature usage corresponding to occurrence of the feature by users who use all or a specified set of the specified features, feature user count corresponding to a number of users who use the feature, program user count corresponding to a number of users who use the application, and occurrence information corresponding to a count of occurrence of the feature by feature users.

6. The method of claim 1 wherein determining the feature usage metrics comprises determining feature user characteristics for a set of users, including determining at least one of:

session count information corresponding to a number of application sessions by the users, session frequency information corresponding to a time between consecutive sessions by users, running time information corresponding to a session time by users, session length information corresponding to session time by users, depth of usage information corresponding to a percentage of total commands of the application used by users.

7. The method of claim 1 wherein determining the feature usage metrics comprises performing a session analysis for a specified set of one or more features, comprising determining at least one of:

session count information corresponding to a number of program sessions by users, occurrence information corresponding to a number of sessions in which the specified feature or features occurred in the session, percentage of sessions information corresponding to a percentage of application sessions in which the specified feature or features occurred, user count information corresponding to a total number of program users, user same session information corresponding to a number of users who used the specified feature or features in the same session, percentage of users information corresponding to a percentage of program users who use the specified feature or features in the same session, frequency of sessions by users information, and distribution information corresponding to a number and percentage of users who use specified feature or features in the same session whose frequency of such sessions falls within each interval.

8. The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least one feature usage trend over time.

9. The method of claim 8 wherein analyzing the information corresponding to the software instrumentation data to determine at least one feature usage trend over time comprises determining information corresponding to at least one of:

feature user count information corresponding to a number of users using a feature during a reporting interval, feature session count information corresponding to a number of sessions in which a feature occurred during a reporting interval, feature occurrence information corresponding to a number of times a feature occurred during a reporting interval, feature occurrence and user information corresponding to the feature occurrence divided by the feature user count during a reporting interval, feature occurrence and session information corresponding to the feature occurrence divided by the feature session count during a reporting interval, program count information corresponding to a number of users using a program with the feature during a reporting interval, percentage of users information corresponding to the feature user count divided by the program count during a reporting interval, program session count information corresponding to a number of sessions of a program with the feature during a reporting interval, percentage of sessions information corresponding to the feature session count divided by the program session count during a reporting interval, cumulative feature occurrence information corresponding to a number of times a feature occurred from a start time of a period to analyze to an end of each reporting interval, cumulative percentage of users information corresponding to a percentage of feature users from a start time of a period to analyze to an end of each reporting interval, and cumulative percentage of sessions information corresponding to a percentage of feature sessions from a start time of a period to analyze to an end of each reporting interval.

10. The method of claim 1, wherein analyzing the information corresponding to the software instrumentation data further comprises determining at least one potential outlier corresponding to feature usage that appears different from feature usage by other users.

11. A computer-readable medium having computer executable instructions, which when executed, perform operations comprising:

analyzing information related to software instrumentation data collected during program usage sessions of a program including one or more features, analyzing information including determining feature usage metrics associated with the usage of one or more features by at least some users during at least some of the program usage sessions, wherein the feature usage metrics include feature usage, cross-feature usage, and feature usage trends; and

upon determining feature usage metrics, outputting information representative of the determined feature usage metrics associated with the usage of one or more features by at least some users during at least some of the program usage sessions of the analysis.

12. The computer-readable medium of claim 11 wherein analyzing the information comprises performing one or more of:

a feature usage analysis, a cross-feature usage analysis, an analysis of characteristics of feature users, and a session analysis.

13. The computer-readable medium of claim 11 wherein outputting information representative of the determined feature usage metrics associated with the feature of the program comprises outputting a report to a user interface.

14. The computer-readable medium of claim 11 having further computer-executable instructions which when executed, perform operations comprising, receiving analysis parameters via a user interface, including at least one of receiving a selection of at least one feature and a user source.

15. In a computing environment, a system comprising:

an analyzer that processes information corresponding to software instrumentation data recorded from user software program usage sessions, to produce an output, wherein the information includes information corresponding to one or more features of one or more software programs

wherein the analyzer includes

means for performing a feature usage analysis, a cross-feature usage analysis, and a feature usage trend analysis on the information corresponding to one or more features of one or more software programs, and

means for outputting information representative of the feature usage analysis, the cross-feature usage analysis, and the feature usage trend analysis corresponding to one or more features of one or more software programs.

16. The system of claim 15 further comprising a user interface coupled to the analyzer for facilitating selection of one or more features.

17. The system of claim 16 wherein the user interface further facilitates selection of a group of users.

18. The system of claim 15 further comprising a user interface coupled to the analyzer for outputting results corresponding to the analysis.

19. The system of claim 15 further comprising means for filtering at least some of the information corresponding to the software instrumentation data.

20. The system of claim 15 wherein the analyzer is coupled to a recognition process that recognizes features from the information corresponding to the software instrumentation data.