Analyzing software usage with instrumentation data

• US 8,176,476 B2
• Filed: 06/15/2007
• Issued: 05/08/2012
• Est. Priority Date: 06/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Yantao, Mahmud, Adnan Azfar, Zhu, Wenli, Zhang, Haidong, Ye, Shuguang, Sun, Bing, Wang, Qiang, Dang, Yingnong, Liu, Guowei, Wang, Min, Wang, Jian
• Primary Examiner(s): Das, Chameli
• Application Number:
  US11/818,611
• Publication Number:
  US 20080313149A1
• Time in Patent Office: 1,789 Days
• Field of Search: None
• US Class Current: 717/130
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence

Abstract

Described is a technology by which software instrumentation data collected from user program sessions are analyzed to output an analysis report or the like via example methods and an architecture configured for efficient operation. A client component queries a service for analysis related information. To process the query, the service works with a data manager, and via a high dimensional analysis component may use information processed from the software instrumentation data, such as in the form of one or more inverted indexes and/or raw value files. The service may include a usage analysis component, a feature recognition component that locates features from command sequences, a user recognition component and/or a program reliability component. One or more counterpart components at the client may generate analysis reports or the like based on the query results. The client also may maintain user libraries and feature libraries to facilitate analyses.

Claims

1. A system comprising:

at least one processor;

a memory, communicatively coupled to the at least one processor and including computer executable components, comprising;

a service configured to analyze collected software instrumentation data and generate a result set using the collected software instrumentation data, the service including at least one analysis mechanism of a set of analysis mechanism, the set including at least one of;

a program usage analysis mechanism, a command usage analysis mechanism, a feature usage analysis mechanism, a user analysis mechanism, or a reliability analysis mechanism;

a data manager;

a data infrastructure component configured to interact with the service to access the collected software instrumentation data, the data infrastructure component including a data importer component configured to process at least some data corresponding to the collected software instrumentation data into an inverted index and a raw value file; and

an interface, wherein the interface is incorporated as part of an explorer component for querying the service for information corresponding to software quality metrics data related to the collected software instrumentation data from user sessions of one or more programs.

2. The system of claim 1 wherein the service includes a query receiving mechanism, a query queue, a result pool, and a scheduler that sends query data from the query pool to the data manager and receives query responses into the result pool for returning to the interface.

3. The system of claim 1 wherein the interface outputs information for human or machine review, or both human and machine review, the information corresponding to at least one query response to a report server or in a format for outputting via a spreadsheet program, or both to a report server and in a format for outputting via a spreadsheet program.

4. The system of claim 1 wherein the service includes a high dimensional analysis component which works in conjunction with the data manager to process the inverted index and the raw value file into query results based on a query received from the interface.

5. The system of claim 1 wherein the collected software instrumentation data includes command sequences collected from actual program usage, and wherein the service includes a feature recognition component that locates one or more specified features from information corresponding to at least some of the command sequences to provide feature-related results in response to a client component query.

6. The system of claim 5 wherein the interface includes a usage analysis component for processing the feature-related results to output a feature usage report.

7. The system of claim 1 wherein the collected software instrumentation data includes data related to users collected from actual program usage, and wherein the service includes a user recognition component that locates one or more groups of users from information corresponding to at least some of the users to provide user-related results in response to a client component query.

8. The system of claim 1 wherein the collected software instrumentation data includes data related to program usage or program commands, or both program usage and program commands, and wherein the service includes a usage recognition component that recognizes program usage or command usage, or both program usage and command usage, from information corresponding to at least some of the program usage or program commands data to provide program usage-related results in response to a client component query.

9. The system of claim 8 wherein the interface includes a usage analysis component for processing the program usage-related results to output a program usage report or a command usage report, or both a program usage report and a command usage report.

10. The system of claim 1 wherein the collected software instrumentation data includes data related to program reliability collected from actual program usage, and wherein the service includes a reliability component that analyzes program usage, command usage, or users, or any combination of program usage, command usage and users to provide reliability results in response to a client component query.

11. The system of claim 10 wherein the interface includes a reliability analysis component for processing the reliability results to output a reliability report.

12. The system of claim 1 wherein the interface includes a feature library for maintaining feature-related information, or a user library for maintaining user related information, or both a feature library for maintaining feature-related information and a user library for maintaining user related information.

13. The system of claim 1 further comprising means for collecting software instrumentation data.

14. At least one computer-readable storage medium storing computer-executable instructions, which in response to execution by a computer, cause the computer to perform steps comprising:

receiving a query directed towards analyzing information corresponding to at least some software instrumentation data collected from program usage sessions;

performing an analysis based at least in part on the software instrumentation data collected to obtain information corresponding to the query in the form of at least one inverted index and at least one raw value file;

processing the information obtained, by performing at least one of a filtering, grouping, aggregating or sorting operation; and

returning a result in response to the query.

15. The computer readable storage medium of claim 14 having further computer-executable instructions, which in response to execution by the computer, cause the computer to perform further steps comprising, queuing the query when received, dequeing the query for processing, and returning the result via a result pool.

16. A system comprising:

a client component having a user interface;

a service, implemented at least in part on at least one processing unit, coupled to the client component via a query and query response mechanism, the client component configured to query the service for results related to software program data based on software instrumentation data collected from user sessions of one or more programs, the client and the service configured to provide at least one of a program usage analysis, a command usage analysis, a feature usage analysis, a user analysis, or a reliability analysis, and wherein the service is further configured to generate a result set using the software instrumentation data collected and at least one of the program usage analysis, the command usage analysis, the feature usage analysis, the user analysis, or the reliability analysis;

a data manager; and

a data infrastructure component including a data importer component configured to process at least some data corresponding to the collected software instrumentation data into an inverted index and a raw value file.

17. The system of claim 16 wherein the collected software instrumentation data includes data related to program reliability collected from actual program usage, and wherein the service includes a reliability component that analyzes program usage, command usage, or users, or any combination of program usage, command usage and users to provide reliability results in response to a client component query.

18. The system of claim 16 wherein the client component includes a feature library for maintaining feature-related information, or a user library for maintaining user related information, or both a feature library for maintaining feature-related information and a user library for maintaining user related information.

19. The system of claim 16 wherein the service further includes a query receiving mechanism, a query queue, a result pool, and a scheduler that sends query data from the query pool to a data manager and receives query responses into the result pool for returning to the client component.

20. The system of claim 16 wherein the collected software instrumentation data includes data related to program usage or program commands, or both program usage and program commands, and wherein the service includes a usage recognition component that recognizes program usage or command usage, or both program usage and command usage, from information corresponding to at least some of the program usage or program commands data to provide program usage-related results in response to a client component query.