Service metric analysis from structured logging schema of usage data

• US 9,354,963 B2
• Filed: 02/26/2014
• Issued: 05/31/2016
• Est. Priority Date: 02/26/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ranganathan, Aravind, Gite, Sanghmitra
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US14/191,058
• Publication Number:
  US 20150242262A1
• Time in Patent Office: 825 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors

Abstract

Technologies are generally described to provide a passive monitoring system employing a logging schema to track usage data in order to analyze performance and reliability of a service. The logging schema may be configured to track user requests as each request is received and processed at individual subsystems of the collaborative service. A logging entry may be created at a data store of the service, where the logging entry includes a subsystem name, an operation performed by the subsystem to fulfill the request, and start and end times of the operation. The logging schema may also detect errors fulfilling the requests, and may classify detected errors into a bucket, where each bucket denotes a failure scenario. Reliability of the service may be calculated based on analysis of the buckets to compute error rates. Reports may be generated to enable continuous monitoring of a performance and reliability of the system.

Claims

1. A method executed at least in part in a computing device to provide a passive monitoring system employing, a logging schema to track requests at a service, the method comprising:

detecting a request received by the service;

creating a logging entry for the request at a data store associated with the service;

detecting an error in processing the request at the service;

recording the detected error with the logging entry for the request;

classifying the detected error into an error bucket, wherein the error bucket denotes a failure scenario; and

determining a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

2. The method of claim 1, further comprising:

identifying a subsystem receiving the request;

identifying an operation performed by the subsystem to fulfill the request; and

identifying a start time and an end time for the operation performed to fulfill the request.

3. The method of claim 2, further comprising:

including the subsystem, the operation, the start time and the end time for the operation performed to fulfill request with the logging entry at the data store.

4. The method of claim 1, further comprising:

recording an error description for the detected error with the logging entry at the data store.

5. The method of claim 1, further comprising:

detecting whether the request is fulfilled; and

if the request is fulfilled, classifying the request as a success in a success bucket.

6. The method of claim 1, further comprising:

generating a report including the determined reliability of one or more of;

the service, one or more features, and one or more subsystems of the service.

7. The method of claim 6, wherein the report includes an analysis of a performance of the service.

8. The method of claim 6, wherein the report includes raw data for one or more detected errors associated with the service.

9. The method of claim 1, further comprising:

if a detected error does not belong to a pre-existing error bucket, creating a new error bucket for the detected error.

10. The method of claim 1, further comprising:

generating an alert if a number of detected errors exceeds a predefined threshold value.

11. A computing device to provide a passive monitoring system employing a logging schema to track requests at a service, the computing device comprising:

a memory;

a processor coupled to the memory, the processor executing a logging application, wherein the logging application is configured to;

detect a request received by the service;

create a logging entry for the request at a data store associated with the service;

determine whether the request is fulfilled by the service;

if the request is fulfilled, classify the request as a success in a success bucket;

if the request is not fulfilled, classify the request as a detected error in an error bucket, wherein the error bucket denotes a defined failure scenario; and

determine a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

12. The computing device of claim 11, wherein the service is a collaboration service facilitating one or more of:

a communication exchange, a document sharing, an enterprise management, a document management, a file management, a collaboration, a social networking contacts management, a calendar management, a data sharing, and an application sharing.

13. The computing device of claim 11, wherein the request is one or more of:

initiating an application, opening a document, initiating a conversation, interacting with a document or application, and retrieving data associated with an application.

14. The computing device of claim 11, wherein the logging application is further configured to:

record an error description for the detected error with the logging entry at the data store, wherein the error description includes an internal error code and an error type.

15. The computing device of claim 14, wherein the logging application is configured to:

generate a report based on the determined reliability and a performance of the service.

16. A computer-readable memory device with instructions stored thereon to provide a passive monitoring system employing a loping schema to track requests at a service, the instructions comprising:

detecting a request received by the service;

creating a logging entry for the request at a data store associated with the service;

determining whether the request is fulfilled by the service;

if the request is fulfilled, classifying the request as a success in a success bucket;

if the request is not fulfilled, classifying the request as a detected error in an error bucket, wherein the error bucket denotes a defined failure scenario; and

determining a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

17. The computer-readable memory device of claim 16, wherein the instructions further comprise:

employing a bug tracking component to continuously examine one or more error buckets to detect a bug;

automatically repairing a detected bug; and

providing an alert to indicate a presence of the detected bug.

18. The computer-readable memory device of claim 17, wherein the instructions include:

enabling an administrator of the service to customize the logging schema to define one or more subsystems to monitor, a monitoring frequency, and a type of data to be monitored.