Service metric analysis from structured logging schema of usage data

• US 9,503,335 B2
• Filed: 05/13/2016
• Issued: 11/22/2016
• Est. Priority Date: 02/26/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ranganathan, Aravind, Gite, Sanghmitra
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US15/153,738
• Publication Number:
  US 20160254969A1
• Time in Patent Office: 193 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors

Abstract

Technologies are generally described to provide a passive monitoring system employing a logging schema to track usage data in order to analyze performance and reliability of a service. The logging schema may be configured to track user requests as each request is received and processed at individual subsystems of the collaborative service. A logging entry may be created at a data store of the service, where the logging entry includes a subsystem name, an operation performed by the subsystem to fulfill the request, and start and end times of the operation. The logging schema may also detect errors fulfilling the requests, and may classify detected errors into a bucket, where each bucket denotes a failure scenario. Reliability of the service may be calculated based on analysis of the buckets to compute error rates. Reports may be generated to enable continuous monitoring of a performance and reliability of the system.

Claims

1. A method executed at least in part in a computing device to provide a passive monitoring system employing a logging schema to track requests at a service, the method comprising:

detecting a request received by the service from a user;

creating a logging entry for the request at a data store associated with the service;

in response to a determination that the request is not fulfilled, detecting an error in processing the request at the service;

recording the detected error with the logging entry for the request;

determining an error type of the detected error;

classifying the detected error into an error bucket by;

classifying the detected error into a pre-existing error bucket based on the error type;

else

in response to a determination that the error type is unique from prior detected errors, create a new error bucket and classify the detected error into the new error bucket;

providing an alert to the user if a number of detected errors exceeds a predefined threshold value; and

determining a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

2. The method of claim 1, further comprising:

identifying a subsystem of the service receiving the request;

identifying an operation performed by the subsystem to fulfill the request; and

identifying a start time and an end time for the operation performed to fulfill the request.

3. The method of claim 2, further comprising:

including the subsystem, the operation, the start time and the end time for the operation performed to fulfill request with the logging entry at the data store.

4. The method of claim 1, further comprising:

recording an error description for the detected error with the logging entry at the data store.

5. The method of claim 4, further comprising:

mapping an internal error code to the error description to provide a user friendly error description within an error message provided to the user.

6. The method of claim 5, further comprising:

localizing the error message such that for a same internal error code, a different error message is generated based on user localization.

7. The method of claim 1, further comprising:

in response to a determination that the request is fulfilled, classifying the request as a success in a success bucket.

8. The method of claim 1, wherein providing the alert to the user if the number of detected errors exceeds the predefined threshold value further comprises:

recognizing repeatedly detected errors of a same error type such that a same alert is not provided multiple times to the user for the same error type.

9. The method of claim 1, further comprising:

providing a troubleshooting message to the user to address the detected error.

10. The method of claim 1, further comprising:

providing healing scripts to resolve the detected error at the service.

11. The method of claim 1, further comprising:

generating a report including the determined reliability of one or more of;

the service, one or more features, and one or more subsystems of the service.

12. A computing device to provide a passive monitoring system employing a logging schema to track requests at a service, the computing device comprising:

a memory;

a processor coupled to the memory, the processor executing a logging application, wherein the logging application is configured to;

detect a request received by the service from a user;

create a logging entry for the request at a data store associated with the service;

determine whether the request is fulfilled by the service;

in response to a determination that the request is fulfilled, classify the request as a success in a success bucket; and

in response to a determination that the request is not fulfilled, classify the request as a detected error in one of a pre-existing error bucket and a newly created error bucket based on an error type of the detected error;

provide at least one of an alert to the user, a troubleshooting message to the user, and a healing script to the service, wherein the alert, the troubleshooting message, and healing script are associated with the detected error; and

determine a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

13. The computing device of claim 12, wherein the success bucket denotes a success scenario and the error bucket denotes a failure scenario.

14. The computing device of claim 12, wherein the request is one or more of:

initiating an application, opening a document, initiating a conversation, interacting with a document or application, and retrieving data associated with an application.

15. The computing device of claim 12, wherein the logging application is further configured to:

record an error description for the detected error with the logging entry at the data store, wherein the error description includes an internal error code, a subsystem of the service where the detected error occurred, and the error type.

16. The computing device of claim 15, wherein the error type includes a locale agnostic string to categorize the error type of the detected error for the request and an associated operation at the subsystem of the service where the detected error occurred.

17. The computing device of claim 12, wherein the logging application is configured to:

generate a report based on a determined reliability and performance of the service, wherein the report includes at least one of an analysis of a performance of the service and raw data for one or more detected errors associated with the service.

18. A method executed at least in part in a computing device to provide a passive monitoring system employing a logging schema to track requests at a service, the instructions comprising:

detecting a request received by the service from a user;

creating a logging entry for the request at a data store associated with the service;

determining whether the request is fulfilled by the service;

if the request is fulfilled, classifying the request as a success in a success bucket;

if the request is not fulfilled, classifying the request as a detected error in an error bucket by one of;

classifying the detected error into a pre-existing error bucket based on an error type;

else

in response to a determination that the error type is unique from prior detected errors, creating a new error bucket and classify the detected error into the new error bucket;

providing at least one of an alert to the user, a troubleshooting message to the user, and a healing script to the service, wherein the alert, the troubleshooting message, and healing script are associated with the detected error; and

determining a reliability of the service based on a percentage of success buckets and error buckets, wherein synthetic requests are distinguished from real user requests and synthetic request data is removed to provide an accurate determination of the reliability of the service.

19. The method of claim 18, wherein the instructions further comprise:

employing a bug tracking component to continuously examining one or more of the error buckets to detect a bug;

automatically repairing a detected bug; and

providing another alert to the user indicate a presence of the detected bug.

20. The method of claim 19, wherein the instructions include:

enabling an administrator of the service to customize the logging schema to define one or more subsystems to monitor, a monitoring frequency, and a type of data to be monitored.