Service metric analysis from structured logging schema of usage data

• US 10,289,470 B2
• Filed: 11/18/2016
• Issued: 05/14/2019
• Est. Priority Date: 02/26/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ranganathan, Aravind, Gite, Sanghmitra
• Primary Examiner(s): Guyton, Philip
• Application Number:
  US15/355,420
• Publication Number:
  US 20170123880A1
• Time in Patent Office: 907 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors

Abstract

Technologies are generally described to provide a passive monitoring system employing a logging schema to track usage data in order to analyze performance and reliability of a service. The logging schema may be configured to track user requests as each request is received and processed at individual subsystems of the collaborative service. A logging entry may be created at a data store of the service, where the logging entry includes a subsystem name, an operation performed by the subsystem to fulfill the request, and start and end times of the operation. The logging schema nay also detect errors fulfilling the requests, and may classify detected errors into a bucket, where each bucket denotes a failure scenario. Reliability of the service may be calculated based on analysis of the buckets to compute error rates. Reports may be generated to enable continuous monitoring of a performance and reliability of the system.

Claims

1. A method to monitor request errors at a service, the method comprising:

detecting a request received by the service;

creating a logging entry for the request, wherein creating the logging entry for the request comprises for each subsystem to process the request, recording one or more of;

a subsystem name,

a sub-operation name,

an entry and an exit time from the subsystem,

a start time and an end time for the sub-operation performed by the subsystem to fulfill the request,

user locale information for the request, or

error information associated with processing of the request at the subsystem;

if the request is fulfilled by the service, logging an entry in a success bucket;

if an error is detected in processing the request at the service;

recording the detected error with the logging entry for the request;

determining an error type of the detected error;

classifying the detected error into a new error bucket if the detected error is unique from prior detected errors;

classifying the detected error into a pre-existing error bucket if the error is known;

and

generating a report on a reliability of the service based on the success bucket and one or more error buckets.

2. The method of claim 1, wherein generating the report on the reliability of the service based on the success bucket and the one or more error buckets comprises:

determining the reliability of the service based on a ratio of the success bucket to a sum of the pre-existing error bucket and the new error bucket.

3. The method of claim 1, wherein generating the report on the reliability of the service further comprises:

including raw data for one or more detected errors associated with the service.

4. The method of claim 1, further comprising:

providing a user interface to customize a logging schema according to which entries for requests and errors are logged.

5. The method of claim 4, further comprising:

receiving definitions of one or more subsystems of the service to be monitored, a monitoring frequency, and a type of data to be monitored.

6. The method of claim 1, further comprising:

recording the detected error in real-time before the request is processed and a response is returned.

7. The method of claim 1, further comprising:

monitoring sub-operations within each subsystem of the service that processes the request in order to detect a success or an error.

8. The method of claim 1, further comprising:

determining a processing time of each subsystem to provide a response to the request from a recorded start time and a recorded end time; and

using the processing time to determine service reliability.

9. The method of claim 1, further comprising providing an alert based on one or more of a number of detected errors exceeding a predefined threshold value and the detected error being a new error.

10. A server to monitor request errors at a service, the server comprising:

a memory;

a processor coupled to the memory, the processor executing an application, wherein the application is configured to;

detect a request received by the service;

create a logging entry for the request;

if the request is fulfilled by the service, log an entry in a success bucket;

if an error is detected in processing the request at the service;

record the detected error with the logging entry for the request;

determine an error type of the detected error;

classify the detected error into a new error bucket if the detected error is unique from prior detected errors;

classify the detected error into a pre-existing error bucket if the error is known;

and

generate a report on a reliability of the service based on the success bucket and one or more error buckets, wherein the report includes one or more of;

a list of subsystems experiencing least usage, a list of subsystems experiencing most usage, an observation about a nature of user interaction with the service, or a pattern of operations performed by users under different scenarios.

11. The server of claim 10, wherein the application is configured to monitor processing of requests and log errors at a subsystem level.

12. The server of claim 11, wherein subsystems of the service are grouped under one or more of a front end, a middle layer, or a back end.

13. The server of claim 11, wherein the request is processed across multiple subsystems of the service sequentially or in parallel.

14. The server of claim 10, wherein the logging entry is stored in a data store and the data store includes a data store interface configured to receive usage data from multiple cloud-based services as part of a distributed system.

15. The server of claim 10, wherein the service is a collaboration service configured to facilitate one or more of:

a communication exchange, a document sharing, an enterprise management, a document management, a file management, a collaboration, a social networking contacts management, a calendar management, a data sharing, or an application sharing.

16. The server of claim 10, wherein the request is one or more of:

initiating an application, opening a document, initiating a conversation, interacting with a document or application, or retrieving data associated with an application.

17. The server of claim 10, wherein the application is further configured to provide an alert based on one or more of a number of detected errors exceeding a predefined threshold value and the detected error being a new error.

18. A method to provide monitoring of errors to one or more cloud-based services, the method comprising:

detecting a request received by the one or more cloud-based services;

creating a logging entry for the request;

if the request is fulfilled by the one or more cloud-based services, logging an entry in a success bucket;

if an error is detected in processing the request at the one or more cloud-based services;

recording the detected error with the logging entry for the request;

determining an error type of the detected error;

classifying the detected error into a new error bucket if the detected error is unique from prior detected errors;

classifying the detected error into a pre-existing error bucket if the error is known;

classifying the detected error independent of usage data localization; and

anonymizing user information for protection of user privacy; and

generating a report on a performance and a reliability of the one or more cloud-based services based on an analysis of the processing of the request.

19. The method of claim 18, further comprising:

generating the report based on a customizable time period and based on default and customizable parameters.

20. The method of claim 18, wherein the report on the performance and the reliability of the one or more cloud-based services is generated based in part on a ratio of the success bucket to one or more error buckets.