Anomaly detection and classification using telemetry data

• US 10,187,328 B2
• Filed: 04/20/2018
• Issued: 01/22/2019
• Est. Priority Date: 02/26/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Patil, Nagaraj, Nallabothula, Kiran, Barnes, Christopher, Palla, Nagaraju
• Primary Examiner(s): Duong, Oanh
• Application Number:
  US15/958,293
• Publication Number:
  US 20180241693A1
• Time in Patent Office: 277 Days
• CPC Class Codes: H04L 41/064 involving time analysis
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 47/827 Aggregation of resource allocation or reservation requests
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]

Abstract

Historical telemetry data can be used to generate predictions for various classes of data at various aggregates of a system that implements an online service. An anomaly detection process can then be utilized to detect anomalies for a class of data at a selected aggregate. An example anomaly detection process includes receiving telemetry data originating from a plurality of client devices, selecting a class of data from the telemetry data, converting the class of data to a set of metrics, aggregating the set of metrics according to a component of interest to obtain values of aggregated metrics over time for the component of interest, determining a prediction error by comparing the values of the aggregated metrics to a prediction, detecting an anomaly based at least in part on the prediction error, and transmitting an alert message of the anomaly to a receiving entity.

Claims

1. One or more computer-readable storage media storing instructions that, when executed by one or more processors, perform operations comprising:

receiving telemetry data originating from a plurality of client devices;

converting a class of data from the telemetry data to a set of metrics;

aggregating the set of metrics according to a component of interest to obtain values of aggregated metrics over time for the component of interest;

determining a prediction error by comparing the values of the aggregated metrics to a prediction, the prediction being based at least in part on historical telemetry data pertaining to the class of data and the component of interest;

detecting an anomaly based at least in part on the prediction error; and

transmitting an alert message of the anomaly.

2. The one or more computer-readable storage media of claim 1, wherein:

the component of interest comprises an internet service provider (ISP) that enables the plurality of client devices to access to an online service; and

the ISP is outside of a domain of a service provider of the online service.

3. The computer-readable storage media of claim 2, wherein the class of data corresponds to an error indicative of a problem with the ISP.

4. The computer-readable storage media of claim 3, wherein the alert message includes an identification of the error and contact information of the ISP.

5. The computer-readable storage media of claim 1, the operations further comprising generating the prediction by computing a Fast Fourier Transform (FFT) on historical values pertaining to the class of data accessed from the historical telemetry data.

6. A computer-implemented method comprising:

receiving telemetry data originating from a plurality of client devices, wherein the telemetry data was generated in response to the plurality of client devices accessing an online service provided by a service provider;

generating a set of metrics based on the telemetry data;

aggregating the set of metrics according to a component that is used to implement the online service;

obtaining values of aggregated metrics over time for the component based at least in part on aggregating the set of metrics;

determining a prediction error by comparing the values of the aggregated metrics to a prediction, the prediction being based at least in part on historical telemetry data pertaining to the component;

detecting an anomaly based at least in part on the prediction error; and

transmitting an alert message of the anomaly.

7. The computer-implemented method of claim 6, further comprising initiating an automatic recovery action with respect to the component.

8. The computer-implemented method of claim 6, further comprising determining that the component is an internet service provider (ISP) that is outside of a domain of the service provider.

9. The computer-implemented method of claim 6, wherein generating the set of metrics based on the telemetry data includes converting a class of data from the telemetry data to the set of metrics.

10. The computer-implemented method of claim 9, wherein aggregating the set of metrics comprises:

determining the ISP from an Internet Protocol (IP) address in the telemetry data, the IP address corresponding to a subset of data in the class of data; and

aggregating the set of metrics for the subset of data in the class of data that corresponds to the IP address.

11. The computer-implemented method of claim 9, wherein the class of data corresponds to an error indicative of a problem with the ISP.

12. The computer-implemented method of claim 9, further comprising generating the prediction by computing a Fast Fourier Transform (FFT) on historical values pertaining to the class of data accessed from the historical telemetry data.

13. A system comprising:

one or more processors;

memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving telemetry data originating from a plurality of client devices;

generating a set of metrics based on the telemetry data;

aggregating the set of metrics according to a component of interest to obtain values of aggregated metrics over time for the component of interest;

determining a prediction error by comparing the values of the aggregated metrics to a prediction, the prediction being based at least in part on historical telemetry data pertaining to the component of interest;

detecting an anomaly based at least in part on the prediction error; and

initiating an automatic recovery action with respect to the component of interest.

14. The system of claim 13, the operations further comprising determining that the component of interest is within a domain of a service provider of an online service.

15. The system of claim 14, wherein:

the component of interest comprises a server in a data center that implements the online service; and

the automatic recovery action comprises at least one of updating code of the server with a code patch, rebooting the server, or transitioning a plurality of user sessions from the server to a different server of the data center.

16. The system of claim 14, wherein:

the component of interest comprises the online service; and

the automatic recovery action comprises at least one of recycling the online service, or extending a subscription for a tenant of the online service.

17. The system of claim 14, wherein:

determining the prediction error comprises determining a difference between the prediction and the values of the aggregated metrics at a time of day; and

detecting the anomaly comprises determining that the difference is greater than a threshold difference.

18. The system of claim 14, wherein:

determining the prediction error comprises determining a series of differences between the prediction and the values of the aggregated metrics over a period of time; and

detecting the anomaly further comprises determining that the series of differences exceeds a threshold difference over the period of time.

19. The system of claim 14, wherein:

the telemetry data corresponds to users who access an online service; and

the set of metrics corresponds to counts of the users who access the online service.

20. The system of claim 14, the operations further comprising generating the prediction by computing a Fast Fourier Transform (FFT) based on the historical telemetry data.