Determining a likelihood of a resource experiencing a problem based on telemetry data

• US 10,503,580 B2
• Filed: 06/15/2017
• Issued: 12/10/2019
• Est. Priority Date: 06/15/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Musuvathi, Madanlal S., Mytkowicz, Todd D., Maleki, Saeed, Ding, Yufei
• Primary Examiner(s): Amoroso, Anthony J
• Application Number:
  US15/624,660
• Publication Number:
  US 20180365093A1
• Time in Patent Office: 908 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0769 Readable error formats, e.g. cross-platform generic formats, human understandable formats
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 43/0817 by checking functioning

Abstract

Described herein is a system that transmits and combines local models, that individually comprise a set of local parameters computed via stochastic gradient descent (SGD), into a global model that comprises a set of global model parameters. The local models are computed in parallel at different geographic locations along with symbolic representations. Network transmission of the local models and the symbolic representations, rather than transmission of the large training data subsets processed to compute the local models and symbolic representations, conserves resources and decreases latency. The global model can then be used as a model to determine a likelihood of a monitored resource or a user of the monitored resource experiencing a problem with respect to performance or completion of one or more operations. The system can also implement an action to assist in resolving or avoiding the problem.

Claims

1. A method comprising:

defining a feature set, an individual feature in the feature set being related to telemetry information associated with a monitored resource, the monitored resource comprising at least one of an application, a device, or a network of devices;

receiving, via a network at a first processing node of first computing infrastructure and from a plurality of other processing nodes of a plurality of other computing infrastructures, a plurality of local models that individually comprise a set of local model parameters computed via stochastic gradient descent (SGD) based at least in part on a training data subset that includes multiple data instances of the feature set and, for each data instance of the feature set, a label indicating whether the monitored resource or a user of the monitored resource experiences a problem with respect to performance or completion of one or more operations or tasks, wherein the plurality of local models and the sets of local model parameters comprised therein are computed in parallel by the plurality of other processing nodes based at least in part on a set of starting model parameters;

receiving, at the first processing node and from the plurality of other processing nodes, a plurality of symbolic representations associated with the plurality of local models, wherein an individual symbolic representation associated with an individual local model is computed to represent how an adjustment to the set of starting model parameters affects the set of local model parameters computed for the individual local model by shifting the set of starting model parameters to a known set of starting model parameters associated with an output of another local model;

combining, at the first processing node using the plurality of symbolic representations to honor sequential dependencies of SGD, the plurality of local models received from the plurality of other processing nodes with a local model computed at the first processing node, the combining generating a global model that includes a set of global model parameters, the global model configured to determine, given a new data instance of the feature set, a likelihood of another monitored resource or another user of the other monitored resource experiencing the problem;

generating, at the first processing node, the new data instance of the feature set based on new telemetry data associated with the other monitored resource; and

determining, using the global model and the new data instance of the feature set, the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem.

2. The method of claim 1, further comprising:

determining that the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem exceeds a threshold; and

implementing an action to assist in resolving or avoiding the problem based at least in part on the determining that the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem exceeds the threshold.

3. The method of claim 2, wherein the action comprises issuing a notification to the other monitored resource that recommends instructions to resolve or to avoid the problem.

4. The method of claim 2, wherein the action comprises one or more of:

activating backup or overflow resources;

or

reducing a data traffic load being handled by the monitored resource.

5. The method of claim 1, wherein the feature set comprises one or more of:

a length of time it takes to perform a task or an operation, a number of times execution of a task or an operation is attempted, or a number of times execution of a task or an operation fails.

6. The method of claim 1, wherein the first processing node and the plurality of other processing nodes are configured in different datacenters operating in different geographic locations.

7. The method of claim 1, wherein the set of global model parameters are essentially the same as a corresponding set of model parameters that would have been computed had the local model and the plurality of local models and the corresponding training data subsets been computed sequentially in an order in which the plurality of symbolic representations were applied rather than in parallel.

8. The method of claim 1, wherein the adjustment is an unknown adjustment at a time the individual symbolic representation is computed.

9. The method of claim 8, wherein the output comprises the set of local model parameters computed for the other local model.

10. The method of claim 1, wherein the individual symbolic representation comprises a matrix.

11. The method of claim 10, wherein a dimension of the matrix has been reduced by projecting the matrix from a first dimensional space to a second dimensional space of smaller dimension.

12. The method of claim 11, wherein the second dimensional space is generated on random bases.

13. The method of claim 11, wherein the plurality of local models and the plurality of symbolic representations are received based at least in part on a schedule that ensures that a variance associated with projecting the matrix from the first dimensional space to the second dimensional space is less than a threshold variance.

14. The method of claim 11, wherein reducing the dimension of the matrix comprises removal of an identity matrix from the matrix, the identity matrix comprising a diagonal entry.

15. A system comprising:

one or more processing units; and

a computer-readable medium having encoded thereon computer-executable instructions to cause the one or more processing units to;

receive, via a network at a first processing node and from a plurality of other processing nodes, a plurality of local models that individually comprise a set of local model parameters computed via stochastic gradient descent (SGD) based at least in part on a training data subset that includes multiple data instances of a feature set and, for each data instance of the feature set, a label indicating whether the monitored resource or a user of the monitored resource experiences a problem with respect to performance or completion of one or more operations or tasks, wherein the plurality of local models and the sets of local model parameters comprised therein are computed in parallel by the plurality of other processing nodes based at least in part on a set of starting model parameters;

receive, at the first processing node and from the plurality of other processing nodes, a plurality of symbolic representations associated with the plurality of local models, wherein an individual symbolic representation associated with an individual local model is computed to represent how an adjustment to the set of starting model parameters affects the set of local model parameters computed for the individual local model by shifting the set of starting model parameters to a known set of starting model parameters associated with an output of another local model;

combine, at the first processing node using the plurality of symbolic representations to honor sequential dependencies of SGD, the plurality of local models received from the plurality of other processing nodes with a local model computed at the first processing node, the combining generating a global model that includes a set of global model parameters, the global model configured to determine, given a new data instance of the feature set, a likelihood of another monitored resource or another user of the other monitored resource experiencing the problem;

generate the new data instance of the feature set based on new telemetry data associated with the other monitored resource; and

determine, using the global model and the new data instance of the feature set, the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem.

16. The system of claim 15, wherein the computer-executable instructions further cause the one or more processing units to:

determine that the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem exceeds a threshold; and

implement an action to assist in resolving or avoiding the problem based at least in part on the determining that the likelihood of the other monitored resource or the other user of the other monitored resource experiencing the problem exceeds the threshold.

17. The system of claim 16, wherein the action comprises one or more of:

issuing a notification to the other monitored resource that recommends instructions to resolve or to avoid the problem;

activating backup or overflow resources;

or

reducing a data traffic load being handled by the monitored resource.

18. The system of claim 15, wherein the output comprises the set of local model parameters computed for the other local model.

19. The system of claim 15, wherein the individual symbolic representation comprises a matrix, wherein a dimension of the matrix has been reduced by projecting the matrix from a first dimensional space to a second dimensional space of smaller dimension, wherein reducing the dimension of the matrix comprises removal of an identity matrix from the matrix, the identity matrix comprising a diagonal entry.

20. A system comprising:

one or more processing units; and

a computer-readable medium having encoded thereon computer-executable instructions to cause the one or more processing units to;

compute a local model that comprises a set of local model parameters computed via stochastic gradient descent (SGD) based at least in part on a training data subset that includes multiple data instances of a feature set and, for each data instance of the feature set, a label indicating whether the monitored resource or a user of the monitored resource experiences a problem with respect to performance or completion of one or more operations or tasks;

compute a symbolic representation associated with the local model, wherein the symbolic representation comprises a matrix that represents how an adjustment to a set of starting model parameters affects the set of local model parameters computed for the local model by shifting the set of starting model parameters to a known set of starting model parameters associated with an output of another local model;

reduce a size of the matrix by projecting the matrix from a first dimensional space to a second dimensional space of smaller dimension; and

transmit the local model and the symbolic representation to processing nodes over a network to enable a global model to be generated, the global model useable to determine, given a new data instance of the feature set, a likelihood of another monitored resource or another user of the other monitored resource experiencing the problem.