Privacy-aware model generation for hybrid machine learning systems

• US 10,536,344 B2
• Filed: 06/04/2018
• Issued: 01/14/2020
• Est. Priority Date: 06/04/2018
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Mermoud, Gregory, Vasseur, Jean-Philippe, Di Pietro, Andrea, Zerhouni, Erwan Barry Tarik
• Primary Examiner(s): Won, Michael
• Assistant Examiner(s): Raza, Muhammad
• Application Number:
  US15/996,645
• Publication Number:
  US 20190372859A1
• Time in Patent Office: 589 Days
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 41/28 Restricting access to network management systems or functions, e.g. using authorisation function to access network configuration
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0817 by checking functioning

Abstract

In one embodiment, a network assurance service executing in a local network clusters measurements obtained from the local network regarding a plurality of devices in the local network into measurement clusters. The network assurance service computes aggregated metrics for each of the measurement clusters. The network assurance service sends a machine learning model computation request to a remote service outside of the local network that includes the aggregated metrics for each of the measurement clusters. The remote service uses the aggregated metrics to train a machine learning-based model to analyze the local network. The network assurance service receives the trained machine learning-based model to analyze performance of the local network. The network assurance service uses the receive machine learning-based model to analyze performance of the local network.

Claims

1. A method comprising:

clustering, by a network assurance service executing in a local network, measurements obtained from the local network regarding a plurality of devices in the local network into measurement clusters;

computing, by the network assurance service, aggregated metrics for each of the measurement clusters, wherein the computing includes;

anonymizing the measurement clusters by adding noise to the measurement clusters, and

computing the aggregated metrics for the anonymized measurement clusters;

sending, by the network assurance service, a machine learning model computation request to a remote service outside of the local network that includes the aggregated metrics for each of the measurement clusters, wherein the remote service uses the aggregated metrics to train a machine learning-based model to analyze the local network, wherein the remote service uses the aggregated metrics to train a machine learning-based model to analyze the local network by at least;

selecting measurement clusters computed from measurements associated with devices in one or more other networks that have similar aggregated metrics to the aggregated metrics from the model computation request, and

forming a synthetic training dataset for the model by combining the aggregated metrics from the model computation request with the aggregated metrics for the selected measurement clusters;

receiving, at the network assurance service, the trained machine learning-based model to analyze performance of the local network; and

using, by the network assurance service, the receive machine learning-based model to analyze performance of the local network.

2. The method as in claim 1, wherein clustering the measurements regarding the plurality of devices in the network into measurement comprises:

forming, for each of the devices, a measurement vector of discrete measurements associated with the device; and

applying a clustering process to the vectors, to group the measurement vectors into the measurement clusters.

3. The method as in claim 1, wherein the plurality of devices in the local network comprise one or more of:

wireless access points, wireless access point controllers, or wireless client devices.

4. The method as in claim 1, wherein the machine learning model computation request indicates relative sizes of the measurement clusters.

5. The method as in claim 1, wherein the measurements comprise one or more statistics regarding traffic in the local network, client counts in the local network, or application usages in the local network.

6. A method comprising:

receiving, at a remote service outside of a local network, a machine learning model computation request from a network assurance service executing in the local network, wherein the request includes aggregated metrics for clusters of measurements obtained by the network assurance service from the local network regarding a plurality of devices in the local network, wherein the measurement clusters are anonymized by adding noise to the measurement clusters and the aggregated metrics are computed for the anonymized measurement clusters;

selecting, by the remote service, measurement clusters for use in the training dataset from among the set of measurement clusters that have aggregated metrics similar to the aggregated metrics in the model computation request;

forming, by the remote service, a synthetic training dataset for the model by combining the aggregated metrics from the model computation request with measurements associated with devices in one or more other networks, wherein forming the synthetic training dataset further comprises clustering the measurements associated with devices in the one or more other networks into a set of measurement clusters;

training, by the remote service, the machine learning-based model using the synthetic training dataset; and

sending, by the remote service, the trained machine learning-based model to the network assurance service, wherein the network assurance service uses the model to analyze performance of the local network.

7. The method as in claim 6, wherein forming the synthetic training dataset comprises:

sampling the measurements associated with the devices in the one or more other networks using a kernel density estimator, based on the aggregated metrics from the model computation request.

8. The method as in claim 6, wherein the plurality of devices in the local network comprise one or more of:

wireless access points, wireless access point controllers, or wireless client devices.

9. The method as in claim 6, wherein the measurements obtained from the local network regarding the plurality of devices in the local network comprise one or more statistics regarding traffic in the local network, client counts in the local network, or application usages in the local network.

10. The method as in claim 6, further comprising:

adding, by the remote service, noise to the aggregated metrics from the model computation request, prior to forming the synthetic training dataset.

11. An apparatus, comprising:

one or more network interfaces;

a processor coupled to the network interfaces and configured to execute one or more processes; and

a memory configured to store a process executable by the processor in a local network, the process when executed configured to;

cluster measurements obtained from a local network regarding a plurality of devices in the local network into measurement clusters;

compute aggregated metrics for each of the measurement clusters by at least;

anonymizing the measurement clusters by adding noise to the measurement clusters, and

computing the aggregated metrics for the anonymized measurement clusters;

send a machine learning model computation request to a remote service outside of the local network that includes the aggregated metrics for each of the measurement clusters, wherein the remote service uses the aggregated metrics to train a machine learning-based model to analyze the local network, wherein the remote service uses the aggregated metrics to train a machine learning-based model to analyze the local network by;

selecting measurement clusters computed from measurements associated with devices in one or more other networks that have similar aggregated metrics to the aggregated metrics from the model computation request, and

forming a synthetic training dataset for the model by combining the aggregated metrics from the model computation request with the aggregated metrics for the selected measurement clusters;

receive the trained machine learning-based model to analyze performance of the local network; and

use the receive machine learning-based model to analyze performance of the local network.

12. The apparatus as in claim 11, wherein the apparatus clusters the measurements regarding the plurality of devices in the network into measurement by:

forming, for each of the devices, a measurement vector of discrete measurements associated with the device; and

applying a clustering process to the vectors, to group the measurement vectors into the measurement clusters.

13. The apparatus as in claim 11, wherein the plurality of devices in the local network comprise one or more of:

wireless access points, wireless access point controllers, or wireless client devices.

14. The apparatus as in claim 11, wherein the machine learning model computation request indicates relative sizes of the measurement clusters.

15. The apparatus as in claim 11, wherein the measurements comprise one or more statistics regarding traffic in the local network, client counts in the local network, or application usages in the local network.