METHOD FOR PREVENTING THE EXTRACTION OF A MACHINE LEARNING MODEL

• US 20200134391A1
• Filed: 10/24/2018
• Published: 04/30/2020
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: NXP B.V. (NXP Semiconductors NV)
• Original Assignee: NXP B.V. (NXP Semiconductors NV)
• Inventors: ASSADERAGHI, FARIBORZ, JOYE, MARC
• Granted Patent: US 11,586,860 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

A method and data processing system for detecting tampering of a machine learning model is provided. The method includes training a machine learning model. During a training operating period, a plurality of input values is provided to the machine learning model. In response to a predetermined invalid input value, the machine learning model is trained that a predetermined output value will be expected. The model is verified that it has not been tampered with by inputting the predetermined invalid input value during an inference operating period. If the expected output value is provided by the machine learning model in response to the predetermined input value, then the machine learning model has not been tampered with. If the expected output value is not provided, then the machine learning model has been tampered with. The method may be implemented using the data processing system.

Claims

1. A method comprising:

during a training phase of operation, training a machine learning model using first training data having a first classification;

during the training phase of operation, training the machine learning model using second training data having a second classification, the second classification being different than the first classification;

determining, during an inference phase of operation of the machine learning model, if an input sample belongs to the first classification or to the second classification;

if the input sample is determined to belong to the second classification, the machine learning model outputting a notification; and

if the input sample is determined to belong to the first classification, the machine learning model predicting a property of the input sample.

2. The method of claim 1, wherein the input sample belongs to the first classification if the input sample has a predetermined property, and the input sample belongs to the second classification if the input sample lacks the predetermined property.

3. The method of claim 1, wherein the second classification is an inlier/outlier classification.

4. The method of claim 1, wherein the machine learning model is characterized as being an ensemble of machine learning models, and further comprises a first machine learning model and a second machine learning model, and wherein determining if an input sample belongs in the first classification or the second classification further comprises:

during the training phase, training the first machine learning model using the first training data;

during the training phase, training the second machine learning model using the second training data, wherein the second classification is an inlier/outlier classification;

during the inference phase, providing the input sample to the second machine learning model, if the input sample is determined to be an outlier, providing a notification, and if the input sample is determined to be an inlier, providing the input sample for further computation by the first machine learning model.

5. The method of claim 4, further comprising providing a confidence level with the inlier/outlier determination.

6. The method of claim 1, wherein the notification comprises one of a randomly generated output value or an error notification.

7. The method of claim 1, wherein the first training data is characterized as being normal training data for training the machine learning model to perform a first task, and wherein the second training data is characterized as being abnormal training data that is for training the machine learning model to perform a second task, the second task being unrelated to the first task.

8. The method of claim 1, wherein the machine learning model is provided as a service and stored remotely from a user.

9. A method for protecting a machine learning model from extraction, the method comprising:

during a training phase of operation, training the machine learning model using normal training data, the normal training data for training the machine learning model to perform a predetermined task;

during the training phase of operation, training the machine learning model using abnormal training data, the abnormal training data for training the machine learning model to identify an attempted extraction of the machine learning model;

determining, during an inference phase of operation of the machine learning model, if an input sample is input to extract the machine learning model or if the input sample is input for performance of the predetermined task;

if the input sample is determined by the model to be the attempted extraction, the machine learning model outputting a notification, and

if the input sample is determined by the model to be related to the predetermined task, the machine learning model predicting a property of the input sample.

10. The method of claim 9, wherein determining if the input sample was input to extract the machine learning model further comprises determining if the input sample is within a predetermined distribution, and wherein if the input sample is outside of the predetermined distribution, the notification is output.

11. The method of claim 9, wherein the machine learning model is characterized as being an ensemble of machine learning models and further comprises a first machine learning model and a second machine learning model, the method further comprising:

during the training phase of operation, training the first machine learning model using the normal training data and training the second machine learning model using the abnormal training data;

during the inference phase, providing the input sample to the second machine learning model, if the input sample is determined to be within a predetermined distribution, providing the input sample for further computation by the first machine learning model, and if the input sample is outside the predetermined distribution, providing a notification.

12. The method of claim 9, further comprising providing a confidence level with the notification.

13. The method of claim 9, wherein the notification comprises one of a randomly generated output value or an error notification.

14. The method of claim 9, wherein the machine learning model is provided as a service and stored remotely from a user.

15. A method comprising:

providing a machine learning system having a first machine learning model and a second machine learning model,

during a training phase of operation of the machine learning system, training the first machine learning model using normal training data, the normal training data for training the machine learning model to perform a predetermined task;

during the training phase of operation, training the second machine learning model using abnormal training data, the abnormal training data for training the machine learning model to identify an attempted extraction of the machine learning model;

determining, during an inference phase of operation of the machine learning system, if an input sample inputted to the second machine learning model is inputted to extract the first machine learning model or if the input sample is inputted for performance of the predetermined task;

if the input sample is determined by the model to be inputted for the attempted extraction, the machine learning model outputting a notification, and

if the input sample is determined by the model to be related to the predetermined task, the machine learning model predicting a property of the input sample.

16. The method of claim 15, wherein determining if the input sample was inputted to attempt extraction of the first machine learning model further comprises determining if the input sample is inside or outside of a predetermined distribution, and wherein if the input sample is determined to be outside of the predetermined distribution, the notification is output.

17. The method of claim 15, further comprising providing a confidence level with the notification.

18. The method of claim 15, wherein the notification comprises one of a randomly generated output value or an error notification.

19. The method of claim 15, wherein the machine learning model is provided as a service and stored remotely from a user.

20. The method of claim 15, wherein the second machine learning model is characterized as being a neural network having a softmax activation function.