Incremental machine learning for data loss prevention

• US 8,862,522 B1
• Filed: 12/14/2011
• Issued: 10/14/2014
• Est. Priority Date: 12/14/2010
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Jaiswal, Sumesh, DiCorpo, Phillip, Sawant, Shitalkumar S., Kauffman, Sally, Galindez, Alan Dale, Aggarwal, Ashish
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Misir, Dave
• Application Number:
  US13/326,194
• Publication Number:
  US 20140304197A1
• Time in Patent Office: 1,035 Days
• Field of Search: 706/12, 726/27
• US Class Current: 706/12
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2141 Access rights, e.g. capability lists, access control lists, access tables, access matrices
• CPC Class Codes: G06F 2221/2147 Locking files
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A computing device receives a document that was incorrectly classified as sensitive data based on a machine learning-based detection (MLD) profile. The computing device modifies a training data set that was used to generate the MLD profile by adding the document to the training data set as a negative example of sensitive data to generate a modified training data set. The computing device then analyzes the modified training data set using machine learning to generate an updated MLD profile.

Claims

1. A method comprising:

receiving a plurality of first documents that were incorrectly classified as sensitive data based on a machine learning-based detection (MLD) profile;

modifying a training data set that was used to generate the MLD profile by adding the first documents to the training data set as negative examples of sensitive data to generate a modified training data set;

determining that there are at least a threshold number of the first documents; and

analyzing, by a processing device, the modified training data set using machine learning to generate an updated MLD profile in response to determining that there are at least the threshold number of the first documents.

2. The method of claim 1, wherein the analyzing is performed on a periodic basis in accordance with an MLD profile retraining schedule.

3. The method of claim 1, further comprising:

assigning a first quality rating to the training data set and a second quality rating to the modified training data set;

determining that the first quality rating is lower than or equal to the second quality rating; and

analyzing the modified training data set in response to determining that the first quality rating is lower than or equal to the second quality rating.

4. The method of claim 1, further comprising:

assigning a first quality rating to the training data set and a second quality rating to the modified training data set;

determining that the second quality rating is higher than or equal to the first quality rating; and

deploying the updated MLD profile in response to determining that the second quality rating is higher than or equal to the first quality rating.

5. The method of claim 3, wherein the first quality rating and the second quality rating are assigned based on performing at least one of latent semantic indexing or k-fold cross validation.

6. The method of claim 1, further comprising:

receiving a second document that was incorrectly classified as non-sensitive data based on the MLD profile; and

modifying the training data set that was used to generate the MLD profile by adding the second document to the training data set as a positive example of sensitive data to generate the modified training data set.

7. A non-transitory computer-readable storage medium having instructions stored therein that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving a plurality of first documents that were incorrectly classified as sensitive data based on a machine learning-based detection (MLD) profile;

modifying a training data set that was used to generate the MLD profile by adding the first documents to the training data set as negative examples of sensitive data to generate a modified training data set;

determining that there are at least a threshold number of the first documents; and

analyzing, by the processing device, the modified training data set using machine learning to generate an updated MLD profile in response to determining that there are at least the threshold number of the first documents.

8. The non-transitory computer-readable storage medium of claim 7, wherein the analyzing is performed on a periodic basis in accordance with an MLD profile retraining schedule.

9. The non-transitory computer-readable storage medium of claim 7, wherein the operations further comprise:

assigning a first quality rating to the training data set and a second quality rating to the modified training data set;

determining that the first quality rating is lower than or equal to the second quality rating; and

analyzing the modified training data set in response to determining that the first quality rating is lower than or equal to the second quality rating.

10. The non-transitory computer-readable storage medium of claim 7, wherein the operations further comprise:

assigning a first quality rating to the training data set and a second quality rating to the modified training data set;

determining that the second quality rating is higher than or equal to the first quality rating; and

deploying the updated MLD profile in response to determining that the second quality rating is higher than or equal to the first quality rating.

11. The non-transitory computer-readable storage medium of claim 9, wherein the first quality rating and the second quality rating are assigned based on performing at least one of latent semantic indexing or k-fold cross validation.

12. The non-transitory computer-readable storage medium of claim 7, wherein the operations further comprise:

receiving a second document that was incorrectly classified as non-sensitive data based on the MLD profile; and

modifying the training data set that was used to generate the MLD profile by adding the second document to the training data set as a positive example of sensitive data to generate the modified training data set.

13. A system comprising:

a memory to store instructions; and

a processing device, coupled to the memory, to execute the instructions to;

receive a plurality of first documents that were incorrectly classified as sensitive data based on a machine learning-based detection (MLD) profile;

modify a training data set that was used to generate the MLD profile by adding the first documents to the training data set as negative examples of sensitive data to generate a modified training data set;

determine that there are at least a threshold number of the first documents and

analyze the modified training data set using machine learning to generate an updated MLD profile in response to the determination that there are at least the threshold number of the first documents.

14. The computing device of claim 13, wherein the processing device is further to:

assign a first quality rating to the training data set and a second quality rating to the modified training data set;

determine that the first quality rating is lower than or equal to the second quality rating; and

analyze the modified training data set in response to the determination that the first quality rating is lower than or equal to the second quality rating.

15. The system of claim 13, wherein the analyzing is performed on a periodic basis in accordance with an MLD profile retraining schedule.

16. The system of claim 13, wherein the processing device is further to:

assign a first quality rating to the training data set and a second quality rating to the modified training data set;

determine that the second quality rating is higher than or equal to the first quality rating; and

deploy the updated MLD profile in response to the determination that the second quality rating is higher than or equal to the first quality rating.

17. The system of claim 14, wherein the first quality rating and the second quality rating are assigned based on performing at least one of latent semantic indexing or k-fold cross validation.

18. The system of claim 13, wherein the processing device is further to:

receive a second document that was incorrectly classified as non-sensitive data based on the MLD profile; and

modify the training data set that was used to generate the MLD profile by adding the second document to the training data set as a positive example of sensitive data to generate the modified training data set.