Strategies for sanitizing data items

• US 20060080554A1
• Filed: 10/09/2004
• Published: 04/13/2006
• Est. Priority Date: 10/09/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: De Barros, Marcelo M., McDonald, Orville C., Ke, Qi, Wang, Xiaoan, Bonilla, Richard K.
• Granted Patent: US 7,509,684 B2
• Time in Patent Office: Days
• US Class Current: 713/189
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification

Abstract

Strategies are described for sanitizing a data set, having the effect of obscuring restricted data in the data set to maintain its secrecy. The strategies operate by providing a production data set to a sanitizer. The sanitizer applies a data directory table to identify the location of restricted data items in the data set and to identify the respective sanitization tools to be applied to the restricted data items. The sanitizer then applies the identified sanitization tools to the identified restricted data items to produce a sanitized data set. A test environment receives the sanitized data set and performs testing, data mining, or some other application on the basis of the sanitized data set. Performing sanitization on a sanitized version of the production data set is advantageous because it preserves the state of the production data set. The data directory table also provides a flexible mechanism for applying sanitization tools to the production data set.

Claims

1. A method for sanitizing restricted data items in a data set to prevent the revelation of the restricted data items, comprising:

transferring an original data set from a production environment to a sanitizer, the original data set characterized by a state;

sanitizing the original data set using the sanitizer, while preserving the state of the original data set, comprising;

identifying the locations of the restricted data items in the original data set;

identifying at least one sanitizing tool to apply to the restricted data items which have been located in the original data set; and

applying said at least one sanitizing tool to the restricted data items which have been located in the original data set to provide a sanitized data set; and

forwarding the sanitized data set to a target environment.

2. The method of claim 1, wherein the transferring comprises:

receiving, by a facilitator, a request from a requestor to initiate the sanitization of the restricted data items, wherein the requestor does not have access to the original data set in the production environment, whereas the facilitator does have access to the original data set in the production environment; and

commanding, by the facilitator, the transfer of the original data set from the production environment to the sanitizer.

3. The method of claim 2, further comprising sending the requestor a notification upon the completion of the sanitizing of the original data set, wherein, upon receiving such notification, the requestor initiates the forwarding of the sanitized data set to the target environment.

4. The method of claim 1, wherein the target environment is a testing environment in which a tester applies a test to the sanitized data set.

5. The method of claim 1, wherein the target environment is a data mining environment in which an analyst applies mining analysis to the sanitized data set.

6. The method of claim 1, wherein the identifying of the locations of the restricted data items comprises using a data directory table to identify the locations of the restricted data items in the original data set.

7. The method of claim 6, further comprising generating the data directory table by:

identifying data items within the original data set;

classifying the data items as having restricted or non-restricted status; and

mapping the restricted data items to their respective locations within the original data set.

8. The method of claim 1, wherein the identifying of said at least one sanitizing tool comprises using a data directory table to identify said at least one sanitizing tool.

9. The method of claim 1, wherein said at least one sanitization tool obscures the restricted data items in a manner which preserves at least one statistical characteristic of the restricted data items.

10. The method of claim 1, wherein the sanitizing of the data set comprises a bulk sanitization operation, and further comprising, subsequent to the bulk sanitization operation, performing a delta sanitization operation to sanitize data items in the original data set which have changed subsequent to the bulk sanitization operation.

11. The method of claim 1, wherein the sanitizing of the original data set comprises an initial sanitization operation to produce the sanitized data set, and further comprising, subsequent to the initial sanitization operation, performing another sanitization operation on the sanitized data set to produce a re-sanitized data set.

12. The method of claim 1, wherein the sanitizing of the original data set comprises identifying a subset of the original data set, and performing the sanitizing on only on that subset of the original data set.

13. The method of claim 1, where the sanitizing of the original data set comprises producing the sanitized data set so that the sanitized data set has a reduced size relative to the original data set.

14. The method of claim 1, wherein the sanitizing of the original data set comprises producing the sanitized data so that the sanitized data set has an expanded size relative to the original data set.

15. One or more machine-readable media for implementing the method of claim 1.

16. A system for sanitizing restricted data items in a data set to prevent the revelation of the restricted data items, comprising:

a production environment which relies on a production data set to perform its allotted functions;

a sanitizer configured to receive an original data set based on the production data set and to sanitize the original data set by;

(a) identifying the locations of the restricted data items in the original data set;

(b) identifying at least one sanitizing tool to apply to the restricted data items which have been located in the original data set; and

(c) applying said at least one sanitizing tool to the restricted data items which have been located in the original data set to provide a sanitized data set; and

a target environment configured to receive the sanitized data set,

wherein the sanitizing preserves a state of the original data set.

17. A sanitizer for sanitizing restricted data items in a data set to prevent the revelation of the restricted data items, comprising:

a sanitizing module configured to receive an original data set based on a production data set used in a production environment, and to sanitize the original data set by;

(a) identifying the locations of the restricted data items in the original data set;

(b) identifying at least one sanitizing tool to apply to the restricted data items which have been located in the original data set; and

(c) applying said at least one sanitizing tool to the restricted data items which have been located in the original data set to provide a sanitized data set,

wherein the sanitizing module is configured to sanitize the original data set while preserving a state of the original data set.

18. The sanitizer of claim 17, further comprising a data directory table, wherein the sanitizing module uses the data directory table to identify the locations of the restricted data items in the original data set.

19. The sanitizer of claim 18, further comprising a table generating module configured to generate the data directory table by:

identifying data items within the original data set;

classifying the data items as having restricted or non-restricted status; and

mapping the restricted data items to their respective locations within the original data set.

20. The sanitizer of claim 17, further comprising a data directory table, wherein the sanitizing module uses the data directory table to identify said at least one sanitizing tool.

21. The sanitizer of claim 17, further comprising a suite of modular sanitizing tools, wherein said at least one sanitizing tool is selected from the suite of sanitizing tools.

22. The sanitizer of claim 17, wherein said at least one sanitization tool is configured to obscure the restricted data items in a manner which preserves at least one statistical characteristic of the restricted data items.

23. The sanitizer of claim 17, further comprising a transformation module configured to identify a subset of the original data set, and wherein the sanitizing module is configured to perform the sanitizing on only on that subset of the original data set.

24. The sanitizer of claim 17, further comprising a transformation module, which, together with the sanitizing module, is configured to produce the sanitized data set so that the sanitized data set has a reduced size relative to the original data set.

25. The sanitizer of claim 17, further comprising a transformation module, which, together with the sanitizing module, is configured to produce the sanitized data set so that the sanitized data set has an expanded size relative to the original data set.