Method and system for masking data in a consistent manner across multiple data sources

• US 8,055,668 B2
• Filed: 02/13/2008
• Issued: 11/08/2011
• Est. Priority Date: 02/13/2008
• Status: Expired due to Fees

• Current Assignee: Imperva Incorporated (Thales SA)
• Original Assignee: Camouflage Software, Inc. (Thales SA)
• Inventors: Spearns, Michael Paul, Morgan, David Isaac, Pomroy, Steven Patrick, Lake, Robert Raymond
• Primary Examiner(s): Channavajjala; Srirama
• Assistant Examiner(s): Hosseini; Reza
• Application Number:
  US12/030,695
• Publication Number:
  US 20090204631A1
• Time in Patent Office: 1,364 Days
• Field of Search: 707/655, 707672-682, 707756-758, 707/769, 707/781, 707/792, 707/802, 707/803, 707/55, 709/200, 713/162, 713/190, 713/191
• US Class Current: 707/757
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

A method and system for data masking a series of interrelated data records is disclosed. A lockable translation matrix repository resource is provided to contain both masked data as well as appropriate key information that provides links between respective copies of respective interlinked databases and maintains the data integrity of masking data inserted therein. Records are masked on a column by column or table by table basis. Records for which masking data is already in the repository are masked by making use of such data, while remaining records are segregated, masked and the masking data updated to the repository. Preferably a backup copy of the masked data records is stored in the repository to permit de-masking of the data records at a later stage. Pivot tables are applied where keys do not match exactly but still exhibit a one-to-one relationship.

Claims

1. A method for data masking at least one data field of at least one data record in a primary database and at least one data field of at least one data record in a secondary database that has a key field having a correspondence with a key field in the at least one data record in the primary database, using a table of translation matrix records comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record, the method comprising the actions of:

with respect of the primary database;

comparing the key field of the at least one data record in the corresponding database against the unmasked key field of each of the translation matrix records;

populating a first sub-table record, if the unmasked key field of a translation matrix record matches the key field of the at least one data record, the first sub-table record comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record, using the unmasked key field and the at least one masked data field of the matching translation matrix record;

populating a second sub-table record, if no unmasked key field of any translation matrix record matches the key field of the at least one data record, the second sub-table record comprising a key field and at least one data field corresponding to the at least one data field of the at least one data record, using the corresponding key field and the at least one data field of the at least one data record;

masking the at least one data field of the second sub-table record;

adding the first sub-table record or the masked second sub-table record as a masked record in a masked copy of the corresponding database; and

if the masked second sub-table record was created, populating a translation matrix record corresponding thereto using the key field and the at least one data field for the unmasked key field and the at least one masked data field;

repeating the actions performed with respect of the primary database with respect to the secondary database and further comprising;

if the key field in the at least one data record in the primary database and the key field in the at least one data record in the secondary database are different, setting out the correspondence therebetween in a pivot table including both of the key fields; and

comparing the key field of the at least one data record in the secondary database against the unmasked key field of each of the translation matrix records further comprises looking up the at least one key field of the at least one data record in the secondary database in the corresponding key field of the pivot table and comparing the other key field thereof against the unmasked key field of each of the translation matrix records;

whereby at least one data field of the at least one data record of the primary database and the at least one data field of the at least one data record of the secondary database are masked in a consistent manner.

2. The method according to claim 1, wherein a value of the key field in the at least one data record in the primary database and of the key field in the at least one data record in the second database are the same.

3. The method according to claim 1, wherein the at least one data field to be masked includes the key field.

4. The method according to claim 1, wherein the masked copies of the primary and secondary databases behave the same as the primary and secondary databases respectively.

5. The method according to claim 4, wherein the masked copies of the primary and secondary databases maintain the relational integrity of the primary and secondary databases.

6. The method according to claim 1, wherein masking the at least one data field of the second sub-table record further comprises performing at least one data transformation on the at least one data field of the second sub-table entry.

7. The method according to claim 6, wherein the data transformation is pre-selected before commencing the actions with respect of the primary or secondary databases.

8. The method according to claim 1, further comprising:

before performing the actions with respect to one of the primary or secondary databases;

ensuring that the table of translation matrix records is not being accessed; and

reserving exclusive access to the table of translation matrix records until released;

after performing the actions with respect to the one of the primary or secondary databases, releasing exclusive access to the table of translation matrix records.

9. The method according to claim 8, comprising:

after reserving exclusive access to the table of translation matrix records until released, copying the table of translation matrix records from a secure repository; and

before releasing exclusive access to the table of translation matrix records, replacing the table of translation matrix records into the secure repository.

10. The method according to claim 9, further comprising removing and replacing a pivot table comprising the two key fields and establishing a one to one correspondence therebetween.

11. The method according to claim 9, wherein replacing the table of translation matrix records into the secure repository further comprises replacing a backup record comprising the key field and the at least one data field of the at least one data record to be masked into the secure repository.

12. The method according to claim 11, comprising adding a masked key field to the backup record containing the masked value of the key field of the at least one data record.

13. A method for de-masking the at least one data record in at least one of the primary database and the secondary database that has been masked according to the method of claim 12, comprising:

reserving exclusive access to the table of translation matrix records until released;

removing at least one of the backup record stored in the repository from the secure repository;

reviewing the table of translation matrix records to determine if the key field of the at least one data record has been masked;

if the key field of the at least one data record has been masked;

comparing the key field of the at least one data record against the masked key field of each of the backup records; and

populating a de-masked data record comprising the key field optionally corresponding to the at least one data field of the at least one data record, using the masked key field and optionally the at least one data field of the backup record;

if the key field of the at least one data record has not been masked;

comparing the key field of the at least one data record against the key field of each of the backup records; and

populating a de-masked data record comprising the key field and at least one de-masked data field corresponding to the at least one data field of the at least one data record, using the key field and the at least one data field of the backup record; and

releasing exclusive access to the table of translation matrix records.

14. The method according to claim 1, wherein the at least one data field is a confidential data field.

15. The method of claim 1 comprising repeating the actions with respect to the primary database for all records in each table of the primary database, and repeating the actions with respect of the secondary database for all records in each table of the secondary database until the masked copy of the corresponding databases comprises masked copies of all records in all tables of the primary and secondary databases.

16. A system for data masking at least one data field of at least one data record in a primary database and at least one data field of at least one data record in a secondary database that has a key field having a correspondence with a key field in the at least one data record in the primary database, the system comprising:

a memory for storing data records, including a table of translation matrix records comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record;

a processor coupled to the memory for;

in respect of the primary database, executing instructions for;

comparing the key field of the at least one data record in the corresponding database against the unmasked key field of each of the translation matrix records;

populating a first sub-table record, if the unmasked key field of a translation matrix record matches the key field of the at least one data record, the first sub-table record comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record, using the unmasked key field and the at least one masked data field of the matching translation matrix record;

populating a second sub-table record, if no unmasked key field of any translation matrix record matches the key field of the at least one data record, the second sub-table record comprising a key field and at least one data field corresponding to the at least one data field of the at least one data record, using the corresponding key field and the at least one data field of the at least one data record;

masking the at least one data field of the second sub-table record;

adding the first sub-table record or the masked second sub-table record as a masked record in a masked copy of the corresponding database;

if the masked second sub-table record was created, populating a translation matrix record corresponding thereto using the key field and the at least one data field for the unmasked key field and the at least one masked data field; and

repeating the instructions in respect of the primary database with respect to the secondary database and further comprising;

if the key field in the at least one data record in the primary database and the key field in the at least one data record in the secondary database are different, setting out the correspondence therebetween in a pivot table including both of the key fields; and

comparing the key field of the at least one data record in the secondary database against the unmasked key field of each of the translation matrix records further comprises looking up the at least one key field of the at least one data record in the secondary database in the corresponding key field of the pivot table and comparing the other key field thereof against the unmasked key field of each of the translation matrix records;

whereby at least one data field of the at least one data record of the primary database and the at least one data field of the at least one data record of the secondary database are masked in a consistent manner.

17. A processor in a system for data masking at least one data field of at least one data record in a primary database and at least one data field of at least one data record in a secondary database that has a key field having a correspondence with a key field in the at least one data record in the primary database, the processor being operatively coupled to a memory for storing data records including a table of translation matrix records comprising an unmasked key field and at least one masked data field corresponding to at least one data field of the at least one data record for:

in respect of the primary database, executing instructions for;

comparing the key field of the at least one data record in the corresponding database against the unmasked key field of each of the translation matrix records;

populating a first sub-table record, if the unmasked key field of a translation matrix record matches the key field of the at least one data record, the first sub-table record comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record, using the unmasked key field and the at least one masked data field of the matching translation matrix record;

populating a second sub-table record, if no unmasked key field of any translation matrix record matches the key field of the at least one data record, the second sub-table record comprising a key field and at least one data field corresponding to the at least one data field of the at least one data record, using the corresponding key field and the at least one data field of the at least one data record;

masking the at least one data field of the second sub-table record;

adding the first sub-table record or the masked second sub-table record as a masked record in a masked copy of the corresponding database;

if the masked second sub-table record was created, populating a translation matrix record corresponding thereto using the key field and the at least one data field for the unmasked key field and the at least one masked data field; and

repeating the instructions in respect of the primary database with respect to the secondary database and further comprising;

if the key field in the at least one data record in the primary database and the key field in the at least one data record in the secondary database are different, setting out the correspondence therebetween in a pivot table including both of the key fields; and

comparing the key field of the at least one data record in the secondary database against the unmasked key field of each of the translation matrix records further comprises looking up the at least one key field of the at least one data record in the secondary database in the corresponding key field of the pivot table and comparing the other key field thereof against the unmasked key field of each of the translation matrix records;

whereby at least one data field of the at least one data record of the primary database and the at least one data field of the at least one data record of the secondary database are masked in a consistent manner.

18. A computer-readable medium coupled to a processor, in a system for data masking at least one data field of at least one data record in a primary database and at least one data field of at least one data record in a secondary database that has a key field having a correspondence with a key field in the at least one data record in the primary database, the medium for storing data records including a table of translation matrix records comprising an unmasked key field and at least one masked data field corresponding to at least one data field of the at least one data record, and having stored thereon, computer-readable and computer-executable instructions which, when executed by the processor, cause the processor to:

in respect of the primary database;

compare the key field of the at least one data record in the corresponding database against the unmasked key field of each of the translation matrix records;

populate a first sub-table record, if the unmasked key field of a translation matrix record matches the key field of the at least one data record, the first sub-table record comprising an unmasked key field and at least one masked data field corresponding to the at least one data field of the at least one data record, using the unmasked key field and the at least one masked data field of the matching translation matrix record;

populate a second sub-table record, if no unmasked key field of any translation matrix record matches the key field of the at least one data record, the second sub-table record comprising a key field and at least one data field corresponding to the at least one data field of the at least one data record, using the corresponding key field and the at least one data field of the at least one data record;

mask the at least one data field of the second sub-table record;

add the first sub-table record or the masked second sub-table record as a masked record in a masked copy of the corresponding database;

if the masked second sub-table record was created, populate a translation matrix record corresponding thereto using the key field and the at least one data field for the unmasked key field and the at least one masked data field; and

repeat the instructions in respect of the primary database with respect to the secondary database and further comprising;

if the key field in the at least one data record in the primary database and the key field in the at least one data record in the secondary database are different, setting out the correspondence therebetween in a pivot table including both of the key fields; and

comparing the key field of the at least one data record in the secondary database against the unmasked key field of each of the translation matrix records further comprises looking up the at least one key field of the at least one data record in the secondary database in the corresponding key field of the pivot table and comparing the other key field thereof against the unmasked key field of each of the translation matrix records;

whereby at least one data field of the at least one data record of the primary database and the at least one data field of the at least one data record of the secondary database are masked in a consistent manner.