Data archiving system

• US 7,801,871 B2
• Filed: 08/09/2006
• Issued: 09/21/2010
• Est. Priority Date: 08/09/2005
• Status: Active Grant

• Current Assignee: Storcentric DIP Lender LLC
• Original Assignee: Nexsan Technologies Canada Inc. (StorCentric, Inc.)
• Inventors: Gosnell, Thomas F.
• Primary Examiner(s): Ali; Mohammad
• Assistant Examiner(s): Hocker; John P
• Application Number:
  US11/463,461
• Publication Number:
  US 20070038857A1
• Time in Patent Office: 1,504 Days
• Field of Search: 707 1- 10, 707/204, 707/698, 707/747
• US Class Current: 707/698
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: H04L 63/061 for key exchange, e.g. in peer-to-peer networks cryptographic mechanisms or cryptographic arrangements for key agreement H04L9/0838
• CPC Class Codes: H04L 9/083 involving central third party, e.g. key distribution center [KDC] or trusted third party [TTP]
• CPC Class Codes: H04L 9/0897 involving additional devices, e.g. trusted platform module [TPM], smartcard or USB
• CPC Class Codes: H04L 9/3239 involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
• CPC Class Codes: H04L 9/3242 involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC

Abstract

An encrypted file storage solution consists of a cluster of processing nodes, external data storage, and a software agent (the “File System Watcher”), which is installed on the application servers. Cluster sizes of one node up to many hundreds of nodes are possible. There are also remote “Key Servers” which provide various services to one or more clusters. The preceding describes a preferred embodiment, though in some cases it may be desirable to “collapse” some of the functionality into a smaller number of hardware devices, typically trading off cost versus security and fault-tolerance.

Claims

1. A computer-implemented method of auditing stored data in an auditable data storage system, comprising:

recording, in a manifest, data associated with files received for storage in the auditable data storage system, each of the files received for storage having associated therewith a unique file identifier,

wherein the step of recording includes storing the unique file identifier, a file size and a serial number associated with the each of the files received for storage, and retention information associated with each of the files, and

wherein the unique file identifier is determined in accordance with at least two cryptographic hashes of the associated file, and includes a concatenation of at least two cryptographic hashes of the associated file;

closing the manifest;

storing the manifest as a file in the auditable data storage system;

creating a new manifest associated with files received for storage in the auditable data storage system, the new manifest recording, as a file entry, data, including a unique file identifier, associated with the closed manifest;

examining the new manifest to verify the existence of a unique file identifier for each file recorded in the new manifest;

determining that each file having associated data stored in the new manifest is present in the auditable storage system; and

verifying that each file having associated data stored in the manifest present in the system has not been altered by recomputing the unique file identifier and comparing the computed unique file identifier to the unique file identifier stored in the manifest.

2. The computer-implemented method of claim 1, wherein each file has an associated timestamp.

3. The computer-implemented method of claim 2 wherein the manifest further includes the timestamp.

4. The computer-implemented method of claim 1 wherein the unique file identifier includes a concatenation of an MD-5 cryptographic hash and a SHA-1 cryptographic hash.

5. The computer-implemented method of claim 1 wherein each of the plurality of cryptographic hashes is selected from a list including MD-5, SHA-1, SHA-224, SHA-256, SHA-384 and SHA-512.

6. The computer-implemented method of claim 1 wherein the step of storing the manifest includes:

assembling a set of metadata associated with the manifest;

encrypting the manifest to create an asset;

storing the asset; and

creating an entry in the new manifest associating the metadata associated with the manifest with the stored asset.

7. The computer-implemented method of claim 1 wherein the step of recording includes associating a consecutive serial number with each of the files received for storage.

8. The computer-implemented method of claim 1 wherein at least one of the files received for storage is encrypted, and the step of recording data includes recording an encryption key container serial number associated with an encryption key used to encrypt the at least one file.

9. The computer-implemented method of claim 1 wherein the step of closing the manifest is performed after a predetermined time interval from the creation of the manifest.

10. The computer-implemented method of claim 1 wherein the step of creating the new manifest includes recording a unique file identifier associated with the previous manifest as the first entry in the new manifest, the unique file identifier including a concatenation of cryptographic hashes of the closed manifest.

11. The computer-implemented method of claim 1 wherein the step of closing the manifest includes forwarding a copy of the stored manifest to an external site.

12. The computer-implemented method of claim 11 wherein the external site is administered by a trusted third party.

13. The computer-implemented method of claim 1 further including:

generating and storing a unique encryption key associated with each file in a remote key server; and

obtaining the unique encryption key from the remote key server.

14. The computer-implemented method of claim 1 wherein the step of deleting the key is performed in response to obtaining authorization to render the asset inaccessible.

15. The computer-implemented method of claim 13 further comprising:

creating a disposition manifest listing a file stored in the auditable data storage system marked for deletion;

sending the disposition manifest to the remote key server; and

obtaining authorization from the remote key server to render the file inaccessible prior to deleting the key.

16. The computer-implemented method of claim 13, further comprising:

rendering any given file inaccessible by selectively deleting the key associated with the given file without deleting any other encryption keys each uniquely associated with a plurality of other files.

17. The computer-implemented method of claim 13 wherein the encryption key is a symmetric encryption key.

18. The computer-implemented method of claim 13 wherein the step of creating a data record includes associating an encryption key serial number with the stored asset.