System and method for record retention date in a write once read many storage system

• US 20050097260A1
• Filed: 11/03/2003
• Published: 05/05/2005
• Est. Priority Date: 11/03/2003
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: McGovern, William P., Heller, Jeffrey L.
• Granted Patent: US 7,590,807 B2
• Time in Patent Office: Days
• US Class Current: 711/100
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/181 providing write once read many [WORM] semantics
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2216/09 Obsolescence
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

This invention provides a specified retention date within a data set that is locked against deletion or modification within a WORM storage implementation. This retention date scheme does not utilize any proprietary application program interfaces (APIs) or protocols, but rather, employs native functionality within conventional file (or other data containers, data sets or block-based logical unit numbers) properties available in commonly used operating systems. In an illustrative embodiment, the retention date/time is calculated by querying the file'"'"'s last-modified time prior to commit, adding the retention period to this value and thereby deriving a retention date after which the file can be released from WORM. Prior to commit, the computed retention date is stored in the file'"'"'s “last access time” property/attribute field, or another metadata field that remains permanently associated with the file and that, in being used for retention date, does not interfere with file management in a WORM state. Since this field is not utilized in a WORM context, it can be adapted to store this date. Once stored, the retention date in this field is locked against modification. Where extension (never reduction) of a retention period is desired, the last access time field be updated, wherein the new retention period is added to the existing last access time value to derive a new, later retention date for the file. Upon expiry of the retention date, the system allows deletion of the expired WORM file/data set.

Claims

1. A method for implementing a retention period to a data set in a write-once-read-many (WORM) storage system comprising the steps of:

creating a WORM data set and saving the WORM data set to a WORM data structure;

setting a retention period in a data of the data set that remains permanently associated with the data set; and

committing the WORM data set to the WORM data structure after which the storage system prevents shortening of the retention period and the storage system prevents any modification to the WORM data set until after until a time after expiration of the retention period.

2. The method as set forth in claim 1 wherein, after the expiration of the retention period, the storage system permits only deletion of the WORM data set from the WORM data structure and prevents any other modification to the WORM data set.

3. The method as set forth in claim 2 wherein the data into which the retention period is set is a file attribute or file property related to a time value associated with the data set.

4. The method as set forth in claim 3 wherein the file attribute or file property is a last accessed time attribute or property.

5. The method as set forth in claim 3 wherein the step of committing includes transitioning a read-only file attribute associated with the data set from a non-read-only value to a read-only value.

6. The method as set forth in claim 1 further comprising extending the retention period bay adding an extension period to the retention period to derive an extended retention period and storing the extended retention period in the data into which the retention period is set.

7. The method as set forth in claim 6 wherein the step of extending includes verifying that the extension period is later in time than the retention period and rejecting an extension period that is earlier in time than the retention period.

8. The method as set forth in claim 1 further comprising comparing the retention period to a time value in a compliance clock associated with the storage system and enabling only deletion of the WORM data set after the time value is equal to or later than the retention period.

9. The method as set forth in claim 8 further comprising counting regular repetitive operations of the storage system and producing the time value based upon a count of the repetitive operations.

10. The method as set forth in claim 9 further comprising storing the time value in a time-storing WORM data set in the WORM data structure and accessing the time-storing WORM data set to restart a count of the time value after an interruption of the count.

11. The method as set forth in claim 1 wherein the storage system is adapted to prevent renaming of directory paths that include the WORM data set in the worm data structure.

12. The method as set forth in claim 1 wherein the step of committing is performed by an application on a client interconnected to the storage system.

13. The method as set forth in claim 1 further comprising creating the WORM data structure at a mirror destination and transmitting and committing the created WORM data set to the destination WORM data structure.

14. A write-once-read-many (WORM) storage system having a file system comprising:

a WORM utility residing in the file system that commits files to a designated volume based upon an open protocol file attribute provided to the file system after the file is stored in the designated volume and that provides a selectable retention period during which the file cannot be modified after being committed.

15. The WORM storage system as set forth in claim 14 wherein the open protocol file attribute comprises a read-only file attribute.

16. The WORM storage system as set forth in claim 14 wherein the file system is adapted to prevent deletion of a directory containing the file.

17. The WORM storage system as set forth in claim 14 wherein the retention period is stored in an open protocol file attribute.

18. The WORM storage system as set forth in claim 17 wherein the file attribute containing the retention period comprises a last access time attribute.

19. The WORM storage system as set forth in claim 14 wherein the WORM utility is adapted to compare the retention period to a time value of a compliance clock and enable predetermined modification to the file after the retention period is earlier than the time value.

20. The WORM storage system as set forth in claim 19 wherein the predetermined modification comprises only deletion of the file.

21. A computer-readable medium for implementing a retention period to a data set in a write-once-read-many (WORM) storage system, the computer-readable medium including program instructions for performing the steps of:

creating a WORM data set and saving the WORM data set to a WORM data structure;

setting a retention period in a data of the data set that remains permanently associated with the data set; and

committing the WORM data set to the WORM data structure after which the storage system prevents shortening of the retention period and the storage system prevents any modification to the WORM data set until after until a time after expiration of the retention period.

22. The computer-readable medium as set forth in claim 21 wherein, after the expiration of the retention period, the storage system permits only deletion of the WORM data set from the WORM data structure and prevents any other modification to the WORM data set.

23. The computer-readable medium as set forth in claim 22 wherein the data into which the retention period is set is a file attribute or file property related to a time value associated with the data set.

24. The computer-readable medium as set forth in claim 23 wherein the file attribute or file property is a last accessed time attribute or property.

25. The computer-readable medium as set forth in claim 23 wherein the step of committing includes transitioning a read-only file attribute associated with the data set from a non-read-only value to a read-only value.

26. The computer-readable medium as set forth in claim 21 further comprising extending the retention period bay adding an extension period to the retention period to derive an extended retention period and storing the extended retention period in the data into which the retention period is set.

27. The computer-readable medium as set forth in claim 26 wherein the step of extending includes verifying that the extension period is later in time than the retention period and rejecting an extension period that is earlier in time than the retention period.

28. The computer-readable medium as set forth in claim 21 further comprising comparing the retention period to a time value in a compliance clock associated with the storage system and enabling only deletion of the WORM data set after the time value is equal to or later than the retention period.

29. The computer-readable medium as set forth in claim 28 further comprising counting regular repetitive operations of the storage system and producing the time value based upon a count of the repetitive operations.

30. The computer-readable medium as set forth in claim 29 further comprising storing the time value in a time-storing WORM data set and accessing the time-storing WORM data set to restart a count of the time value after an interruption of the count.

31. The computer-readable medium as set forth in claim 21 wherein the storage system is adapted to prevent renaming of directory paths that include the WORM data set in the worm data structure.

32. The computer-readable medium as set forth in claim 31 wherein the step of committing is performed by an application on a client interconnected to the storage system.

33. The computer-readable medium as set forth in claim 31 further comprising creating the WORM data structure at a mirror destination and transmitting and committing the created WORM data set to the destination WORM data structure.

34. A record on a storage system that is recorded so as to have write-once-read-many (WORM) characteristics comprising:

a retention period value stored in a permanent open protocol data field associated with the record, the retention period thereby providing an instruction to the storage system so that the record cannot be modified in any way until the retention period expires.

35. The record as set forth in claim 34 wherein the permanent data field is a file attribute that cannot be modified after the record is committed to storage.

36. The record as set forth in claim 35 wherein the file attribute comprises a last access time file attribute.

37. The record as set forth in claim 35 wherein the record includes an open protocol read/read-only attribute, the read/read-only attribute, when set to read-only, providing an instruction to the storage system to prevent modification of the record until the retention period expires.

38. The record as set forth in claim 34 wherein the record is stored on a disk drive in a data structure.

39. The record as set forth in claim 38 wherein the data structure is a WORM data structure that is recognized by the storage system as containing records with WORM characteristics.

40. The record as set forth in claim 34 wherein the storage system is adapted to enable extension of the retention period by storing an extended retention period value that specifies a time greater than the retention period.

41. A record on a storage system that is recorded so as to have write-once-read-many (WORM) characteristics comprising:

retention period value stored in permanent association with the record, the retention period thereby providing an instruction to the storage system so that the record cannot be modified in any way until the retention period expires and the record being individually deletable from the storage system after the retention period expires.

42. A compliance clock for ensuring that data sets in a write-once-read-many (WORM) storage system are not modified prior to a retention date thereof comprising:

a counter that counts a plurality of repetitive events in the storage system and assigning a time value relative to a count of the events; and

a generator that compares the count to an external clock time input and that adjusts the time value based upon the external clock time input based upon a bounded drift rate between the external clock input and the time value.

43. The compliance clock as set forth in claim 42 wherein the generator is adapted to compare the external clock time input and the time value and issue an alert if the external clock time input and the time value differ by more than a predetermined value.

44. The compliance clock as set forth in claim 43 wherein the alert causes the storage system to prevent modification of all data sets in the storage system.

45. The compliance clock as set forth in claim 42 wherein the external clock comprises a trusted third-party clock.

46. The compliance clock as set forth in claim 42 further comprising a time-storing data set on the storage system that receives at periodic times and stores the time value and that is adapted to be accessed when the counter is interrupted to derive a starting time value.

47. The compliance clock as set forth in claim 46 wherein the time storing data set and the data sets are each stored on a WORM volume so that the time-storing data set is permanently associated with the data sets.

48. The compliance clock as set forth in claim 47 wherein the generator is adapted to review each of a plurality of time storing data sets associated with each of a plurality of volumes, respectively, and select the earliest time value of the time storing data sets as a start time.

49. A method for establishing a write-once-read-many (WORM) volume in a storage system wherein predetermined of the data sets therein are each locked against modification until expiration of a respective retention date comprising the steps of:

identifying a type of WORM volume as at least one of either an untrusted administrator WORM volume or a trusted-administrator WORM volume; and

allowing only untrusted administrator activities to be performed on each of the predetermined of the data sets in an untrusted administrator WORM volume and allowing trusted administrator activities to be performed on each of the predetermined of the data sets in a trusted administrator WORM volume.

50. The method as set forth in claim 49 wherein the step of allowing untrusted administrator activities include only allowing deletion of each of the data sets after the respective retention date has expired and providing a default minimum retention date to each of the data sets.

51. The method as set forth in claim 50 wherein the step of allowing trusted administrator activities includes providing a default retention date that enables immediate modification to each of the data sets.

52. The method as set forth in claim 50 wherein the step of allowing trusted administrator activities includes enabling deletion of each of the data sets and other modification to each of the data sets after the respective retention date has expired.

53. The method as set forth in claim 50 wherein the step of allowing trusted administrator activities includes enabling destruction of the trusted administrator volume.

54. A method for automatically deleting records stored in a data structure of a write-once-read-many storage system comprising the steps of:

establishing retention dates for predetermined of the records in the data structure;

scanning the data structure and comparing the retention dates to a trusted clock time value; and

automatically deleting each of the records with a retention date that is equal to or earlier than the trusted clock time.

55. The method as set forth in claim 54 wherein the steps of scanning, comparing and automatically deleting are performed by a process running on a client interconnected to the storage system.

56. The method as set forth in claim 54 wherein the trusted clock generates the time value based upon a periodic process running on the storage system and that persistently stores the time value in the data structure at predetermined times.

57. The method as set forth in claim 56 wherein the trusted clock receives a time input from another clock and adjusts the time value at a bounded drift rate to approach a value of the time input.

58. A WORM storage system adapted to store records in a write-once-read-many configuration comprising:

an application program interface for applying a respective retention date in the form of metadata to each of the records; and

a trusted clock, separate from a system hardware clock that is used to unlock WORM features on each of the records when a respective retention date expires.

59. The WORM storage system as set forth in claim 58 wherein the trusted clock derives a time value based upon a count of periodic events performed by the storage system.

60. The WORM storage system as set forth in claim 59 wherein the trusted clock is adjusted based upon a time input from another clock at a bounded shift rate that limits adjustment to a predetermined amount of time shift for a predetermined overall time.

61. The WORM storage system as set forth in claim 60 wherein the other clock is a system clock.

62. The WORM storage system as set forth in claim 58 wherein the storage system comprises a cluster of interconnected computers and is a content addressable storage system.