History preservation in a computer storage system

• US 7,747,583 B2
• Filed: 01/07/2004
• Issued: 06/29/2010
• Est. Priority Date: 02/26/2003
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Permabit Technology Corporation (International Business Machines Corporation)
• Inventors: Margolus, Norman H., Homsy, George E. II, Keller, Jeffrey M., Floyd, Jered J.
• Primary Examiner(s): Pham; Khanh B
• Application Number:
  US10/752,981
• Publication Number:
  US 20040167934A1
• Time in Patent Office: 2,365 Days
• Field of Search: 709/220, 713/187, 711/114
• US Class Current: 707/689
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Claims

1. A method by which a disk-based data storage system attached to a network is organized for protecting historical records of stored data entities, wherein storage clients can increase the level of protection but not reduce it, the method comprising:

recording distinct states of the stored data entities, corresponding to different moments of time, as a plurality of entity versions coexisting within the data storage system;

designating at least some of the plurality of entity versions as historical versions; and

assigning a finite expiration time to an entity version that is one of the historical versions, before which time both modification and deletion are prohibited and after which time deletion is allowed;

wherein the assigning is done by the disk-based data storage system, in response to a request communicated to it over the network by a client program;

wherein, after the expiration time has been assigned and before the expiration time has passed, the client program requests that the expiration time be set to a specified later time and it is changed to the later time;

wherein, after the expiration time has been assigned and before the expiration time has passed, the client program requests that the expiration time be set to a specified earlier time and the request is denied and the expiration time is not changed; and

wherein, once the expiration time has been assigned and as long as the expiration time has not yet passed, the client program can change the expiration time to later times but cannot cause the disk-based data storage system to change the expiration time to an earlier time, nor can the client program cause the entity version to be deleted from the disk-based data storage system at an earlier time than the expiration time.

2. The method of claim 1 in which the entity version is deposited into the disk-based data storage system, and the expiration time for the entity version is specified by the client program.

3. The method of claim 1 in which the entity version is deposited into the disk-based data storage system, and a current time interval is assigned to the entity version by the client program.

4. The method of claim 3 in which the expiration time is assigned by the disk-based data storage system and depends on the current time interval.

5. The method of claim 3 in which some of the historical versions record distinct states, staffing at a start time, of a stored data entity that is a file in a source file system, and the entity version records an historical state of the file that was copied from an archival record of the source file system'"'"'s contents.

6. The method of claim 5 in which the archival record was created before the start time and the current time interval of the entity version predates the start time.

7. The method of claim 1 in which separate expiration times have been assigned to each of the historical versions, each of which can be extended independently.

8. The method of claim 1 in which the disk based data storage system is adapted for storing an unstructured-set of entities.

9. The method of claim 1 in which, during times when the entity version is not being accessed, no connection is visible to servers that make up the disk-based data storage system between the entity version and a stored block of content that is a constituent of the entity version.

10. The method of claim 1 in which a first stored data entity is one of the stored data entities and a plurality of first entity versions of the first stored data entity are all deposited during a time interval, and each of the first entity versions has its expiration time extended to at least a first expiration time.

11. The method of claim 10 in which a second stored data entity is one of the stored data entities, and the second stored data entity contains hierarchical directory information including information about the first stored data entity, and a second entity version that records the state of the second stored data entity is deposited during the time interval, and the second entity version expires earlier than the first expiration time.

12. The method of claim 11 in which a different entity version of the second stored data entity exists that does not expire before the first expiration time, and summary information is added to the different entity version that is sufficient to recreate hierarchical directory information of the second entity version.

13. The method of claim 1 in which the plurality of entity versions comprise constituent blocks of stored content, with a block assigned a reference count that reflects a count of occurrences of the block as a constituent of the plurality of entity versions.

14. The method of claim 13 in which the block is also assigned an expiration time that depends on the latest expiration time associated with any entity version of which it is a constituent.

15. The method of claim 14 in which the block has a reference count of zero and an expiration time which has passed, and it is discarded and its storage space is reused.

16. The method of claim 1 in which, at a time after the expiration time has passed, the client program communicates with the disk-based data storage system to request that the entity version be deleted, and the disk-based data storage system compares the expiration time against a current time and, based at least in part on this comparison, causes the entity version to be deleted.

17. The method of claim 16 in which the current time is determined by clocks under the control of the disk-based data storage system which operate either independently of outside clocks, or with strict limits on the rate at which outside clocks can influence them.

18. The method of claim 1 in which, once the expiration time has been assigned and as long as the expiration time has not yet passed, no action taken by any client program that only communicates with the disk-based data storage system over a wide area network can cause the expiration time assigned to the entity version to be changed to an earlier time.

19. The method of claim 1 in which the expiration time is a year after the time when the expiration time was assigned.

20. the method of claim 1 in which the entity version is initially deposited into the disk-based data storage system designated as being a current version and without an expiration time assigned, and as long as it is a current version it cannot be deleted.

21. the method of claim 20 in which the client program communicates a request to the disk-based data storage system to end the period during which it is a current version, and the entity version becomes a historical version and it is assigned the expiration time.

22. The method of claim 21 in which the client program assigns an initial expiration time to the entity version while it remains a current version, and when it is subsequently designated to be a historical version the expiration time assigned is the maximum of the initial expiration time and the expiration time that would have been assigned if no initial expiration time had been assigned.

23. The method of claim 1 in which the historical versions record states of a single stored data entity at different times, with each of the historical versions associated with a current time interval during which the recorded state was the state of the single stored data entity.

24. The method of claim 23 in which the expiration time assigned to the entity version depends at least in part on the length of the current time interval associated with the entity version.

25. The method of claim 23 in which the expiration time assigned to the entity version depends at least in part on whether or not the current time interval associated with the entity version includes a specified moment of time.

26. The method of claim 23 in which both the current time interval associated with the entity version and the expiration time are assigned by the disk-based data storage system.

27. The method of claim 1 in which the stored data entities are files in a file system or records in a database or objects in an object storage system.

28. The method of claim 1 in which the disk-based data storage system comprises a plurality of storage sites and a set of rules for assigning expiration times are distributed to each of the plurality of storage sites as part of the process of storing the plurality of entity versions.

29. The method of claim 1 in which an entity version which has not been assigned an expiration time and has not been designated historical can be deleted.

30. The method of claim 1 in which the expiration time is assigned as a time period relative to an event.

31. The method of claim 30 in which the expiration time is assigned at a moment when the entity version is first designated as being a historical version, in response to a request by the client program.

32. The method of claim 1 in which after the expiration time has passed, the entity version is deleted in response to a request by the client program.

33. The method of claim 1 in which the assigning is done by the disk-based data storage system, according to a rule communicated to it over the network by the client program.