History preservation in a computer storage system

• US 9,104,716 B2
• Filed: 01/07/2004
• Issued: 08/11/2015
• Est. Priority Date: 02/26/2003
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Permabit, Inc. (International Business Machines Corporation)
• Inventors: Margolus, Norman H.
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Kim, Paul
• Application Number:
  US10/752,839
• Publication Number:
  US 20040167902A1
• Time in Patent Office: 4,234 Days
• Field of Search: 707/3, 707/10, 707/100, 707/1, 707/9, 707/663, 707/662, 707/661, 709/226
• US Class Current: 1/1
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Claims

1. A data storage system in which a plurality of client programs running on processors connected to a communications network share space on storage hardware used to store a data item in a data repository that uses processors at a plurality of storage sites connected to the communications network to enforce retention requirements governing the data item, the data storage system comprising one or more processors configured for:

determining a digital fingerprint from the content of the data item using a hash function;

storing the data item on storage hardware at the plurality of storage sites of the data repository, in response to a request from a first client program to deposit the data item in the data repository;

ensuring that the data item is stored on storage hardware in the data repository, in response to a request from a second client program to deposit content identical to the data item;

assigning an expiration time to the data item by the processors at each of the plurality of storage sites;

reacting to a request from the second client program to delete the data item from the data repository, the reacting step comprising;

receiving the request by the processors at each of the plurality of storage sites;

separately comparing, on a processor at each of the plurality of storage sites, the expiration time assigned to the data item at the storage site with a current time value; and

deleting the data item from the storage site if the expiration time has passed, and not deleting the data item if the expiration time has not yet passed;

wherein the expiration time assigned by the processors at each of the plurality of storage sites defines the end of a finite time interval during which the processor at the storage site does not allow the second client program to delete the data item and does not allow the second client program to change the expiration time to an earlier time, but does allow the second client program to change the expiration time to a later time.

2. The data storage system of claim 1 in which the expiration time assigned to the data item by a processor at a storage site depends upon a plurality of expiration times assigned by the plurality of client programs.

3. The data storage system of claim 1 further comprising encrypting the data item using a key derived from the content of the data item.

4. The data storage system of claim 1 wherein the ensuring step further comprises:

comparing the digital fingerprint with digital fingerprints of data items already stored on storage hardware in the data repository; and

storing the data item on storage hardware in the data repository if comparing establishes that the data item is not already stored there, and not storing the data item on the storage hardware in the data repository if comparing establishes that the data item is already stored there.

5. The data storage system of claim 2 in which the expiration time assigned to the data item by the processor at the storage site is the latest of all of the plurality of expiration times assigned by the plurality of client programs.

6. The data storage system of claim 1 in which the hash function is a cryptographic hash function.

7. The data storage system of claim 1 in which the storing step further comprises:

selecting at least one of a plurality of storage devices at each of the plurality of storage sites of the data repository, with the selection based at least in part on the digital fingerprint; and

storing the data item on the selected storage devices.

8. The data storage system of claim 1 in which the second client program specifies an expiration time for the data item and the specified expiration time is assigned to the data item by the processors at each of the plurality of storage sites.

9. The data storage system of claim 1 in which the plurality of client programs can extend the expiration time assigned by the processor at a storage site but no action taken by any of the plurality of client programs can cause the expiration time to be changed to an earlier time or cause the stored data item to be deleted from the storage site at an earlier time than the expiration time.

10. The data storage system of claim 1 in which the data item has been assigned a permanent status that indicates that it will never expire and the data repository should never delete the data item.

11. The data storage system of claim 1 in which a set of rules governing the assignment of an expiration time to the data item are communicated to the processors at the plurality of storage sites.

12. The data storage system of claim 7 in which at least part of the digital fingerprint is used as a source of pseudo-randomness in the selecting step.

13. The data storage system of claim 1 in which two of the plurality of storage sites are located in different cities.

14. The data storage system of claim 7 in which the selected storage devices comprise magnetic disks.

15. The data storage system of claim 11 in which a one of the set of rules depends, at least in part, on the time at which a storage system client makes a request that causes the expiration time to be assigned.

16. The data storage system of claim 1 in which the first client program transmits the data item over the network to the processor that is a part of a one of the plurality of storage sites, and that storage site communicates redundant information about the content of the data item to others of the plurality of storage sites.

17. The data storage system of claim 1 in which fault tolerance against the loss of a storage site is achieved using redundancy.

18. The data storage system of claim 1 wherein software running on the processors at the plurality of storage sites protects the data item from premature deletion from the data repository even if the correct operation of the data repository is maliciously subverted at a one of the plurality of storage sites.

19. The data storage system of claim 18 wherein policies governing the operation of the data repository require that an entity that has administrative authority or physical access at a one of the plurality of storage sites is not granted special privileges or physical access that would allow the entity to globally circumvent rules governing how long the data item must be retained in the plurality of storage sites.

20. The data storage system of claim 19 in which the entity is an ordinary operator or an ordinary administrator of the data repository.

21. The data storage system of claim 1 in which the second client program communicates over the network with a server of the data repository using a defined storage protocol that makes use of the digital fingerprint.

22. The data storage system of claim 21 in which the use of digital fingerprints reduces the amount of bandwidth required for depositing data items.

23. The data storage system of claim 21 in which the digital fingerprint is used to verify the integrity of the content of the data item.

24. The data storage system of claim 1 wherein, even after the expiration time has passed, the data item is retained at the plurality of storage sites until a new request to delete the data item is initiated by one of the plurality of client programs that share space used to store the data item in the data repository, and communicated to the processors at each of the plurality of storage sites.