Template-based creation and archival of file systems

• US 6,618,736 B1
• Filed: 03/09/2001
• Issued: 09/09/2003
• Est. Priority Date: 03/09/2001
• Status: Expired due to Term

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: Ensim Corporation (Techstep ASA)
• Inventors: Menage, Paul
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US09/802,613
• Time in Patent Office: 914 Days
• Field of Search: 707/202, 707/201, 707/203, 707/204, 707/8
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

File systems are created and archived by providing a set of shared storage units and one or more templates, each template including a set of private storage units and a corresponding usage map.

Claims

1. A method for file system creation and archival comprising:

providing a first set of storage units and a second set of storage units, each storage unit of the first set corresponding to a storage unit of the second set;

providing a first usage map for indicating which storage units of the second set contain valid data;

intercepting an attempt to write a data item to a storage unit of the first set;

writing the data item to the corresponding storage unit of the second set; and

storing an indication in the first usage map that the corresponding storage unit of the second set contains valid data.

2. The method of claim 1, wherein the first and second sets of storage units are stored in a same storage device.

3. The method of claim 1, wherein the first set of storage units is stored in a local storage device and the second set of storage units is stored in a remote storage device.

4. The method of claim 1, wherein the first usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding storage unit of the second set contains valid data.

5. The method of claim 1, wherein providing a first usage map comprises:

initializing the first usage map to indicate that none of the storage units of the second set contain valid data.

6. The method of claim 1, further comprising:

intercepting an attempt to read a data item from a storage unit of the first set; and

in response to the first usage map indicating that the corresponding storage unit of the second set contains valid data, reading the data item from the corresponding storage unit of the second set.

7. The method of claim 1, further comprising:

intercepting an attempt to read a data item from a storage unit of the first set; and

in response to the first usage map indicating that the corresponding storage unit of the second set does not contain valid data, reading the data item from the storage unit of the first set.

8. The method of claim 1, further comprising:

providing a third set of storage units, each storage unit of the third set corresponding to one of the storage units of the first set;

providing a second usage map for indicating which storage units of third set contain valid data;

intercepting an attempt to write a data item to a storage unit of the first set;

writing the data item to the corresponding storage unit of the third set; and

storing an indication in the second usage map that the corresponding storage unit of the third set contains valid data.

9. The method of claim 8, wherein the second usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding storage unit of the third set contains valid data.

10. The method of claim 8, wherein providing a second usage map comprises:

initializing the second usage map to indicate that none of the storage units of the third set contain valid data.

11. The method of claim 8, further comprising:

intercepting an attempt to read a data item from a storage unit of the first set; and

in response to the second usage map indicating that the corresponding storage unit of the third set contains valid data, reading the data item from the corresponding storage unit of the third set.

12. The method of claim 8, further comprising:

intercepting an attempt to read a data item from a storage unit of the first set; and

in response to the second usage map indicating that the corresponding storage unit of the third set does not contain valid data and the first usage map indicating that the corresponding storage unit of the second set contains valid data, reading the data item from the corresponding storage unit of the second set.

13. The method of claim 8, further comprising:

intercepting an attempt to read a data item from a storage unit of the first set; and

in response to the second usage map indicating that the corresponding storage unit of the third set does not contain valid data and the first usage map indicating that the corresponding storage unit of the second set does not contain valid data, reading the data item from the storage unit of the first set.

14. The method of claim 8, further comprising:

archiving each storage unit of the second set for which an indication of valid data is stored in the first usage map.

15. The method of claim 14, further comprising:

merging the second set of storage units with the third set of storage units.

16. The method of claim 15, wherein merging comprises:

copying the storage units of the second set that contain valid data to those corresponding storage units of the third set that do not contain valid data; and

for each storage unit of the second set that is copied, storing an indication of valid data in the second usage map.

17. A method for creating and archiving file systems of a plurality of servers, the method comprising:

providing a set of shared storage units;

for each of the plurality of servers;

providing a first set of private storage units, each of the private storage units corresponding to a shared storage unit; and

providing a first usage map for indicating which of the private storage units contain valid data;

intercepting an attempt to write a data item to a shared storage unit;

writing the data item to the corresponding private storage unit; and

storing an indication in the first usage map that the corresponding private storage unit contains valid data.

18. The method of claim 17, wherein the file system of each server comprises a combination of the set of shared storage units and the first set of private storage units for the server.

19. The method of claim 17, further comprising:

storing a template comprising a copy of the first set of private storage units and the first usage map of a first server.

20. The method of claim 19, further comprising:

creating a file system for a second server using the stored template.

21. The method of claim 17, wherein the shared and private storage units are stored in a same storage device.

22. The method of claim 17, wherein the shared storage units are stored a local storage device and the private storage units are stored in a remote storage device.

23. The method of claim 17, wherein the first usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding private storage unit contains valid data.

24. The method of claim 17, wherein providing a first usage map comprises:

initializing the first usage map to indicate that none of the private storage units contain valid data.

25. The method of claim 17, further comprising:

intercepting an attempt to read a data item from a shared storage unit; and

in response to the first usage map indicating that the corresponding private storage unit contains valid data, reading the data item from the corresponding private storage unit.

26. The method of claim 17, further comprising:

intercepting an attempt to read a data item from a shared storage unit; and

in response to the first usage map indicating that the corresponding private storage unit does not contain valid data, reading the data item from the shared storage unit.

27. The method of claim 17, further comprising:

providing a second set of private storage units, each private storage unit of the second set corresponding to one of the shared storage units;

providing a second usage map for indicating which private storage units of second set contain valid data;

intercepting an attempt to write a data item to a shared storage unit;

writing the data item to the corresponding storage unit of the second set; and

storing an indication in the second usage map that the corresponding storage unit of the second set contains valid data.

28. The method of claim 27, wherein the second usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding private storage unit of the second set contains valid data.

29. The method of claim 27, wherein providing a second usage map comprises:

initializing the second usage map to indicate that none of the private storage units of the second set contain valid data.

30. The method of claim 27, further comprising:

intercepting an attempt to read a data item from a shared storage unit; and

in response to the second usage map indicating that the corresponding private storage unit of the second set contains valid data, reading the data item from the corresponding private storage unit of the second set.

31. The method of claim 27, further comprising:

intercepting an attempt to read a data item from a shared storage unit; and

in response to the second usage map indicating that the corresponding private storage unit of the second set does not contain valid data and the first usage map indicating that the corresponding private storage unit of the first set contains valid data, reading the data item from the corresponding private storage unit of the first set.

32. The method of claim 27, further comprising:

intercepting an attempt to read a data item from a shared storage unit; and

in response to the second usage map indicating that the corresponding private storage unit of the second set does not contain valid data and the first usage map indicating that the corresponding private storage unit of the first set does not contain valid data, reading the data item from the shared storage unit.

33. The method of claim 27, further comprising:

archiving each private storage unit of the first set for which an indication of valid data is stored in the first usage map.

34. The method of claim 33, further comprising:

merging the first set of private storage units with the second set of private storage units.

35. The method of claim 34, wherein merging comprises:

copying the private storage units of the first set that contain valid data to those corresponding private storage units of the second set that do not contain valid data; and

for each private storage unit of the first set that is copied, storing an indication of valid data in the second usage map.

36. A system for creating and archiving a file system within at least one storage device comprising a first set of storage units and a second set of storage units, each storage unit of the first set corresponding to a storage unit of the second set, the system comprising:

a first usage map for indicating which storage units of the second set contain valid data;

an interception module for intercepting an attempt to write a data item to a storage unit of the first set;

a storage unit writing module for writing the data item to the corresponding storage unit of the second set; and

a usage map updating module for storing an indication in the first usage map that the corresponding storage unit of the second set contains valid data.

37. The system of claim 36, wherein the first and second sets of storage units are stored in a same storage device.

38. The system of claim 36, wherein the first set of storage units is stored in a local storage device and the second set of storage units is stored in a remote storage device.

39. The system of claim 36, wherein the first usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding storage unit of the second set contains valid data.

40. The system of claim 36, wherein the first usage map is initially reset to indicate that none of the storage units of the second set contain valid data.

41. The system of claim 36, wherein the interception module is further configured to intercept an attempt to read a data item from a storage unit of the first set, the system further comprising:

a storage unit reading module configured, in response to the first usage map indicating that the corresponding storage unit of the second set contains valid data, to read the data item from the corresponding storage unit of the second set.

42. The system of claim 36, wherein the interception module is further configured to intercept an attempt to read a data item from a storage unit of the first set, the system further comprising:

a storage unit reading module configured, in response to the first usage map indicating that the corresponding storage unit of the second set does not contain valid data, to read the data item from the storage unit of the first set.

43. The system of claim 36, wherein the at least one storage device further comprises a third set of storage units, each storage unit of the third set corresponding to one of the storage units of the first set, the system further comprising a second usage map for indicating which storage units of third set contain valid data.

44. The system of claim 43, wherein the interception module is further configured to intercept an attempt to write a data item to a storage unit of the first set;

wherein the storage unit writing module is further configured to write the data item to the corresponding storage unit of the third set; and

wherein the usage map updating module is further configured to store an indication in the second usage map that the corresponding storage unit of the third set contains valid data.

45. The system of claim 44, wherein the second usage map comprises a bitmap, each bit of the bitmap indicating whether a corresponding storage unit of the third set contains valid data.

46. The system of claim 44, wherein the second usage map is initially reset to indicate that none of the storage units of the third set contain valid data.

47. The system of claim 44, wherein the interception module is further configured to intercept an attempt to read a data item from a storage unit of the first set, the system further comprising:

a storage unit reading module configured, in response to the second usage map indicating that the corresponding storage unit of the third set contains valid data, to read the data item from the corresponding storage unit of the third set.

48. The system of claim 44, wherein the interception module is further configured to intercept an attempt to read a data item from a storage unit of the first set, the system further comprising:

a storage unit reading module configured, in response to the second usage map indicating that the corresponding storage unit of the third set does not contain valid data and the first usage map indicating that the corresponding storage unit of the second set contains valid data, to read the data item from the corresponding storage unit of the second set.

49. The system of claim 44, wherein the interception module is further configured to intercept an attempt to read a data item from a storage unit of the first set, the system further comprising:

a storage unit reading module configured, in response to the second usage map indicating that the corresponding storage unit of the third set does not contain valid data and the first usage map indicating that the corresponding storage unit of the second set does not contain valid data, to read the data item from the storage unit of the first set.

50. The system of claim 44, further comprising:

an archival module configured to archive each storage unit of the second set for which an indication of valid data is stored in the first usage map.

51. The system of claim 50, further comprising:

a merging module configured to merge the second set of storage units with the third set of storage units.

52. The system of claim 51, wherein the merging module is further configured to copy the storage units of the second set that contain valid data to those corresponding storage units of the third set that do not contain valid data;

and wherein the storage unit update module is configured, for each storage unit of the second set that is copied, to store an indication of valid data in the second usage map.

53. An computer program product for creating and archiving a file system within at least one storage device comprising a first set of storage units and a second set of storage units, each storage unit of the first set corresponding to a storage unit of the second set, the computer program product comprising:

a first usage map for indicating which storage units of the second set contain valid data;

program code for intercepting an attempt to write a data item to a storage unit of the first set;

program code for writing the data item to the corresponding storage unit of the second set; and

program code for storing an indication in the first usage map that the corresponding storage unit of the second set contains valid data.