Log device layered transparently within a filesystem paradigm

• US 5,832,515 A
• Filed: 09/12/1996
• Issued: 11/03/1998
• Est. Priority Date: 09/12/1996
• Status: Expired due to Term

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Veritas Software Corporation (Whitehouse Group Ltd.)
• Inventors: Colgrove, John A., Koren, Dan, Ledain, Joel E.
• Primary Examiner(s): Kulik, Paul V.
• Application Number:
  US08/711,889
• Time in Patent Office: 782 Days
• Field of Search: 395/681, 395/618, 395/619, 395/621, 395/622, 395/438-441, 395/497.01, 395/497.04, 707/202, 707/203, 707/205, 707/206
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0674 Disk device
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A log device based data storage subsystem provides for the efficient storage and retrieval of data with respect to an operating system executing on a computer system coupled to the data storage system. The data storage system includes a storage device providing for the storage of predetermined file and system data, as provided by the computer system, within a main filesystem layout established in the storage device. The data storage system also includes a log device coupled in the logical data transfer path between storage device and the computer system. The log device provides for the storage of the predetermined file and system data within a log structured filesystem layout established in the log device. A control program, included as part of the data storage system, is executed in connection with the log device and provides log structured filesystem management over the log device to store the predetermined file and system data in one of a plurality of data segments, delimited by a first free data segment and an oldest filled data segment, to selectively clean the oldest filled data segment to the first free data segment, and to selectively transfer the predetermined file and system data from the log device to the storage device. The control program utilizes location data provided in the predetermined file and system data to identify a destination storage location for the predetermined file and system data within the main filesystem layout.

Claims

1. A data storage subsystem coupleable to computer system providing for the storage and retrieval of data with respect to a primary mass storage device as defined in accordance with a filesystem of said computer system, said data storage system comprising:

a) a first mass storage device providing for the storage and retrieval of data with respect to said computer system; and

b) a log device driver coupleable to said filesystem of said computer system and said first mass storage device, said log device driver implementing a translation map defined relationship between predetermined data storage portions of said primary mass storage device and said first mass storage device that is transparent to the operation of said filesystem.

2. The data storage subsystem of claim 1 wherein said log device driver internally implements said translation map defined relationship wherein data stored to said first mass storage device preserves an identification of a corresponding predetermined data storage portion of said primary mass storage device.

3. The data storage subsystem of claim 1 wherein said filesystem is operable through a primary device driver to store and retrieve data with respect to predetermined primary addresses within the logical data storage space of said primary mass storage device and wherein said log device driver is coupleable between said filesystem and said primary device driver to provide for the selective transfer of data with respect to said first mass storage device subject to a translation mapping between said predetermined primary addresses and predetermined first addresses within the logical data storage space of said first mass storage device.

4. The data storage subsystem of claim 3 wherein said predetermined first addresses are selectively accessible through said log device driver independent of said filesystem.

5. The data storage subsystem of claim 4 wherein said log device driver provides for the selective storage and retrieval of data from said first mass storage device and said primary mass storage device, said log device driver selectively defining data transfer paths optimized for the writing of data to said first mass storage device and the reading of data from said primary mass storage device.

6. A data storage subsystem for use cooperatively with a computer system including a filesystem and a main device driver coupling said filesystem to a main mass storage disk drive for the storage and retrieval of data identifiable by disk identification and main storage address, said data storage subsystem comprising:

a) a log mass storage disk drive providing for the storage of data in a plurality of segments; and

b) a log device driver selectively coupleable between said filesystem and said main device driver, said log device driver providing for the selective construction and storage of a predetermined data block within a predetermined one of said segments from data directed by said filesystem to be stored at a predetermined main storage address, said log device driver providing for the establishment of a translation map relating said predetermined main storage address with said predetermined one of said segments.

7. The data storage subsystem of claim 6 wherein said predetermined one of said segments includes a plurality of data blocks, wherein said predetermined data block is one of said plurality of data blocks within said predetermined one of said segments, and wherein said predetermined main storage address is associated with said predetermined data block as stored by said log mass storage disk drive.

8. The data storage subsystem of claim 7 wherein said log device driver provides support for reordering said predetermined data block within said predetermined one of said segments.

9. The data storage subsystem of claim 8 wherein said log device driver supports reordering as a segment cleaning operation separate from the operation of said filesystem.

10. The data storage subsystem of claim 6 wherein said log device driver selectively directs the routing of data between said filesystem, said main mass storage disk drive, and said log mass storage disk when said log device driver is coupled between said filesystem and said main device driver.

11. The data storage subsystem of claim 6 wherein said log device driver comprises:

a) a command interface coupleable to said operating system;

b) a data interface coupleable between said filesystem and said main device driver; and

c) a segment manager selectively coupleable between said data interface and said main device driver, said segment manager providing for the organization of predetermined write data as received by said data interface as a predetermined data block and for organizing a plurality of data blocks including said predetermined data block into a predetermined segment storable as said predetermined one of said segments.

12. The data storage subsystem of claim 11 wherein said segment manager includes:

a) a log cleaner that selectively directs the reading of a trailing one of said plurality of segments from said log disk drive and determines whether to rewrite said trailing one of said plurality of segments to said log disk drive; and

b) a log manager that directs the organizing of said plurality of data blocks into said predetermined segment, said log manager being coupled to said log cleaner wherein said plurality of data blocks may be obtained from said trailing one of said plurality of segments.

13. A data storage subsystem providing for the storage and retrieval of data with respect to an operating system including a filesystem executing on a computer system, said data storage system comprising:

a) a main storage device providing for the storage of predetermined data, as provided by said filesystem, at a predetermined main store address within a main filesystem layout in said storage device as managed by said filesystem;

b) a log storage device coupled in a logical data transfer path between said filesystem and said main storage device, said log storage device providing for the storage of said predetermined data within a log structured layout in said log storage device including a plurality of data segment storage locations; and

c) a control program executable by said computer system and providing log structured layout management over said log storage device to store said predetermined data in a predetermined data segment with predetermined location data indicative of said predetermined main store address and to further store said predetermined data segment to said log storage device at a predetermined data segment storage location.

14. The data storage subsystem of claim 13 wherein said control program further provides for the reading of said predetermined data segment from said log storage device transparent to the operation of said filesystem and wherein said control program provides for the writing of said predetermined data to said main store device at said predetermined main store address as determinable from said predetermined location data.

15. The data storage subsystem of claim 14 wherein said control program provides for an atomic write of said predetermined data and said predetermined location data to said log storage device.

16. The data storage subsystem of claim 15 wherein said control program selectively provides for the storage of first predetermined data by said log storage device and for the storage of second predetermined data by said main storage device subject to the control of said filesystem, said control program further providing for the transfer of said first predetermined data to said main storage device transparently with respect to the storage of said second predetermined data to said main storage device by said filesystem.

17. The data storage subsystem of claim 16 wherein said control program utilizes a translation map to store said predetermined location data and wherein said control program provides for the writing of said translation map to said log storage device as a translation map data segment.

18. The data storage subsystem of claim 17 wherein said control program provides for at least a predetermined portion of said translation map to be written to said log storage device as part of said predetermined data segment.

19. The data storage subsystem of claim 18 wherein said control program provides for the log ordered reading and writing of data segments, including said translation map data segment and said predetermined data segment, with respect to an ordered set of said plurality of data segment storage locations.

20. A data storage subsystem transparently and selectively providing an optimized write data path for logical data blocks provided by a filesystem of an operating system executable by a computer system that nominally provides for the storage of logical data blocks on a main filesystem disk drive at locations defined by logical data block numbers, said data storage subsystem comprising:

a) log device disk drive providing for the storage of a plurality of data segments, each including a respective plurality of logical data blocks, and wherein said plurality of data segments are stored by said log device disk drive at locations defined by segment numbers; and

b) a device driver executable in support of said filesystem to organize a plurality of predetermined logical data blocks into a first predetermined data segment and directing the storage of said predetermined data segment to said log device disk drive, said device driver providing for the construction of a location translation map system relating the logical data block numbers of said predetermined logical data blocks to a first predetermined segment number identifying said first predetermined data segment, said device driver supporting the partitioning of said translation map system into map blocks substitutable as logical data blocks within a second predetermined data segment storable by said log device disk drive.

21. The data storage subsystem of claim 20 wherein said device driver generates and uses said first predetermined segment number independent of said filesystem.

22. The data storage subsystem of claim 21 wherein said device driver provides for the lookup of said predetermined segment number using any of the logical block numbers of said predetermined logical data blocks utilizing said location translation map system.

23. The data storage subsystem of claim 22 wherein said device driver provides for the rewriting of said first predetermined data segment to said log device disk drive at a second predetermined segment number, said device driver providing for the selective substitution of logical data blocks within a first respective plurality of logical data blocks of said first predetermined data segment, wherein selectively substituted logical data blocks of said first respective plurality of logical data blocks are transferable to said filesystem and said main filesystem disk drive.

24. A data storage subsystem providing a write optimized data storage path in a computer system having a main storage device provided to store data blocks at respective main store addresses, said data storage system comprising:

a) a log device providing a log structured storage space for a data segment, said log device being coupleable to the computer system to establish a data transfer path between the computer system and said log device; and

b) a control program, executable by said computer system, providing for the processing of a predetermined data block and a corresponding predetermined main store address into a user data block and transfer of said user data block to said log device for storage in said data segment, and further providing for the transfer of said user data block to said computer system to allow storage of said predetermined data block in said main storage device at said corresponding main store address.

25. The data storage subsystem of claim 24 wherein said control program provides for the storage of said user data block in said data segment through an atomic write of said user data block.

26. The data storage subsystem of claim 25 wherein said control program selects between a first data block and a second data block to provide for storage of said first data block, as a first user data block, in said data segment and provides for said second data block to be stored in said main storage device.

27. The data storage subsystem of claim 26 wherein said computer system executes a filesystem to provide for the storage and retrieval of data with respect to said main storage device and wherein said control program operates transparently with respect to said filesystem.