Filesystem-aware block storage system, apparatus, and method

• US 7,873,782 B2
• Filed: 05/03/2007
• Issued: 01/18/2011
• Est. Priority Date: 11/05/2004
• Status: Active Grant

• Current Assignee: Storcentric DIP Lender LLC
• Original Assignee: Data Robotics Incorporated
• Inventors: Clarkson, Neil A., Terry, Julian M., Barrall, Geoffrey S.
• Primary Examiner(s): Bragdon; Reginald G
• Assistant Examiner(s): Sadler; Nathan
• Application Number:
  US11/743,967
• Publication Number:
  US 20070266037A1
• Time in Patent Office: 1,356 Days
• Field of Search: None
• US Class Current: 711/114
• CPC Class Codes: G06F 11/1084 Degraded mode, e.g. caused by single or multiple storage removals or disk failures
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2087 with a common controller
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/325 by lamps or LED's
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: G06F 2211/1023 Different size disks, i.e. non uniform size of disks in RAID systems with parity
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11B 2220/415 Redundant array of inexpensive disks [RAID] systems
• CPC Class Codes: G11B 2220/65 wherein solid state memory is used for storing indexing information or metadata
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/329 on a disc [VTOC]

Abstract

A filesystem-aware storage system locates and analyzes host filesystem data structures in order to determine storage usage of the host filesystem. To this end, the storage system might locate an operating system partition, parse the operating system partion to locate its data structures, and parse the operating system data structures to locate the host filesystem data structures. The storage system manages data storage based on the storage usage of the host file system. The storage system can use the storage usage information to identify storage areas that are no longer being used by the host filesystem and reclaim those areas for additional data storage capacity. Also, the storage system can identify the types of data stored by the host filesystem and manage data storage based on the data types, such as selecting a storage layout and/or an encoding scheme for the data based on the data type.

Claims

1. A method of supplementing functionality of a block-level storage system that is capable of storing fixed-size blocks of data under the control of a host filesystem, the host filesystem currently operating in a host computer, the method comprising:

in response to receiving, from the currently operating file system, a client request having fixed-sized blocks of data, analyzing the received blocks of data to identify the host filesystem type from among a plurality of supported host filesystem types;

locating host filesystem data structures stored in the block-level storage system based on the identified host filesystem type;

analyzing the host filesystem data structures to determine usage, by the host filesystem, of the fixed-size blocks of data stored in the block-level storage system; and

managing storage of the fixed-size blocks of data in the block-level storage system based on the analysis;

the first analyzing, locating, second analyzing, and managing steps being performed, outside the control of the host computer, by a storage controller disposed in the block-level storage system.

2. A method according to claim 1, wherein locating host filesystem data structures in storage comprises:

maintaining a partition table;

parsing the partition table to locate an operating system partition;

parsing the operating system partition to identify the operating system and locate operating system data structures; and

parsing the operating system data structures to identify the host filesystem and locate the host filesystem data structures.

3. A method according to claim 2, wherein the host filesystem data structures include a superblock, and the method further comprises parsing the superblock to identify used and unused blocks.

4. A method according to claim 2, wherein parsing the host filesystem data structures comprises:

making a working copy of a host filesystem data structure; and

parsing the working copy.

5. A method according to claim 1, wherein analyzing the host filesystem data structures comprises identifying fixed-size blocks of data that have been released by the host filesystem, and wherein managing storage comprises making the blocks of data so identified available for additional storage.

6. A method according to claim 1, wherein analyzing the host filesystem data structures comprises identifying a data type associated with data to be stored, and wherein managing storage comprises storing such data using a storage scheme selected based on the data type, data having different data types being stored according to different storage schemes.

7. A method according to claim 6, wherein storing the data includes using a physical storage layout selected based on the data type.

8. A method according to claim 6, wherein storing the data includes storing frequently accessed data so as to provide enhanced storage accessibility.

9. A method according to claim 8, wherein storing the data includes storing the data uncompressed, in sequential storage, or both.

10. A method according to claim 6, wherein storing the data includes storing infrequently accessed data so as to provide enhanced storage efficiency.

11. A method according to claim 10, wherein storing the data includes using at least one of data compression and non-sequential storage.

12. A method according to claim 6, wherein storing the data includes encoding the data according to an encoding scheme selected based on the data type.

13. A method according to claim 12, wherein encoding the data comprises data compression or encryption.

14. A storage controller capable of storing and retrieving fixed-size blocks of data in a block-level storage system under the control of a host filesystem currently operating in a separate host computer, the data including host filesystem data structures, the storage controller comprising:

a parser configured to identify a host filesystem type from among a plurality of supported host filesystem types in response to receiving, from the currently-operating host filesystem, a client request having fixed-size blocks of data, and further configured to locate host filesystem data structures stored in the block-level storage system based on the identification;

an analyzer configured to analyze the host filesystem data structures to determine usage, by the host filesystem, of the fixed-size blocks of data stored in the block-level storage system; and

a storage manager configured to manage storage of the fixed-size blocks of data in the block-level storage system based on the analysis,

the storage controller being configured to identify, locate, analyze, and manage outside the control of the host computer.

15. A system according to claim 14, wherein the parser is further configured to:

maintain a partition table in the block-level storage system;

parse the partition table to locate an operating system partition;

parse the operating system partition to identify the operating system and to locate operating system data structures; and

parse the operating system data structures to identify a host filesystem type and to locate the host filesystem data structures.

16. A system according to claim 15, wherein the host filesystem data structures include a superblock, and wherein the parser is further configured to parse the superblock to identify used and unused blocks.

17. A system according to claim 14, wherein the analyzer is further configured to analyze a host filesystem data structure by copying the data structure and parsing the copy.

18. A system according to claim 14, wherein the analyzer is further configured to identify fixed-size blocks of data that have been released by the host filesystem, and the storage manager is further configured to make the released blocks so identified available for additional storage.

19. A system according to claim 14, wherein the analyzer is further configured to identify a data type associated with data to be stored, and wherein the storage manager is further configured to store such data using a storage scheme selected based on the data type, the storage manager storing data having different data types according to different storage schemes.

20. A system according to claim 19, wherein the storage controller is further configured to store the data using a physical storage layout selected based on the data type.

21. A method according to claim 19, wherein the storage controller is further configured to store frequently accessed data so as to provide enhanced storage accessibility.

22. A system according to claim 21, wherein the storage controller is further configured to store frequently requested data uncompressed, in sequential storage, or both.

23. A method according to claim 19, wherein the storage controller is further configured to store infrequently accessed data so as to provide enhanced storage efficiency.

24. A system according to claim 23, wherein the storage controller is further configured to store infrequently requested data using at least one of data compression and non-sequential storage.

25. A system according to claim 19, wherein the storage controller is further configured to encode the data according to an encoding scheme selected based on the data type.

26. A system according to claim 25, wherein the encoding scheme comprises data compression or encryption.