System and method for multi-tiered meta-data caching and distribution in a clustered computer environment

• US 7,698,334 B2
• Filed: 04/29/2005
• Issued: 04/13/2010
• Est. Priority Date: 04/29/2005
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Sanzi, Richard N. Jr., Jernigan, Richard P. IV, Hyer, Robert Wyckoff Jr., Kazar, Michael
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): KUDDUS, DANIEL A
• Application Number:
  US11/119,277
• Publication Number:
  US 20060248088A1
• Time in Patent Office: 1,810 Days
• Field of Search: None
• US Class Current: 707/737
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A system and method caches and distributes meta-data for one or more data containers stored on a plurality of volumes configured as a striped volume set (SVS) and served by a plurality of nodes interconnected as a cluster. The SVS comprises one meta-data volume (MDV) configured to store a canonical copy of certain meta-data, including access control lists and directories, associated with all data containers stored on the SVS, and one or more data volumes (DV) configured to store, at least, data content of those containers. In addition, for each data container stored on the SVS, one volume is designated a container attribute volume (CAV) and, as such, is configured to store (“cache”) a canonical copy of certain, rapidly-changing attribute meta-data, including time stamps and container length, associated with that container.

Claims

1. A computer-implemented method for distributing meta-data relating to a plurality of data containers in a striped volume set, comprising:

striping the data containers across a plurality of volumes configured to form the striped volume set, wherein the plurality of volumes are stored across a plurality of computers with each computer storing at least one volume and wherein each volume is a logical arrangement of a plurality of storage devices;

storing a canonical copy of the meta-data of each of the plurality of data containers in the striped volume set in a meta-data volume, wherein the canonical copy of the meta-data comprises an access control list;

organizing the meta-data into a plurality of categories, wherein a first category comprises meta-data that changes for every data access request served by the striped volume set, and wherein a second category comprises meta-data that is read for every data access request served by the striped volume set; and

designating each data volume as a container attribute volume for at least one data container in the striped volume set, wherein the container attribute volume comprises the meta-data of the first category and the second category, and wherein the container attribute volume serves as a first tier caching and distribution point within the striped volume set.

2. The method of claim 1 further comprising:

receiving a request to modify/create the meta-data by a volume striping module serving the meta-data volume;

invalidating a cached copy of the meta-data stored on the container attribute volume; and

processing the request to modify/create the canonical copy of the meta-data.

3. The method of claim 2 wherein invalidating the cached copy of the meta-data further comprises setting a meta-data invalidation flag to identify that the cached copy of the meta-data is invalid.

4. The method of claim 2 wherein invalidating the cached copy of the meta-data further comprises transmitting a request to invalidate a cached copy of the meta-data on each data volumes.

5. The method of claim 1 wherein the container attribute volume is selected as the data volume storing a first stripe of data of the data container.

6. The method of claim 1 wherein the data container comprises a file.

7. The method of claim 1 wherein the data container comprises a logical unit number.

8. The method of claim 1 further comprising:

receiving, by a volume striping module serving one of the data volumes, a data access operation;

determining whether a copy of the meta-data stored on the data volume is valid;

in response to determining that the copy of the meta-data is not valid, requesting a volume striping module serving the container attribute volume to send a copy of cached meta-data to the volume striping module serving the data volume;

determining, by the volume striping module serving the container attribute volume, whether cached copy of the meta-data is valid; and

in response to determining that the container attribute volume'"'"'s cached copy of the meta-data is not valid, updating the cached copy of the meta-data by querying the meta-data volume.

9. The method of claim 1 wherein the data container is a file and the meta-data comprises information about the file.

10. The method of claim 1 wherein the first category of meta-data comprises time stamps.

11. A computer-implemented system for distributing meta-data relating to a data container in a striped volume set, comprising:

a storage system operatively interconnected with one or more processors configured to store a canonical copy of the meta-data in a meta-data volume, wherein the canonical copy of the meta-data comprises an access control list;

the one or more processors configured to organize the meta-data into a plurality of categories, wherein a first category comprises meta-data that changes for every data access request served by the striped volume set, and wherein a second category comprises meta-data that is read for every data access request served by the striped volume set; and

one or more data volumes configured to organized and to store stripes of data of the data container, wherein each of the data volumes is further configured to be designated as a container attribute volume for at least one data container in the striped volume set, wherein the container attribute volume comprises the meta-data of the first category and the second category, and wherein the container attribute volume serves as a first tier caching and distribution point within the striped volume set.

12. The system of claim 11 wherein the container attribute volume is selected as the data volume storing a first stripe of data of the data container.

13. The system of claim 11 wherein the data container comprises a file.

14. The system of claim 11 wherein the data container comprises a logical unit number.

15. The computer implemented system of claim 11 wherein the first category of meta-data comprises time stamps.

16. The computer implemented system of claim 11 wherein the data container is a file and the meta-data comprises information about the file.

17. A computer readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that stripe a plurality of data containers across a plurality of volumes configured to form a striped volume set, wherein the plurality of volumes are stored across a plurality of computers with each computer storing at least one volume and wherein each volume is a logical arrangement of a plurality of storage devices;

program instructions that store a canonical copy of meta-data of each of the plurality of data containers in the striped volume set in a meta-data volume, wherein the canonical copy of the meta-data comprises an access control list;

program instructions that organize the meta-data into a plurality of categories, wherein a first category comprises meta-data that changes for every data access request served by the striped volume set, and wherein a second category comprises meta-data that is read for every data access request served by the striped volume set; and

program instructions that designate each data volume as a container attribute volume for at least one data container in the striped volume set, wherein the container attribute volume comprises the meta-data of the first category and the second category, and wherein the container attribute volume serves as a first tier caching and distribution point within the striped volume set.

18. The computer readable storage medium of claim 17 further comprising:

program instructions that receive a request to modify/create the meta-data by a volume striping module serving the meta-data volume;

program instructions that invalidate a cached copy of the meta-data stored on the container attribute volume; and

program instructions that process the received request to modify/create the canonical copy of the meta-data.

19. The computer readable storage medium of claim 18 wherein the program instructions that invalidate the cached copy of the meta-data comprises:

program instructions that a meta-data invalidation flag to identify that the cached copy of the meta-data is invalid.

20. The computer readable storage medium of claim 18 wherein the program instructions that invalidate the cached copy of the meta-data comprises:

program instructions that transmit a request to one or more data volumes to invalidate a cached copy of the meta-data on each of the data volumes.

21. The computer readable storage medium of claim 17 wherein the container attribute volume is selected as the data volume storing a first stripe of data of the data container.

22. The computer readable storage medium of claim 17 wherein at least one data container comprises a file.

23. The computer readable storage medium of claim 17 wherein at least one data container comprises a logical unit number.

24. The computer readable storage medium of claim 17 further comprising:

program instructions that receive, by a volume striping module serving one or more data volumes, a data access operation;

program instructions that determine whether a copy of the meta-data stored on the one or more data volumes is valid;

in response to determining that the copy of the meta-data is not valid, program instructions that request a volume striping module serving the container attribute volume to send a copy of cached meta-data to the volume striping module serving the one or more data volumes;

program instructions that determine, by the volume striping module serving the container attribute volume, whether cached copy of the meta-data is valid; and

in response to determining that the container attribute volume'"'"'s cached copy of the meta-data is not valid, program instructions that update the cached copy of the meta-data by querying the meta-data volume.

25. The computer readable storage medium of claim 17 wherein the at least one data container is a file and the meta-data comprises information about the file.

26. The computer readable storage medium of claim 17 wherein the first category of meta-data comprises time stamps.