System and method for multi-tiered meta-data caching and distribution in a clustered computer environment

• US 8,713,077 B2
• Filed: 01/27/2010
• Issued: 04/29/2014
• Est. Priority Date: 04/29/2005
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Kazar, Michael, Jernigan, Richard P. IV, Sanzi, Richard N. Jr., Hyer, Robert Wyckoff Jr.
• Primary Examiner(s): KUDDUS, DANIEL A
• Application Number:
  US12/694,714
• Publication Number:
  US 20100138605A1
• Time in Patent Office: 1,553 Days
• Field of Search: 707/812, 707/823, 707/999.104, 707/737
• US Class Current: 707/812
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A system and method caches and distributes meta-data for one or more data containers stored on a plurality of volumes configured as a striped volume set (SVS) and served by a plurality of nodes interconnected as a cluster. The SVS comprises one meta-data volume (MDV) configured to store a canonical copy of certain meta-data, including access control lists and directories, associated with all data containers stored on the SVS, and one or more data volumes (DV) configured to store, at least, data content of those containers. In addition, for each data container stored on the SVS, one volume is designated a container attribute volume (CAV) and, as such, is configured to store (“cache”) a canonical copy of certain, rapidly-changing attribute meta-data, including time stamps and container length, associated with that container.

Claims

1. A computer-implemented method for distributing meta-data of a data container, comprising:

striping a first data volume across a plurality of logical volumes configured to form a striped volume set, wherein each logical volume is a logical arrangement of a plurality of storage devices coupled to a computer, and wherein the first data volume includes the data container;

storing a copy of the meta-data of the data container in a meta-data volume of the plurality of logical volumes,

designating a second data volume of the plurality of logical volumes as a container attribute volume for the data container, wherein the container attribute volume serves as a first tier caching and distribution point within the striped volume set for the meta-data in the meta-data volume in response to receiving a data access request, and wherein the data container and container attribute volume are stored on different data volumes;

invalidating a cached copy of the meta-data stored on the container attribute volume in response to receiving the data access request, wherein invalidating the cached copy of the meta-data comprises setting a meta-data invalidation flag of a data structure of the data container to identify that the cached copy of the meta-data is invalid; and

updating the cached copy of the meta-data stored on the container attribute volume, wherein the data access request is one of a request to modify and to create the meta-data.

2. The method of claim 1 further comprising transmitting a command to invalidate the cached copy of the meta-data on each data volume.

3. The method of claim 1 wherein the container attribute volume is selected as the data volume storing a first stripe of data of the data container.

4. The method of claim 1 wherein the meta-data comprises an access control list.

5. The method of claim 1 wherein the data container comprises a file.

6. The method of claim 1, further comprising:

requesting an update of meta-data from the copy of the meta-data in the meta-data volume, wherein the copy of the meta-data in the meta-data volume is a canonical copy.

7. The method of claim 1 further comprising:

receiving the data access request by a first module of the computer serving the first data volume;

determining whether a first local copy of the meta-data stored on the first data volume is valid;

requesting a second module serving the container attribute volume to send a second local copy of the meta-data to the first module in response to determining that the first local copy is not valid;

determining whether the second local copy on the container attribute volume is valid; and

in response to determining that the second local copy is not valid, querying the meta-data volume for an updated version of the meta-data to be stored on at least one of the first data volume and the container attribute volume.

8. The method of claim 1 wherein a portion of the meta-data is not modified in response to the data access request.

9. The method of claim 1 wherein the copy of the meta-data is a canonical copy.

10. A computer-implemented system for distributing meta-data of a data container, comprising:

a hardware processor operatively connected to a computer configured to execute a storage operating system to stripe the data container across a plurality of logical volumes to form a striped volume set, wherein each logical volume is a logical arrangement of a plurality of storage devices coupled to the computer;

a meta-data volume of the plurality of logical volumes configured to store a copy of meta-data of the data container;

one or more data volumes of the plurality of logical volumes configured as a container attribute volume for the data container, wherein the container attribute volume is configured to serve as a first tier caching and distribution point within the striped volume set for the meta-data in the meta-data volume in response to receiving a data access request, and wherein the data container and container attribute volume are stored on different data volumes; and

a module of the storage operating system serving the meta-data volume configured to invalidate a cached copy of the meta-data stored on the container attribute volume, a data structure of the data container comprising a meta-data invalidation flag to identify that the cached copy of the meta-data is invalid, the module of the storage operating system serving the meta-data volume configured to update the copy of the meta-data stored on the container attribute volume.

11. The computer-implemented system of claim 10 further comprising a volume striping module of the storage operating system configured to transmit a command to invalidate the cached copy of the meta-data on each data volume.

12. The computer-implemented system of claim 10 wherein the container attribute volume is selected as the data volume storing a first stripe of data of the data container.

13. The computer-implemented system of claim 10 wherein the meta-data comprises an access control list.

14. The computer-implemented system of claim 10 wherein the data container comprises a file.

15. The computer-implemented system of claim 10 wherein an update of meta-data from the copy of the meta-data in the meta-data volume is requested.

16. The computer-implemented system of claim 10 further comprising:

a first volume striping module of the computer configured to serve one of the data volumes and configured to receive the data access request at the first volume striping module and further configured to determine whether a first local copy of the meta-data stored on the data volume is valid;

a second volume striping module configured to serve the container attribute volume and further configured, in response to determining that the first local copy is not valid, to receive a command from the first volume striping module to send a second local copy of cached meta-data stored on the container attribute volume to the first volume striping module, the second volume striping module further configured to determine whether the second local copy is valid; and

the second volume striping module further configured, in response to determining that the second local copy is not valid, to query the meta-data volume for an updated version of the meta-data to be stored on at least one of the data volume and the container attribute volume.

17. The computer-implemented system of claim 10 wherein the meta-data comprises a portion of meta-data that is not modified on the data access request.

18. The computer-implemented system of claim 10 wherein the copy of the meta-data is a canonical copy.

19. A non-transitory computer readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that stripe a data container across a plurality of logical volumes configured to form a striped volume set, wherein each logical volume is a logical arrangement of a plurality of storage devices coupled to a computer;

program instructions that store a copy of meta-data of the data container in a meta-data volume of the plurality of logical volumes, wherein the meta-data is organized into a plurality of categories based on a frequency that meta-data changes in response to receiving a data access request directed to the data container, wherein a first category of the plurality of categories includes an access control list;

program instructions that designate a container attribute volume for the data container, wherein the container attribute volume is formed from the stripe volume set, wherein the container attribute volume serves as a first tier caching and distribution point within the striped volume set for the meta-data in the meta-data volume in response to receiving the data access request directed to the data container, and wherein the data container and container attribute volume are stored on different logical volumes; and

program instructions that invalidate a cached copy of the meta-data stored on the container attribute volume in response to receiving the data access request.

20. A computer-implemented method, comprising:

striping a data container across a plurality of logical volumes, wherein each logical volume is a logical arrangement of a plurality of storage devices;

organizing meta-data of the data container into a plurality of categories based on a frequency that meta-data changes in response to receiving a data access request directed to the data container, wherein a first category of the plurality of categories includes an access control list;

storing a copy of meta-data in the first category of the plurality of categories at a meta-data volume, a data volume, and a container attribute volume, wherein the container attribute volume is striped across the plurality of logical volumes, wherein the container attribute volume serves as a first tier caching and distribution point for the meta-data in the meta-data volume in response to receiving the data access request directed to the data container, wherein the data container and container attribute volume are stored on different logical volumes;

determining that a first local copy of the meta-data stored at the container attribute volume is invalid;

querying the meta-data volume for a valid copy of the meta-data in response to determining the first local copy is invalid; and

storing the valid copy of the meta-data at the container attribute volume.

21. The computer-implemented method of claim 20 wherein determining that the local copy of the meta-data stored at the container attribute volume is invalid further comprises:

determining that a second local copy of the meta-data stored at the data volume is invalid; and

querying the container attribute volume for the first local copy in response to determining the second local copy is invalid.