Apparatus and method for maintaining a file system index

• US 8,135,763 B1
• Filed: 09/30/2005
• Issued: 03/13/2012
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Compton, James T., Doohkan, Surrendranath R., Fridella, Stephen A., Gupta, Uday K., Popov, Nikolay, Rago, Stephen A.
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Khoshnoodi, Fariborz
• Application Number:
  US11/241,054
• Time in Patent Office: 2,356 Days
• Field of Search: 707/100, 707/2, 707/822, 707/823, 714/6
• US Class Current: 707/822
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06

Abstract

An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Claims

1. A method for maintaining an index to files in a file system, the index including identifiers of files having key attribute values of interest, the method including the step of:

passing an Index Build Command from a policy server to the file system;

constructing, within the said file system having said files, the index in response to receipt of the Index Build command received from the policy server, such policy server is configured to apply policy rules generated to control the placement of data and metadata on various storage systems, the index is associated with at least one policy of the policy server;

monitoring, in said file system, said files to detect changes in one or more key attribute values of interest; and

updating the index within the said file system in accordance with the key attribute value changes.

2. The method of claim 1, wherein the step of updating the index is synchronized with the change in key attribute value.

3. The method of claim 1, wherein the step of updating the index is performed asynchronously of the key attribute value change.

4. The method of claim 1, wherein the step of monitoring the file system files includes the steps of:

periodically storing file system snapshots;

differentiating sequential file system snapshot pairs to identify changed key attribute values; and

updating the index in accordance with the changed key attribute values.

5. The method of claim 1, wherein the step of updating the index in accordance with the changed key attribute values includes the step of maintaining consistency between the index and the file system.

6. The method of claim 5, wherein the step of maintaining consistency includes the step of logging changes to the index in a journal.

7. The method of claim 6, wherein the step of updating the index is synchronized with the key attribute value change, and wherein the step of journaling index updates includes the steps of:

logging a file event in response to a file access by a client;

executing the file event;

determining whether the index is affected by the file event; and

updating the index in accordance with key attribute value changes resulting from the file event; and

following the updating of the index, returning a result of the file access to the client.

8. The method of claim 6, wherein the step of updating the index occurs asynchronously of the key attribute value change, and wherein the step of journaling index updates includes the steps of:

logging a file event in response to a file access by a client;

executing the file event;

returning a result of the file access to the client;

following the return of the result to the client, determining whether the index is affected by the file event; and

updating the index in accordance with key attribute value changes resulting from the file event.

15. The method recited in claim 1 wherein one of the key attributes is an access frequency attribute that tracks the number of accesses to a file within a certain time delta.

16. The method recited in claim 1 wherein one of the key attributes is an index access frequency array attribute which tracks the number of accesses to a file within a certain time delta to previous X time deltas.

9. An apparatus, comprising:

a file server;

a processor for maintaining an index to files in a file system of the file server, the index including identifiers of files having one or more key attribute values of interest, the processor;

passing means for passing an Index Build Command from a policy server to the file system;

generating means for generating the index within said file system having said files in response to the Index Build command received from a the policy server, such policy server is configured to apply policy rules generated to control the placement of data and metadata on various storage systems the Index Build command including the one or more key attribute values of interest;

file system monitoring means for monitoring in said file system events to detect key attribute value changes; and

index update means for updating the index in said file system in accordance with detected key attribute value changes.

10. The apparatus of claim 9, wherein index updates are synchronized to associated key attribute value changes.

11. The apparatus of claim 9, wherein index updates operate asynchronously of associated key attribute value changes.

12. The apparatus of claim 9, wherein the file system monitoring means further comprises:

storage for storing periodic file system snapshots;

means for generating a delta-set for use by the update means, the delta set identifying files having changed key attribute value changes in a sequential file system snapshot pair.

13. The apparatus of claim 9, wherein the updating means includes consistency logic for maintaining consistency between the index and the file system.

14. The apparatus of claim 9, wherein the consistency logic includes:

a journal for logging intended changes to an index prior to changing the index.