File system management with enhanced searchability

• US 8,417,746 B1
• Filed: 04/02/2007
• Issued: 04/09/2013
• Est. Priority Date: 04/03/2006
• Status: Active Grant

• Current Assignee: F5 Networks, Inc. (F5 Incorporated)
• Original Assignee: F5 Networks, Inc. (F5 Incorporated)
• Inventors: Berger, Michael A., Nicklin, Jonathan C., Cain, Bradley E., Gillett, Jr., Richard B.
• Primary Examiner(s): Singh, Amresh
• Application Number:
  US11/732,492
• Time in Patent Office: 2,199 Days
• Field of Search: 707/200, 707/829
• US Class Current: 707/829
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata

Abstract

A primary (e.g., master) file system stores multiple files and corresponding metadata. A view generator function receives template information (e.g., view configuration information) specifying types of metadata information associated with the multiple files stored in the primary file system. Based on processing of the metadata information in the primary file system as specified by the template information, the generator function produces metadata for inclusion in a secondary file system, which is used to satisfy search inquiries associated with the multiple files stored in the primary file system. According to one arrangement, the generator function replicates metadata information in the primary file system as specified by the template information for inclusion in the secondary file system. The secondary file system can include metadata from other sources as well that produced by the generator function. The secondary file system enables more efficient handling of search inquiries than applying search inquiries to the master file system.

Claims

1. A method comprising:

maintaining a primary repository to store multiple files and metadata information associated with the multiple files;

receiving template information specifying types of the metadata information associated with the multiple files stored in the primary repository; and

applying the template information to the metadata information in the primary repository to produce a secondary repository of information comprising one or more metadata files with a subset of the metadata information, the metadata files having filenames comprising at least filenames of the corresponding multiple files in the primary repository; and

receiving a search inquiry, utilizing a hierarchical file system in the secondary repository of information to perform a search, and identifying at least one of the multiple files in a hierarchical file system of the primary repository that satisfies the search inquiry based at least in part on at least a partial match of the filename of at least one corresponding metadata file.

2. A method as in claim 1, wherein applying the template information to the metadata information in the primary repository includes replicating the metadata information as specified by the template information for inclusion in the secondary repository.

3. A method as in claim 1 further comprising:

in response to receiving search criteria, utilizing the secondary repository of information to produce a reference to a corresponding file in the primary repository that has associated metadata satisfying the search criteria.

4. A method as in claim 1 further comprising:

maintaining the primary repository to include a first hierarchical tree structure for storing the multiple files and keeping track of corresponding metadata associated with the multiple files;

maintaining the secondary repository of information to include a second hierarchical tree structure based upon the first hierarchical tree structure; and

utilizing the template information to identify what, if any, portion of the multiple files and the corresponding metadata in the first hierarchical tree structure to include in the second hierarchical tree structure.

5. A method as in claim 1 further comprising:

based on application of the template information to a respective file in the primary repository, replicating a portion of the respective file in the primary repository for inclusion in the secondary repository.

6. A method as in claim 1, wherein applying the template information to the metadata information in the primary repository includes:

retrieving a given file from a file system in the primary repository;

based on a text string as specified in the template information, searching for instances of the text string in the given file; and

in response to detecting an instance of the text string in the file, maintaining the secondary repository of information to include an indication that the file includes at least one instance of the text string.

7. A method as in claim 1 further comprising:

for a given file of the multiple files in the primary repository, creating a corresponding view file of metadata information associated with the given file based on metadata information in the primary repository associated with the given file; and

storing the corresponding view file in the secondary repository, the corresponding view file having same filename and filepath information as the given file in the first repository.

8. A method as in claim 1, wherein applying the template information includes:

based on processing of the template information, identifying selection of path information as a parameter associated with a given file in the primary repository to copy to the secondary repository; and

replicating identified path information associated with the given file from the primary repository to the secondary repository.

9. A method as in claim 1 further comprising:

logging occurrence of file operations associated with the multiple files stored in a file system of the primary repository; and

initiating modification of the secondary repository in accordance with the logged file operations such that search inquiries applied to the secondary repository yield corresponding search results reflecting a current state of the primary repository including application of the file operations to the multiple files in the primary repository.

10. A method as in claim 1 further comprising:

logging occurrence of file operations applied to the multiple files stored in a file system of the primary repository;

initiating storage of modification information as searchable parameters in the secondary repository in accordance with the file operations; and

activating application of searches to the storage modification information in the second repository to search for files in the primary repository based on operations applied to the multiple files in the primary repository.

11. A method as in claim 1 further comprising:

activating a logging function that keeps track of changes to the primary repository to convert stored records into update operations for modifying the secondary repository such that the primary repository and the secondary repository are automatically synchronized with each other in real-time.

12. A method as in claim 1 further comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information associated with multiple corresponding files in the primary repository;

receiving a search request; and

utilizing the composite file in the file system of the secondary repository to identify multiple files in the primary repository that satisfy the search request.

13. A method as in claim 1 further comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information, each group of metadata information being associated with a corresponding file in the primary repository;

receiving a search request;

fetching the composite file;

comparing search criteria in the search request to each group of metadata in the composite file; and

for each respective group of metadata that matches the search criteria, including a corresponding file name of a corresponding file in the primary repository for inclusion in a reply to the search request.

14. A method as in claim 1 further comprising:

activating a multi-tiered search including;

i) a first search applied to the secondary repository to identify a group of files in the primary repository; and

ii) a second search that utilizes results of the first search to apply to the primary repository and identify a subset of the identified group of files.

15. An apparatus comprising:

at least one processor; and

memory coupled to the at least one processor which is configured to execute program instructions stored in the memory comprising;

maintaining a primary repository to store multiple files and metadata information associated with the multiple files;

receiving template information specifying types of the metadata information associated with the multiple files stored in the primary repository;

applying the template information to the metadata information in the primary repository to produce a secondary repository of information comprising one or more metadata files with a subset of the metadata information, the metadata files having filenames comprising at least filenames of the corresponding multiple files in the primary repository; and

receiving a search inquiry, utilizing a hierarchical file system in the secondary repository of information to perform a search, and identifying at least one of the multiple files in a hierarchical file system of the primary repository that satisfies the search inquiry based at least in part on at least a partial match of the filename of at least one corresponding metadata file.

16. An apparatus as in claim 15, wherein applying the template information to the metadata information in the primary repository includes replicating metadata information as specified by the template information for inclusion in the secondary repository.

17. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

in response to receiving search criteria, utilizing the secondary repository of information to produce a reference to a corresponding file in the primary repository that has associated metadata satisfying the search criteria.

18. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

maintaining the primary repository to include a first hierarchical tree structure for storing the multiple files and keeping track of corresponding metadata associated with the multiple files;

maintaining the secondary repository of information to include a second hierarchical tree structure based upon the first hierarchical tree structure; and

utilizing the template information to identify what, if any, portion of the multiple files and the corresponding metadata in the first hierarchical tree structure to include in the second hierarchical tree structure.

19. An apparatus as in claim 15, wherein applying the template information to the metadata information in the primary repository includes:

retrieving a given file from a file system in the primary repository;

based on a text string as specified in the template information, searching for instances of the text string in the given file; and

in response to detecting an instance of the text string in the file, maintaining the secondary repository of information to include an indication that the file includes at least one instance of the text string.

20. An apparatus as in claim 15, wherein applying the template information includes:

based on processing of the template information, identifying selection of path information as a parameter associated with a given file in the primary repository to copy to the secondary repository; and

replicating identified path information associated with the given file from the primary repository to the secondary repository.

21. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

logging occurrence of file operations associated with the multiple files stored in a file system of the primary repository; and

initiating modification of the secondary repository in accordance with the logged file operations such that search inquiries applied to the secondary repository yield corresponding search results reflecting a current state of the primary repository including application of the file operations to the multiple files in the primary repository.

22. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

logging occurrence of file operations applied to the multiple files stored in a file system of the primary repository;

initiating storage of modification information as searchable parameters in the secondary repository in accordance with the file operations; and

enabling application of searches to the storage modification information in the second repository to search for files in primary repository based on operations applied to the multiple files in the primary repository.

23. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

enabling a logging function that keeps track of changes to the primary repository to convert stored records into update operations for modifying the secondary repository such that the primary repository and the secondary repository are automatically synchronized with each other in real-time.

24. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information associated with multiple corresponding files in the primary repository;

receiving a search request; and

utilizing the composite file in the file system of the secondary repository to identify multiple files in the primary repository that satisfy the search request.

25. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information, each group of metadata information being associated with a corresponding file in the primary repository;

receiving a search request;

fetching the composite file;

comparing search criteria in the search request to each group of metadata in the composite file; and

for each respective group of metadata that matches the search criteria, including a corresponding file name of a corresponding file in the primary repository for inclusion in a reply to the search request.

26. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising, based on application of the template information to a respective file in the primary repository, replicating a portion of the respective file in the primary repository for inclusion in the secondary repository.

27. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising:

for a given file of the multiple files in the primary repository, creating a corresponding view file of metadata information associated with the given file based on metadata information in the primary repository associated with the given file; and

storing the corresponding view file in the secondary repository, the corresponding view file having same filename and filepath information as the given file in the first repository.

28. An apparatus as in claim 15 wherein the at least one processor is further configured to execute programmed instructions stored in the memory further comprising activating a multi-tiered search including:

i) a first search applied to the secondary repository to identify a group of files in the primary repository; and

ii) a second search that utilizes results of the first search to apply to the primary repository and identify a subset of the identified group of files.

29. A non-transitory computer readable medium having stored thereon instructions comprising machine executable code which when executed by at least one processor, causes the processor to perform steps comprising:

maintaining a primary repository to store multiple files and metadata information associated with the multiple files;

receiving template information specifying types of the metadata information associated with the multiple files stored in the primary repository;

applying the template information to the metadata information in the primary repository to produce a secondary repository of information comprising one or more metadata files with a subset of the metadata information, the metadata files having filenames comprising at least filenames of the corresponding multiple files in the primary repository; and

receiving a search inquiry, utilizing a hierarchical file system in the secondary repository of information to perform a search, and identifying at least one of the multiple files in a hierarchical file system of the primary repository that satisfies the search inquiry based at least in part on at least a partial match of the filename of at least one corresponding metadata file.

30. A medium as in claim 29, wherein applying the template information to the metadata information in the primary repository includes replicating the metadata information as specified by the template information for inclusion in the secondary repository.

31. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising, in response to receiving search criteria, utilizing the secondary repository of information to produce a reference to a corresponding file in the primary repository that has associated metadata satisfying the search criteria.

32. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

maintaining the primary repository to include a first hierarchical tree structure for storing the multiple files and keeping track of corresponding metadata associated with the multiple files;

maintaining the secondary repository of information to include a second hierarchical tree structure based upon the first hierarchical tree structure; and

utilizing the template information to identify what, if any, portion of the multiple files and the corresponding metadata in the first hierarchical tree structure to include in the second hierarchical tree structure.

33. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising, based on application of the template information to a respective file in the primary repository, replicating a portion of the respective file in the primary repository for inclusion in the secondary repository.

34. A medium as set forth in claim 29, wherein applying the template information to the metadata information in the primary repository includes:

retrieving a given file from a file system in the primary repository;

based on a text string as specified in the template information, searching for instances of the text string in the given file; and

in response to detecting an instance of the text string in the file, maintaining the secondary repository of information to include an indication that the file includes at least one instance of the text string.

35. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

for a given file of the multiple files in the primary repository, creating a corresponding view file of metadata information associated with the given file based on metadata information in the primary repository associated with the given file; and

storing the corresponding view file in the secondary repository, the corresponding view file having same filename and filepath information as the given file in the first repository.

36. A medium as set forth in claim 29, wherein applying the template information includes:

based on processing of the template information, identifying selection of path information as a parameter associated with a given file in the primary repository to copy to the secondary repository; and

replicating identified path information associated with the given file from the primary repository to the secondary repository.

37. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

logging occurrence of file operations associated with the multiple files stored in a file system of the primary repository; and

initiating modification of the secondary repository in accordance with the logged file operations such that search inquiries applied to the secondary repository yield corresponding search results reflecting a current state of the primary repository including application of the file operations to the multiple files in the primary repository.

38. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

logging occurrence of file operations applied to the multiple files stored in a file system of the primary repository;

initiating storage of modification information as searchable parameters in the secondary repository in accordance with the file operations; and

activating application of searches to the storage modification information in the second repository to search for files in the primary repository based on operations applied to the multiple files in the primary repository.

39. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising activating a logging function that keeps track of changes to the primary repository to convert stored records into update operations for modifying the secondary repository such that the primary repository and the secondary repository are automatically synchronized with each other in real-time.

40. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information associated with multiple corresponding files in the primary repository;

receiving a search request; and

utilizing the composite file in the file system of the secondary repository to identify multiple files in the primary repository that satisfy the search request.

41. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising:

producing a file system in the secondary repository to include a composite file including groups of metadata information, each group of metadata information being associated with a corresponding file in the primary repository;

receiving a search request;

fetching the composite file;

comparing search criteria in the search request to each group of metadata in the composite file; and

for each respective group of metadata that matches the search criteria, including a corresponding file name of a corresponding file in the primary repository for inclusion in a reply to the search request.

42. A medium as set forth in claim 29 further having stored thereon instructions that when executed by the processor causes the process to perform steps comprising activating a multi-tiered search including:

i) a first search applied to the secondary repository to identify a group of files in the primary repository; and

ii) a second search that utilizes results of the first search to apply to the primary repository and identify a subset of the identified group of files.