System and method for efficient file content searching within a file system

• US 7,636,710 B2
• Filed: 03/04/2004
• Issued: 12/22/2009
• Est. Priority Date: 03/04/2004
• Status: Expired due to Fees

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Harmer, Craig K., Premo, Nur, Borthakur, Dhrubajyoti
• Primary Examiner(s): Pham; Khanh B
• Application Number:
  US10/793,070
• Publication Number:
  US 20050198010A1
• Time in Patent Office: 2,119 Days
• Field of Search: 707/102, 707/3, 707/200, 707/205
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A system and method for efficient file content searching within a file system. In one embodiment, the system may include a storage device configured to store data and a file system configured to manage access to the storage device. The file system may be configured to detect a search operation to determine the presence of a given data pattern within a first file stored on the storage device, and to store an indication of the given data pattern and an indication of result data of the search operation in a first record associated with the first file.

Claims

1. A system, comprising:

a computer-accessible storage device configured to store file system content, said file system content comprising a plurality of files and a plurality of metadata records associated with said files; and

a host computing device configured to implement an operating system including a file system, wherein said file system comprises data structures configured to store information about said file system content, and wherein said file system further comprises executable routines configured to access and modify said file system content, wherein said executable routines of said file system are configured to store said file system content to said computer-accessible storage device and to manage accesses to file system content stored by said computer-accessible storage device, wherein said accesses are generated by applications distinct from said operating system, and wherein said file system is further configured to;

detect a given search operation generated by a first one of said applications, wherein the given search operation is operable to determine whether or not a given data pattern that is specified by the given search operation occurs within a first one of said files stored on said computer-accessible storage device;

in response to detecting said given search operation, access said computer-accessible storage device to determine whether one or more of said metadata records associated with said first file indicate a result of a previously detected search operation, wherein the previously detected search operation is detected by said file system prior to said given search operation, wherein the previously detected search operation is operable to determine whether or not a particular data pattern that is specified by the previously detected search operation occurs within said first file, and wherein the particular data pattern specified by the previously detected search operation includes the given data pattern specified by the given search operation;

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, return a result of said given search operation to said first application dependent upon said result of said previously detected search operation without retrieving the entirety of said first file from said computer-accessible storage device; and

in response to determining that said one or more metadata records associated with said first file do not indicate a result of a previously detected search operation to determine the presence of a data pattern including said given data pattern within said first file, instruct that said first file be searched to determine the presence of said given data pattern within said first file and subsequently store an indication, within one or more of said metadata records associated with said first file, of a result of instructing that said first file be searched.

2. The system as recited in claim 1, wherein in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system is further configured to determine whether said result of said previously detected search operation indicates that said given data pattern is present within said first file, and in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system is further configured to access said computer-accessible storage device to retrieve only a portion of said first file and to return said retrieved portion to said first application as said result of said given search operation.

3. The system as recited in claim 1, wherein said one or more of said metadata records that indicate said result of said previously detected search operation include an index corresponding to one or more portions of said first file in which said given data pattern occurs, and wherein in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system is further configured to determine whether said result of said previously detected search operation indicates that said given data pattern is present within said first file, and in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system is further configured to access said computer-accessible storage device to retrieve only the one or more portions of said first file corresponding to said index and to return said one or more retrieved portions to said first application as said result of said given search operation.

4. The system as recited in claim 1, wherein in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system is further configured to determine whether said result of said previously detected search operation indicates that said given data pattern is not present within said first file, and in response to determining that said given data pattern is not present within said first file according to said result of said previously detected search operation, said file system is further configured to return a negative indication to said first application as said result of said given search operation without accessing said computer-accessible storage device to retrieve any portion of said first file.

5. The system as recited in claim 1, wherein to store said indication of said result of said instructing that said first file be searched, if said given data pattern is determined to be present in said first file, said file system is further configured to store an indication of said given data pattern and pattern location information in a metadata record within a positive search named stream associated with said first file and stored on said computer-accessible storage device, and wherein if said given data pattern is determined not to be present in said first file, said file system is further configured to store said indication of said given data pattern in a metadata record within a negative search named stream associated with said first file and stored on said computer-accessible storage device.

6. The system as recited in claim 1, wherein in response to detecting an operation to modify data stored in said first file, said file system is further configured to clear metadata records associated with said first file that indicate results of previously detected search operations corresponding to said first file.

7. The system as recited in claim 1, wherein to instruct that said first file be searched to determine the presence of said given data pattern within said first file, said file system is further configured to call an external search application configured to execute within user space.

8. The system as recited in claim 1, wherein to instruct that said first file be searched to determine the presence of said given data pattern within said first file, said file system is further configured to execute a search algorithm within kernel space.

9. A method, comprising:

a file system storing file system content to a computer-accessible storage device, said file system content comprising a plurality of files and a plurality of metadata records associated with said files, wherein said file system is implemented by an operating system executing on a host computing device, wherein said file system comprises data structures configured to store information about said file system content, and wherein said file system further comprises executable routines configured to access and modify said file system content;

said file system managing accesses to said computer-accessible storage device, wherein said accesses are generated by applications distinct from said operating system;

said file system detecting a given search operation generated by a first one of said applications, wherein the given search operation is operable to determine whether or not a given data pattern that is specified by the given search operation occurs within a first one of said files stored on said computer-accessible storage device;

in response to detecting said given search operation, said file system accessing said computer-accessible storage device and determining whether one or more of said metadata records associated with said first file indicate a result of a previously detected search operation, wherein the previously detected search operation is detected by said file system prior to said given search operation, wherein the previously detected search operation is operable to determine whether or not a particular data pattern that is specified by the previously detected search operation occurs within said first file, and wherein the particular data pattern specified by the previously detected search operation includes the given data pattern specified by the given search operation;

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system returning a result of said given search operation to said first application dependent upon said result of said previously detected search operation without retrieving the entirety of said first file from said computer-accessible storage device; and

in response to determining that said one or more metadata records associated with said first file do not indicate a result of a previously detected search operation to determine the presence of a data pattern including said given data pattern within said first file, said file system instructing that said first file be searched to determine the presence of said given data pattern within said first file and said file system subsequently storing an indication, within one or more of said metadata records associated with said first file, of a result of said instructing that said first file be searched.

10. The method as recited in claim 9, further comprising:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is present within said first file; and

in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system accessing said computer-accessible storage device to retrieve only a portion of said first file and returning said retrieved portion to said first application as said result of said given search operation.

11. The method as recited in claim 9, wherein said one or more of said metadata records that indicate said result of said previously detected search operation include an index corresponding to one or more portions of said first file in which said given data pattern occurs, and wherein the method further comprises:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is present within said first file;

in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system accessing said computer-accessible storage device to retrieve only the one or more portions of said first file corresponding to said index; and

said file system returning said one or more retrieved portions to said first application as said result of said given search operation.

12. The method as recited in claim 9, further comprising:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is not present within said first file; and

in response to determining that said given data pattern is not present within said first file according to said result of said previously detected search operation, said file system returning a negative indication to said first application as said result of said given search operation without accessing said computer-accessible storage device to retrieve any portion of said first file.

13. The method as recited in claim 9, wherein said file system storing said indication of said result of said instructing that said first file be searched further comprises:

if said given data pattern is determined to be present in said first file, said file system storing an indication of said given data pattern and pattern location information in a metadata record within a positive search named stream associated with said first file and stored on said computer-accessible storage device; and

if said given data pattern is determined not to be present in said first file, said file system storing said indication of said given data pattern in a metadata record within a negative search named stream associated with said first file and stored on said computer-accessible storage device.

14. The method as recited in claim 9, further comprising said file system clearing metadata records associated with said first file that indicate results of previously detected search operations corresponding to said first file, in response to said file system detecting an operation to modify data stored in said first file.

15. The method as recited in claim 9, wherein said file system instructing that said first file be searched to determine the presence of said given data pattern within said first file comprises said file system calling an external search application configured to execute within user space.

16. A computer-accessible storage medium comprising program instructions, wherein the program instructions are computer-executable to implement:

a file system of an operating system storing file system content to a computer-accessible storage device, said file system content comprising a plurality of files and a plurality of metadata records associated with said files, wherein said file system comprises data structures configured to store information about said file system content, and wherein said file system further comprises executable routines configured to access and modify said file system content;

said file system managing accesses to said computer-accessible storage device, wherein said accesses are generated by applications distinct from said operating system;

said file system detecting a given search operation generated by a first one of said applications to determine the presence of a given data pattern within a first one of said files stored on said computer-accessible storage device;

in response to detecting said given search operation, said file system accessing said computer-accessible storage device and determining whether one or more of said metadata records associated with said first file indicate a result of a previously detected search operation, wherein the previously detected search operation is detected by said file system prior to said given search operation, wherein the previously detected search operation is operable to determine whether or not a particular data pattern that is specified by the previously detected search operation occurs within said first file, and wherein the particular data pattern specified by the previously detected search operation includes the given data pattern specified by the given search operation;

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system returning a result of said given search operation to said first application dependent upon said result of said previously detected search operation without retrieving the entirety of said first file from said computer-accessible storage device; and

in response to determining that said one or more metadata records associated with said first file do not indicate a result of a previously detected search operation to determine the presence of a data pattern including said given data pattern within said first file, said file system instructing that said first file be searched to determine the presence of said given data pattern within said first file and said file system subsequently storing an indication, within one or more of said metadata records associated with said first file, of a result of said instructing that said first file be searched.

17. The computer-accessible storage medium as recited in claim 16, wherein the program instructions are further computer-executable to implement:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is present within said first file; and

in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system accessing said computer-accessible storage device to retrieve only a portion of said first file and returning said retrieved portion to said first application as said result of said given search operation.

18. The computer-accessible storage medium as recited in claim 16, wherein said one or more of said metadata records that indicate said result of said previously detected search operation include an index corresponding to one or more portions of said first file in which said given data pattern occurs, and wherein the program instructions are further computer-executable to implement:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is present within said first file;

in response to determining that said given data pattern is present within said first file according to said result of said previously detected search operation, said file system accessing said computer-accessible storage device to retrieve only the one or more portions of said first file corresponding to said index; and

said file system returning said one or more retrieved portions to said first application as said result of said given search operation.

19. The computer-accessible storage medium as recited in claim 16, wherein the program instructions are further computer-executable to implement:

in response to determining that said one or more of said metadata records indicate said result of said previously detected search operation, said file system determining whether said result of said previously detected search operation indicates that said given data pattern is not present within said first file; and

in response to determining that said given data pattern is not present within said first file according to said result of said previously detected search operation, said file system returning a negative indication to said first application as said result of said given search operation without accessing said computer-accessible storage device to retrieve any portion of said first file.

20. The computer-accessible storage medium as recited in claim 16, wherein said file system storing said indication of said result of said instructing that said first file be searched further comprises:

if said given data pattern is determined to be present in said first file, said file system storing an indication of said given data pattern and pattern location information in a metadata record within a positive search named stream associated with said first file and stored on said computer-accessible storage device; and

if said given data pattern is determined not to be present in said first file, said file system storing said indication of said given data pattern in a metadata record within a negative search named stream associated with said first file and stored on said computer-accessible storage device.

21. The computer-accessible storage medium as recited in claim 16, wherein the program instructions are further computer-executable to implement said file system clearing metadata records associated with said first file that indicate results of previously detected search operations corresponding to said first file, in response to said file system detecting an operation to modify data stored in said first file.

22. The computer-accessible storage medium as recited in claim 16, wherein said file system instructing that said first file be searched to determine the presence of said given data pattern within said first file comprises said file system calling an external search application configured to execute within user space.