System and method for indexing user data on storage systems

• US 8,868,495 B2
• Filed: 08/30/2007
• Issued: 10/21/2014
• Est. Priority Date: 02/21/2007
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Batterywala, Yusuf
• Primary Examiner(s): Hoang, Son T
• Application Number:
  US11/847,925
• Publication Number:
  US 20080201384A1
• Time in Patent Office: 2,609 Days
• Field of Search: None
• US Class Current: 707/613
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata

Abstract

A system and method for indexing user data on storage systems is provided. A client of the storage system executes a novel client side agent that is configured to detect changes to data stored by the client on the storage system. In response to detecting that data has been modified, the agent examines modified data containers and parses the modified data to identify new and/or modified index terms or the creation/deletion of data containers. Once the data has been parsed by the agent to identify new/modified index terms, the parsed data is transmitted from the agent to a management module executing on a management server. The management module receives the parsed data and updates a search database using the received parsed data.

Claims

1. A method for indexing data overlaid on a storage space exported by a storage system, including a processor and a memory, to a client, the method comprising:

determining, by an agent executing on the client, that the data, associated with data containers of the storage system managed by the client, has been modified by the client, wherein the agent utilizes functionality of a client-based file system to access the data overlaid on the storage space exported by the storage system to determine the modified data, wherein the client-based file system has a format different than that of the storage system'"'"'s native file system format, the native file system configured to perform indexing functionality; and

in response to determining that the data has been modified by the client, utilizing the functionality of the client-based file system, to access the data overlaid on the storage space exported by the storage system to;

parse, by the agent, the modified data,

transmit, from the client to a management server, the parsed data, and

update a search database associated with the management server utilizing the parsed data, whereby the data stored on the storage system is indexed, wherein updating the search database includes;

(i) generating one or more word entries, for the parsed data, within a word table data structure of the search database that associates each word entry with a numeric identifier,

(ii) creating associations within a content table data structure of the search database that provides at least a mapping between a particular word entry stored in the word table data structure and file entries stored in a file table data structure of the search database utilizing the numeric identifier, and

(iii) generating a new entry for the file table data structure when the parsed data indicates that a new data container has been created.

2. The method of claim 1, wherein parsing the modified data further comprises identifying new index terms within the modified data.

3. The method of claim 1, wherein parsing the modified data further comprises identifying modified index terms within the modified data.

4. The method of claim 1, wherein parsing the modified data further comprises identifying one or more additional new data containers.

5. The method of claim 4, wherein the new data container comprises a file.

6. The method of claim 1, wherein parsing the modified data further comprises identifying deleted data containers.

7. The method of claim 1, wherein parsing the modified data further comprises reading the modified data by the agent using the client-based file system functionality.

8. The method of claim 1, wherein parsing the modified data further comprises reading the modified data by the agent using an application program executing on the client.

9. The method of claim 8, wherein the application program comprises a volume manager.

10. The method of claim 8, wherein the application program comprises a database program.

11. The method of claim 1, wherein a data management module executes on the management server operatively interconnected with the client.

12. The method of claim 1, wherein the agent that utilizes the client based file system functionality to access the data overlaid on the storage space exported by the storage system, further comprises:

querying, by the agent, the file system of an operating system executing on the client utilizing a first protocol to access the data overlaid on the storage space, that includes logical unit numbers (luns), exported by the storage system, wherein the first protocol is different from a second protocol being utilized by the storage system.

13. A system for indexing data, the system comprising:

a storage system, including a computer processor and a computer memory, configured to export the data overlaid on storage space for use by a client; and

an agent executing on the client, the agent configured to identify modified data by utilizing functionality of a client-based file system to access the data overlaid on the storage space exported by the storage system, the client-based file system having a format that is different than that of the storage system'"'"'s native file system format that is configured to perform indexing functionality, and

in response to identifying the modified data utilizing the functionality of the client-based file system, to access the data overlaid on the storage space exported by the storage system, the computer processor configured to;

parse the modified data, and

forward the parsed modified data from the client to a management server, wherein the management server is configured to update a search database associated with the data management server using the parsed data, wherein updating the search database includes;

(i) generating one or more word entries, for the parsed data, within a word table data structure of the search database that associates each word entry with a numeric identifier,

(ii) creating associations within a content table data structure of the search database that provides at least a mapping between a particular word entry stored in the word table data structure and file entries stored in a file table data structure of the search database utilizing the numeric identifier, and

(iii) generating a new entry for the file table data structure when the parsed data indicates that a new data container has been created.

14. The system of claim 13, wherein the client utilizes the exported storage space to store structured data from an application.

15. The system of claim 13, wherein the agent parses the modified data utilizing the client-based file system functionality.

16. The system of claim 13, wherein the agent parses the modified data using an application executing on the client.

17. The system of claim 13, wherein a data management module executes on the management server operatively interconnected with the client.

18. The system of claim 13, wherein the management server is further configured to, in response to a user request, perform a query of the search database and display results of the query.

19. A non-transitory computer readable medium containing executable program instructions executed by a processor, comprising:

program instructions that determine, by an agent executing on a client, that data has been modified by the client, wherein the agent utilizes functionality of a client-based file system to access the data overlaid on the storage space exported by the storage system to determine the modified data, wherein the client-based file system has a format different than that of the storage system'"'"'s native file system format that is configured to perform indexing functionality; and

program instructions that, in response to identifying the modified data, utilizes the functionality of client-based file system to access the data overlaid on the storage space exported by the storage system, the program instructions executable to;

parse the modified data,

transmit, from the client to a management server, the parsed data, and

update a search database associated with the management server utilizing the parsed data, wherein updating the search database includes;

(i) generating one or more word entries, for the parsed data, within a word table data structure of the search database that associates each word entry with a numeric identifier,

(ii) creating associations within a content table data structure of the search database that provides at least a mapping between word entries stored in the word table data structure and file entries stored in a file table data structure of the search database utilizing the numeric identifier, and

(iii) generating a new entry for the file table data structure when the parsed data indicates that a new data container has been created.