Computer system, file management method and metadata server

• US 9,396,198 B2
• Filed: 09/21/2011
• Issued: 07/19/2016
• Est. Priority Date: 09/21/2011
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Takaoka, Nobumitsu, Kodama, Shoji
• Primary Examiner(s): Vital, Pierre
• Assistant Examiner(s): Ho, Andrew N
• Application Number:
  US14/003,972
• Publication Number:
  US 20140040331A1
• Time in Patent Office: 1,763 Days
• Field of Search: 707/827, 707/821
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A computer system, comprising: a file server; a metadata server; and a business server, the metadata server being coupled to a storage apparatus for providing a save area for storing at least one file that is removed from the file server, the metadata server storing a metadata repository for managing metadata of a file and the files stored in the save area, the metadata server being configured to: store a file as a saved file in the save area, in a case of detecting that the file stored in the file server is to be removed; and store information indicating a location of the file in the file server and information indicating the location of the saved file in the save area in association with each other in the metadata repository.

Claims

1. A computer system, comprising:

a file server configured to manage a plurality of files;

a metadata server configured to manage a plurality of piece of metadata of the plurality of files;

a business server configured to use the plurality of files to execute predetermined business processing,

the file server, the metadata server, and the business server being coupled to each other via a network,

the file server including a first processor, a first memory coupled to the first processor, a first network interface coupled to the first processor, and a first storage medium coupled to the first processor, and configured to store the plurality of files,

the metadata server including a second processor, a second memory coupled to the second processor, a second network interface coupled to the second processor, and a second storage medium coupled to the second processor,

the business server including a third processor, a third memory coupled to the third processor, and a third network interface coupled to the third processor,

the metadata server being coupled to a storage apparatus including a controller and a plurality of storage media, and configured to provide a save area for storing at least one file that is removed from the file server,

the second storage medium storing a metadata repository for managing at least one of the plurality of piece of metadata of the plurality of files and at least one location of one of the plurality of files stored in the save area,

the metadata server being configured to;

store the one of the plurality of files as a saved file in the save area, in a case of detecting that the one of the plurality of files stored in the file server is to be removed by the predetermined business processing executed by the business server; and

store information indicating a location of the one of the plurality of files in the file server and information indicating the location of the saved file in the save area in association with each other in the metadata repository,

wherein the computer system further comprises an analysis server configured to execute predetermined analytical processing on the plurality of files,

wherein the analysis server includes a fourth processor, a fourth memory coupled to the fourth processor, and a fourth network interface coupled to the fourth processor,

wherein the metadata server identifies a location of a file to be read in response to an inquiry request for the file, and notifies the analysis server of the identified location of the file to be read, in a case of receiving the inquiry request for the plurality of files from the analysis server,

wherein the analysis server obtains the file to be read from one of the file server and the save area to execute the predetermined analytical processing, based on the notification received from the metadata server,

wherein the file server includes a first storage space allocated to the file server, the metadata server, the business server, and the analysis server, and a second storage space allocated to the file server, the metadata server, and the analysis server,

wherein the one of the plurality of files before being removed is stored in the first storage space,

wherein the file server detects that the one of the plurality of files is to be removed from the file server by the predetermined business processing executed by the business server, and moves the one of the plurality of files from the first storage space to the second storage space, and

wherein the metadata server is configured to;

obtain the one of the plurality of files stored in the second storage space and store the obtained one of the plurality of files as the saved file in the save area;

store, as the information indicating the location of the one of the plurality of files in the file server, information indicating a location of the one of the plurality of files in the first storage space, in the metadata repository; and

transmit an instruction to remove the one of the plurality of files to the file server, and

wherein the file server removes the one of the plurality of files from the second storage space, in a case of receiving the instruction to remove.

2. The computer system according to claim 1, wherein the metadata server is configured to:

generate, based on the information stored in the metadata repository, list information including information on locations of the one of the plurality of files stored in the file server and the saved file stored in the save area, in a case of receiving the inquiry request for the plurality of files; and

transmit the generated list information to the analysis server.

3. The computer system according to claim 1,

wherein the file server transmits a notification that the one of the plurality of files is to be removed to the metadata server, in a case of detecting that the one of the plurality of files is to be removed from the file server by the predetermined business processing executed by the business server, and

wherein the metadata server detects that the one of the plurality of files is to be removed from the file server by receiving the notification from the file server.

4. The computer system according to claim 3, wherein, the file server transmits a notification that at least one of the plurality of files has been removed to the metadata server, one of in a case of detecting that a predetermined number of the plurality of files are to be removed from the file server, and in a case where a predetermined period of time has elapsed since last transmission of the notification that the one of the plurality of files is to be removed.

5. The computer system according to claim 1,

wherein the plurality of files include a plurality of pieces of data,

wherein the file server is configured to:

determine whether removed data is generated, the removed data is data to be removed from the one of the plurality of files by the predetermined business processing executed by the business server; and

move the removed data included in the one of the plurality of files to the second storage space, in a case where it is determined that the removed data is generated, and

wherein the metadata server stores the removed data, which is stored in the second storage space, as the saved file in the save area.

6. The computer system according to claim 5, wherein the metadata server is configured to:

refer to the metadata repository to determine whether there is the removed data corresponding to the file to be read, in a case of receiving the inquiry request for the plurality of files from the analysis server;

read the file to be read stored in the file server, in a case where it is determined that there is the removed data corresponding to the file to be read;

restore the file to be read by obtaining the saved file corresponding to the removed data stored in the save area and overwriting the read file to be read with the removed data obtained from the saved file; and

transmit the restored file to be read to the analysis server.

7. The computer system according to claim 1,

wherein the computer system further comprises a backup storage apparatus including a controller and a plurality of storage media, for storing backup files of the plurality of files stored in the file server, and

wherein the metadata server is configured to:

determine whether the backup storage apparatus stores a backup file having the same content as that of the one of the plurality of files to be removed, in a case of detecting that the one of the plurality of files stored in the file server is to be removed by the predetermined business processing executed by the business server; and

store a location of the backup file as the location of the saved file in the metadata repository, in a case where it is determined that the backup storage apparatus stores the backup file having the same content as that of the one of the plurality of files to be removed.

8. The computer system according to claim 1,

wherein the file server has a snapshot generating function for generating a snapshot recording a status of the file server at an arbitrary time point, and

wherein the metadata server is configured to:

transmit, to the file server, an instruction to generate a first snapshot recording a current status of the file server;

refer to the generated first snapshot to update the plurality of piece of metadata stored in the metadata repository;

extract at least one of the plurality of piece of metadata that is not updated from the plurality of piece of metadata to determine whether there is a file corresponding to the extracted at least one of the plurality of piece of metadata in the first snapshot;

obtain a second snapshot that precedes the first snapshot in time series, and obtain the file corresponding to the extracted at least one of the plurality of piece of the metadata from the second snapshot, in a case where it is determined that there is no file corresponding to the extracted at least one piece of the metadata in the first snapshot; and

store the obtained file as the saved file in the save area.

9. A file management method for a computer system,

the computer system including a file server for managing a plurality of files, a metadata server for managing a plurality of piece of metadata of the plurality of files, a business server for using the plurality of files to execute predetermined business processing, and an analysis server for executing predetermined analytical processing on the plurality of files,

the file server, the metadata server, the business server, and the analysis server being coupled to each other via a network,

the file server including a first processor, a first memory coupled to the first processor, a first network interface coupled to the first processor, and a first storage medium coupled to the first processor, for storing the plurality of files,

the metadata server including a second processor, a second memory coupled to the second processor, a second network interface coupled to the second processor, and a second storage medium coupled to the second processor,

the business server including a third processor, a third memory coupled to the third processor, and a third network interface coupled to the third processor,

the analysis server including a fourth processor, a fourth memory coupled to the fourth processor, and a fourth network interface coupled to the fourth processor,

the metadata server being coupled to a storage apparatus including a controller and a plurality of storage media, for providing a save area for storing at least one file that is removed from the file server,

the second storage medium storing a metadata repository for managing at least one of the plurality of piece of metadata of the plurality of files and at least one location of one of the plurality of files stored in the save area,

the file management method comprising:

a first step of storing, by the metadata server, the one of the plurality of files as a saved file in the save area, in a case of detecting that the one of the plurality of files stored in the file server is to be removed by the predetermined business processing executed by the business server;

a second step of storing, by the metadata server, information indicating a location of the one of the plurality of files in the file server and information indicating the location of the saved file in the save area in association with each other in the metadata repository;

a third step of identifying, by the metadata server, a location of a file to be read in response to an inquiry request for the plurality of files, and notifying the analysis server of the identified location of the file to be read, in a case of receiving the inquiry request for the plurality of files from the analysis server; and

a fourth step of obtaining, by the analysis server, the file to be read from one of the file server and the save area to execute the predetermined analytical processing, based on the notification received from the metadata server,

wherein the file server includes a first storage space allocated to the file server, the metadata server, the business server, and the analysis server, and a second storage space allocated to the file server, the metadata server, and the analysis server,

wherein the one of the plurality of files before being removed is stored in the first storage space,

wherein the first step further includes;

detecting, by the file server, that the one of the plurality of files is to be removed from the file server by the predetermined business processing executed by the business server, and moving the one of the plurality of files from the first storage space to the second storage space; and

obtaining, by the metadata server, the one of the plurality of files stored in the second storage space and storing the obtained one of the plurality of files as the saved file in the save area, and

wherein the second step further includes;

storing, by the metadata server, as the information indicating the location of the one of the plurality of files in the file server, information indicating a location of the one of the plurality of files in the first storage space, in the metadata repository;

transmitting, by the metadata server, an instruction to remove the one of the plurality of files to the file server; and

removing, by the file server, the one of the plurality of files from the second storage space, in a case of receiving the instruction to remove.

10. The file management method according to claim 9, wherein the third step includes:

generating, by the metadata server, list information including information on locations of the plurality of files stored in the file server and the saved file stored in the save area based on the information stored in the metadata repository; and

transmitting, by the metadata server, the generated list information to the analysis server.

11. The file management method according to claim 9, wherein the first step includes detecting that the one of the plurality of files is to be removed from the file server by receiving a notification that the one of the plurality of files is to be removed transmitted from the file server.

12. The file management method according to claim 11, further including transmitting, by the file server, a notification that at least one of the plurality of files has been removed to the metadata server, one of in a case of detecting that a predetermined number of the plurality of files are to be removed from, and in a case where a predetermined period of time has elapsed since last transmission of the notification that the one of the plurality of files is to be removed.

13. The file management method according to claim 9,

wherein the plurality of files include a plurality of pieces of data, and

wherein the first step includes:

determining, by the file server, whether removed data is generated, being data to be removed from the one of the plurality of files by the predetermined business processing executed by the business server;

moving, by the file server, the removed data included in the one of the plurality of files to the second storage space, in a case where it is determined that the removed data is generated; and

storing, by the metadata server, the removed data, which is stored in the second storage space, as the saved file in the save area.

14. The file management method according to claim 13, wherein the third step includes:

referring, by the metadata server, to the metadata repository to determine whether there is the removed data corresponding to the file to be read;

reading, by the metadata server, the file to be read stored in the file server, in a case where it is determined that there is the removed data corresponding to the file to be read;

restoring, by the metadata server, the file to be read by obtaining the saved file corresponding to the removed data stored in the save area and overwriting the read file to be read with the removed data obtained from the saved file; and

transmitting, by the metadata server, the restored file to be read to the analysis server.

15. The file management method according to claim 9,

wherein the computer system further includes a backup storage apparatus including a controller and a plurality of storage media, for storing backup files of the plurality of files stored in the file server,

wherein the first step includes determining, by the metadata server, whether the backup storage apparatus stores a backup file having the same content as that of the one of the plurality of files to be removed, in a case of detecting that the one of the plurality of files stored in the file server is to be removed by the predetermined business processing executed by the business server, and

wherein the second step includes storing, by the metadata server, a location of the backup file as the location of the saved file in the metadata repository, in a case where it is determined that the backup storage apparatus stores the backup file having the same content as that of the one of the plurality of files to be removed.

16. The file management method according to claim 9,

wherein the file server has a snapshot generating function for generating a snapshot recording a status of the file server at an arbitrary time point,

wherein the first step includes:

transmitting, by the metadata server, to the file server, an instruction to generate a first snapshot recording a current status of the file server;

referring, by the metadata server, to the generated first snapshot to update the plurality of piece of metadata stored in the metadata repository; and

extracting, by the metadata server, at least one of the plurality of piece of metadata that is not updated from the plurality of piece of metadata to determine whether there is a file corresponding to the extracted at least one of the plurality of piece of metadata in the first snapshot, and

wherein the second step includes;

obtaining, by the metadata server, a second snapshot that precedes the first snapshot in time series, and obtaining the file corresponding to the extracted at least one of the plurality of piece of metadata from the second snapshot, in a case where it is determined that there is no file corresponding to the extracted at least one of the plurality of piece of metadata in the first snapshot; and

storing, by the metadata server, the obtained file as the saved file in the save area.

17. A metadata server, comprising:

a processor;

a memory coupled to the processor;

a network interface coupled to the processor; and

a local storage coupled to the processor, and configured to manage a plurality of piece of metadata of a plurality of files stored in a file server coupled via a network,

the metadata server being coupled to an analysis server for executing predetermined analytical processing on the plurality of files, and a saving storage apparatus configured to store at least one file that is removed from the file server,

the memory storing a metadata management program for managing the plurality of piece of metadata, a file removal detection program for detecting that the one of the plurality of files is to be removed from the file server, a file save program for moving the one of the plurality of files to be removed from the file server to the saving storage, and an inquiry processing program for processing an inquiry request for the plurality of files from the analysis server,

the local storage storing a metadata repository including a metadata management table for managing the plurality of piece of metadata of the plurality of files, and a saved file management table for managing the one of the plurality of files stored in the save area,

the processor executing the metadata management program obtaining the plurality of piece of metadata of the plurality of files stored in the file server to update the metadata management table,

the processor executing the file removal detection program detecting that the one of the plurality of files stored in the file server is to be removed,

the processor executing the file save program copying the one of the plurality of files as a saved file in the saving storage apparatus,

the processor executing the file save program storing information indicating a location of the saved file in the saving storage apparatus, in the saved file management table,

the processor executing the inquiry processing program identifying, a location of a file to be read in response to the inquiry request for the plurality of files and generating a list including the identified location of the file to be read, in a case of receiving the inquiry request for the plurality of files,

the processor executing the inquiry processing program transmitting the generated list to the analysis server, and

wherein the file server includes a first storage space allocated to the file server, the metadata server, a business server, and the analysis server, and a second storage space allocated to the file server, the metadata server, and the analysis server,

wherein the one of the plurality of files before being removed is stored in the first storage space,

wherein the file server is configured to detect that the one of the plurality of files is to be removed from the file server by a predetermined business processing executed by the business server, and move the one of the plurality of files from the first storage space to the second storage space; and

wherein the metadata server is configured to;

obtain the one of the plurality of files stored in the second storage space and store the obtained one of the plurality of files as the saved file in the save area,

store, as the information indicating the location of the one of the plurality of files in the file server, information indicating a location of the one of the plurality of files in the first storage space, in the metadata repository, and

transmit an instruction to remove the one of the plurality of files from the second storage space to the file server,

wherein the file server removes the one of the plurality of files from the second storage space, in a case of receiving the instruction to remove.