Network file migration system

• US 5,367,698 A
• Filed: 10/31/1991
• Issued: 11/22/1994
• Est. Priority Date: 10/31/1991
• Status: Expired due to Term

• Current Assignee: Epoch Systems, Inc. (Dell Technologies Inc.)
• Original Assignee: Epoch Systems, Inc. (Dell Technologies Inc.)
• Inventors: Israel, Robert K., Kenley, Gregory, Webber, Neil F., Foster, Antony W., Taylor, Tracy M.
• Primary Examiner(s): Heckler, Thomas M.
• Application Number:
  US07/786,248
• Time in Patent Office: 1,118 Days
• Field of Search: 395/575, 395/600, 395/800, 364/DIG. 1 MS File
• US Class Current: 709/203
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647

Abstract

A networked digital data processing system has two or more client devices, a network including a set of interconnections for transferring information between the client devices, at least one of which has a local data file storage element for locally storing and providing access to digital data files arranged in one or more client filesystems. A migration file server includes a migration storage element that stores data portions of files from the client devices, a storage level detection element that detects a storage utilization level in the storage element, and a level-responsive transfer element that selectively transfers data portions of files from the client device to the storage element.

Claims

1. A networked digital data processing system comprising

A. two or more client devices,

B. a network comprising a set of interconnections coupled to said client devices for transferring information therebetween,

C. at least a selected one of the client devices having a local data file storage element for locally storing and providing access to digital data files arranged in one or more client filesystems,

said local data file storage clement including, for each such filesystem, at least one file directory containing an entry for each data file stored in that filesystem,

D. a migration file server, coupled to client devices through said network, including

(i) migration storage means, coupled to said network, for storing at least data portions of data files from said selected client device,

(ii) storage level detection means, coupled to said local data file storage element, for detecting a storage utilization level therein,

(iii) level-responsive transfer means, coupled to the storage level detection means and with said migration storage means and with said local data file storage element, for

responding to storage utilization level exceeding a selected high watermark level for transferring data portions of one or more data files from such local data file storage element to the migration storage means, and

(iv) said level-responsive transfer means further including means for effecting release, by said local data file storage element, of storage space previously occupied in that local data file storage element by those one or more data files.

2. A networked digital data processing system according to claim 1, wherein the migration file server includes dump means for generating a local backup copy of data file, or data portions thereof, stored in any of said local data storage element and said the migration storage means.

3. A networked digital data processing system according to claim 2, wherein the dump means includes means for

(i) determining whether a given data file has been transferred to the migration storage means, and responding to a negative such determination for generating a local backup copy of that data file, and

(ii) determining, for each data file transferred to the migration storage means, whether the migration file server has already generated a backup copy and responding to an affirmative such determination for omitting that data file from a subsequent local backup copy, while responding to a negative such determination for including the file in the local backup copy.

4. A networked digital data processing system according to claim 3, wherein the migration file server includes restore means for restoring a data file to the client filesystem from a backup copy.

5. A networked digital data processing system according to claim 4, wherein the restore means comprises locating means for determining (i) whether the data portion of a given file to be restored has been transferred to the migration storage means, and (ii) whether the data are present in the migration storage means, and

link means for responding to the data portion of a given data file being present in the migration storage means, for restoring a logical link between a directory portion of that data file in the client filesystem and the data portion of that same data file in the migration storage means.

6. A networked digital data processing system according to claim 5, wherein the restore means further comprises means

(i) responding to the locating means determining that the data portion of a data file was previously transferred to, but not currently stored in, the migration storage means for

(a) retrieving the data portion of that data file to be restored from the backup copy maintained by the migration file server,

(b) transferring the data to the migration storage means, and

(ii) enabling the link means to restore the logical link between the directory portion of the given data file and the data portion of that same data file retrieved from the backup copy.

7. A networked digital data processing system according to claim 1, wherein the migration file server includes client store means for establishing designated client stores in selected areas in the migration storage means, each client store being designated for storing the data portion of tiles that have been transferred from at least one client filesystem.

8. A networked digital data processing system according to claim 7, wherein the client store means includes means for controlling access to the client stores so that data files in a given client store are accessible only by the filesystems for which that client store is designated.

9. A networked digital data processing system according to claim 8, wherein said client store means permits data files in a given client store to be accessed solely for READ operations.

10. A networked digital data processing system according to claim 7, wherein the client store means includes means for deleting selected client stores.

11. A networked digital data processing system according to claim 7, wherein the client store means includes means for moving a client store from a first are of the migration storage means to a second area of the migration storage means.

12. A networked digital data processing system according to claim 1, wherein the migration file server includes return file transfer means, coupled to the migration storage means and with said selected client device, for responding to a file access request by that client device for transferring the data portion of a requested file stored in the migration storage means to the local file data storage element of that client device.

13. A networked digital data processing system according to claim 12, wherein the migration file server comprises intercept means, coupled to said selected client device, for intercepting and handling selected file access requests transmitted between elements of the client filesystem thereof, the intercept means including

A. detection means for detecting selected file access requests transmitted between elements of such client filesystem,

B. means, coupled to the detection means for determining whether a requested file has been transferred to the migration storage means, and

C. control means, coupled to the return file transfer means.

(i) for responding to a requested file being stored in the local data storage element for transmitting a detected access request to the succeeding element of the client filesystem in a normal fashion, and

(ii) for responding to the data portion of the requested file having been previously transferred to the migration storage means and not being stored in a local storage element for

(a) causing the return file transfer means to transfer the data portion of the requested file from the migration storage means to the client filesystem, and for

(b) transmitting the access request to the succeeding element of the client filesystem for processing without altering the client filesystem.

14. A networked digital data processing system according to claim 13, wherein

A. the migration file server includes means for storing a set of digital flags representative of the state of data files stored in the migration storage means, and

B. the control means of the intercept means includes means updating selected ones of said digital flags upon transfer of the data portion of a requested data file in response to an intercepted access request.

15. A networked digital data processing system according to claim 13, wherein the control means includes means for

(i) determining, in response to an intercepted write request, whether said local data file storage element has sufficient storage space to store the data portion of a new data file, and

(ii) responding to a negative such determination for causing the level-responsive transfer means to transfer the data portions of one or more data files from a client filesystem on that local data file storage element to said migration storage means and for effecting release of storage space previously occupied by those data portions.

16. A networked digital data processing system according to claim 1, wherein the migration file server includes means for generating an extended file attribute table containing a set of entries, each of which corresponds to an entry in a directory associated with a client filesystem, and each of which includes a set of digital flags representative of the state of the corresponding data file.

17. A networked digital data processing system according to claim 16, wherein the set of digital flags includes a flag representative of whether the data portion of the corresponding data file has been transferred from the local data file storage element to the migration storage means, transfer from said one or more client filesystems to the migration storage means.

18. A networked digital data processing system according to claim 1, wherein the level-responsive transfer means includes means for responding to said storage utilization level exceeding said selected high watermark level, for transferring data portions of one or more data files from said local data file storage element to the migration storage means until the storage utilization level attains a selected low watermark level.

19. A networked digital data processing system according to claim 18, wherein the level-responsive transfer means includes prestaging means for

(i) copying, at scheduled times, the data portions of data files from one or more client filesystems to the migration storage means until the storage utilization level has attained a selected prestage reserve level that is lower than said low watermark level, while

(ii) maintaining copies of those data files in the local data file storage element, so that client devices can continue to operate upon them, and so that storage space utilized by unmodified ones of them can be released in response to a storage utilization level exceeding a selected high watermark level.

20. A networked digital data processing system according to claim 1, wherein the migration file server includes

A. means for installing a first subset of routines in the client device operating system for execution thereby, server means, and means for transferring the first subset of routines to the client,

B. means for executing a predetermined set of file transfer routines including said first subset of routines.

21. A networked digital data processing system according to claim 20, wherein the migration file server includes de-installation means for disconnecting said migration file server from said client devices without loss of data, the de-installation means including

i) means for transferring data stored in the migration storage means to other storage elements, and

ii) means for removing the first subset of routines from the client device operating system.

22. A networked digital data processing system according to claim 1, wherein the migration file server includes dump means for generating a local backup copy of data file, or data portions thereof, stored in any of said local data storage element and said the migration storage means.

23. A networked digital data processing system according to claim 1, wherein said level-responsive transfer means comprises

means for maintaining in the file directory of said local data file storage element, upon transfer of the data-portion of said data file from said local data file storage element to the migration storage means, an entry for that data file.

24. A networked digital data processing system according to claim 1, wherein the level-responsive transfer means includes scheduler means for establishing a schedule of periodic data file transfer from said one or more client filesystems to the migration storage means.

25. A networked digital data processing system according to claim 1, wherein the level-responsive transfer means includes means, responsive to commands asserted by a user, for transferring data portions of selected files from said local data file storage element to the migration storage means.

26. A networked digital data processing system according to claim 1, wherein the level-responsive transfer means includes means for establishing criteria for establishing a priority for file transfer.