Data storage management for network interconnected processors

• US 5,832,522 A
• Filed: 05/22/1996
• Issued: 11/03/1998
• Est. Priority Date: 02/25/1994
• Status: Expired due to Term

• Current Assignee: Clouding Corporation (Marathon Patent Group, Inc.)
• Original Assignee: Kodak Limited (Eastman Kodak Company)
• Inventors: Kirchner, Anton H., Montez, Jennifer Kay, Dodd, Paul David, Winter, Richard Allen, Blickenstaff, Ronald L., Brant, Catherine Irlam, Trede, Brian Eldred
• Primary Examiner(s): Lintz, Paul R.
• Application Number:
  US08/650,114
• Time in Patent Office: 895 Days
• Field of Search: 395/620, 395/621, 395/440, 395/441, 395/444, 395/800, 707/204, 707/205, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup
• CPC Class Codes: Y10S 707/99956 File allocation

Abstract

The data storage system is connected to a local area network and includes a storage server that on a demand basis and/or on a periodically scheduled basis audits the activity on each volume of each data storage device that is connected to the network. Low priority data files are migrated via the network and the storage server to backend data storage media, and the directory resident in the data storage device is updated with a placeholder entry to indicate that this data file has been migrated to backend storage. When the processor requests this data file, the placeholder entry enables the storage server to recall the requested data file to the data storage device from which it originated.

Claims

1. A data storage management system for a data network which functions to interconnect a plurality of file servers, each of which stores data files, said data storage management system comprising:

directory means, located in each of said plurality of file servers, for identifying a storage location of each data file stored on said file server;

secondary storage means, comprising a multi-layer hierarchical memory, wherein said layers in said multi-layer hierarchical memory comprise media of differing characteristics, for storing data files migrated from said plurality of file servers;

storage server means connected to said network for automatically managing transfer of data files between said plurality of file servers and said secondary storage means, comprising;

means for migrating selected data files from said plurality of file servers to said secondary storage means;

means for writing said migrated selected data files received from said plurality of file servers into selected available memory space in said multi-layer hierarchical memory, absent reservation of memory space in said multi-layer hierarchical memory on a file server basis;

means for writing in said directory means at a directory location for each of said migrated selected data files, data indicating that said migrated selected data file has been migrated to said secondary storage means and data identifying a physical data storage location in said storage server means for said migrated selected data file, which physical data storage location contains data indicative of a locus in said multi-layer hierarchical memory which contains said migrated selected data file;

means for collecting a plurality of data files, that are transmitted to said secondary storage means, into a transfer unit; and

means for storing said transfer unit on a first layer of said hierarchy.

2. The system of claim 1 wherein said means for storing data comprises:

transfer unit directory means for storing data indicative of a correspondence between a data file and a transfer unit in which said data file is located; and

media object directory means for storing data indicative of a correspondence between a transfer unit and a media on which said transfer unit is located.

3. The system of claim 1 further comprising:

means, located in each of said plurality of file servers, for intercepting a call at a selected file server to a data file that has been stored in said file server;

means, responsive to said data written in said directory means indicating that said requested data file has been migrated to said secondary storage means, for recalling said requested data file from said secondary storage means to said file server, comprising;

means for reading said data stored in said directory means to identify a physical data storage location in said storing means that contains data which identifies a locus in said secondary storage means of said requested migrated data file,

means for retrieving said data stored in said identified physical storage location in said storing means, and

means, responsive to said retrieved data, for transmitting said requested migrated data file from said locus in said secondary storage means to said selected file server.

4. The system of claim 1 wherein said data, written by said writing means in said directory means indicating that said migrated selected data file has been migrated to said secondary storage means, is stored as part of the data file attributes.

5. The system of claim 1 wherein said means for migrating data files comprises:

means for ordering data files stored on said selected file server into a priority ordering by selected characteristics of said data files.

6. The system of claim 5 wherein each said file server contains a plurality of volumes of data storage, said means for migrating data files further comprises:

means for reviewing each volume of said at least one file server to identify lowest priority data files stored thereon.

7. The system of claim 6 wherein said means for migrating data files further comprises:

means for transmitting at least one of said identified lowest priority data files to said secondary storage means.

8. The system of claim 7 wherein said storage server means further comprises:

means for activating said data file migration means for successive lowest priority data files until available memory in a volume of said selected file server is at least as great as a predefined threshold.

9. The system of claim 7 wherein said storage server means further comprises:

means for scheduling activation of said means for migrating data files on a temporal basis.

10. The system of claim 7 wherein said storage server means further comprises:

means for activating said means for migrating data files as a function of volume space utilization.

11. The system of claim 7 wherein said storage server means further comprises:

means for activating said means for migrating data files as a function of activity on said data network.

12. The system of claim 6 wherein said means for migrating data files further comprises:

means for copying at least one of said priority ordered data files from said selected file server to said secondary storage means; and

means responsive to a subsequent determination of insufficient available data storage space on said selected file server for utilizing data storage space occupied by said copied at least one said priority ordered data files as available data storage space.

13. The system of claim 1 wherein at least one of said layers comprises:

a plurality of data storage elements for storing data files migrated from said file servers;

at least one data storage element drive means for reading/writing data on a data storage element mounted in said data storage element drive means; and

automated data storage element management means for robotically mounting a selected one of said plurality of data storage elements in said data storage element drive means.

14. In a data storage management system for a data network which functions to interconnect a plurality of file servers, each of which stores data files, a method of data storage management comprising the steps of:

storing data in a directory, located in each of said plurality of file servers, for identifying a storage location of each data file stored on said file server;

storing, in a secondary storage system, comprising a multi-layer hierarchical memory, wherein said layers in said multi-layer hierarchical memory comprise media of differing characteristics, data files migrated from said plurality of file servers;

automatically managing transfer of data files between said plurality of file servers and said secondary storage system, comprising the step of;

migrating selected data files from said plurality of file servers to said secondary storage system;

writing said migrated selected data files received from said plurality of file servers into selected available memory space in said multi-layer hierarchical memory, absent reservation of memory space in said multi-layer hierarchical memory on a file server basis;

writing in said directory at a directory location for each of said migrated selected data files, data indicating that said migrated selected data file has been migrated to said secondary storage system and data identifying a physical data storage location in said storage server for said migrated selected data file, which physical data storage location contains data indicative of a locus in said multi-layer hierarchical memory which contains said migrated selected data file;

a multi-layer hierarchical memory, wherein said layers in said hierarchical memory comprise media of differing characteristics, said method further comprises;

collecting a plurality of data files, that are transmitted to said secondary storage system, into a transfer unit; and

storing said transfer unit on a first layer of said hierarchy.

15. The method of claim 14 wherein said step of storing data comprises:

storing, in a transfer unit directory, data indicative of a correspondence between a data file and a transfer unit in which said data file is located; and

storing, in a media object directory, data indicative of a correspondence between a transfer unit and a media on which said transfer unit is located.

16. The method of claim 14 further comprising:

intercepting, in each of said plurality of file servers, a call at a selected file server to a data file that has been stored in said file server;

recalling, in response to said data written in said directory indicating that said requested data file has been migrated to said secondary storage system, said requested data file from said secondary storage system to said file server, comprising the steps of;

reading said data stored in said directory to identify a physical data storage location in said memory that contains data which identifies a locus in said secondary storage system of said requested migrated data file,

retrieving said data stored in said identified physical storage location in said memory, and

transmitting, in response to said retrieved data, said requested migrated data file from said locus in said secondary storage system to said selected file server.

17. The method of claim 14 wherein said data, written in said directory indicating that said migrated selected data file has been migrated to said secondary storage system, is stored as part of the data file attributes.

18. The method of claim 14 wherein said step of migrating data files comprises:

ordering data files stored on said selected file server into a priority ordering by selected characteristics of said data files.

19. The method of claim 18 wherein each said file server contains a plurality of volumes of data storage, said step of migrating data files further comprises:

reviewing each volume of said at least one file server to identify lowest priority data files stored thereon.

20. The method of claim 19 wherein said step of migrating data files further comprises:

transmitting at least one of said identified lowest priority data files to said secondary storage system.

21. The method of claim 20 further comprising:

activating said data file migration step for successive lowest priority data files until available memory in a volume of said selected file server is at least as great as a predefined threshold.

22. The method of claim 20 further comprising:

scheduling activation of said step of migrating data files on a temporal basis.

23. The method of claim 20 further comprising:

activating said step of migrating data files as a function of volume space utilization.

24. The method of claim 20 further comprising:

activating said step of migrating data files as a function of activity on said data network.

25. The method of claim 19 wherein said step of migrating data files further comprises:

copying at least one of said priority ordered data files from said selected file server to said secondary storage system; and

utilizing, in response to a subsequent determination of insufficient available data storage space on said selected file server, data storage space occupied by said copied at least one said priority ordered data files as available data storage space.