Distributed file system using scatter-gather

• US 20030204571A1
• Filed: 04/24/2002
• Published: 10/30/2003
• Est. Priority Date: 04/24/2002
• Status: Active Grant

• Current Assignee: Lenovo PC International Limited (Lenovo Group Ltd.)
• Original Assignee: International Business Machines Corporation
• Inventors: Waltermann, Rod David
• Granted Patent: US 6,813,632 B2
• Time in Patent Office: Days
• US Class Current: 709/214
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Storage capability otherwise going underutilized in a LAN is made available for sharing among workstations connected to the LAN. Systems connected to a LAN are surveyed for storage capability potentially available for sharing, a weighting function is derived for each system which is indicative of shared system storage capability, and data files to be stored are scattered among and gathered from the connected systems.

Claims

1. A method comprising the steps of:

surveying a plurality of computer systems associated one to another through a local area network and determining the free file storage capability of each surveyed system;

determining for each surveyed system a weighting function based on available storage capacity, network connectivity, and system resources of the respective system;

responding to instruction at one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file.

2. Apparatus comprising:

a plurality of computer systems associated one to another through a local area network, each of said systems having a processing unit and data storage capability; and

a program module stored accessibly to said processing unit of each of said plurality of computer systems;

said program module cooperating with each of said processing units when executing thereon to scatter a data file to be stored from one of said plurality of computer systems across the storage capabilities of a plurality of said computer systems, said program module and the processing unit of said one of said plurality of computer systems responding to instruction at said one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file.

3. Apparatus comprising:

a computer readable medium; and

a program module stored on said medium accessibly to a computer system;

said program module, when executing on a computer system which is one of a plurality of computer systems associated one to another through a local area network, surveying the plurality of computer systems and determining the free file storage capability of each surveyed system;

determining for each surveyed system a weighting function based on available storage capacity, network connectivity, and system resources of the respective system; and

responding to instruction at one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file.

4. A method comprising the steps of:

responding to instruction at one of a plurality of computer systems associated one to another through a local area network to retrieve from storage a data file stored in scattered portions in a plurality of the computer systems by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.

5. Apparatus comprising:

a plurality of computer systems associated one to another through a local area network, each of said systems having a processing unit and data storage capability; and

a program module stored accessibly to said processing unit of each of said plurality of computer systems;

said program module cooperating with each of said processing units when executing thereon to retrieve a data file which has been stored in scattered form across the storage capabilities of a plurality of said computer systems, said program module and the processing unit of said one of said plurality of computer systems responding to instruction at said one of said plurality of systems to retrieve a data file by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.

6. Apparatus comprising:

a computer readable medium; and

a program module stored on said medium accessibly to a computer system;

said program module, when executing on a computer system which is one of a plurality of computer systems associated one to another through a local area network, responding to instruction at one of said plurality of computer systems to retrieve from storage a data file stored in scattered portions in a plurality of the computer systems by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.

7. A method comprising the steps of:

surveying a plurality of computer systems associated one to another through a local area network and determining the free file storage capability of each surveyed system;

determining for each surveyed system a weighting function based on available storage capacity, network connectivity, and system resources of the respective system;

responding to instruction at one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file; and

responding to instruction at one of said plurality of computer systems to retrieve from storage a data file stored in scattered portions in a plurality of the computer systems by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.

8. Apparatus comprising:

a plurality of computer systems associated one to another through a local area network, each of said systems having a processing unit and data storage capability; and

a program module stored accessibly to said processing unit of each of said plurality of computer systems;

said program module cooperating with each of said processing units when executing thereon to scatter a data file to be stored from one of said plurality of computer systems across the storage capabilities of a plurality of said computer systems, said program module and the processing unit of said one of said plurality of computer systems responding to instruction at said one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file; and

said program module and the processing unit of one of said plurality of computer systems responding to instruction at said one of said plurality of systems to retrieve a data file by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.

9. Apparatus comprising:

a computer readable medium; and

a program module stored on said medium accessibly to a computer system;

said program module, when executing on a computer system which is one of a plurality of computer systems associated one to another through a local area network, surveying the plurality of computer systems and determining the free file storage capability of each surveyed system;

determining for each surveyed system a weighting function based on available storage capacity, network connectivity, and system resources of the respective system; and

responding to instruction at one of said plurality of systems to store a data file by;

dividing the data file to be stored into a plurality of portions to be scattered among the plurality of systems for storage, each portion being sized to accommodate the weighting function of a corresponding one of the plurality of systems;

tagging each portion with encoded data identifying its sequence in the data file

and system assigned for storage;

creating at said one system at which an instruction for storing the data file is given an index table identifying each tagged portion and the system to which that tagged portion is assigned for storage; and

transmitting the portions and the table to each of the systems at which a tagged portion is stored for retention and use in retrieval of the data file; and

said program module, when executing on a computer system which is one of the plurality of computer systems, responding to instruction at one of said plurality of computer systems to retrieve from storage a data file stored in scattered portions in a plurality of the computer systems by;

accessing a table stored accessibly to the one computer system which identifies a plurality of tagged portions and the identity of the computer system to which the respective tagged portion is assigned for storage, each of the portions having been sized to accommodate a weighting function of a corresponding one of the plurality of systems; and

gathering scattered portions from the plurality of computer systems to the one computer system and assembling the retrieved portions into the data file.