Data storage systems
First Claim
Patent Images
1. A distributed computing environment system comprising:
- a first set of a plurality of nodes (Client Nodes) each such node being defined by processing means including a “
requester”
means, a “
locator”
means, a “
transmitter”
means and a “
data-splitter”
means;
data storage means;
memory means;
input/output means and each said node being further defined and addressed by a globally unique identifier hereafter referred to as the “
parameters”
for each of said client node;
a second set of a plurality of nodes (Storage Nodes) differentiated form the first set by having enhanced data-storage capabilities and further defined by memory means;
Input/output means; and
processing means including “
receiver”
means, “
data-retrieval”
means and a “
transmitter”
means;
each said node of the second set being further defined by a plurality of variable identifiers, a measurement of the total storage space and a measurement of the total available storage space in the node, together hereafter referred to as the “
parameters”
for each of said storage nodes;
at least one meta-data node adapted to manage data, including data in the form of data files, being inputted into and outputted form the second set of nodes [storage nodes] and having a database of all data files and their respective identifier stored in the said environment, processing means including “
prefix generation”
means, “
file-identifier generation”
means adapted to generate a globally unique identifier for a data file;
said file identifier being defined by a “
head”
element that'"'"'s derived from a hash of a pre-determined set of properties of the file a “
next-prefix”
element that is derived from the “
prefix-generation”
means a pre-determined “
striping”
element a pre-determined “
sizing”
element at least one Administrative node having tool and utilities for accessing and configuring abovementioned parameters associated with each of said nodes and disseminating modified parameters of one node to all other nodes irrespective of their set;
an interconnect means that is connected to the Input/Output means of the first set of nodes [client nodes], the second set of nodes [storage nodes], the meta-data node and the Administrative node; and
adapted to facilitate one-to-one, one-to-many, many-to-one and many-to-many data communication between the nodes;
said nodes in the first set adapted to initiate file operations within the computing environment;
said file operation including file-create, file-read and file-write operations;
said “
data splitter”
means within the each node of the first set adapted to split a file into smaller “
stripes”
of size defined by the “
sizing”
element dependent upon the file-identifier generated for that file;
said “
locator”
means within each node of the first set adapted to receive a “
file-identifier”
of a file and further adapted to provide the “
requestor” and
“
transmitter”
means within the node with a designated sequence of nodes from the second set to which the requestor or transmitter means respectively must request from or transmit data to respectively;
said “
transmitter”
means within the each node of the first set adapted to receive the stripes of the file from the “
data-splitter” and
the sequence of nodes in the second set from the “
locator” and
to transmit entire data contained in a file to a designated sequence of nodes from the second set;
said “
requestor”
means within the each node of the first set adapted to receive a designated sequence of nodes from the second set from the “
locator”
means and further adapted to send a request for file stripe to an appropriate node in the second set;
said “
prefix-generation”
means at the meta-data node adapted to generate a “
next-prefix”
element for each node in the second set by using the parameters associated with that individual node;
said “
identifier-generation”
means at the meta-data node adapted to generate a unique identifier for a file and transmitting the identifier to the node within the first set responsible for creating a file;
said “
receiver”
means within the each node of the second set adapted to receive a request for data from the “
requester”
or stripes of new or modified data form the “
transmitter”
within a node in the first set;
said “
retrieval”
means, upon trigger from the “
receiver”
means adapted to retrieve data form the storage media associated with the storage nodes and to deliver it to the “
transmitter”
means of the second set;
said “
storage”
means adapted to receive data from the “
receiver” and
store data onto a storage media; and
said “
transmitter”
means adapted to receive data from the “
retrieval”
means and further adapted to transmit data to the “
requestor”
means of the node from the first set that requested for that data.
0 Assignments
0 Petitions
Accused Products
Abstract
The invention discloses an apparatus for storage of data in a distributed storage environment and a method of efficiently storing data in such an environment which includes a set of client nodes, a set of storage nodes, meta data nodes and an administrator node, each node having processing and memory capability. The client nodes can directly access data files stored in the form of “stripes” across the storage nodes without querying the meta data node by using specially created file identifiers and storage identifiers.
62 Citations
18 Claims
-
1. A distributed computing environment system comprising:
-
a first set of a plurality of nodes (Client Nodes) each such node being defined by processing means including a “
requester”
means, a “
locator”
means, a “
transmitter”
means and a “
data-splitter”
means;
data storage means;
memory means;
input/output means and each said node being further defined and addressed by a globally unique identifier hereafter referred to as the “
parameters”
for each of said client node;
a second set of a plurality of nodes (Storage Nodes) differentiated form the first set by having enhanced data-storage capabilities and further defined by memory means;
Input/output means; and
processing means including “
receiver”
means, “
data-retrieval”
means and a “
transmitter”
means;
each said node of the second set being further defined by a plurality of variable identifiers, a measurement of the total storage space and a measurement of the total available storage space in the node, together hereafter referred to as the “
parameters”
for each of said storage nodes;
at least one meta-data node adapted to manage data, including data in the form of data files, being inputted into and outputted form the second set of nodes [storage nodes] and having a database of all data files and their respective identifier stored in the said environment, processing means including “
prefix generation”
means, “
file-identifier generation”
means adapted to generate a globally unique identifier for a data file;
said file identifier being defined bya “
head”
element that'"'"'s derived from a hash of a pre-determined set of properties of the filea “
next-prefix”
element that is derived from the “
prefix-generation”
meansa pre-determined “
striping”
elementa pre-determined “
sizing”
elementat least one Administrative node having tool and utilities for accessing and configuring abovementioned parameters associated with each of said nodes and disseminating modified parameters of one node to all other nodes irrespective of their set;
an interconnect means that is connected to the Input/Output means of the first set of nodes [client nodes], the second set of nodes [storage nodes], the meta-data node and the Administrative node; and
adapted to facilitate one-to-one, one-to-many, many-to-one and many-to-many data communication between the nodes;
said nodes in the first set adapted to initiate file operations within the computing environment;
said file operation including file-create, file-read and file-write operations;
said “
data splitter”
means within the each node of the first set adapted to split a file into smaller “
stripes”
of size defined by the “
sizing”
element dependent upon the file-identifier generated for that file;
said “
locator”
means within each node of the first set adapted to receive a “
file-identifier”
of a file and further adapted to provide the “
requestor” and
“
transmitter”
means within the node with a designated sequence of nodes from the second set to which the requestor or transmitter means respectively must request from or transmit data to respectively;
said “
transmitter”
means within the each node of the first set adapted to receive the stripes of the file from the “
data-splitter” and
the sequence of nodes in the second set from the “
locator” and
to transmit entire data contained in a file to a designated sequence of nodes from the second set;
said “
requestor”
means within the each node of the first set adapted to receive a designated sequence of nodes from the second set from the “
locator”
means and further adapted to send a request for file stripe to an appropriate node in the second set;
said “
prefix-generation”
means at the meta-data node adapted to generate a “
next-prefix”
element for each node in the second set by using the parameters associated with that individual node;
said “
identifier-generation”
means at the meta-data node adapted to generate a unique identifier for a file and transmitting the identifier to the node within the first set responsible for creating a file;
said “
receiver”
means within the each node of the second set adapted to receive a request for data from the “
requester”
or stripes of new or modified data form the “
transmitter”
within a node in the first set;
said “
retrieval”
means, upon trigger from the “
receiver”
means adapted to retrieve data form the storage media associated with the storage nodes and to deliver it to the “
transmitter”
means of the second set;
said “
storage”
means adapted to receive data from the “
receiver” and
store data onto a storage media; and
said “
transmitter”
means adapted to receive data from the “
retrieval”
means and further adapted to transmit data to the “
requestor”
means of the node from the first set that requested for that data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
Specification