Distributed file system and node, storage method and storage controlling method
Distributed file system and node, storage method and storage controlling method
 CN 102,387,179 B
 Filed: 09/02/2010
 Issued: 08/10/2016
 Est. Priority Date: 09/02/2010
 Status: Active Grant
First Claim
1. a storage method for distributed file system, described distributed file system includes N number of depositingStorage node；
 Described method includes;
According to predetermined segmentation strategy, the file that is to be saved is cut into M data object；
Use checking algorithm that described M data object is verified, obtain corresponding to described literary composition to be savedK verification data object of part, described K is more than or equal to 1；
By described M the data object corresponding with described file to be saved and described K verification data pairAs being saved in described N number of storage node on S available storage node, wherein, S is less than or equal toN；
Described by described M data object with described K verification data object is saved in described N number of depositIn storage node, the step on S available storage node includes;
The storage node that S in described N number of storage node available is divided into the first storage node and secondStorage node；
Described M data object is saved on described first storage node；
Described K verification data object is saved on described second storage node；
If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node；
If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.
Chinese PRB Reexamination
Abstract
The invention discloses a kind of distributed file system and node, storage method and storage controlling method；Described storage method includes: according to predetermined segmentation strategy, the file that is to be saved is cut into M data object；Using checking algorithm to verify described M data object, obtain K the verification data object corresponding to described file to be saved, described K is more than or equal to 1；Being saved in described N number of storage node on S available storage node by described M the data object corresponding with described file to be saved and described K verification data object, wherein, S is less than or equal to N.The present invention can improve memory space utilization rate.
6 Claims

1. a storage method for distributed file system, described distributed file system includes N number of depositingStorage node；
 Described method includes;
According to predetermined segmentation strategy, the file that is to be saved is cut into M data object； Use checking algorithm that described M data object is verified, obtain corresponding to described literary composition to be savedK verification data object of part, described K is more than or equal to 1； By described M the data object corresponding with described file to be saved and described K verification data pairAs being saved in described N number of storage node on S available storage node, wherein, S is less than or equal toN； Described by described M data object with described K verification data object is saved in described N number of depositIn storage node, the step on S available storage node includes; The storage node that S in described N number of storage node available is divided into the first storage node and secondStorage node； Described M data object is saved on described first storage node； Described K verification data object is saved on described second storage node； If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node； If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.
 Described method includes;

2. a storage controlling method for distributed file system, described distributed file system includes NIndividual storage node, service station and metadata management node；
 Described method includes;
Receiving the solicited message that described service station sends, described solicited message is for the first file to be saved M data object of cutting and corresponding K verification data object request storage node；
DescribedVerification data object is to use checking algorithm that described M data object is verified the result obtained, instituteState K more than or equal to 1；According to described solicited message, it is thus achieved that the state of described N number of storage node, from described N number of storage knotFinding out S available storage node in point, wherein, S is less than or equal to N； According to predetermined deposit strategy, described M data object and K verification data object are distributed to instituteState S available storage node, produce an allocation result； Feed back described allocation result； Described according to predetermined deposit strategy by described M data object and K verification data object distributionInclude to the step of described S available storage node; According to described predetermined deposit strategy the storage node that described S is available is divided into the first storage node andSecond storage node； According to described predetermined deposit strategy, described M data object is distributed to described first storage node,Described K verification data object is distributed to described second storage node； If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node； If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.
 Described method includes;

3. a distributed file system, it is characterised in that including:

N number of storage node, is used for storing data； Metadata management node； Service station, for cutting into M data pair by a file to be saved according to predetermined segmentation strategy As, use checking algorithm that described M data object is verified, obtain corresponding to described literary composition to be savedK verification data object of part, described K is more than or equal to 1；
Ask to described metadata management nodeStorage node for described file to be saved；Described metadata management node is for the request according to described service station, it is thus achieved that described storage nodeState, the storage node that S is available is returned to described service station, and according to predetermined deposit strategyControl described service station by described M the data object corresponding with described file to be saved and described KIndividual verification data object is saved on the individual available storage node of S in described N number of storage node；
Wherein,S is less than or equal to N；Described metadata management node controls described service station by described M according to predetermined deposit strategyData object and described K verification data object are saved in S available depositing in described N number of storage nodeRefer on storage node; Described metadata management node can by the S in described N number of storage node according to predetermined deposit strategyStorage node be divided into the first storage node and the second storage node；
According to predetermined deposit strategy by describedM data object distributes to described first storage node, and described K verification data object is distributed to instituteState the second storage node, produce an allocation result；
Described service station is controlled according to described allocation resultDescribed M data object is saved on described first storage node, by described K verification data pairAs being saved on described second storage node；If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node； If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.


4. system as claimed in claim 3, it is characterised in that:
Described service station is additionally operable to when reading file from described storage node, if preserving data objectStorage node occur abnormal, then use the checking algorithm data object to being read and described check number Verify according to object, obtain the data object occurring depositing on abnormal storage node.

5. a metadata management node for distributed file system, described distributed file system is also wrappedInclude N number of storage node, service station；
 It is characterized in that, described metadata management node includes;
Select module, for when receiving the M that described service station cuts for the first file to be savedAfter the information of individual data object and corresponding K verification data object request storage node, it is thus achieved that described NThe state of individual storage node, finds out the available storage node of S from described N number of storage node, wherein,S is less than or equal to N；
Described verification data object is to use checking algorithm to enter described M data objectThe result that row verification obtains, described K is more than or equal to 1；Distribution module, is used for described M data object and K check number according to predetermined deposit strategyDistribute to described S available storage node according to object, produce an allocation result； Feedback module, is used for feeding back described allocation result； Described M data object and K are verified data according to predetermined deposit strategy by described distribution moduleObject is distributed to described S available storage node and is referred to; The storage node that described S is available is divided into the according to described predetermined deposit strategy by described distribution moduleOne storage node and the second storage node, distribute to described first storage knot by described M data objectPoint, distributes to described second storage node by described K verification data object； If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node； If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.
 It is characterized in that, described metadata management node includes;

6. a service station for distributed file system, described distributed file system includes N number of depositingStorage node；
 It is characterized in that, described service station includes;
Segmentation module, for cutting into M data by a file to be saved according to predetermined segmentation strategyObject； Correction verification module, is used for using checking algorithm to verify described M data object, obtains correspondenceIn K verification data object of described file, described K is more than or equal to 1； Request module, for asking the storage node for file to be saved to metadata management node； Preserve module, after arriving S available storage node when request, will be with described file to be savedCorresponding described M data object and described K verification data object are saved in described N number of storage knotIn point on S available storage node, wherein, S is less than or equal to N； Wherein, described by M corresponding for described file to be saved data object with described K check numberIt is saved on the individual available storage node of S in described N number of storage node according to object, refers to; The storage node that S in described N number of storage node available is divided into the first storage node and secondStorage node； Described M data object is saved on described first storage node； Described K verification data object is saved on described second storage node； If described verification data object has multiple, the part of described verification data object is saved in describedOn second storage node, part is saved on described first storage node； If there have the storage node preserving data object to occur to be abnormal, then select in the checking algorithm usedSelect multiple, use each checking algorithm respectively, the verification data object corresponding to this checking algorithm and this articleOther data object that part cuts into verifies, and obtains the data occurring preserving on abnormal storage nodeObject；
When selecting multiple checking algorithm, also the data object that each verification obtains is compared, to testThe correctness of the data object obtained by card.
 It is characterized in that, described service station includes;
Specification(s)