DISTRIBUTED OBJECT STORAGE SYSTEM COMPRISING PERFORMANCE OPTIMIZATIONS
8 Assignments
0 Petitions
Accused Products
Abstract
A distributed object storage system includes several performance optimizations with respect to storing very small data objects, very large data objects and CRC calculations.
45 Citations
30 Claims
-
1-15. -15. (canceled)
-
16. A distributed object storage system which, according to a first storage and retrieval option, comprising:
-
a plurality of redundant storage elements, operable to store and retrieve a data object comprising a data object identifier in the form of a predetermined number of redundant sub blocks comprising said data object identifier, said predetermined number corresponding to a predetermined multiple of a desired spreading width, which consists of the sum of; (a) a minimal spreading requirement, corresponding to the minimal number of storage elements that must store sub blocks of said data object and are not allowed to fail;
supplemented with(b) a maximal concurrent failures tolerance, corresponding to the number of storage elements that must store sub blocks of said data object and are allowed to fail concurrently; each one of said redundant sub blocks comprising; (a) encoded data of equal size of the data object divided by a factor equal to said predetermined multiple of said minimal spreading requirement; and (b) decoding data, such that said data object can be decoded from any combination of said redundant sub blocks of which the number corresponds to predetermined multiple of said minimal spreading requirement; a plurality of storage nodes each comprising a share of said plurality of redundant storage elements; and at least one controller node, operably connected to or comprised within said storage nodes when storing or retrieving said data object, comprising; (a) an encoding module operable to disassemble said data object into said predetermined number of redundant sub blocks; (b) a spreading module operable to store said predetermined number of said redundant sub blocks on a number of said storage elements being larger or equal to said desired spreading width; (c) a clustering module operable to retrieve at least said predetermined multiple of said minimal spreading requirement of said redundant sub blocks from a plurality of said storage elements; and (d) a decoding module operable to assemble said data object from any combination of said redundant sub blocks of which the number corresponds to said predetermined multiple of said minimal spreading requirement; wherein, according to a second storage and retrieval option of said distributed object storage system; (a) said plurality of redundant storage elements, are further operable to store and retrieve said data object comprising a data object identifier in the form of a predetermined plurality of replication copies of said data object comprising said data object identifier, said predetermined plurality corresponding to said desired spreading width, each one of said replication copies comprising an exact copy of said data object; (b) said encoding module is further operable to replicate said data object into said predetermined plurality of said replication copies; (c) said spreading module is further operable to store said predetermined plurality of said replication copies on a corresponding plurality of said storage elements; (d) said clustering module is further operable to retrieve at least one of said predetermined plurality of said replication copies from said corresponding plurality of said storage elements; and (e) said decoding module is further operable to provide said data object (500) from any one of said replication copies; and wherein said distributed object storage system is operated according to said first storage and retrieval option if the size of said data object is equal to or larger than a predetermined lower data object size threshold, and is operated according to said second storage and retrieval option if the size of said data object is smaller then said predetermined lower data object size threshold. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. An apparatus for storage or retrieval of a data object on a storage medium, which is unreliable, said storage medium, wherein said apparatus is operable to store and retrieve on said storage medium a data object comprising a data object identifier in the form of a predetermined number of redundant sub blocks comprising said data object identifier, said predetermined number corresponding to a predetermined multiple of a desired spreading width, which consists of the sum of:
-
(a) a minimal spreading requirement, which when multiplied with said predetermined multiple, corresponds to the minimal number of sub blocks of said data object which are not allowed to fail;
supplemented with(b) a maximal concurrent failures tolerance, which when multiplied with said predetermined multiple, corresponds to the number of sub blocks of said data object which are allowed to fail concurrently; each one of said redundant sub blocks comprising; (a) encoded data of equal size of the data object divided by a factor equal to said predetermined multiple of said minimal spreading requirement; and (b) decoding data, such that said data object can be decoded from any combination of said redundant sub blocks of which the number corresponds to predetermined multiple of said minimal spreading requirement; said apparatus operably being connected to said storage medium when storing or retrieving said data object, comprising; (a) an encoding module operable to disassemble said data object into said predetermined number of redundant sub blocks; (b) a spreading module operable to store said predetermined number of said redundant sub blocks on said storage medium; (c) a clustering module operable to retrieve at least said predetermined multiple of said minimal spreading requirement of said redundant sub blocks from said storage medium; and (d) a decoding module operable to assemble said data object from any combination of said redundant sub blocks of which the number corresponds to said predetermined multiple of said minimal spreading requirement. - View Dependent Claims (29, 30)
-
Specification