Method for lock-free clustered erasure coding and recovery of data across a plurality of data stores in a network
First Claim
Patent Images
1. An erasure encoding method comprising:
- (a) providing a plurality of computer systems forming a distributed computer network, each computer system of the plurality of computer systems having a management process operating thereon, thereby forming a plurality of management processes;
(b) providing a plurality of distributed persistent memories, each persistent memory of the plurality of distributed persistent memories interoperably coupled to at least one of the plurality of computer systems and operative to receive constant updates;
(c) distributing, via an erasure encoding process, a plurality of data sets, wherein each data set of the plurality of data sets comprises at least one data block and at least one checksum block, across a plurality of the plurality of distributed persistent memories;
(d) initializing the plurality of management processes into and operating as a cluster, wherein during the initializing of the cluster, each management process;
agreeing to enter into an initialization mode;
entering into the initialization mode;
suspending regular runtime processing; and
registering data-set management-process management responsibility;
(e) performing a consistency check by the cluster of the plurality of data sets; and
(f) managing the plurality of data sets, comprising;
communicating by each management process to other management processes of the plurality of management processes to determine a responsible management process of each data set of the plurality of data sets, each data set managed by one of the management processes; and
determining by the responsible management process, via a sequencing indicator, whether each data block and each checksum block of each data set of the plurality of data sets is modified completely.
26 Assignments
0 Petitions
Accused Products
Abstract
The present invention provides a distributed clustering method to allow multiple active instances of consistency management processes that apply the same encoding scheme to be cooperative and function collectively. The techniques described herein facilitate an efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. The technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.
-
Citations
30 Claims
-
1. An erasure encoding method comprising:
-
(a) providing a plurality of computer systems forming a distributed computer network, each computer system of the plurality of computer systems having a management process operating thereon, thereby forming a plurality of management processes; (b) providing a plurality of distributed persistent memories, each persistent memory of the plurality of distributed persistent memories interoperably coupled to at least one of the plurality of computer systems and operative to receive constant updates; (c) distributing, via an erasure encoding process, a plurality of data sets, wherein each data set of the plurality of data sets comprises at least one data block and at least one checksum block, across a plurality of the plurality of distributed persistent memories; (d) initializing the plurality of management processes into and operating as a cluster, wherein during the initializing of the cluster, each management process; agreeing to enter into an initialization mode; entering into the initialization mode; suspending regular runtime processing; and registering data-set management-process management responsibility; (e) performing a consistency check by the cluster of the plurality of data sets; and (f) managing the plurality of data sets, comprising; communicating by each management process to other management processes of the plurality of management processes to determine a responsible management process of each data set of the plurality of data sets, each data set managed by one of the management processes; and determining by the responsible management process, via a sequencing indicator, whether each data block and each checksum block of each data set of the plurality of data sets is modified completely. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A computer-program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, the computer-readable program code executed to implement an erasure encoding method comprising:
-
(a) providing a plurality of computer systems forming a distributed computer network, each computer system of the plurality of computer systems having a management process operating thereon, thereby forming a plurality of management processes; (b) providing a plurality of distributed persistent memories, each persistent memory of the plurality of distributed persistent memories interoperably coupled to at least one of the plurality of computer systems and operative to receive constant updates; (c) distributing, via an erasure encoding process, plurality of data sets, wherein each data set of the plurality of data sets comprises at least one data block and at least one checksum block, across a plurality of the plurality of distributed persistent memories; (d) initializing the plurality of management processes into and operating as a cluster; (e) performing a consistency check by the cluster of the plurality of data sets, wherein performing the consistency check comprises; communicating by each management process with other management processes of the plurality of management processes to determine a responsible consistency-check management process of each data set of the plurality of data sets; performing by the responsible consistency check management process the consistency check; communicating by each management process with the other management processes to confirm a completion of the consistency check; and entering the cluster into an active mode; and (f) managing, the plurality of data sets, comprising; communicating by each management process to the other management processes of the plurality of management processes to determine a responsible management process of each data set of the plurality of data sets, each data set managed by one of the management process; and determining by the responsible management process, via a sequencing indicator, whether each data block and each checksum block of each data set of the plurality of data sets is modified completely. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
-
Specification