DECENTRALIZED DATA MANAGEMENT ACROSS HIGHLY DISTRIBUTED SYSTEMS

• US 20190327180A1
• Filed: 04/23/2018
• Published: 10/24/2019
• Est. Priority Date: 04/23/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Todd, Stephen J., Tabet, Said, Natanzon, Assaf, Savic, Dragan
• Granted Patent: US 10,841,237 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/783 Distributed allocation of resources, e.g. bandwidth brokers
• CPC Class Codes: H04L 47/803 Application aware
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 9/3239 involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
• CPC Class Codes: H04L 9/50 using hash chains, e.g. blockchains or hash trees

Abstract

In a system environment comprising a plurality of computing resources, wherein at least a portion of the computing resources are mobile, a method maintains a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes. Each of the plurality of computing resources is associated with a given messaging node and a given data node. The method manages transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network. Managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set and inserting a message into the decentralized messaging network instructing implementation of the one or more policies.

Claims

1. A method comprising:

in a system environment comprising a plurality of computing resources, wherein at least a portion of the computing resources are mobile, maintaining a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources is associated with a given messaging node and a given data node; and

managing transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing implementation of the one or more policies, such that each of the plurality of computing resources obtains the policy file and implements the one or more policies;

wherein the method is implemented via one or more processing devices each comprising a processor coupled to a memory.

2. The method of claim 1, wherein the step of managing transfer of the data set further comprises assigning a content address to the policy file and inserting a pointer to the content address into the decentralized messaging network to direct each of the plurality of computing resources where to obtain the policy file.

3. The method of claim 1, wherein the step of managing transfer of the data set further comprises logging a confirmation message indicating implementation of the one or more policies of the policy file by the plurality of computing resources as each of the computing resources implements the one or more policies.

4. The method of claim 3, wherein the confirmation messages are retrievable to audit the percentage of the plurality of computing resources that have implemented the one or more policies.

5. The method of claim 2, wherein the policy file specifies an amount of the data set that given types of the computing resources are instructed to store.

6. The method of claim 5, wherein a first type of computing resource is instructed to store the entire data set.

7. The method of claim 6, wherein the first type of computing resource is a stationary computing resource.

8. The method of claim 6, wherein a second type of computing resource is instructed to store at least a predetermined fraction of the entire data set.

9. The method of claim 8, wherein the second type of computing resource is a mobile computing resource.

10. The method of claim 1, wherein the step of managing transfer of the data set further comprises partitioning the data set into chunks.

11. The method of claim 10, wherein the step of managing transfer of the data set further comprises inserting one or more of the chunks into the decentralized data network and inserting a message into the decentralized messaging network instructing at least a portion of the plurality of computing resources to obtain the one or more chunks.

12. The method of claim 11, wherein the step of managing transfer of the data set further comprises assigning a content address to each of the chunks and inserting a pointer to the content address of each chunk into the decentralized messaging network to direct the portion of the plurality of computing resources where to obtain the one or more chunks.

13. The method of claim 11, wherein the step of managing transfer of the data set further comprises a given one of a given type of the plurality of computing resources sharing one or more chunks of the data set which it currently stores with one or more others of the given type of the plurality of computing resources that do not currently store the one or more chunks.

14. The method of claim 13, wherein the sharing step is implemented using a distributed hash table architecture.

15. The method of claim 11, wherein the step of managing transfer of the data set further comprises managing an instance wherein a given computing resource does not have the storage capacity to store the one or more chunks.

16. The method of claim 1, wherein the step of managing transfer of the data set further comprises maintaining a reachable status for each of the plurality of computing resources and setting one of the one or more policies in the policy file to ensure that the entire data set is reachable by any given computing resource.

17. The method of claim 1, wherein the step of managing transfer of the data set further comprises aggregating functionalities from two or more of the computing resources to ensure that the aggregation of two or more computing resources obtains the entire data set.

18. The method of claim 1, wherein the plurality of computing resources comprises a cloud platform at a first level of the system environment, a plurality of edge servers coupled to the cloud platform at a second level of the system environment, and a plurality of mobile compute platforms coupled to the plurality of edge servers at a third level of the system environment.

19. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by one or more processing device causes the one or more processing devices to perform steps of:

in a system environment comprising a plurality of computing resources, wherein at least a portion of the computing resources are mobile, maintaining a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources is associated with a given messaging node and a given data node; and

managing transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing implementation of the one or more policies, such that each of the plurality of computing resources obtains the policy file and implements the one or more policies.

20. A system comprising:

one or more processing devices configured to;

in a system environment comprising a plurality of computing resources, wherein at least a portion of the computing resources are mobile, maintain a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources is associated with a given messaging node and a given data node; and

manage transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing implementation of the one or more policies, such that each of the plurality of computing resources obtains the policy file and implements the one or more policies.