Systems and methods for distributed storage

• US 9,542,466 B2
• Filed: 03/15/2013
• Issued: 01/10/2017
• Est. Priority Date: 05/10/2012
• Status: Active Grant

• Current Assignee: Aetherstore, Inc.
• Original Assignee: Aetherstore, Inc.
• Inventors: MacInnis, Robert Francis
• Primary Examiner(s): Rostami, Mohammad S
• Application Number:
  US13/840,017
• Publication Number:
  US 20130304706A1
• Time in Patent Office: 1,397 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Techniques for distributed storage using a plurality of computing devices connected to a network can include storing an electronic file in a local storage layer of one of the computing devices. The stored electronic file can be asymmetrically transmitted, in portions, over the network to other computing devices to store the file across the other computing devices in a distributed storage layer. The electronic file can be asynchronously transmitted over the network to a cloud storage layer such that the electronic file is mirrored in the cloud storage layer. The local storage layer of each computing device can store, for each electronic file stored in the distributed storage layer, metadata having pointers to locations of the portions the electronic files stored in the local storage layer and distributed storage layer. The electronic files stored in the distributed storage layer can be displayed as stored in a single logical drive.

Claims

1. A method for distributed storage using a plurality of computing devices communicatively coupled via a network, comprising:

storing an electronic file in a local storage layer of one of the plurality of computing devices;

asynchronously transmitting the electronic file stored in the local storage layer of one of the plurality of computing devices, in portions, over the network to others of the plurality of computing devices such that the electronic file is stored across the other computing devices in a distributed storage layer;

asynchronously transmitting the electronic file stored in the distributed storage layer over the network to a cloud storage layer such that the electronic file is mirrored in the cloud storage layer;

storing in the local storage layer of each of the plurality of computing devices, for each electronic file stored in the distributed storage layer, metadata having pointers to locations of the portions the electronic files stored in the local storage layer and distributed storage layer,

wherein asynchronously transmitting the electronic file stored in the local storage layer of one of the plurality of computing devices includes;

partitioning the electronic file into fixed or variable sized portions;

hashing the portions onto the storage devices of the other computing devices, via the network;

updating the metadata to include pointers to the hashed portions'"'"' locations; and

transmitting, via the network, the metadata to the other computing devices using a gossip protocol; and

storing and retrieving an edited version of the electronic file, wherein storing includes, at one of the plurality of computing devices;

comparing the edited version with the electronic file to generate a diff file, the diff file containing differences between the edited version and the electronic file;

partitioning the diff file into fixed or variable sized portions;

hashing the portions of the diff file onto the storage devices of the other computing devices, via the network;

updating the metadata to include pointers to the locations of the hashed portions of the diff file;

transmitting, via the network, the metadata to the other computing devices using a gossip protocol, and

wherein retrieving the edited version at one of the plurality of computing devices, comprises;

determining locations of portions of the electronic file and portions of the diff file from the metadata;

retrieving the portions of the electronic file and the diff file from said locations, via the network; and

assembling the edited version based on the retrieved portions of the electronic file and the diff file on the local storage layer.

2. The method of claim 1, further comprising displaying, with a display device coupled with one of the computing devices, the electronic files stored in the distributed storage layer as stored in a single logical drive.

3. The method of claim 1, wherein the plurality of computing devices include one or more of a desktop computer, a laptop computer, or a handheld computing device.

4. The method of claim 1, wherein the network includes one or more of a wired local area network, a wireless local area network, an intranet, or the internet.

5. The method of claim 1, further comprising replicating the hashed portions onto one or more other computing devices through further hashing.

6. The method of claim 1, further comprising remotely accessing the electronic file, wherein remotely accessing includes, at a remote computing device retrieving the electronic file from the cloud storage layer.

7. The method of claim 1, wherein the metadata further includes one or more of version information, access information, permission information, and logical drive path information about the electronic file.

8. The method of claim 1, further comprising encrypting the electronic file.

9. A non-transitory computer-readable medium containing computer-executable instructions that when executed cause one or more computer devices to perform a method for distributed storage using a plurality of computing devices communicatively coupled via a network, comprising:

storing an electronic file in a local storage layer of one of the plurality of computing devices;

asynchronously transmitting the electronic file stored in the local storage layer of one of the plurality of computing devices, in portions, over the network to others of the plurality of computing devices such that the electronic file is stored across the other computing devices in a distributed storage layer;

asynchronously transmitting the electronic file stored in the distributed storage layer over the network to a cloud storage layer such that the electronic file is mirrored in the cloud storage layer;

storing in the local storage layer of each of the plurality of computing devices, for each electronic file stored in the distributed storage layer, metadata having pointers to locations of the portions the electronic files stored in the local storage layer and distributed storage layer,

wherein asynchronously transmitting the electronic file stored in the local storage layer of one of the plurality of computing devices includes;

partitioning the electronic file into fixed or variable sized portions;

hashing the portions onto the storage devices of the other computing devices, via the network;

updating the metadata to include pointers to the hashed portions'"'"' locations; and

transmitting, via the network, the metadata to the other computing devices using a gossip protocol; and

storing and retrieving an edited version of the electronic file, wherein storing includes, at one of the plurality of computing devices;

comparing the edited version with the electronic file to generate a diff file, the diff file containing differences between the edited version and the electronic file;

partitioning the diff file into fixed or variable sized portions;

hashing the portions of the diff file onto the storage devices of the other computing devices, via the network;

updating the metadata to include pointers to the locations of the hashed portions of the diff file;

transmitting, via the network, the metadata to the other computing devices using a gossip protocol, and

wherein retrieving the edited version at one of the plurality of computing devices, comprises;

determining locations of portions of the electronic file and portions of the diff file from the metadata;

retrieving the portions of the electronic file and the diff file from said locations, via the network; and

assembling the edited version based on the retrieved portions of the electronic file and the diff file on the local storage layer.

10. The non-transitory computer readable medium of claim 9, further comprising displaying, with a display device coupled with one of the computing devices, the electronic files stored in the distributed storage layer as stored in a single logical drive.

11. The non-transitory computer readable medium of claim 9, wherein the plurality of computing devices include one or more of a desktop computer, a laptop computer, or a handheld computing device.

12. The non-transitory computer readable medium of claim 9, wherein the network includes one or more of a wired local area network, a wireless local area network, an intranet, or the internet.

13. The non-transitory computer readable medium of claim 9, further comprising replicating the hashed portions onto one or more other computing devices through further hashing.

14. The non-transitory computer readable medium of claim 9, further comprising remotely accessing the electronic file, wherein remotely accessing includes, at a remote computing device retrieving the electronic file from the cloud storage layer.

15. The non-transitory computer readable medium of claim 9, wherein the metadata further includes one or more of version information, access information, permission information, and logical drive path information about the electronic file.

16. The non-transitory computer readable medium of claim 9, further comprising encrypting the electronic file.

17. A system for distributed storage, comprising:

a plurality of computing devices communicatively coupled with a network, each computing device having associated therewith one or more processors and one or more storage devices;

a distributed storage layer including a portion of the one or more storage devices of each of the plurality of computing devices;

a local storage layer defined on the one or more storage devices of each computing device having stored thereon metadata having pointers to locations of portions of electronic files stored in the local storage layer and distributed storage layer;

wherein the one or more processors of each computing device are configured to communicate, via the network, with others of the one or more computing devices such that portions of an electronic file stored in the local storage layer of a first computing device are asynchronously stored across the other computing devices in the distributed storage layer;

wherein the one or more processors of each computing device are configured to communicate, via the network, with a cloud storage layer such that the electronic file stored in the distributed storage layer is asynchronously mirrored in the cloud layer,

wherein the one or more processors of the first computing device are further configured to partition the electronic file into fixed or variable sized portions, hash the portions onto the storage devices of the other computing devices, via the network, update the metadata to include pointers to the hashed portions'"'"' locations, and transmit, via the network, the metadata to the other computing devices using a gossip protocol such that the portions of the electronic file are asynchronously stored across the other computing devices in the distributed storage layer, and

wherein the one or more processors of the first computing device are further configured to store an edited version of the electronic file, wherein said storing includes;

comparing the edited version with the electronic file to generate a diff file, the diff file containing differences between the edited version and the electronic file;

partitioning the diff file into fixed or variable sized portions;

hashing the portions of the diff file onto the storage devices of the other computing devices, via the network;

updating the metadata to include pointers to the locations of the hashed portions of the diff file;

transmitting, via the network, the metadata to the other computing devices using a gossip protocol; and

and wherein the one or more processors of a second computing device are configured to retrieve the edited version of the electronic file, wherein said retrieving the edited version comprises;

determining locations of portions of the electronic file and portions of the diff file from the metadata;

retrieving the portions of the electronic file and the diff file from said locations, via the network; and

assembling the edited version based on the retrieved portions of the electronic file and the diff file on the local storage layer.

18. The system of claim 17, wherein the one or more processors of each computing device are configured to display, with a display device coupled with one of the computing devices, the electronic files stored in the distributed storage layer as stored in a single logical drive.

19. The system of claim 17, wherein the plurality of computing devices include one or more of a desktop computer, a laptop computer, or a handheld computing device.

20. The system of claim 17, wherein the network includes one or more of a wired local area network, a wireless local area network, an intranet, or the internet.

21. The system of claim 17, wherein the one or more processors of each of the plurality of computing devices are further configured to replicate the hashed portions onto one or more other computing devices through further hashing.

22. The system of claim 17, further comprising a remote computing device having one or more processors configured to remotely access the electronic file by retrieving the electronic file from the cloud storage layer.

23. The system of claim 17, wherein the metadata further includes one or more of version information, access information, permission information, and logical drive path information about the electronic file.

24. The system of claim 17, wherein the one or more processors each of the plurality of computing devices is further configured to encrypt the electronic file.