Distributed File System

• US 20120254116A1
• Filed: 03/31/2011
• Published: 10/04/2012
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Thereska, Eno, Gunawardena, Dinan Srilal, Scott, James W., Harper, Richard
• Granted Patent: US 9,384,199 B2
• Time in Patent Office: Days
• US Class Current: 707/640
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3209 Monitoring remote activity, e.g. over telephone lines or network connections
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 9/4418 Suspend and resume; Hibernate and awake
• CPC Class Codes: H04L 63/205 involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved negotiation of communication capabilities H04L69/24

Abstract

A distributed file system for devices is described. In an embodiment, each data element stored on one of the devices has an associated location and availability attribute. The location attribute is stored co-located with the data element. The availability attribute and a copy of the location attribute are stored by a metadata service. When a client on a device needs to access a data element, it sends a request to the metadata service to find the location of the data element. If the data element is available, this information is provided to the client and this may involve waking a dormant device which holds the data element. Where the data element is not available, read only access may be granted to a cached copy of the data element. Where replication is used and one of the devices holding a replica is unavailable, the system may use write off-loading.

Claims

1. A computer-implemented method of managing a distributed data storage system, the data storage system comprising a plurality of different devices and the method comprising:

storing location and availability information for a plurality of data elements stored on the devices;

receiving a request from a device for access to a data element;

determining if the data element is available based on stored availability information; and

if the data element is available, providing location information for the data element to the device.

2. A method according to claim 1, further comprising:

if the data element is not available, determining if a copy of the data element is available; and

if a copy of the data element is available, allowing read only access to the copy of the data element by the device.

3. A method according to claim 1, wherein at least one of the data elements is replicated and wherein:

storing location and availability information for a plurality of data elements stored on the devices comprises storing location and availability information for each replica of the plurality of data elements,

determining if the data element is available based on stored availability information comprises determining if a primary replica of the data element is available, and

providing location information for the data element to the device comprises providing location information for the primary replica of the data element to the device.

4. A method according to claim 3, further comprising:

if the primary replica of the data element is not available, determining if any other replica of the data element is available; and

if another replica of the data element is available, allowing read only access to the other replica of the data element by the device.

5. A method according to claim 3, wherein the request from the device comprises user location data, the method further comprising:

determining if an access control policy for the data element is satisfied, the access control policy being defined in terms of at least one of;

the user location data and location information for the primary replica of the data element; and

wherein location information for the primary replica of the data element is provided to the device only if the primary replica is available and the access control policy is satisfied.

6. A method according to claim 5, further comprising:

if the access control policy for the data element is not satisfied, determining if any available replica of the data element satisfies the access control policy; and

if an available replica of the data element satisfies the access control policy, migrating a primary designation from the current primary replica to the available replica which satisfies the access control policy prior to providing location information for the primary replica of the data element to the device.

7. A method according to claim 3, further comprising:

on receipt of a message from a device indicating that the device is entering a reduced power mode, identifying any primary replicas of data elements stored on that device; and

for each primary replica of a data element stored on that device, migrating the primary designation from the current primary replica to an available replica stored on another device.

8. A method according to claim 1, further comprising:

updating the location and availability information based on data received from the devices.

9. A method according to claim 8, further comprising:

detecting availability of devices; and

updating availability information based on the detected availability of devices.

10. A method according to claim 1, wherein the distributed data storage system comprises a home distributed data storage system.

11. A device for use in a distributed data storage system, the device comprising:

a data store arranged to store a plurality of data elements, each data element having an associated location attribute identifying a location of the data element and availability attribute identifying availability of the data element, wherein the location attribute is stored in a location which is co-located with the data element and the availability attribute is stored by a metadata service in the distributed storage system;

a communication interface;

a main processor arranged to send requests for location information to the metadata service and to send read/write requests to a remote device in the distributed storage system based on location information received from the metadata service, the requests being sent via the communication interface; and

a processing element arranged to maintain network connectivity when the device is in a reduced power mode and the main processor is off and to wake the main processor on receipt of a read/write request from a remote device in the distributed storage system.

12. A device according to claim 11, wherein the main processor is arranged to provide read only access to the plurality of data elements when communication with the metadata service is not possible.

13. A device according to claim 11, wherein one of the plurality of data elements comprises one of a set of replicas of a data element, each replica being stored on a different device in the distributed data storage system and the replica stored in the data store being designated a primary replica, and wherein the main processor is further arranged to send a message to the metadata service prior to entering the reduced power mode to trigger migration of the primary designation to another of the replicas of the data element.

14. A device according to claim 11, wherein the main processor is further arranged to:

write data to a data element in the data store in response to receipt of a write request relating to the data element from a remote device;

determine if all remote devices storing replicas of the data element are available; and

send the write request to a short-term store in the distributed storage system if all the remote devices are not available.

15. A device according to claim 11, wherein the main processor is further arranged to:

receive a read/write request relating to a data element in the data store from a remote device;

determine a user location associated with the write request;

determine if an access control policy associated with the data element is satisfied, the access control policy being defined in terms of at least one of the user location and the location attribute associated with the data element; and

service the read/write request if the access control policy is satisfied.

16. A device according to claim 11, wherein the processing element is mounted within a module, the module further comprising a network interface and wherein the processing element is arranged to maintain network connectivity when the main processor is off via the network interface and to receive read/write requests via the network interface.

17. A device according to claim 11, wherein the data store is further arranged to store a cached copy of metadata received from the metadata service, the metadata comprising the location and availability attributes associated with each data element stored in the distributed data storage system and wherein the main processor is arranged to send a request for updated metadata in response to a failure of a read/write request sent to a remote device based on location information from the cached copy of metadata.

18. A device according to claim 11, which comprises a consumer device and wherein the distributed data storage system comprises a plurality of different consumer devices.

19. A distributed data storage system comprising a plurality of different devices, each device comprising:

a data store arranged to store data elements and a location attribute associated with each data element, the location attribute indicating a geographical location of the data element and wherein the location attribute is co-located with the data element; and

a main processor;

a processing element arranged to maintain network connectivity when the device is in a reduced power mode and the main processor is off and to wake the main processor on receipt of a read/write request from another device in the system,

and the system further comprising;

a metadata service arranged to store both a copy of the location attribute and an availability attribute associated with each data element stored in the distributed data storage system, the availability attribute indicating an availability status of the associated data element and to service access requests for data elements received from devices based on the location and availability attributes.

20. A distributed data storage system according to claim 19, further comprising a short-term store and wherein the main processor in a device is further arranged to:

write data to a data element in the data store in response to receipt of a write request relating to the data element from a remote device;

determine if all remote devices storing replicas of the data element are available; and

if all the remote devices are not available, send the write request to the short-term store,

wherein the metadata service and short-term store are hosted in a cloud computing system.