Distributed file system

• US 9,384,199 B2
• Filed: 03/31/2011
• Issued: 07/05/2016
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Thereska, Eno, Gunawardena, Dinan Srilal, Scott, James W., Harper, Richard
• Primary Examiner(s): Jacob, Ajith
• Application Number:
  US13/077,198
• Publication Number:
  US 20120254116A1
• Time in Patent Office: 1,923 Days
• Field of Search: 707/652
• US Class Current: 1/1
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3209 Monitoring remote activity, e.g. over telephone lines or network connections
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 9/4418 Suspend and resume; Hibernate and awake
• CPC Class Codes: H04L 63/205 involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved negotiation of communication capabilities H04L69/24

Abstract

A distributed file system for devices is described. In an embodiment, each data element stored on one of the devices has an associated location and availability attribute. The location attribute is stored co-located with the data element. The availability attribute and a copy of the location attribute are stored by a metadata service. When a client on a device needs to access a data element, it sends a request to the metadata service to find the location of the data element. If the data element is available, this information is provided to the client and this may involve waking a dormant device which holds the data element. Where the data element is not available, read only access may be granted to a cached copy of the data element. Where replication is used and one of the devices holding a replica is unavailable, the system may use write off-loading.

Claims

1. A computer-implemented method of managing a distributed data storage system, the data storage system comprising a plurality of different devices and the method comprising:

storing location and availability information for a plurality of data elements stored on the devices, the location data of each data element being co-located with the associated data element and copies of the location data being co-located with the availability information for the plurality of data elements, availability information comprising at least one of battery level of a device or bandwidth of a network connection;

receiving a request from a device for access to a data element; and

,

in response to receiving the request,

determining if the data element is available based on stored availability information, the determination based at least in part on a first cost function that accounts for the availability information; and

if the data element is available, providing location information for the data element to the device.

2. A method according to claim 1, further comprising:

if the data element is not available, determining if a copy of the data element is available; and

if a copy of the data element is available, allowing read only access to the copy of the data element by the device.

3. A method according to claim 1, wherein at least one of the data elements is replicated and wherein:

storing location and availability information for a plurality of data elements stored on the devices comprises storing location and availability information for each replica of the plurality of data elements,

determining if the data element is available based on stored availability information comprises determining if a primary replica of the data element is available, and

providing location information for the data element to the device comprises providing location information for the primary replica of the data element to the device.

4. A method according to claim 3, further comprising:

if the primary replica of the data element is not available, determining if any other replica of the data element is available; and

if another replica of the data element is available, allowing read only access to the other replica of the data element by the device.

5. A method according to claim 3, wherein the request from the device comprises user location data determined based at least in part on at least one of GPS or cell site data, the method further comprising:

determining if an access control policy for the data element is satisfied, the access control policy being defined in terms of at least one of;

the user location data and location information for the primary replica of the data element; and

wherein location information for the primary replica of the data element is provided to the device only if the primary replica is available and the access control policy is satisfied.

6. A method according to claim 5, further comprising:

if the access control policy for the data element is not satisfied, determining if any available replica of the data element satisfies the access control policy; and

if an available replica of the data element satisfies the access control policy, migrating a primary designation from the current primary replica to the available replica which satisfies the access control policy prior to providing location information for the primary replica of the data element to the device.

7. A method according to claim 3, further comprising:

on receipt of a message from a device indicating that the device is entering a reduced power mode, identifying any primary replicas of data elements stored on that device; and

for each primary replica of a data element stored on that device, migrating the primary designation from the current primary replica to an available replica stored on another device.

8. A method according to claim 1, wherein at least one of the plurality of data elements is replicated and wherein:

one of the replicas is designated as a primary replica, the designation being based at least in part on expiration of a primary lease.

9. A method according to claim 1, further comprising:

receiving a write request to the data element; and

sending the write request to at least one of a plurality of short-term data stores, the selection of the at least one short-term data store being based at least in part on a second cost function, the second cost function being related to at least one of location, bandwidth of a network connection, or battery level of a device.

10. A method according to claim 1, wherein the distributed data storage system comprises a home distributed data storage system and providing location information for the data element to the device includes determining whether the device is awake and, if the device is not awake, waking the device.

11. A system for managing distributed data storage, the system comprising:

a plurality of devices, each comprising tangible storage media, storing a plurality of data elements;

a data store arranged to store data elements and a location attribute associated with each data element, the location attribute being co-located with the data element;

a processor arranged to operate a metadata service, the metadata service arranged to store both a copy of the location attribute and an availability attribute associated with each data element stored on the plurality of devices, the availability attribute indicating an availability status of the associated data element and to service access requests for data elements received from devices based on the location and availability attributes; and

the metadata service further arranged to determine whether to retrieve at least metadata associated with at least one of the plurality of data elements from a device storage or from a network storage, the determination being based at least in part on at least one of bandwidth of a network connection or a battery level of a device.

12. A system according to claim 11, in which the availability attribute of the data elements located on a device is related to the power state of the device.

13. A system according to claim 11, further comprising:

in response to a request for access to a data element from a device, accessing the stored availability attribute for the device; and

if the data element is not available, determining if a copy of the data element is available; and

if a copy of the data element is available, allowing read only access to the copy of the data element by the device.

14. A system according to claim 11, in which at least one of the data elements is replicated and:

storing location and availability information for the plurality of data elements stored on the devices comprises storing location and availability information for each replica of the plurality of data elements,

determining if the data element is available based on stored availability information comprises determining if a primary replica of the data element is available, and

providing location information for the data element to the device comprises providing location information for the primary replica of the data element to the device.

15. A system according to claim 14, further comprising:

if the primary replica of the data element is not available, determining if any other replica of the data element is available; and

if another replica of the data element is available, allowing read only access to the other replica of the data element by the device.

16. A system according to claim 14, further comprising:

a processor arranged to allow a user to select a consistency attribute for at least one of the plurality of data elements, the consistency attribute defining whether or not write access is provided to the primary replica of the data element.

17. One or more tangible storage media, having stored thereon executable instructions comprising:

device-executable instructions to store location and availability information for a plurality of data elements and a plurality of replicas of data elements, the location data of each data element and each replica being co-located with the associated data element or replica and copies of the location data being co-located with the availability information for the plurality of data elements and plurality of replicas, the plurality of data elements and plurality of replicas being stored on a plurality of devices and the location information including the geographic location of the data element;

device-executable instructions to receiver a request from a device for access to a data element; and

,

device-executable instructions to, in response to receiving the request,

determine if the data element is available based on the stored availability information, determining comprising calculating a cost of accessing at least one of the plurality of data elements and at least one of the plurality of replicas, the cost based at least in part on at least one of a battery level of a device or a bandwidth of a network connection; and

if the data element is available, provide location information for the data element to the device.

18. One or more tangible storage media according to claim 17, having stored thereon executable instructions comprising:

device-executable instructions to, if a requested data element is not available, determine if a primary replica of the data element is available; and

if the primary replica of the data element is available, provide location information for the primary replica to the device; and

if the primary replica of the data element is not available, determine if a secondary replica of the data element is available and provide location information for the available secondary replica.

19. One or more tangible storage media according to claim 17, having stored thereon executable instructions comprising:

device-executable instructions to receive location data related to the requesting device; and

device-executable instructions to migrate the primary replica based on the received location data.

20. One or more tangible storage media according to claim 17, having stored thereon executable instructions comprising:

device-executable instructions to determine a degree of replication for a data element, the degree of replication being related to the location attribute for the data element.