Distributed file systems on content delivery networks

• US 10,353,873 B2
• Filed: 07/10/2018
• Issued: 07/16/2019
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Desai, Rajiv
• Primary Examiner(s): Meky, Moustafa M
• Application Number:
  US16/031,775
• Publication Number:
  US 20180322141A1
• Time in Patent Office: 371 Days
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata

Abstract

A method, article of manufacture, and apparatus for accessing a distributed file system using a content delivery network is discussed. An IO request for an object is received at a metadata server comprising metadata mapping a plurality of objects to an object store. An object identifier associated with the object and a CDN domain associated with the object store are determined. A URL is generated from the object identifier and the CDN domain. The URL is then returned to the client.

Claims

1. A method for accessing data using a content delivery network (CDN), the CDN including a plurality of datacenters that are in different locations, the method comprising:

receiving an IO request for an object at a metadata server from a client, wherein the metadata server comprises metadata mapping a plurality of objects to an object store;

determining an object identifier associated with the requested object and the CDN domain associated with the object store;

generating a URL from the object identifier and the CDN domain; and

returning the URL to the client;

requesting the data object from the CDN domain using the URL, wherein the CDN domain is configured to access the data object from a datacenter that is optimal for the client.

2. The method of claim 1, further comprising mapping the plurality of objects to multiple object stores, wherein the CDN comprises a plurality of different datacenters that are part of a logical object store.

3. The method of claim 1, wherein the CDN comprises at least some servers that act as both a proxy and as a cache in the object store.

4. The method of claim 1, further comprising mounting at least a portion of the CDN domain.

5. The method of claim 4, further comprising changing a directory or location by sending a request to the metadata server.

6. The method of claim 1, wherein the requested object includes a plurality of data segments and the IO request is a write request.

7. The method of claim 6, further comprising:

determining segment identifiers for each of the plurality of data segments;

determining whether the object store contains data segments for each of the segment identifiers;

generating a plurality of URLs from the segment identifiers and the CDN domain; and

returning the plurality of URLs to the client.

8. The method of claim 7, further comprising determining at the metadata server whether the object store contains each of the data segments.

9. The method of claim 8, wherein the URL comprises a plurality of write URLs for the data segments the object store does not contain.

10. The method of claim 9, further comprising writing the data segments the object store does not contain to the object store and updating the metadata server by creating a new entry for each of the data segments the object store does not contain at the metadata server.

11. A non-transitory computer readable storage medium comprising processor instructions for accessing data using a content delivery network (CDN), the instructions comprising:

receiving an IO request for an object at a metadata server from a client, wherein the metadata server comprises metadata mapping a plurality of objects to an object store;

determining an object identifier associated with the requested object and the CDN domain associated with the object store;

generating a URL from the object identifier and the CDN domain; and

returning the URL to the client;

requesting the data object from the CDN domain using the URL, wherein the CDN domain is configured to access the data object from a datacenter that is optimal for the client.

12. The method of claim 11, further comprising mapping the plurality of objects to multiple object stores, wherein the CDN comprises a plurality of different datacenters that are part of a logical object store.

13. The method of claim 11, wherein the CDN comprises at least some servers that act as both a proxy and as a cache in the object store.

14. The method of claim 11, further comprising mounting at least a portion of the CDN domain.

15. The method of claim 14, further comprising changing a directory or location by sending a request to the metadata server.

16. The method of claim 11, wherein the requested object includes a plurality of data segments and the IO request is a write request.

17. The method of claim 16, further comprising:

determining segment identifiers for each of the plurality of data segments;

determining whether the object store contains data segments for each of the segment identifiers;

generating a plurality of URLs from the segment identifiers and the CDN domain; and

returning the plurality of URLs to the client.

18. The method of claim 17, further comprising determining at the metadata server whether the object store contains each of the data segments.

19. The method of claim 18, wherein the URL comprises a plurality of write URLs for the data segments the object store does not contain.

20. The method of claim 19, further comprising writing the data segments the object store does not contain to the object store and updating the metadata server by creating a new entry for each of the data segments the object store does not contain at the metadata server.