Distributed data storage and access systems

• US 8,099,402 B2
• Filed: 07/10/2008
• Issued: 01/17/2012
• Est. Priority Date: 07/10/2008
• Status: Expired due to Fees

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Carver, David C., Gerovac, Branko J.
• Primary Examiner(s): Ali, Mohammad
• Assistant Examiner(s): HOCKER, JOHN P
• Application Number:
  US12/170,657
• Publication Number:
  US 20100011003A1
• Time in Patent Office: 1,286 Days
• Field of Search: 707/705, 707/999.01, 707/913, 707/706
• US Class Current: 707/705
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A distributed system for content storage and access includes a storage platform having at least a first storage component, and an access platform having one or multiple access components. Each access component is associated with at least one access service. The access service may be an administrative service for receiving a service request initiated by a first user, the service request being associated with a first item of content, and for identifying a characteristic of the service request, a content management service for determining, based at least in part on the characteristic of the service request, a specification of a data transfer operation to be executed in association with the first storage component, and a directory service for maintaining information associated with the first item of content.

Claims

1. A distributed system for content storage and access, the system comprising:

an access platform, connected, via a first network, to a storage platform comprising at least a first storage component, the access platform comprising one or multiple access components, where the access platform is to provide a plurality of access services including;

an administrative service to;

receive, via a second network, that is different from the first network, an upload request, initiated by a first user, the upload request being associated with a first item of content, where the upload request includes a first set of attributes that characterize one or more operational objectives, of the first item of content, and the first item of content, and

identify a characteristic of the upload request;

a content management service to;

determine, based on the characteristic of the upload request, a specification of a data transfer operation to be executed in association with the first storage component, where the specification of the data transfer operation includes a description of a complete resource allocation arrangement and;

where the content management service comprises;

an upload service to;

 

allocate, in response to the upload request, resources of the first storage component to determine a partial resource allocation arrangement,

 

determine the complete resource allocation arrangement based on one or more of the first set of attributes, and

 

commit the complete resource allocation arrangement according to the partial resource allocation arrangement, and

an access service to;

determine, based on the specification of the data transfer operation, whether to grant a request to access the first item of content, where the request to access the first item of content is received from a second user via a third network that is different from the first network and the second network, and where an execution of the data transfer operation includes elements of the first item of content being stored in respective elements of the allocated resources of the first storage component according to the complete resource allocation arrangement.

2. The distributed system of claim 1, where the specification of the data transfer operation includes an identifier of a data transfer operation type and a persistent fully-resolvable identifier for an element of the first storage component.

3. The distributed system of claim 1, where the request to access the first item of includes a download request, the content management service further comprising:

a download service to;

identify, based on an interaction with a directory service that maintains information associated with the first item of content, an existing resource allocation arrangement associated with the first item of content in response to the download request.

4. The distributed system of claim 3, where the directory service maintains a description of the existing resource allocation arrangement that includes associations of elements of the first item of content with respective elements of the first storage component allocated to the first item, and

where the download service is further to:

perform a set of inquires to determine whether a condition for initiating a download operation in response to the download request is satisfied.

5. The distributed system of claim 4, where the set of inquires include an assessment of one or more of:

an overall workload condition of the first storage component,

a deviation in load across the respective elements of the first storage component associated with the elements of the first item,

availability of resources to be involved with the download operation, or

a present level of resource utilization, by various operations in the distributed system, associated with the first item of content.

6. The distributed system of claim 4, where, upon satisfaction of the condition for initiating the download operation, the download service is to:

determine a specification of the download operation, where the specification of the download operation includes a selected set of elements, among the elements of the first storage component associated with the first item, to be accessed for the download operation.

7. The distributed system of claim 1, where the plurality of access services further comprises:

a system management service to coordinate operations performed by one or more of the plurality of access services in the distributed system.

8. The distributed system of claim 1, further comprising:

a plurality of access service platforms, each of the plurality of access service platforms comprising one or more access components, where at least one of the plurality of access services provided by the one or more access components is distributed in multiple instances across one or more of the plurality of access service platforms.

9. The distributed system of claim 1, where the directory service provides an interface to a memory that is used to store information associated with content being handled by the distributed system.

10. The distributed system of claim 1, where the administrative service receives a particular type of service request based on a verification of an identity of the first user.

11. A method comprising:

receiving, by a network device and via a first network, a request to upload content to another network device, where the request includes the content, an external name associated with the content, and a set of attributes associated with the content;

creating, by the network device and in response to receiving the request, an object associated with the content, where the object is stored in a memory associated with the network device;

determining, in response to the object being created, a mapping for storing the content on the other network device, where determining the mapping is based on the set of attributes;

allocating, based on the mapping, resources of one or more sets of storage devices of the other network device to determine a partial resource allocation arrangement;

determining a complete resource allocation arrangement based on the partial resource allocation arrangement and one or more attributes of the set of attributes;

writing, by the network device and via a second network that is different from the first network, the content to the other network device based on the mapping, where writing the content includes storing elements of the content in respective elements of the allocated resources of the one or more sets of storage devices based on the complete resource allocation arrangement;

updating, by the network device, the memory to associate the mapping with the object; and

receiving, by the network device and via a third network that is different from the first network and the second network, a request to download the content, where the request to download the content is received from a client device.

12. The method of claim 11, where determining the mapping for storing the content includes:

identifying the one or more sets of storage devices, of the other network device; and

determining, based on the set of attributes, whether to store the content on the one or more sets of storage devices.

13. The method of claim, 11 where receiving the request to download the content includes:

identifying, based on the object stored in the updated memory, a location of the content on the other network device;

retrieving, based on identifying the location and via the second network, the content from the other network device; and

providing, via the third network, the content to the client device.

14. A method comprising:

receiving, by a network device and via a first network, an upload request from a first user, where the upload request includes a first item of content and a set of attributes that characterize one or more operational objectives of the first item of content;

identifying a characteristic of the upload request;

determining, by the network device, based on the characteristic, a specification associated with a data transfer operation, where the data transfer operation includes a transfer of the first item of content between the network device and another network device via a second network that is different than the first network;

allocating, by the network device, resources of the other network device based on the specification to determine a partial resource allocation;

determining, based on specification and one or more of the set of attributes, a complete resource allocation;

maintaining, by the network device, information associated with the partial resource allocation and the complete resource allocation; and

transferring, based on the maintained information and via the second network, the first item of content between the network device and the other network device.

15. The method of claim 14, further comprising:

receiving, from a client device and via a third network that is different than the first network and the second network, a download request associated with the first item of content;

determining, based on the maintained information, whether to permit the download of the first item of content from the other network device to the client device.

16. The method of claim 15, where determining whether to permit the download of the first item of content includes:

identifying, based on the maintained information, allocated resources associated with downloading the first item of content;

determining a current allocation of resources associated with the other network device; and

determining, based on identifying the allocated resources and determining the current allocation of resources, whether to permit the download of the first item of content from the other network device to the client device.