Large scale storage system

• US 9,407,516 B2
• Filed: 01/10/2012
• Issued: 08/02/2016
• Est. Priority Date: 01/10/2011
• Status: Active Grant

• Current Assignee: StorONE, Ltd.
• Original Assignee: StorONE, Ltd.
• Inventors: Gordon, Raz
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Roumi, Mahran Abu
• Application Number:
  US13/978,959
• Publication Number:
  US 20130290399A1
• Time in Patent Office: 1,666 Days
• Field of Search: 709/239
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/284 being distributed
• CPC Class Codes: G06F 2212/314 In storage network, e.g. network attached cache
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/5025 by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade
• CPC Class Codes: H04L 41/5051 Service on demand, e.g. definition and deployment of services in real time
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

A distributed storage system comprising interconnected computer nodes; each one of the computer nodes comprising at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent; at least one of the computer nodes comprising one or more resources including at least one cache resource configured to cache objects and having corresponding cache-related parameters; at least one UDSP agent of a respective computer node having the at least one cache resource is configured to: monitor cache-related parameters of the at least one cache resource connected to the respective computer node, for determining whether the cache-related parameters meet at least one first SLS criteria; and in the case the at least one first SLS criteria is not met, initiate handoff of at least part of one or more cache object spaces of the at least one cache resource to at least one other computer node, which after receiving the at least part of one or more cache object spaces, its cache-related parameters meet at least one second SLS criteria.

Claims

1. A distributed storage system comprising interconnected computer nodes;

each one of said interconnected computer nodes comprising at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent;

wherein a first computer node among said interconnected computer nodes comprises a first cache resource, said first computer node is responsible for handling one or more cache object spaces related to one or more logical storage entities and is operatively connected with a UDSP data repository comprising data on first cache-related parameters corresponding to said first cache resource and cache-related parameters corresponding to one or more cache resources of one or more other computer nodes among said interconnected computer nodes; and

wherein said UDSP agent of said first computer node is configured to;

monitor said first cache-related parameters to determine whether said first cache-related parameters meet at least one first Service Level Specification (SLS) criterion;

when said at least one first SLS criterion is not met by said first cache-related parameters, initiate a handoff of responsibility over at least part of said one or more cache object spaces to at least one second computer node among said interconnected computer nodes, said at least one second computer node comprising a second cache resource with second cache-related parameters corresponding thereto,

wherein said UDSP agent is further configured to

select said at least one second computer node such that, after receiving responsibility over said at least part of said one or more cache object spaces, said second cache-related parameters meet at least one second SLS criterion, wherein said second cache-related parameters are comprised in said UDSP data repository;

create a post-handoff cache mapping at least indicative, for said at least part of said one or more cache object spaces, of said at least one second computer node selected to be responsible thereof;

send said post-handoff cache mapping to said at least one second computer node;

transfer responsibility over said at least part of said one or more cache object spaces to said at least one second computer node;

use data comprised in said UDSP repository to monitor said second cache-related parameters and to determine whether said second cache-related parameters meet at least one first SLS criterion; and

when said at least one first SLS criterion is not met, initiate a message to said selected at least one second computer node, said message informative of at least one of said cache-related parameters extracted from said UDSP data repository, corresponding to said second cache resource and not meeting said at least one first SLS criterion.

2. The distributed storage system of claim 1, wherein said UDSP agent of said first computer node is further configured to:

monitor said cache-related parameters corresponding to said first cache resource to determine whether said parameters meet at least one third SLS criterion indicative of underutilization of said first cache resource; and

when said at least one third SLS criterion is met, initiate a handoff of responsibility over at least part of said one or more cache object spaces to at least one third computer node among said interconnected computer nodes, said third computer node comprising a third cache resource, wherein said UDSP agent is further configured to select said at least one third computer node such that after receiving responsibility over said at least part of one or more cache object spaces, cache-related parameters corresponding to said third cache resource meet said at least one second SLS criterion.

3. The distributed storage system of claim 1, wherein said UDSP agent is further configured, in response to a failure to initiate said handoff, to provide a user with a failure notification or with a recommendation indicative of a need in addition of at least one additional cache resource.

4. The distributed storage system of claim 1 wherein said UDSP repository is comprised in at least one of the group consisted of said first computer node, a gateway resource operatively coupled to said first computer node and a client server operatively coupled to said first computer node.

5. The distributed storage system of claim 1, wherein said UDSP agent of said first computer node is further configured to send said post-handoff cache mapping to one or more client servers associated with said at least part of one or more cache object spaces.

6. The distributed storage system of claim 1 wherein said UDSP agent of said first computer node is further configured to:

receive, during said handoff, a request relating to a cache object space within said post-handoff cache mapping; and

relay said request to said second computer node when, according to said post-handoff cache mapping, said first computer node is not responsible for the respective cache object space.

7. A computer node configured to operatively connected to an infrastructure layer including interconnected computer nodes, said computer node comprising:

at least one first cache resource having corresponding cache-related parameters and configured to be responsible for handling one or more cache object spaces related to one or more logical storage entities, the computer node further operatively connected to a UDSP data repository comprising data on first cache-related parameters corresponding to said first cache resource and cache-related parameters corresponding to one or more cache resources of one or more other computer nodes among said interconnected computer nodes;

at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent, wherein said UDSP agent is configured to;

monitor the first cache-related parameters to determine whether the first cache-related parameters meet at least one first Service Level Specification (SLS) criterion;

when said at least one first SLS criterion is not met by the first cache-related parameters, initiate a handoff of responsibility over at least part of said one or more cache object spaces to at least one second computer node among said interconnected computer nodes, said at least one second computer node comprising a second cache resource with second cache-related parameters corresponding thereto,

wherein said UDSP agent is further configured to

select said at least one second computer node such that, after receiving responsibility over said at least part of said one or more cache object spaces, the second cache-related parameters meet at least one second SLS criterion, wherein the second cache-related parameters are comprised in said UDSP data repository;

create a post-handoff cache mapping at least indicative, for said at least part of said one or more cache object spaces, of said at least one second computer node selected to be responsible thereof;

send said post-handoff cache mapping to said at least one second computer node;

transfer responsibility over said at least part of said one or more cache object spaces to said at least one second computer node;

use data comprised in said UDSP repository to monitor the second cache-related parameters and to determine whether the second cache-related parameters meet at least one first SLS criterion; and

when said at least one first SLS criterion is not met, initiate a message to said selected at least one second computer node, said message informative of at least one of the cache-related parameters extracted from said UDSP data repository, corresponding to said second cache resource and not meeting said at least one first SLS criterion.

8. The computer node of claim 7, wherein said UDSP agent is further configured to:

monitor the cache-related parameters corresponding to said first cache resource to determine whether the parameters meet at least one third SLS criterion indicative of underutilization of said first cache resource; and

when said at least one third SLS criterion is met, initiate a handoff of responsibility over at least part of said one or more cache object spaces to at least one third computer node among said interconnected computer nodes, said third computer node comprising a third cache resource, wherein said UDSP agent is further configured to select said at least one third computer node such that after receiving responsibility over said at least part of one or more cache object spaces, cache-related parameters corresponding to said third cache resource meet said at least one second SLS criterion.

9. The computer node of claim 7, wherein said UDSP agent is further configured, in response to a failure to initiate said handoff, to provide a user with a failure notification or with a recommendation indicative of a need in addition of at least one additional cache resource.

10. The computer node of claim 7, wherein said UDSP repository is comprised in at least one of the group consisted of said first computer node, a gateway resource operatively coupled to said first computer node and a client server operatively coupled to said first computer node.

11. The computer node of claim 7, wherein said UDSP agent is further configured to send said post-handoff cache mapping to one or more client servers associated with said at least part of one or more cache object spaces.

12. The computer node of claim 7, wherein said UDSP agent is further configured to:

receive, during said handoff, a request relating to a cache object space within said post-handoff cache mapping; and

relay said request to said second computer node when, according to said post-handoff cache mapping, said first computer node is not responsible for the respective cache object space.

13. A method of operating a computer node having at least one first cache resource connected thereto and configured to be responsible for handling one or more cache object spaces related to one or more logical storage entities, the computer node configured to be operatively connected to an infrastructure layer including interconnected computer nodes and to a UDSP data repository comprising data on first cache-related parameters corresponding to said first cache resource and cache-related parameters corresponding to one or more cache resources of one or more other computer nodes among said interconnected computer nodes, the method comprising:

monitoring the first cache-related parameters to determine whether the first cache-related parameters meet at least one first Service Level Specification (SLS) criterion;

when said at least one first SLS criterion is not met by the first cache-related parameters, initiating handoff of responsibility over at least part of said one or more cache object spaces to at least one second computer node of said interconnected computer nodes, said at least one second computer node comprising a second cache resource with second cache-related parameters corresponding thereto, wherein said at least one second computer node is selected such that after receiving responsibility over said at least part of said one or more cache object spaces, the second cache-related parameters meet at least one second SLS criterion, wherein the second cache-related parameters are comprised in said UDSP data repository;

creating a post-handoff cache mapping at least indicative, for said at least part of said one or more cache object spaces, of said at least one second computer node selected to be responsible thereof;

sending said post-handoff cache mapping to said at least one second computer node;

transferring responsibility over said at least part of said one or more cache object spaces to said at least one second computer node;

using data comprised in said UDSP repository to monitor the second cache-related parameters and to determine whether the second cache-related parameters meet at least one first SLS criterion; and

when said at least one first SLS criterion is not met, initiating a message to said selected at least one second computer node, said message informative of at least one of the cache-related parameters extracted from said UDSP data repository, corresponding to said second cache resource and not meeting said at least one first SLS criterion.

14. The method of claim 13 further comprising:

monitoring the cache-related parameters corresponding to said first cache resource to determine whether the parameters meet at least one third SLS criterion indicative of underutilization of said first cache resource; and

when said at least one third SLS criterion is met, initiating a handoff of responsibility over at least part of said one or more cache object spaces to at least one third computer node among said interconnected computer nodes, said third computer node comprising a third cache resource, wherein said at least one third computer node is selected such that after receiving responsibility over said at least part of one or more cache object spaces, cache-related parameters corresponding to said third cache resource meet said at least one second SLS criterion.

15. The method of claim 13 further comprising:

in response to a failure to initiate said handoff, providing a user with a failure notification or with a recommendation indicative of a need in addition of at least one additional cache resource.

16. The method of claim 13, wherein said UDSP repository is comprised in at least one of the group consisted of said first computer node, a gateway resource operatively coupled to said first computer node and a client server operatively coupled to said first computer node.

17. The method of claim 13 further comprising:

sending said post-handoff cache mapping to one or more client servers associated with said at least part of one or more cache object spaces.

18. The method of claim 13 further comprising:

receiving, during said handoff, a request relating to a cache object space within said post-handoff cache mapping; and

relaying, said request to said second computer node when, according to said post-handoff cache mapping, said first computer node is not responsible for the respective cache object space.

19. A non-transitory computer readable storage medium having instructions that, when executed by a processing device, cause the processing device to perform operating a computer node having at least one first cache resource connected thereto and configured to be responsible for handling one or more cache object spaces related to one or more logical storage entities, the computer node configured to connected to an infrastructure layer including interconnected computer nodes and to a UDSP data repository comprising data on first cache-related parameters corresponding to said first cache resource and cache-related parameters corresponding to one or more cache resources of one or more other computer nodes among said interconnected computer nodes, the operating comprising:

monitoring the first cache-related parameters to determine whether the cache-related parameters meet at least one first Service Level Specification (SLS) criterion; and

when said at least one first SLS criterion is not met by the first cache-related parameters, initiating handoff of responsibility over at least part of said one or more cache object spaces to at least one second computer node of said interconnected computer nodes, said at least one second computer node comprising a second cache resource with second cache-related parameters corresponding thereto, wherein said at least one second computer node is selected such that after receiving responsibility over said at least part of said one or more cache object spaces, the second cache-related parameters meet at least one second SLS criterion, wherein the second cache-related parameters are comprised in said UDSP data repository;

creating a post-handoff cache mapping at least indicative, for said at least part of said one or more cache object spaces, of said at least one second computer node selected to be responsible thereof;

sending said post-handoff cache mapping to said at least one second computer node;

transferring responsibility over said at least part of said one or more cache object spaces to said at least one second computer node;

using data comprised in said UDSP repository to monitor the second cache-related parameters and to determine whether the second cache-related parameters meet at least one first SLS criterion; and

when said at least one first SLS criterion is not met, initiating a message to said selected at least one second computer node, said message informative of at least one of the cache-related parameters extracted from said UDSP data repository, corresponding to said second cache resource and not meeting said at least one first SLS criterion.