Systems and methods for implementing content sensitive routing over a wide area network (WAN)

• US 8,255,477 B2
• Filed: 08/17/2010
• Issued: 08/28/2012
• Est. Priority Date: 11/28/2001
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Hayward, Geoff
• Primary Examiner(s): Moore, Ian N
• Assistant Examiner(s): Towfighi, Afshawn
• Application Number:
  US12/857,703
• Publication Number:
  US 20110035430A1
• Time in Patent Office: 742 Days
• Field of Search: 709213-219, 709/248, 711141-146
• US Class Current: 709/213
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/2885 Hierarchically arranged intermediate devices, e.g. for hierarchical caching
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Systems and methods for optimizing storage network functionality. The methods and systems of the present invention are particularly useful for optimizing storage network performance for cases in which some components of the network may be separated by significant distances and/or which include communication links with relatively limited bandwidth. In certain aspects, the present invention provides methods and systems for implementing access to and management of geographically distributed storage resources through multiple peer-to-peer storage network array management functions (AMFs) that may also be geographically distributed. The methods and systems of the present invention, in certain aspects, provide geographically aware cache sharing, cache replication, cache coherence, traffic routing, redundancy group structure, source and destination selection, pre-fetching of data, message gathering and other useful features.

Claims

1. An array management function (AMF) apparatus involved in implementing hierarchical distributed cache coherence in a storage network having a first local AMF access group (LAAG) and a second LAAG, the first LAAG including first AMF devices and a first proxy agent, and the second LAAG including second AMF devices and a second proxy agent, the AMF apparatus comprising:

instruction execution circuitry; and

instruction storage circuitry operative to store computer instructions and provide them to the instruction execution circuitry for execution to realize functional circuitry including;

write update receiving circuitry for receiving a write update from a host server, the write update identifying a set of storage blocks;

first sending circuitry for sending, in a first send operation, a first write invalidate command to all of the first AMF devices of the first LAAG, each first write invalidate command directing a respective first AMF device of the first LAAG to locally invalidate the set of storage blocks identified by the write update;

second sending circuitry for sending, in a second send operation, a proxy agent write invalidate command to the first proxy agent of the first LAAG, the proxy agent write invalidate command directing the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG for distribution of a second write invalidate command to all of the second AMF devices of the second LAAG, each second write invalidate command directing a respective second AMF device of the second LAAG to locally invalidate the set of storage blocks identified by the write update;

first receiving circuitry for receiving, in a first receive operation, a first write invalidate acknowledgement from all of the first AMF devices of the first LAAG, each first write invalidate acknowledgement indicating that a respective first AMF device of the first LAAG has successfully locally invalidated the set of storage blocks identified by the write update;

second receiving circuitry for receiving, in a second receive operation, a proxy agent write invalidate acknowledgement from the first proxy agent of the first LAAG, the proxy agent write invalidate acknowledgement indicating that all of the second AMF devices of the second LAAG have acknowledged successful local invalidation of the set of storage blocks identified by the write update; and

write update sending circuitry for sending, upon completion of the first and second receive operations, a write update acknowledgement to the host server.

2. An AMF apparatus as in claim 1 wherein the first AMF devices and the first proxy agent are constructed and arranged to communicate with each other through a high bandwidth communications medium;

wherein the first and second proxy agents are constructed and arranged to communicate with each other through a low bandwidth communications medium, the high bandwidth communications medium having a bandwidth which is higher than that of the low bandwidth communications medium; and

wherein the first sending circuitry, when sending the first write invalidate command to all of the first AMF devices of the first LAAG, is constructed and arranged to transmit the first write invalidate command to all of the first AMF devices of the first LAAG at the bandwidth which is higher than that of the low bandwidth communications medium.

3. An AMF apparatus as in claim 2 wherein the first sending circuitry, when transmitting the first write invalidate command to all of the first AMF devices of the first LAAG at the bandwidth which is higher than that of the low bandwidth communications medium, is constructed and arranged to broadcast the first write invalidate command to all of the first AMF devices of the first LAAG.

4. An AMF apparatus as in claim 2 wherein the first LAAG resides at a first location;

wherein the second LAAG resides at a second location;

wherein the first and second locations are different and remotely separated from each other; and

wherein the second sending circuitry, when sending the proxy agent write invalidate command to the first proxy agent of the first LAAG, is constructed and arranged to direct the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG via a point-to-point communications protocol.

5. An AMF apparatus as in claim 4 wherein the second AMF devices and the second proxy agent are constructed and arranged to communicate with each other through another high bandwidth communications medium having a bandwidth which is higher than that of the low bandwidth communications medium;

and

wherein the second sending circuitry, when directing the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG, is constructed and arranged to direct the second proxy agent of the second LAAG to broadcast the second write invalidate command to all of the second AMF devices of the second LAAG.

6. An AMF apparatus as in claim 2 wherein the proxy agent write invalidate command further directs the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to a third proxy agent of a third LAAG for distribution of a third write invalidate command to all of the third AMF devices of the third LAAG, each third write invalidate command directing a respective third AMF device of the third LAAG to locally invalidate the set of storage blocks identified by the write update.

7. An AMF apparatus as in claim 2 wherein the write update identifies, as the set of storage blocks, a block range of consecutive storage blocks;

and

wherein local invalidation of the set of storage blocks identified by the write update includes invalidating a local cache for the block range of the consecutive storage blocks.

8. An AMF apparatus as in claim 2 wherein the functional circuitry further includes AMF management circuitry constructed and arranged to perform array management functions as part of the first LAAG.

9. An AMF apparatus as in claim 8 wherein each AMF device couples to a set of storage devices to form a redundancy group;

and

wherein each AMF device, including the AMF apparatus, operates as a front-end interface between a set of host servers and the redundancy group.

10. An AMF apparatus as in claim 8 wherein the AMF management circuitry and the first proxy agent are disposed within an AMF device of the first LAAG.

11. A method of operating an array management function (AMF) apparatus to facilitate hierarchical distributed cache coherence in a storage network having a first local AMF access group (LAAG) and a second LAAG, the first LAAG including first AMF devices and a first proxy agent, and the second LAAG including second AMF devices and a second proxy agent, the method comprising:

receiving a write update from a host server, the write update identifying a set of storage blocks;

sending, in a first send operation, a first write invalidate command to all of the first AMF devices of the first LAAG, each first write invalidate command directing a respective first AMF device of the first LAAG to locally invalidate the set of storage blocks identified by the write update;

sending, in a second send operation, a proxy agent write invalidate command to the first proxy agent of the first LAAG, the proxy agent write invalidate command directing the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG for distribution of a second write invalidate command to all of the second AMF devices of the second LAAG, each second write invalidate command directing a respective second AMF device of the second LAAG to locally invalidate the set of storage blocks identified by the write update;

receiving, in a first receive operation, a first write invalidate acknowledgement from all of the first AMF devices of the first LAAG, each first write invalidate acknowledgement indicating that a respective first AMF device of the first LAAG has successfully locally invalidated the set of storage blocks identified by the write update;

receiving, in a second receive operation, a proxy agent write invalidate acknowledgement from the first proxy agent of the first LAAG, the proxy agent write invalidate acknowledgement indicating that all of the second AMF devices of the second LAAG have acknowledged successful local invalidation of the set of storage blocks identified by the write update; and

sending, upon completion of the first and second receive operations, a write update acknowledgement to the host server.

12. A method as in claim 11 wherein the first AMF devices and the first proxy agent are constructed and arranged to communicate with each other through a high bandwidth communications medium;

wherein the first and second proxy agents are constructed and arranged to communicate with each other through a low bandwidth communications medium, the high bandwidth communications medium having a bandwidth which is higher than that of the low bandwidth communications medium; and

wherein sending the first write invalidate command to all of the first AMF devices of the first LAAG includes transmitting the first write invalidate command to all of the first AMF devices of the first LAAG at the bandwidth which is higher than that of the low bandwidth communications medium.

13. A method as in claim 12 wherein transmitting the first write invalidate command to all of the first AMF devices of the first LAAG at the bandwidth which is higher than that of the low bandwidth communications medium includes broadcasting the first write invalidate command to all of the first AMF devices of the first LAAG.

14. A method as in claim 12 wherein the first LAAG resides at a first location;

wherein the second LAAG resides at a second location;

wherein the first and second locations are different and remotely separated from each other; and

wherein sending the proxy agent write invalidate command to the first proxy agent of the first LAAG includes directing the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG via a point-to-point communications protocol.

15. A method as in claim 14 wherein the second AMF devices and the second proxy agent are constructed and arranged to communicate with each other through another high bandwidth communications medium having a bandwidth which is higher than that of the low bandwidth communications medium;

and

wherein directing the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to the second proxy agent of the second LAAG includes directing the second proxy agent of the second LAAG to broadcast the second write invalidate command to all of the second AMF devices of the second LAAG.

16. A method as in claim 12 wherein the proxy agent write invalidate command further directs the first proxy agent of the first LAAG to forward the proxy agent write invalidate command to a third proxy agent of a third LAAG for distribution of a third write invalidate command to all of the third AMF devices of the third LAAG, each third write invalidate command directing a respective third AMF device of the third LAAG to locally invalidate the set of storage blocks identified by the write update.

17. A method as in claim 12 wherein the write update identifies, as the set of storage blocks, a block range of consecutive storage blocks;

and

wherein local invalidation of the set of storage blocks identified by the write update includes invalidating a local cache for the block range of the consecutive storage blocks.

18. A method as in claim 12, further comprising:

performing a set of array management functions as part of the first LAAG.

19. A method as in claim 18 wherein each AMF device couples to a set of storage devices to form a redundancy group;

and

wherein performing a set of array management functions as part of the first LAAG includes operating as a front-end interface between a set of host servers and the redundancy group.

20. A method as in claim 18, wherein the AMF apparatus and the first proxy agent are co-located within an AMF device of the first LAAG.