Method and apparatus for scalable distributed storage

• US 20040139145A1
• Filed: 03/16/2004
• Published: 07/15/2004
• Est. Priority Date: 12/21/2000
• Status: Abandoned Application

• Current Assignee: Amnon Strasser, Gigy Bar-Or, Nir Peleg
• Original Assignee: Amnon Strasser, Gigy Bar-Or, Nir Peleg
• Inventors: Strasser, Amnon, Bar-or, Gigy, Peleg, Nir
• Application Number:
  US10/451,180
• Publication Number:
  US 20040139145A1
• Time in Patent Office: Days
• US Class Current: 709/200
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Independent nodes (66) providing storage services can be networked together, such that client devices (60, 61) can be attached to any independent node (66c, 66d), while independent nodes (66) identify themselves to client devices (60, 61) uniformly. Each independent node (66) would have the same name, address or other identification data with respect to each client device (60, 61). When data stored in a specific independent node (66) are accessed by a client device (60, 61) connected to a different independent node (66c, 66d), the request is forwarded to the independent node that where the requested data is stored. That independent node (66) can either respond to the client device (60, 61) directly or forward the response to another independent node (66) which can send the response back to the client device (60, 61).

Claims

1. A scalable distributed storage apparatus comprising a network, the apparatus further comprising:

a plurality of independent nodes connected to each other through the network, each independent node comprising at least one storage device;

wherein each independent node responds with the same identifier when a client device attaches to any one independent node from the plurality of independent nodes.

2. The scalable distributed storage apparatus as claimed in claim 1, wherein each independent node is a server.

3. The scalable distributed storage apparatus as claimed in claim 1, wherein the at least one storage device is a disk device.

4. The scalable distributed storage apparatus as claimed in claim 1, wherein the at least one storage device is a redundant array of independent disks device.

5. The scalable distributed storage apparatus as claimed in claim 1, wherein communications protocol between the attached-client device and the independent node to which the client device is attached is the InfiniBand protocol.

6. The scalable distributed storage apparatus as claimed in claim 1, further comprising at least one network storage device is connected to the network independent of the plurality of independent nodes.

7. The scalable distributed storage apparatus as claimed in claim 6, wherein the communications protocol between the plurality of independent nodes and the at least one network storage device is the InfiniBand protocol.

8. The scalable distributed storage apparatus as claimed in claim 1, wherein a data retrieval request from the attached client device is routed to the independent node storing the requested data.

9. The scalable distributed storage apparatus as claimed in claim 8, wherein the independent node caching the requested data broadcasts a data caching notification to a subset of independent nodes.

10. The scalable distributed storage apparatus as claimed in claim 8, wherein the subset of independent nodes is a random grouping of independent nodes.

11. The scalable distributed storage apparatus as claimed in claim 10, wherein the subset of independent nodes are those independent nodes directly connected to the independent node caching the requested data.

12. The scalable distributed storage apparatus as claimed in claim 1, wherein an independent node receiving new data input from the attached client device notifies the attached client device when the new data has been successfully stored.

13. The scalable distributed storage apparatus as claimed in claim 1, wherein an independent node receiving updated data input from the attached client device that affects previously stored data notifies the attached client device that the updated data has been successfully stored only after all cached copies of the previously stored data have been invalidated.

14. The scalable distributed storage apparatus as claimed in claim 1, wherein an independent node receiving updated data input from the attached client device that affects previously stored data notifies the attached client device that the updated data has been successfully stored only after all cached copies of the previously stored data have been updated.

15. A scalable distributed storage apparatus comprising a network, the apparatus further comprising:

a plurality of independent nodes connected to each other through the network;

a plurality of network storage devices connected to each other and the plurality of independent nodes through the network;

wherein each independent node responds with the same identifier when a client device attaches to any one independent node from the plurality of independent nodes.

16. The scalable distributed storage apparatus as claimed in claim 15, wherein each independent node is a server.

17. The scalable distributed storage apparatus as claimed in claim 15, wherein at least one independent node comprises a redundant array of independent disks device.

18. The scalable distributed storage apparatus as claimed in claim 15, wherein communications protocol between the attached client device and the independent node to which the client device is attached is the InfiniBand protocol.

19. The scalable distributed storage apparatus as claimed in claim 15, wherein the communications protocol between the plurality of independent nodes and the plurality of network storage devices is the InfiniBand protocol.

20. The scalable distributed storage apparatus as claimed in claim 15, wherein a data request from the attached client device is routed to the independent node storing the requested data.

21. The scalable distributed storage apparatus as claimed in claim 20, wherein the independent node caching the requested data broadcasts a data caching notification to a subset of independent nodes.

22. The scalable distributed storage apparatus as claimed in claim 21, wherein the subset of independent nodes is a random grouping of the plurality of independent nodes.

23. The scalable distributed storage apparatus as claimed in claim 21, wherein the subset of independent nodes are those independent nodes directly connected to the independent node caching the requested data.

24. The scalable distributed storage apparatus as claimed in claim 15, wherein an independent node receiving new data input from the attached client device notifies the attached client device when the new data input has been successfully stored.

25. The scalable distributed storage apparatus as claimed in claim 15, wherein an independent node, receiving updated data input from the attached client device that affects previously stored data, notifies the attached client device that the updated data input has been successfully stored only after all cached copies of the previously stored data have been invalidated.

26. The scalable distributed storage apparatus as claimed in claim 15, wherein an independent node receiving updated data input from the attached client device that affects previously stored data notifies the attached client device that the updated data has been successfully stored only after all cached copies of the previously stored data have been updated.

27. A scalable distributed storage apparatus comprising a network, the apparatus further comprising:

a plurality of independent computing means connected to each other through the network;

a plurality of network storage means connected to each other and the plurality of independent computing means through the network;

wherein each independent computing means responds with the same identifier when a client means attaches to any one independent computing means from the plurality of independent computing means.

28. A method of handling data on a scalable distributed storage apparatus comprising a plurality of independent nodes, wherein a plurality of client devices can attach to several of the independent nodes, the method comprising:

attaching a client device to an independent node;

transmitting a predetermined identifier to each of the client devices when the client device attaches to a selected one of the plurality of independent nodes;

requesting data from the scalable distributed storage apparatus through the independent node to which the client device is attached;

forwarding the data request to the plurality of independent nodes;

receiving the requested data from at least one of the plurality of independent nodes and caching the requested data at the independent node to which the requesting client device is attached; and

notifying at least one of the plurality of independent nodes of the location of the cached requested data.

29. The method as claimed in claim 28, wherein notifying other independent nodes further comprises notifying a subset of independent nodes of the location of the cached requested data.

30. The method as claimed in claim 29, wherein notifying a subset of independent nodes further comprises notifying a random grouping of the plurality of independent nodes.

31. The method as claimed in claim 29, wherein notifying a subset of independent nodes further comprises notifying those independent nodes directly connected to the independent node caching the requested data.

32. A computer program product for processing data requests on a scalable distributed storage apparatus comprising a plurality of independent nodes, wherein a plurality of client devices can attach to several of the independent nodes, the computer program product comprising:

software instructions for enabling an independent node to perform predetermined operations, and a computer readable medium bearing the software instructions;

the predetermined operations comprising;

processing an attachment request from a client device to the independent node;

transmitting a predetermined identifier to each of the client devices when the client device attaches to a selected one of the plurality of independent nodes;

processing a data request to the scalable distributed storage apparatus through the independent node to which the client device is attached;

forwarding the data request to the plurality of independent nodes;

receiving the requested data from at least one of the plurality of independent nodes and caching the requested data at the independent node to which the requesting client device is attached; and

notifying at least one of the plurality of independent nodes of the location of the cached requested data.

33. The computer program product as claimed in claim 32, wherein the predetermined operation of notifying other independent nodes further comprises notifying a subset of independent nodes of the location of the cached requested data.

34. The computer program product as claimed in claim 32, wherein notifying a subset of independent nodes further comprises notifying a random grouping of the plurality of independent nodes.

35. The computer program product as claimed in claim 32, wherein notifying a subset of independent nodes further comprises notifying the independent nodes directly connected to the independent node caching the requested data.

36. An executable program for an independent node in a scalable distributed storage apparatus, the executable program comprising:

a first executable code portion which, when executed on the independent node, processes an attachment request from a client device to the independent node;

a second executable code portion which, when executed on the independent node, transmits a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

a third executable code portion which, when executed on the independent node, processes a data request to the scalable distributed storage apparatus through the independent node to which the client device is attached;

a fourth executable code portion which, when executed on the independent node, forwards the data request to the plurality of independent nodes;

a fifth executable code portion which, when executed on the independent node, receives the requested data from at least one of the plurality of independent nodes and caches the requested data at the independent node to which the requesting client device is attached; and

a sixth executable code portion which, when executed on the independent node, notifies at least one of the plurality of independent nodes of the location of the cached requested data.

37. An executable program for an independent node in a scalable distributed storage apparatus, the executable program comprising:

software means for attaching a client device to the independent node;

software means for a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

software means for processing a data request to the scalable distributed storage apparatus through the independent node to which the client device is attached;

software means for forwarding the data request to the plurality of independent nodes;

software means for receiving the requested data from at least one of the plurality of independent nodes and caches the requested data at the independent node to which the requesting client device is attached; and

software means for notifying at least one of the plurality of independent nodes of the location of the cached requested data.

38. A computer system adapted to storing data from a plurality of storage systems on a storage medium, the computer system comprising:

a processor;

a memory comprising software instructions adapted to enable the computer system to perform the steps of;

processing an attachment request from a;

client device to the computer system;

transmitting a predetermined identifier to the client device from the computer system, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

processing a data request to a scalable distributed storage apparatus through the computer system to which the client device is attached;

forwarding the data request to the scalable distributed storage apparatus;

receiving the requested data from the scalable distributed storage apparatus and caching the requested data at the computer system to which the requesting client device is attached; and

notifying the scalable distributed storage apparatus of the location of the cached requested data.

39. A method of handling data on a scalable distributed storage apparatus comprising a plurality of independent nodes, wherein a plurality of client devices can attach to several of the independent nodes, the method comprising:

attaching a client device to an independent node;

transmitting a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

receiving a new data input from the client device attached to the scalable distributed storage apparatus through the independent node;

determining whether the new data input is new data to be stored or an update to previously stored data, and based on that determination, storing the new data input on the scalable distributed storage apparatus, if it is new data to be stored, or updating the previously stored data on the scalable distributed is storage apparatus, if it is an update to previously stored data; and

transmitting a notification to the attached client device if the storing of the new data input was successful.

40. The method as claimed in claim 39, wherein the updating the previously stored data on the scalable distributed storage apparatus further comprises invalidating all cached copies of the previously stored data, and storing the new data input on the scalable distributed storage apparatus.

41. The method as claimed in claim 40, wherein the transmitting a notification to the attached client device occurs after all cached copies of the previously stored data is invalidated.

42. The method as claimed in claim 39, wherein the updating the previously stored data on the scalable distributed storage apparatus further comprises replacing all cached copies of the previously stored data with the new data input.

43. The method as claimed in claim 42, wherein the transmitting a notification to the attached client device occurs after all cached copies of the previously stored data have been replaced.

44. A computer program product for processing data requests on a scalable distributed storage apparatus comprising a plurality of independent nodes, wherein a plurality of client devices can attach to several of the independent nodes, the computer program product comprising:

software instructions for enabling an independent node to perform predetermined operations, and a computer readable medium bearing the software instructions;

the predetermined operations comprising;

attaching a client device to an independent node;

transmitting a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

receiving a new data input from the client device attached to the scalable distributed storage apparatus through the independent node;

determining whether the new data input is new data to be stored or an update to previously stored data, and based on that determination, storing the new data input on the scalable distributed storage apparatus, if it is new data to be stored, or updating the previously stored data on the scalable distributed storage apparatus, if it is an update to previously stored data; and

transmitting a notification to the attached client device if the storing of the new data input was successful.

45. The computer program product as claimed in claim 44, wherein the updating the previously stored data on the scalable distributed storage apparatus further comprises invalidating all cached copies of the previously stored data, and storing the new data input-on the scalable distributed storage apparatus.

46. The computer program product as claimed in claim 45, wherein the transmitting a notification to the attached client device occurs after all cached copies of the previously stored data is invalidated.

47. The computer program product as claimed in claim 44, wherein the updating the previously stored data on the scalable distributed storage apparatus further comprises replacing all cached copies of the previously stored data with the new data input.

48. The computer program product as claimed in claim 47, wherein the transmitting a notification to the attached client device occurs after all cached copies of the previously stored data have been replaced.

49. An executable program for an independent node in a scalable distributed storage apparatus, the executable program comprising:

a first executable code portion which, when executed on the independent node, processes an attachment request from client device to the independent node;

a second executable code portion which, when executed on the independent node, transmits a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

a third executable code portion which, when executed on the independent node, receives a new data input from the client device attached to the scalable distributed storage apparatus through the independent node;

a fourth executable code portion which, when executed on the independent node, determines whether the new data input is new data to be stored or an update to previously stored data, and based on that determination, storing the new data input on the scalable distributed storage apparatus, if it is new data to be stored, or updating the previously stored data on the scalable distributed storage apparatus, if it is an update to previously stored data; and

a fifth executable code portion which, when executed on the independent node, transmits a notification to the attached client device if the storing of the new data input was successful.

50. An executable program for an independent node in a scalable distributed storage apparatus, the executable program comprising:

software means for attaching a client device to the independent node;

software means for transmitting a predetermined identifier to the client device from the independent node, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

software means for receiving a new data input from the client device attached to the scalable distributed storage apparatus through the independent node;

software means for determining whether the new data input is new data to be stored or an update to previously stored data, and based on that determination, storing the new data input on the scalable distributed storage apparatus, if it is new data to be stored, or updating the previously stored data on the scalable distributed storage apparatus, if it is an update to previously stored data; and

software means for transmitting a notification to the attached client device if the storing of the new data input was successful.

51. A computer system adapted to storing data from a plurality of storage systems on a storage medium, the computer system comprising:

a processor;

a memory comprising software instructions adapted to enable the computer system to perform the steps of;

attaching a client device to the computer system;

transmitting a predetermined identifier to the client device from the computer system, wherein the predetermined identifier is identical to the identifier transmitted to other attached client devices;

receiving a new data input from the client device attached to the computer system;

determining whether the new data input is new data to be stored or an update to previously stored data, and based on that determination, storing