Balanced computer architecture

• US 20060288080A1
• Filed: 05/17/2006
• Published: 12/21/2006
• Est. Priority Date: 09/12/2000
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Company (HP Inc.)
• Original Assignee: IBRIX, Inc. (HP Inc.)
• Inventors: Srinivasan, Sudhir, Orszag, Steven A.
• Application Number:
  US11/434,928
• Publication Number:
  US 20060288080A1
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Methods and systems are described comprising a plurality of nodes each comprising at least one processor and at least one storage device providing storage for the system along with an interconnect configured to establish connections between pairs of nodes. The nodes may be configured (e.g. programmed) to determine from a file identifier that identifies a particular file that a node desires to access, which of the plurality of nodes stores the desired file. The interconnect may then establish a connection between the node and the node storing the file to permit the node desiring access to access the file (e.g., read or write the file). Further, the system comprising the plurality of nodes (e.g., a cluster computing architecture) may be balanced or nearly balanced.

Claims

1. A system comprising:

a plurality of nodes each comprising at least one processor and at least one storage device providing storage for the system; and

an interconnect configured to establish connections between at least a first node and a second node of the plurality of nodes;

wherein a processor of the first node of the plurality of nodes is configured to determine from a file identifier that identifies a particular file that a second node of the plurality of nodes stores the file in a storage device of the second node;

direct the interconnect to establish a connection between the first node and the second node;

forward a request to the second node indicating that the first node desires access to the file corresponding to the file identifier; and

access the file stored by the second node.

2. The system of claim 1, wherein the average processor time for the processor nodes to solve a problem is the order of the average time of transfers between pairs of computers in solving the problem.

3. The system of claim 2, wherein the plurality of nodes comprise M nodes and wherein the processors for each of the M nodes are configured to provide a processing speed, P, for the node;

and wherein the interconnect is configured to establish one or more point to point connections between one or more pairs of nodes at a cumulative data rate R; and

wherein the data rate, R, of the interconnect is the order of MP/125.

4. The system of claim 1, wherein the storage for the system comprises a plurality of segments each identified by a unique identifier and wherein each segment stores files identified by a range of file identifiers;

and wherein the processor of the first node in determining that the second node stores the file, is further configured to;

identify the segment storing the file using the file identifier for the file; and

identify the node storing the identified segment.

5. The system of claim 4, wherein the processor is further configured to identify the segment by determining the unique identifier for the segment by dividing the file identifier by a predetermined number, and identify the node storing the identified segment by looking up the determined unique identifier in a table.

6. The system of claim 4, wherein the file identifier comprises an Inode number and information regarding the unique identifier for the segment;

and

wherein the processor is further configured to obtain the unique identifier for the segment from the file from the file identifier.

7. The system of claim 4, wherein the file identifier comprises an Inode number and information identifying a computer node responsible for the file associated with the Inode number;

and

wherein the processor is further configured to identify the computer node responsible for the file.

8. The system of claim 1, wherein the interconnect comprises a non-blocking switch.

9. The system of claim 8, wherein the interconnect is selected from the set of an Infiniband interconnect, a Gigabit Ethernet interconnect, 10 Gigabit Ethernet interconnect, Myrinet interconnect, and a Quadrics interconnect.

10. The system of claim 1, wherein the system further comprises:

a storage system selected from the set of a network attached storage (NAS) and a storage area network (SAN); and

a second plurality of nodes each of which does not comprise a storage device providing storage for the system, and;

wherein each of the second plurality of nodes comprises a processor configured to access a file stored by the storage system and to access a file stored by the second node.

11. A method for use in a system comprising a plurality of nodes each comprising at least one processor and at least one storage device providing storage for the system and an interconnect configured to establish connections between at least a first node and a second node of the plurality of nodes, the method comprising:

determining from a file identifier that identifies a particular file that the second node of the plurality of nodes stores the file in a storage device of the second node;

directing the interconnect to establish a connection between the first node and the second node;

forwarding a request to the second node indicating that the first node desires access to the file corresponding to the file identifier; and

accessing the file stored by the second node.

12. The method of claim 11, wherein the average processor time for the processors of the nodes to solve a problem is the order of the average time of transfers between pairs of nodes in solving the problem.

13. The method of claim 12, wherein the plurality of nodes comprise M nodes and wherein the processors for each of the M nodes are configured to provide a processing speed, P, for the node;

and wherein the interconnect is configured to establish one or more point to point connections between one or more pairs of nodes at a cumulative data rate R; and

wherein the data rate, R, of the interconnect is the order of MP/125.

14. The method of claim 11, wherein the storage for the system comprises a plurality of segments each identified by a unique identifier and wherein each segment stores files identified by a range of file identifiers;

wherein in determining that the second node stores the file, the method further comprises;

identifying the segment storing the file using the file identifier for the file; and

identifying the node storing the identified segment.

15. The method of claim 14, wherein identifying the segment further comprises:

determining the unique identifier for the segment by dividing the file identifier by a predetermined number; and

wherein identifying the node storing the identified segment further comprises;

looking up the determined unique identifier in a table.

16. The method claim 14, wherein the file identifier comprises an Inode number and information regarding the unique identifier for the segment, the method further comprising:

obtaining the unique identifier for the segment for the file from the file identifier.

17. The method of claim 14, wherein the file identifier comprises an Inode number and information identifying a computer node responsible for the file associated with the Inode number, the method further comprising:

identifying the computer node responsible for the file.

18. The method of claim 11, wherein the interconnect comprises a non-blocking switch.

19. The method of claim 18, wherein the interconnect is selected from the set of an Infiniband interconnect a Gigabit Ethernet interconnect, 10 Gigabit Ethernet interconnect, Myrinet interconnect, and a Quadrics interconnect.

20. The method of claim 11, wherein the system further comprises a storage system selected from the set of a network attached storage (NAS) and a storage area network (SAN) and a second plurality of nodes each of which does not comprise a storage device providing storage for the system, the method further comprising:

at least one of the second plurality of nodes accessing a file stored by the storage system; and

at least one of the second plurality of nodes accessing a file stored by the second node.

21. An apparatus for use in a system comprising a plurality of nodes each comprising at least one storage device providing storage for the system and an interconnect configured to establish connections between at least a first node and a second node of the plurality of nodes, the apparatus comprising:

means for determining from a file identifier that identifies a particular file that the second node of the plurality of nodes stores the file in a storage device of the second node;

means for directing the interconnect to establish a connection between the first node and the second node;

means for forwarding a request to the second node indicating that the first node desires access to the file corresponding to the file identifier; and

means for accessing the file stored by the second node.

22. The apparatus of claim 21, wherein each node comprises means for solving a problem and wherein the average processor time for the processors of the nodes to solve a problem is the order of the average time of transfers between pairs of nodes in solving the problem.

23. The apparatus of claim 22, wherein the plurality of nodes comprise M nodes and wherein the processors for each of the M nodes are configured to provide a processing speed, P, for the node;

and wherein the interconnect is configured to establish one or more point to point connections between one or more pairs of nodes at a cumulative data rate R; and

wherein the data rate, R, of the interconnect is the order of MP/25.

24. The apparatus of claim 21, wherein the storage for the system comprises a plurality of segments each identified by a unique identifier and wherein each segment stores files identified by a range of file identifiers;

and wherein the means for determining that the second node stores the file comprises;

means for identifying the segment storing the file using the file identifier for the file; and

means for identifying the node storing the identified segment.

25. The apparatus of claim 24, wherein the means for identifying the segment further comprises:

means for determining the unique identifier for the segment by dividing the file identifier by a predetermined number; and

wherein the means for identifying the node storing the identified segment further comprises;

means for looking up the determined unique identifier in a table.

26. The apparatus of claim 24, wherein the file identifier comprises an Inode number and information regarding the unique identifier for the segment, the apparatus further comprising:

means for obtaining the unique identifier for the segment for the file from the file identifier.

27. The apparatus of claim 24, wherein the file identifier comprises an Inode number and information identifying a computer node responsible for the file associated with the Inode number, the method apparatus further comprising:

means for identifying the computer node responsible for the file.

28. The apparatus of claim 21, wherein the interconnect comprises a non-blocking switch.

29. The apparatus of claim 28, wherein the interconnect is selected from the set of an Infiniband interconnect a Gigabit Ethernet interconnect, 10 Gigabit Ethernet interconnect, Myrinet interconnect, and a Quadrics interconnect.

30. The apparatus of claim 21, wherein the system further comprises a storage system selected from the set of a network attached storage (NAS) and a storage area network (SAN) and a second plurality of nodes each of which does not comprise a storage device providing storage for the system, the apparatus further comprising:

means for accessing a file stored by the storage system.