Methods and apparatus for enabling bus connectivity over a data network

• US 7,694,312 B2
• Filed: 09/10/2004
• Issued: 04/06/2010
• Est. Priority Date: 09/10/2004
• Status: Active Grant

• Current Assignee: Pleora Technologies, Inc.
• Original Assignee: Pleora Technologies, Inc.
• Inventors: Rivard, Alain, Dubois, Martin
• Primary Examiner(s): Zhen; Li B
• Application Number:
  US10/937,423
• Publication Number:
  US 20060059287A1
• Time in Patent Office: 2,034 Days
• Field of Search: 719/321, 719/328, 709/223, 709/225, 710/305, 710/306
• US Class Current: 719/321
• CPC Class Codes: G06F 9/4411 Configuring for operating with peripheral devices; Loading of device drivers
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes

Abstract

A method and system for interconnecting peripherals, processor nodes, and hardware devices to a data network to produce a network bus providing OS functionality for managing hardware devices connected to the network bus involves defining a network bus driver at each of the processor nodes that couples hardware device drivers to a network hardware abstraction layer of the processor node. The network bus can be constructed to account for the hot-swappable nature of the hardware devices using a device monitoring function, and plug and play functionality for adding and, removing device driver instances. The network bus can be used to provide a distributed processing system by defining a shared memory space at each processor node. Distributed memory pages are provided with bus-network-wide unique memory addresses, and a distributed memory manager is added to ensure consistency of the distributed memory pages, and to provide a library of functions for user mode applications.

Claims

1. A method for defining a network bus between a plurality of hardware devices connected to each other over a data network by network interface circuitry (NIC) without the need for network stack processing, at least one of said plurality of hardware devices being a processor node having processing capability, the method comprising:

providing, on each processor node, a network bus driver, a bus adapter driver and a NIC, said network bus driver configured to control exchange of network messages by accessing said NIC through said bus adapter driver, said bus adapter driver controlling said NIC;

registering on each processor node, an NIC driver for each of said plurality of hardware devices to permit said network bus driver on that processor node to effect communications with respective NICs for said hardware devices via said data network;

providing, on each processor node, a device driver for each of said plurality of hardware devices for controlling respective hardware devices through said network bus driver; and

creating on each processor node, a software abstraction layer of the network bus for communicating between said plurality of hardware devices by registering said bus adapter driver with said network bus driver instead of an operating system (OS) on said processor node as a network provider thus bypassing network stack processing, and having said bus adapter driver forward received messages to said network bus driver;

wherein said software abstraction layer permits said plurality of hardware devices to be seen as physical resources available to each processor node and configures said hardware devices to communicate as if over a bus to share resources at a network bus level removing need for said network stack processing.

2. The method as claimed in claim 1 wherein providing the network bus driver further comprises installing the NIC driver, which consists of kernel program instructions for controlling the NIC.

3. The method as claimed in claim 1 wherein providing the device drivers comprises:

instantiating drivers for base driver objects that may be added to or removed from the network while the respective processor node remains connected to the data network;

providing a device monitoring function that uses network messaging to detect the connection and disconnection of the hardware devices from the data network; and

providing a plug and play functionality for adding and deleting device driver instances in response to the connection and disconnection of the hardware devices.

4. The method as claimed in claim 3 wherein providing the device monitoring function comprises:

providing a device enumerator registered to receive all unaddressed network messages from the network bus driver, the device enumerator being adapted to broadcast a message requesting all hardware devices to reply with an auto-identification message, and declaring a new hardware device to the plug and play functionality upon receipt of an auto-identification message from a hardware device that is not already enumerated; and

providing a timeout mechanism for each of the enumerated hardware devices.

5. The method as claimed in claim 4 wherein providing a device enumerator further comprises providing the respective processor node with kernel mode device enumerator program instructions.

6. The method as claimed in claim 3 further comprising:

providing a respective processor node with a kernel mode device driver defining a device and service connector in communication with the network bus driver; and

publishing a user mode access interface for each device and service connector for permitting user mode software to access the network bus driver through the device and service connector, the user mode access interface defining a list of functions for an I/O manager of an operating system of the one of the respective processor nodes.

7. The method as claimed in claim 6 further comprising:

permitting a peripheral device of one processor node to be bus-connected to the networked devices; and

providing a user mode software device gateway that uses the published access interface to permit a driver of the peripheral device to communicate with the network bus driver.

8. The method as claimed in claim 6 further comprising:

permitting a special service of one processor node to be bus-connected to the networked devices; and

providing a user mode software service that uses the published user mode access interface to permit a driver of the peripheral device to access the network bus driver.

9. The method as claimed in claim 3 further comprising:

permitting a peripheral device of one processor node to be bus-connected to the networked devices; and

providing a kernel mode plug and play driver defining a software device gateway for interfacing a driver of the peripheral device with the network bus driver.

10. The method as claimed in claim 3 further comprising:

permitting memory sharing and distributed processing among at least two processor nodes connected to the network bus;

providing at each of the at least two processor nodes, a distributed memory manager that cooperates with the operating system to define and maintain a shared memory space of distributed memory pages that is consistent with those of the other of the at least two processor nodes to define a pool of shared objects;

providing an addressing scheme for uniquely identifying each of the distributed memory pages; and

providing a distributed memory driver of the distributed memory manager for effecting communications over the network bus to update pages of the shared memory space in order to maintain the consistency of the data when changes are made at one of the other of the at least two processor nodes.

11. The method as claimed in claim 10 wherein providing the addressing scheme further comprises:

providing program instructions for each processing unit running on the same operating system and connected to the same distributed process, to maintain a code space of addressable program instructions, the code space definable by a list of operations loaded and unloaded by the processing unit; and

providing program instructions for communicating any change in the code space to the corresponding code spaces of processing units of the same distributed process by adding code space change operations to the operation list.

12. The method as claimed in claim 10 wherein providing the distributed memory manager further comprises providing a notification system for permitting shared objects to exchange data.

13. The method as claimed in claim 12 further comprising:

providing a library of user mode program instructions that permits mutual exclusion of the pages by the at least two processor nodes updating a page to prevent two of the at least two processor nodes from concurrently updating the same page.

14. The method as claimed in claim 13 wherein providing the library to produce the distributed memory manager further comprises providing user mode program instructions for selecting an idle processing unit for a distributed process, the selection being made according to a property of at least one of the distributed process, and a processor node running the processing unit.

15. The method as claimed in claim 10 wherein the at least two processor nodes are adapted to share control of one or more of the hardware devices, and the method further comprises:

providing at each of the at least two processor nodes connected to the same network bus, a mirror image of a part of a kernel memory space of each of the other of the at least two processor nodes in respective parts of a shared kernel memory space, the part of the kernel space that is mirrored containing context information, and identifying user mode applications using respective device drivers;

ensuring that a network-wide unique addressing space is used by user mode applications of the at least two processor nodes so that each user mode application is identifiable;

providing the distributed memory driver with program instructions for exchanging local notifications, and for passing local notifications for requesting and accessing kernel mode memory pages; and

providing the distributed memory manager with program instructions for effecting a distributed memory service that controls access to the kernel mode memory space.

16. The method as claimed in claim 14 further comprising:

creating a standby processing unit for a running processing unit, the standby processing unit running on a different processor node from that of the running processing unit;

providing program instructions for enabling the standby processing unit to passively monitor all system-level object notification messages sent to the processor node with the running processing unit; and

communicating any change in the code space to the standby processing unit so that if the primary processing unit fails, the standby processing unit can start all of the threads that were running on the primary processing unit.

17. The method as claimed in claim 14 wherein providing a notification system further comprises permitting remote processor nodes to start, stop, suspend, resume, or kill a thread, by issuing a notification to the thread'"'"'s shared object.

18. The method as claimed in claim 3 further comprising:

providing one processor node with a plurality of NICs;

defining an aggregate network bus supported by the plurality of NICs; and

registering an aggregator to the network bus driver, and registering a NIC driver of each of the plurality of NICs to the aggregator so that the network communications of the aggregator'"'"'s network bus driver is effected through the aggregator, the aggregator being a plug and play driver having program instructions for transmitting all of the packets received from any of the NICs to the aggregate network bus, and to direct packets from the aggregate network bus toward respective ones of the network buses.

19. The method as claimed in claim 18 further comprising:

defining a respective network bus for each of the NICs by providing a corresponding network bus driver for each of the NIC drivers; and

defining an aggregation manager for each of the network buses by providing a plug and play driver for exchanging packets with the aggregator;

wherein registering the NIC driver to the aggregator comprises registering each of the NIC drivers and aggregation managers to the respective network bus driver, and registering the aggregation managers with the aggregator.

20. The method as claimed in claim 18 wherein registering the aggregator and defining the aggregation manager comprises providing program instructions for effecting an aggregator and aggregation manager instance in compliance with an Ethernet standard for load sharing, load balancing and/or higher reliability connections through redundancy.

21. The method as claimed in claim 1 wherein providing the network bus driver further comprises providing kernel mode program instructions for encrypting and decrypting information exchanged between the hardware devices.

22. The method as claimed in claim 1 further comprising:

providing a bridge between two or more network buses with a bus-to-bus bridge, the bus-to-bus bridge comprising a plug and play driver coupled to each of the buses;

communicatively coupling the plug and play drivers to each other; and

providing program instructions at each of the plug and play drivers for defining and applying permissions that control passage of messages between the network buses.

23. The method as claimed in claim 22 wherein communicatively coupling the plug and play drivers comprises establishing a communications link between the two plug and play drivers that are implemented on respective processor nodes.

24. The method as claimed in claim 1 wherein providing said device driver comprises:

instantiating a driver for a base OS network subsystem support allowing communications with standard data networking equipment and processes.

25. A system for defining a network bus between a plurality of hardware devices interconnected over a data network by network interface circuitry (NIC) without the need for network stack processing, at least one of said plurality of hardware devices being a processor node having processing capability, the system comprising said at least one processor node, each processor node comprising:

a network bus driver, a bus adapter driver and a NIC, said network bus driver configured to control exchange of network messages by accessing said NIC through said bus adapter driver, said bus adapter driver controlling said NIC;

an NIC driver for each of said plurality of hardware devices registered on said processor node to permit the network bus driver on that processor node to effect communications with respective NICs for said hardware devices via the data network;

a device driver for each of said plurality of hardware devices for controlling respective hardware devices through said network bus driver; and

program instructions for creating a software abstraction layer of said network bus for communicating between said plurality of hardware devices by registering said bus adapter driver with said network bus driver instead of an operating system (OS) on said processor node as a network provider thus bypassing network stack processing, and having said bus adapter driver forward received messages to said network bus driver;

wherein said software abstraction layer permits said plurality of hardware devices to be seen as physical resources available to each processor node and configures said hardware devices to communicate as if over a bus to share resources at a network bus level removing the need for said network stack processing.

26. The system as claimed in claim 25 wherein the NIC driver is a plug and play driver for controlling the NIC.

27. The system as claimed in claim 25 further comprising:

device drivers instantiated for the hardware devices that may be added to or removed from the data network while at least one processor node remains connected to the data network;

program instructions at the at least one processor node for defining a device monitoring function that uses network messaging to detect the connection and disconnection of the hardware devices from the data network; and

program instructions at the at least one processor node for defining a plug and play functionality for adding and deleting device driver instances in response to detected connection and disconnection of the hardware devices.

28. The system as claimed in claim 27 wherein the device monitoring function program instructions comprise:

a device enumerator registered to receive all network messages from the network bus driver that are not addressed to a plug and play device driver, the device enumerator being adapted to broadcast a message requesting all hardware devices reply with an auto-identification message, and to declare a new hardware device to the plug and play functionality upon receipt of an auto-identification message from a hardware device that is not enumerated;

a timeout mechanism for each of the enumerated hardware devices; and

a monitor for resetting a timer of a corresponding hardware device whenever a message is received from the corresponding hardware device;

issuing a request for a response from the corresponding hardware device whenever the hardware device'"'"'s timer expires; and

declaring the corresponding hardware device dropped to the plug and play functionality if no response is received within a predetermined time after a predetermined number of requests have been issued.

29. The system as claimed in claim 28 wherein the device enumerator comprises kernel mode program instructions.

30. The system as claimed in claim 28 wherein the at least one processor node further comprises:

a kernel mode plug and play device driver defining a device and service connector in communication with the network bus driver;

a user mode access interface for permitting device and service drivers to access the network bus driver through the device and service connector, the user mode access interface defining a list of functions for an I/O manager of an operating system of the at least one processor node; and

the device enumerator comprises user mode program instructions that use the user mode access interface to communicate with the network bus driver in order to exchange the broadcast request message and receive the auto-identification messages.

31. The system as claimed in claim 30 wherein the system further permits a peripheral device of one of the at least one processor node to be accessed by the network bus, and the system further comprises:

a user mode software device gateway for the peripheral that connects to a driver of the peripheral device, and uses the access interface to access the network bus driver, the software device gateway providing for the connection of the peripheral device driver to the network bus.

32. The system as claimed in claim 30 wherein the system further permits a special service of one of the each processor node to be accessed by the network bus, and the system further comprises user mode program instructions defining the special service that uses the access interface to access the network bus driver.

33. The system as claimed in claim 28 further comprising:

a peripheral device of one of the at least one processor node is accessible by the network bus; and

a kernel mode plug and play driver defining a software device gateway for interfacing a driver of the peripheral device with the network bus driver.

34. The system as claimed in claim 27 further comprising:

permitting memory sharing and distributed processing among at least two processor nodes connected to the network bus;

at each of the at least two processor node, a distributed memory manager that cooperates with the operating system to define and maintain a shared memory space of distributed memory pages that is consistent with those of the other of the at least two processor nodes for storing shared objects;

the distributed memory manager including a library of user mode program instructions for maintaining an addressing scheme that uniquely identifies each of the distributed memory pages; and

a plug and play distributed memory driver of the distributed memory manager for effecting communications over the network bus to update pages of the shared memory space, in order to maintain the consistency of the data when changes are made at the other of the at least two processor nodes.

35. The system as claimed in claim 34 further comprising:

program instructions that maintain a code space of addressable program instructions for each processing unit running on a same operating system and connected to the same distributed process, a content of the code space being definable by a list of operations loaded and unloaded by the processing unit; and

program instructions for communicating any change in the code space to corresponding code spaces of other processing units of the same distributed process by adding code space change operations to the operation list.

36. The system as claimed in claim 35 wherein the system permits each processing unit of a same distributed process to call methods to be effected at processing units of remote processor nodes.

37. The system as claimed in claim 35 wherein the distributed memory manager further comprises program instructions for effecting a notification system for permitting shared objects to exchange data.

38. The system as claimed in claim 35 wherein the distributed memory manager library further comprises program instructions that permit mutual exclusion of the data structures by the processing units to prevent two processor units from concurrently updating the same data structure.

39. The system as claimed in claim 38 wherein the distributed memory manager library further comprises user mode program instructions for selecting an idle processing unit for a distributed process, the selection being made according to a property of at least one of the distributed process, and a processor node running the processing unit.

40. The system as claimed in claim 35 wherein each of the at least two processor nodes is adapted to share control of one or more of the hardware devices with each other of the at least two processor nodes.

41. The system as claimed in claim 40 further comprising:

at each of the at least two processor nodes connected to one network bus, a mirror image of a part of a kernel memory space of the each of the other at least two processor nodes in respective parts of a shared kernel memory space, the part of the kernel memory space that is mirrored containing context information, and identifying user mode applications using respective device drivers;

a network-wide unique addressing space for user mode applications of the at least two processor nodes so that each user mode application is identifiable;

program instructions of the distributed memory driver for exchanging local notifications for requesting and accessing kernel mode memory pages; and

program instructions of the distributed memory manager library for effecting a distributed memory service that controls access to the kernel mode memory space.

42. The system as claimed in claim 37 further comprising:

a standby processing unit for at least one running processing unit,

each standby processing unit running on a different processor node from that of the running processing unit;

program instructions for enabling the standby processing unit to passively monitor all system-level object notifications sent to the processor node with the running processing unit; and

program instructions for communicating any change in the code space to the standby processing unit so that if the primary processing unit fails, the standby processing unit can start each thread that was running on the primary processing unit.

43. The system as claimed in claim 27 wherein one of the at least one processor node comprises a plurality of NICs, and the network bus is an aggregate network bus supported by the plurality of NICs.

44. The system as claimed in claim 43 further comprising a plug and play driver enabled to communicate with the network bus driver defining an aggregator, wherein a NIC driver for each of the plurality of NICs are enabled to communicate with the aggregator, so that the network communications of the network bus driver are effected through the aggregator, the aggregator being adapted to transmit all of the packets received from any of the NICs to the aggregate network bus, and to direct packets from the aggregate network bus toward respective ones of the network buses.

45. The system as claimed in claim 43 further comprising:

a network bus driver for each of the NIC drivers defining respective network buses for each of the NICs; and

an aggregation manager for each of the network buses, the aggregation manager comprising a plug and play driver for exchanging packets with the aggregator.

46. The system as claimed in claim 45 wherein the aggregator and the aggregation manager comprises program instructions for effecting an aggregator and aggregation manager instance in compliance with an Ethernet standard for load sharing, load balancing and/or higher reliability connections through redundancy.

47. The system as claimed in claim 25 wherein the network bus driver further comprises kernel mode program instructions for encrypting and decrypting information exchanged between the hardware devices.

48. The system as claimed in claim 25 further comprising:

a bus-to-bus bridge between the network bus and another similar network bus, the bus-to-bus bridge comprising a plug and play driver coupled to the network bus, and another plug and play driver coupled to the other network bus;

a communications link coupling the plug and play drivers to each other; and

program instructions at each of the plug and play drivers for defining and applying permissions that control a passage of messages between the network buses.

49. The system as claimed in claim 48 wherein the communications link coupling the plug and play drivers comprises a network communications link between the two plug and play drivers that are implemented on respective ones of the at least one processor node.

50. The system as claimed in claim 25 wherein the hardware device are drivers instantiated for an OS network subsystem support allowing communications using standard data network equipment and processes.