Apparatus and method for streamlining data transfer with existing interconnect bandwidth

• US 6,098,155 A
• Filed: 10/28/1996
• Issued: 08/01/2000
• Est. Priority Date: 10/28/1996
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Chong, Fay Jr.
• Primary Examiner(s): Yoo, Do Hyun
• Assistant Examiner(s): Nguyen, Hiep T.
• Application Number:
  US08/742,602
• Time in Patent Office: 1,373 Days
• Field of Search: 711/112, 711/113, 711/114, 711/138, 711/162, 714/6, 714/11, 707/204
• US Class Current: 711/138
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

An efficient memory retrieval/storage schemes for a fault tolerant memory control system for a computer system having a host processor, a mass storage memory and a system interconnect. The memory control system includes a primary memory controller and a backup memory controller with taps between the memory controllers to increase fault tolerance. Data is transferred between the host processor and the mass storage memory in the form of data packets. During a write operation, instead of receiving a data packet from the host processor, storing the data content, retrieving the data and then generating a new data packet for the mass storage memory, the memory controller redirects the data packets without the need to copy the data into the controller cache. Similarly, during a read operation, instead of receiving a data packet from the mass storage memory, storing the data content, retrieving the data and then generating a new data packet for the host processor, the memory controller redirects the data packets without the need to copy the data into the controller cache. Upon receipt of the data packets, verification packets are sent from the memories devices to the memory controller which then sends a confirmation packet to the host processor. Advantages of the present invention include reduced load on the system interconnect, reduced latency in backing up data, and reduced latency in recovering from partial system failures. In addition, the change in software and hardware needed to implementation such a solution is minimal since most of the backup functionality is provided by existing hardware.

Claims

1. In a computer system having a host processor, a memory controller, a memory and an interconnect, wherein the memory includes a plurality of memory devices, a method for retrieving data from the memory comprising the steps of:

issuing a controller read command from said host processor to said memory controller via said interconnect;

transforming said controller read command into a plurality of device read commands;

issuing said plurality of device read commands from said memory controller to said plurality of memory devices;

forming a plurality of device data packets destined for said controller in response to said plurality of device read commands; and

redirecting said plurality of device data packets from said controller to said host processor by transforming said plurality of device data packets into a corresponding plurality of controller device packets destined for said host processor without storing said plurality of device data packets at said controller;

wherein said transforming and said issuing said plurality of device read commands are performed without determining whether data included in said plurality of device data packets are in a controller cache.

2. The method for retrieving data from the memory as recited in claim 1 further comprising the steps of:

sending a verification packet to said memory controller from said memory;

checking said verification packet at said memory controller; and

sending a confirmation packet from said memory controller to said host processor.

3. In a computer system having a host processor, a memory controller, a memory and an interconnect, wherein the memory includes a plurality of memory devices, a method for retrieving data from the memory comprising the steps of:

issuing a controller read command from said host processor to said memory controller via said interconnect;

transforming said controller read command into a plurality of device read commands;

issuing said plurality of device read commands from said memory controller to said plurality of memory device; and

forming a plurality of device data packets destined for said host processor in response to said plurality of device read commands;

wherein said transforming and said issuing said plurality of device read commands are performed without determining whether data associated with said controller read command are in a controller cache.

4. The method for retrieving data from the memory as recited in claim 3 further comprising the steps of:

sending a verification packet to said memory controller from said memory;

checking said verification packet at said memory controller; and

sending a confirmation packet from said memory controller to said host processor.

8. The method for storing data to the memory as recited in claim 4 further comprising the steps of:

sending a plurality of verification packets destined for said controller in response to said combined device data packet;

checking said verification packets at said memory controller; and

sending a confirmation packet from said memory controller to said host processor.

5. In a computer system having a host processor, a memory controller, a memory and an interconnect, wherein the memory includes a plurality of memory devices, a method for storing data to the memory comprising the steps of:

issuing a controller write command from said host processor to said memory controller via said interconnect;

sending a plurality of controller data packets destined from said host processor to said controller;

redirecting said plurality of controller data packets from said controller to said plurality of memory devices by transforming said controller data packets into a corresponding plurality of device data packets without storing said plurality of controller data packets at said controller; and

receiving said plurality of device data packets at said memory devices;

wherein said redirecting is performed without determining whether data associated with said controller write command are in a controller cache.

6. The method for storing data to the memory as recited in claim 5 further comprising the steps of:

sending a plurality of verification packets destined for said controller in response to said device data packets;

checking said verification packets at said memory controller; and

sending a confirmation packet from said memory controller to said host processor.

7. In a computer system having a host processor, a memory controller, a memory and an interconnect, wherein the memory includes at least one intermediate memory device and a last memory device, a method for storing data to the memory comprising the steps of:

issuing a controller write command from said host processor to said memory controller via said interconnect;

sending a combined device data packet destined for said last memory device;

tapping said combined device data packet by said at least one intermediate memory device to extract at least one corresponding data portion from said combined device data packet; and

receiving said combined device data packet by said last memory device to extract corresponding data portion from said combined device data packet.

9. A method of transferring data in a fault-tolerant computer system from a storage device to a host processor, comprising:

receiving a data request at a storage device controller from a host processor coupled to said storage device controller, said storage device controller comprising a cache;

dividing said data request into a plurality of device requests;

transmitting said device requests to a set of storage devices;

receiving a first data packet at said storage device controller from a first storage device of said set of storage devices, wherein said first data packet comprises;

a destination field containing an address of said storage device controller; and

a source field containing an address of said first storage device; and

in said storage device controller, without storing said first data packet in said cache;

storing an address of said host processor in said destination field; and

storing an address of said storage device controller in said source field;

wherein said dividing and said transmitting are performed without determining if said requested data is in said cache.

10. The method of claim 9, in which said first data packet further comprises an offset field configured to store an address in a host memory, wherein a first offset retrieved from a first device request is stored in said offset field by said first storage device.

11. A method of receiving a set of data packets at a host processor from a plurality of data storage devices in a fault-tolerant computer, comprising:

receiving a read memory command from a first host processor at a first memory controller of a fault-tolerant computer over a first channel coupling said first host processor to said first memory controller and a plurality of data storage devices, wherein said first memory controller comprises a cache;

intercepting said read memory command at a second memory controller tapped into said first channel, wherein access to said plurality of data storage devices by said first host processor is conducted through said second memory controller if said first memory controller is inoperative;

generating a set of device read commands upon receipt of said read memory command;

issuing said set of device read commands to said plurality of data storage devices;

receiving a series of data packets from said plurality of data storage devices; and

directing said series of data packets to said first host processor without storing any of said data packets in said cache;

wherein said generating, issuing, receiving and directing are performed by said first memory controller unless said first memory controller is inoperative, in which case said generating, issuing, receiving and directing are performed by said second memory controller; and

wherein said generating and issuing are performed without determining if data requested in said read memory command is in said cache.

12. The method of claim 11, wherein said receiving a set of data packets comprises:

receiving at a first data storage device a first device read command;

generating at said first data storage device a first data packet comprising;

a first portion of data, stored on said first data storage device, responsive to said read memory command;

a destination field configured to store an address of a destination of said first data packet;

a source field configured to store a source of said first data packet; and

an offset field configured to store an offset into a host memory; and

transmitting said first data packet from said first data storage device.

13. The method of claim 12, wherein said first data storage device stores an address of said first host processor in said destination field prior to said transmitting.

14. The method of claim 12, wherein said first data storage device further comprises, prior to said transmitting:

storing an address of said first memory controller in said destination field; and

storing an address of said first data storage device in said source field.

15. The method of claim 14, wherein said first data storage device further comprises, prior to said transmitting;

retrieving an offset into a host memory from said first device read command; and

storing said retrieved offset in said offset field.

16. The method of claim 11, wherein said directing comprises:

inserting an address of said first host processor into a destination field of each data packet of said series of data packets; and

inserting an address of one of said first memory controller and said second memory controller into a source field of each said data packet.

17. The method of claim 16, wherein said directing further comprises inserting an offset in a first host memory into an offset field of each data packet of said series of data packets.

18. In a fault tolerant computer system comprising:

a first host processor;

a first interconnect coupled to said first host processor;

a first memory controller coupled to said first host processor via said first interconnect, the first memory controller configured to receive a first data packet from said first host processor via said first interconnect;

a second memory controller coupled to said first host processor via said first interconnect;

a first tap coupled between said first interconnect and said second memory controller, said first tap configured to obtain a backup copy of said first data packet for said second memory controller as said first data packet is being sent from said first host processor to said first memory controller via said first interconnect; and

a first memory;

a method for retrieving data from the first memory comprising the steps of;

issuing a controller read command from said first host processor to said first memory controller via said first interconnect;

transforming said controller read command into a plurality of device read commands;

issuing said plurality of device read commands from said first memory controller to a plurality of memory devices including said first memory;

forming a plurality of device data packets destined for said first memory controller in response to said plurality of device read commands; and

redirecting said plurality of device data packets from said first memory controller to said first host processor by transforming said plurality of device data packets into a corresponding plurality of controller device packets destined for said first host processor without storing said plurality of device data packets at said first memory controller;

wherein said transforming and said issuing are performed without determining if data in said device data packets is in a controller cache.