Methods and apparatus for transferring a data element within a data storage system

• US 6,948,031 B2
• Filed: 12/19/2000
• Issued: 09/20/2005
• Est. Priority Date: 12/19/2000
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Chilton, Kendell A.
• Primary Examiner(s): BRAGDON, REGINALD GLENWOOD
• Application Number:
  US09/741,494
• Publication Number:
  US 20020078292A1
• Time in Patent Office: 1,736 Days
• Field of Search: 711/113, 711/118, 710/30, 710/74
• US Class Current: 711/113
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/312 In storage controller

Abstract

The invention is directed to techniques for transferring data within a data storage system that includes, among other things, an interconnection mechanism having a point-to-point channel between an interface circuit (e.g., a director) and a volatile memory cache circuit. The point-to-point channel allows the interface circuit and the volatile memory cache circuit to have contention free access to each other. One arrangement of the invention is directed to a data storage system having a volatile memory cache circuit that buffers data elements exchanged between a storage device and a host, an interface circuit that operates as an interface between the volatile memory cache circuit and at least one of a storage device and a host, and a point-to-point channel interconnected between the volatile memory cache circuit to the interface circuit.

Claims

1. In an interface circuit of a data storage system, a point-to-point channel directly connecting to both the interface circuit and a volatile memory cache circuit, a method for exchanging data with the volatile memory cache circuit, the method comprising the steps of:

providing a command to the volatile memory cache circuit through the point-to-point channel which directly connects to both the interface circuit and the volatile memory cache circuit, the point-to-point channel including a first link and a second link;

dividing a data element into a first half of the data element and a second half of the data element;

moving the data element through the point-to-point channel in accordance with the command, the step of moving including the step of conveying the first half of the data element through the first link and the second half of the data element through the second link, the step of conveying including the step of transmitting the first half of the data element from a first transmitter and concurrently transmitting the second half of the data element from a second transmitter; and

receiving status from the volatile memory cache circuit through the point-to-point channel in accordance with the data element.

2. The method of claim 1 wherein the first and second links form a set of unidirectional links that carries signals from the interface circuit to the volatile memory cache circuit, and wherein the step of providing the command to the volatile memory cache circuit includes the step of:

sending the command to the volatile memory cache circuit through the set of unidirectional links of the point-to-point channel.

3. The method of claim 2 wherein the point-to-point channel further includes another set of unidirectional links that carries signals from the volatile memory cache circuit to the interface circuit, and wherein the step of receiving status from the volatile memory cache circuit includes the step of:

obtaining the status from the volatile memory cache circuit through the other set of unidirectional links of the point-to-point channel.

4. The method of claim 1 wherein the first link and the second link form a first set of unidirectional serial links that carries signals from the interface circuit to the volatile memory cache circuit, and a second set of unidirectional serial links that carries signals from the volatile memory cache circuit to the interface circuit;

and wherein the step of moving the data element includes the steps of;

sending the data element to the volatile memory cache circuit through the first set of unidirectional serial links when the command indicates a write transaction; and

obtaining the data element from the volatile memory cache circuit through the second set of unidirectional serial links when the command indicates a read transaction.

5. The method of claim 4 wherein each of the first set of unidirectional serial links and each of the second set of unidirectional serial links is an asynchronous link;

wherein the step of sending includes the step of;

outputting a respective portion of the data element framed with synchronization delimiters to the volatile memory cache circuit through each of the first set of unidirectional serial links when the command indicates a write transaction; and

wherein the step of obtaining includes the step of;

inputting a respective portion of the data element framed with synchronization delimiters from the volatile memory cache circuit through each of the second set of unidirectional serial links when the command indicates a read transaction.

6. The method of claim 4 wherein the step of sending includes the step of:

outputting a respective portion of the data element and a corresponding multiple bit error detection code to the volatile memory cache circuit through each of the first set of unidirectional serial links when the command indicates a write transaction; and

wherein the step of obtaining includes the step of;

inputting a respective portion of the data element and a corresponding multiple bit error detection code from the volatile memory cache circuit through each of the second set of unidirectional serial links when the command indicates a read transaction.

7. The method of claim 4 wherein the step of sending includes the step of:

outputting portions of the data element as data codes through the first set of unidirectional serial links when the command indicates a write transaction;

wherein the step of obtaining includes the step of;

inputting portions of the data element as data codes through the second set of unidirectional serial links when the command indicates a read transaction; and

wherein the data codes belong to an 8B/10B encoding/decoding data space.

8. The method of claim 4, further comprising the step of:

receiving a busy signal from the volatile memory cache circuit through each of the second set of unidirectional serial links after the step of providing the command and before the step of moving the data element.

9. The method of claim 4 wherein the step of providing the command includes the step of:

sending a tag indicator to the volatile memory cache circuit through the first set of unidirectional serial links; and

wherein the step of receiving the status includes the step of;

obtaining a copy of the tag indicator from the volatile memory cache circuit through the second set of unidirectional serial links.

10. The method of claim 1 wherein the step of moving the data element through the point-to-point channel includes the step of:

reading the data element from the volatile memory cache circuit; and

wherein the method further comprises the step of;

processing the read data element within the interface circuit during the step of receiving status from the volatile memory cache circuit.

11. The method of claim 1, further comprising the step of:

prior to transmitting the first half of the data element and the second half of the data element, associating a tag within each of the first half of the data element and the second half of the data element so that the tag passes through each of the first link and the second link of the point-to-point channel, the tap being adapted to persist within a subsequent status message to indicate whether communications occurred between a correct interface and volatile memory cache circuits.

12. The method of claim 11 wherein the point-to-point channel further includes a third link and a fourth link, and wherein the step of receiving the status through the point-to-point channel includes the step of:

obtaining (i) a first half of a status message through the third link, and (ii) a second half of the status message through the fourth link, the first half of the status message having a copy of the tag associated with the first half of the data element and the second half of the status message having a copy of the tag associated with the second half of the data element to enable the interface circuit to verify transmission of the status message to a correct interface circuit among multiple interface circuits.

13. The method of claim 1 wherein the step of providing a command to the volatile memory cache circuit includes the step of:

sending the command through the point-to-point channel for immediate processing by the volatile memory cache circuit at an end of the point-to-point channel.

14. The method of claim 13 wherein the volatile memory cache circuit combines with other volatile memory cache circuits to form a cache which is disposed between other interface circuits and non-volatile disk storage;

and wherein the steps of providing, moving and receiving occur concurrently with exchanging of data through other point-to-point channels which directly connect the other interface circuits and the cache.

15. The method of claim 1 wherein the data element includes a series of even and odd numbered bytes;

and wherein the step of dividing the data element into the first half and the second half includes the step of;

putting all of the even numbered bytes of the series into the first half to enable the first transmitter to exclusively transmit only even numbered bytes through the first link of the point-to-point channel; and

putting all of the odd numbered bytes of the series into the second half to enable the second transmitter to exclusively transmit only odd numbered bytes through the second link of the point-to-point channel.

16. The method of claim 15 wherein transmission of the first half of the data element and transmission of the second half of the data element occur at the same time and independently of each other from a clock synchronization standpoint.

17. The method of claim 16 wherein the command is a read command;

wherein the data element includes a data field containing cached host data and a status field containing status indicating whether an earlier-performed read operation completed correctly; and

wherein moving the data element includes the step of;

transferring the data field portion containing the cached host data in front of the status field portion containing the status indicating whether the earlier-performed read operation completed correctly to enable error detection to begin upon receipt of the data field portion while the status field portion is still being transferred.

18. In a volatile memory cache circuit of a data storage system, a point-to-point channel directly connecting to both an interface circuit and the volatile memory cache circuit, a method for exchanging data with the interface circuit, the method comprising the steps of:

receiving a command from the interface circuit through the point-to-point channel which directly connects to both the interface circuit and the volatile memory cache circuit, the point-to-point channel including a first link and a second link;

moving a data element through the point-to-point channel in accordance with the command, the step of moving including the step of conveying a first half of the data element through the first link and a second half of the data element through the second link, the step of conveying including the step of receiving the first half of the data element from a first transmitter of the interface circuit and concurrently receiving the second half of the data element from a second transmitter of the interface circuit;

reconstructing the data element from the first half of the data element and the second half of the data element; and

providing status to the interface circuit through the point-to-point channel in accordance with the data element.

19. The method of claim 18 wherein the first and second links form a set of unidirectional links that carries signals from the interface circuit to the volatile memory cache circuit, and wherein the step of receiving the command from the interface circuit includes the step of:

obtaining the command from the interface circuit through the set of unidirectional links of the point-to-point channel.

20. The method of claim 19 wherein the point-to-point channel further includes another set of unidirectional links that carries signals from the volatile memory cache circuit to the interface circuit, and wherein the step of providing status to the interface circuit includes the step of:

sending the status to the interface circuit through the other set of unidirectional links of the point-to-point channel.

21. The method of claim 18 wherein the first link and the second link form a first set of unidirectional serial links that carries signals from the interface circuit to the volatile memory cache circuit, and a second set of unidirectional serial links that carries signals from the volatile memory cache circuit to the interface circuit;

and wherein the step of moving the data element includes the steps of;

obtaining the data element from the interface circuit through the first set of unidirectional serial links when the command indicates a write transaction; and

sending the data element to the interface circuit through the second set of unidirectional serial links when the command indicates a read transaction.

22. The method of claim 21 wherein each of the first set of unidirectional serial links and each of the second set of unidirectional serial links is an asynchronous link;

wherein the step of obtaining includes the step of;

inputting a respective portion of the data element framed with synchronization delimiters from the interface circuit through each of the first set of unidirectional serial links when the command indicates a write transaction; and

wherein the step of sending includes the step of;

outputting a respective portion of the data element framed with synchronization delimiters to the interface circuit through each of the second set of unidirectional serial links when the command indicates a read transaction.

23. The method of claim 21 wherein the step of obtaining includes the step of:

inputting a respective portion of the data element and a corresponding multiple bit error detection code from the interface circuit through each of the first set of unidirectional serial links when the command indicates a write transaction; and

wherein the step of sending includes the step of;

outputting a respective portion of the data element and a corresponding multiple bit error detection code to the interface circuit through each of the second set of unidirectional serial links when the command indicates a read transaction.

24. The method of claim 21 wherein the step of obtaining includes the step of:

inputting portions of the data element as data codes through the first set of unidirectional serial links when the command indicates a write transaction;

wherein the step of sending includes the step of;

outputting portions of the data element as data codes through the second set of unidirectional serial links when the command indicates a read transaction; and

wherein the data codes belong to an 8B/10B encoding/decoding data space.

25. The method of claim 21, further comprising the step of:

sending a busy signal to the interface circuit through each of the second set of unidirectional serial links after the step of receiving the command and before the step of moving the data element.

26. The method of claim 21 wherein the step of receiving the command includes the step of:

obtaining a tag indicator from the interface circuit through the first set of unidirectional serial links; and

wherein the step of providing status includes the step of;

sending a copy of the tag indicator to the interface circuit through the second set of unidirectional serial links.

27. The method of claim 18 wherein the step of moving the data element through the point-to-point channel includes the step of:

providing the data element to the interface circuit; and

wherein the step of providing the status to the interface circuit includes the step of;

sending a status message to the interface circuit, the status message including a tag originally obtained from the interface circuit during the step of receiving the command to enable the interface circuit to verify that the interface circuit is a proper recipient of the data element.

28. The method of claim 18 wherein the point-to-point channel further includes a third link and a fourth link, and wherein the step of providing the status to the interface circuit includes the steps of:

extracting a tag from the first half of the data element and sending a first half of a status message to the interface circuit through the third link the first half of the status message having a copy of the tag from the first half of the data element; and

extracting a tag from the second half of the data element and sending a second half of the status message to the interface circuit through the fourth link, the second half of the status message having a copy of the tag from the second half of the data element, the tag being copied from the data element to the status message to indicate whether communications occurred between a correct interface and volatile memory cache circuits.

29. The method of claim 18 wherein the step of receiving the command from the interface circuit includes the step of:

obtaining the command at an end of the point-to-point channel for immediate processing by the volatile memory cache circuit.

30. The method of claim 29 wherein the volatile memory cache circuit combines with other volatile memory cache circuits to form a cache which is disposed between other interface circuits and non-volatile disk storage;

and wherein the steps of receiving, moving and providing occur concurrently with exchanging of data through other point-to-point channels which directly connect the other interface circuits and the cache.

31. The method of claim 18 wherein the data element includes a series of even and odd numbered bytes;

and wherein the step of receiving the first half of the data element and the second half of the data element includes the step of;

obtaining all of the even numbered bytes of the series within the first half of the data element exclusively through the first link of the point-to-point channel; and

obtaining all of the odd numbered bytes of the series within the second half of the data element exclusively through the second link of the point-to-point channel.

32. The method of claim 31 wherein acquisition of the first half of the data element and acquisition of the second half of the data element occur at the same time and independently of each other from a clock synchronization standpoint.

33. The method of claim 32 wherein the command is a read command;

wherein the data element includes a data field containing cached host data and a status field containing status indicating whether an earlier-performed read operation completed correctly; and

wherein moving the data element includes the step of;

transferring the data field portion containing the cached host data in front of the status field portion containing the status indicating whether the earlier-performed read operation completed correctly to enable error detection to begin upon receipt of the data field portion while the status field portion is still being transferred.

34. A data storage system, comprising:

a volatile memory cache circuit that buffers data elements exchanged between a storage device and a host;

an interface circuit that operates as an interface between the volatile memory cache circuit and at least one of the storage device and the host; and

a point-to-point channel, interconnected between the volatile memory cache circuit to the interface circuit, that carries the data elements between the volatile memory cache circuit and the interface circuit, the point-to-point channel directly connecting to both the interface circuit and the volatile memory cache circuit, the point-to-point channel including a first link configured to carry first halves of the data elements and a second link configured to carry second halves of the data elements, the interface circuit including a first transmitter and a second transmitter configured to concurrently transmit the first and second halves of the data elements respectively through the first and second links.

35. The data storage system of claim 34 wherein the first and second links form a set of unidirectional links that is capable of carrying a command from the interface circuit to the volatile memory cache circuit.

36. The data storage system of claim 35 wherein the point-to-point channel further includes another set of unidirectional links that is capable of carrying status from the volatile memory cache circuit to the interface circuit.

37. The data storage system of claim 34 wherein the first link and the second link form a first set of unidirectional serial links that carries signals from the interface circuit to the volatile memory cache circuit, and a second set of unidirectional serial links that carries signals from the volatile memory cache circuit to the interface circuit.

38. The data storage system of claim 37 wherein each of the first set of unidirectional serial links and each of the second set of unidirectional serial links is an asynchronous link;

wherein the interface circuit is configured to provide a respective portion of a data element framed with synchronization delimiters to the volatile memory cache circuit through each of the first set of unidirectional serial links during a write transaction; and

wherein the interface circuit is configured to obtain a respective portion of a data element framed with synchronization delimiters from the volatile memory cache circuit through each of the second set of unidirectional serial links during a read transaction.

39. The data storage system of claim 37 wherein the interface circuit is configured to provide a respective portion of a data element and a corresponding multiple bit error detection code to the volatile memory cache circuit through each of the first set of unidirectional serial links during a write transaction;

and wherein the interface circuit is configured to obtain a respective portion of a data element and a corresponding multiple bit error detection code from the volatile memory cache circuit through each of the second set of unidirectional serial links during a read transaction.

40. The data storage system of claim 39 wherein the interface circuit is configured to provide portions of a data element as data codes to the volatile memory cache circuit through the first set of unidirectional serial links during a write transaction;

and wherein the interface circuit is configured to obtain portions of a data element as data codes from the volatile memory cache circuit through the second set of unidirectional serial links during a read transaction; and

wherein the data codes belong to an 8B/10B encodingldecoding data space.

41. The data storage system of claim 37 wherein the volatile memory cache circuit is configured to provide a busy signal to the interface circuit through each of the second set of unidirectional serial links after receiving a command and before a data element moves through the point-to-point channel.

42. The data storage system of claim 37 wherein the interface circuit is configured to provide a tag indicator to the volatile memory cache circuit through the first set of unidirectional serial links when providing a command to the volatile memory cache circuit;

and wherein the volatile memory cache circuit is configured to provide a copy of the tag indicator to the interface circuit through the second set of unidirectional serial links when providing status to the interface circuit.

43. The data storage system of claim 34 wherein the interface circuit is configured to (i) associate tags with the data elements and transmit the tags to the volatile memory cache circuit with the data elements, and (ii) receive tags within status messages from the volatile memory cache circuit in response to the data elements to verify transmission of the status messages to a correct interface circuit among multiple interface circuits.

44. The data storage system of claim 34 wherein the volatile memory cache circuit directly connects to multiple front-end point-to-point channels which directly connect to respective front-end interfaces configured to communicate with external hosts, and further directly connects to multiple back-end point-to-point channels which directly connect to respective back-end interfaces configured to communicate with non-volatile storage devices.

45. The data storage system of claim 34, further comprising:

other volatile memory cache circuits which, in combination with the volatile memory cache circuit, forms a cache;

other interfaces configured to operate as interfaces between hosts and the cache; and

other point-to-point channels that directly connect to both the other interfaces and the cache, the other point-to-point channels being configured to concurrently exchange data between the interfaces and the cache.

46. The data storage system of claim 34 wherein the data elements includes series of even and odd numbered bytes;

and wherein the interface circuit is adapted to divide the data elements into the first halves and the second halves by;

putting all of the even numbered bytes of the series into the first halves to enable the first transmitter to exclusively transmit only even numbered bytes through the first link of the point-to-point channel; and

putting all of the odd numbered bytes of the series into the second halves to enable the second transmitter to exclusively transmit only odd numbered bytes through the second link of the point-to-point channel.

47. The data storage system of claim 46 wherein transmission of the first halves and transmission of the second halves occur at the same time and independently of each other from a clock synchronization standpoint.