PCI express enhancements and extensions

• US 8,230,120 B2
• Filed: 03/28/2011
• Issued: 07/24/2012
• Est. Priority Date: 11/02/2006
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ajanovic, Jasmin, Wagh, Mahesh, Sethi, Prashant, Das Sharma, Debendra, Harriman, David, Rosenbluth, Mark, Bhatt, Ajay, Barry, Peter, Rodgers, Scott Dion, Vasudevan, Anil, Muthrasanallur, Sridhar, Akiyama, James, Blankenship, Robert, Falik, Ohad, Mendelson, Abraham, Pardo, Ilan, Tamari, Eran, Weissmann, Eliezer, Shamia, Doron
• Primary Examiner(s): Lee, Chun-Kuan
• Assistant Examiner(s): MARTINEZ, DAVID E
• Application Number:
  US13/073,149
• Publication Number:
  US 20110238882A1
• Time in Patent Office: 484 Days
• Field of Search: 710/5, 710/6, 710/305
• US Class Current: 710/5
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/324 by lowering clock frequency
• CPC Class Codes: G06F 1/3253 Power saving in bus
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 13/385 for adaptation of a particular data processing system to different peripheral devices
• CPC Class Codes: G06F 13/4045 where the bus bridge performs an extender function
• CPC Class Codes: G06F 13/4068 Electrical coupling
• CPC Class Codes: G06F 13/4265 on a point to point bus G06F13/4247, G06F13/4282 take precedence
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method and apparatus for enhancing/extending a serial point-to-point interconnect architecture, such as Peripheral Component Interconnect Express (PCIe) is herein described. Temporal and locality caching hints and prefetching hints are provided to improve system wide caching and prefetching. Message codes for atomic operations to arbitrate ownership between system devices/resources are included to allow efficient access/ownership of shared data. Loose transaction ordering provided for while maintaining corresponding transaction priority to memory locations to ensure data integrity and efficient memory access. Active power sub-states and setting thereof is included to allow for more efficient power management. And, caching of device local memory in a host address space, as well as caching of system memory in a device local memory address space is provided for to improve bandwidth and latency for memory accesses.

Claims

1. An apparatus comprising:

an input/output (I/O) module including a protocol stack configured to communicate with a device over a serial point-to-point interconnect, the protocol stack being configured to transmit a first packet on the serial point to point interconnect, wherein the first packet is to include a reference to a memroy address and a first caching hint including an intent to write only hint, and wherein in response to the first cache hint including the intent to write only, the device is not to fetch a data element from the memory address and is to transition a memory location associated with the memory address to an Exclusive coherency state.

2. The apparatus of claim 1, wherein the protocol stack comprises a transaction layer, a link layer, and a physical layer, and wherein the I/O module is included in a device selected from a group consisting of a processing element, a microprocessor, a root controller, a switch, a controller hub, a graphics accelerator, and a input/output (I/O) device.

3. The apparatus of claim 1, wherein the first packet is a read/write request packet, and wherein the first caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

4. The apparatus of claim 1, wherein the protocol stack is also to transmit a second packet on the serial point-to-point interconnect, wherein the second packet is a prefetch message packet to include a first prefetching hint, and wherein the first prefetching hint is to include a reference to a prefetch address and a prefetch parameter, and wherein the prefetch parameter is selected from a group consisting of a length, a number of blocks, a number of blocks within a page, a number of cachelines in a prefetch block, a number of cachelines to the start of a next block, an offset, a stride, and a direction.

5. An apparatus comprising:

an input/output (I/O) module configured to implement a protocol stack including at least a transaction layer, a link layer, and a physical layer, the I/O module configured to transmit a first packet, wherein the first packet is to include a reference to an address and a caching hint, and wherein in response to the caching hint indicating the element is to be fetched, an element associated with the address is to be fetched and held in a cache coherency state within a cache memory based on the caching hint.

6. The apparatus of claim 5, wherein the caching hint is selected from a group consisting of an intent to modify hint, an intent to write hint, an intent to writeback hint, an intent to read hint, an intent to read only hint, an intent to read and write hint, an evict hint, a DIO hint, a base read hint, a base write, and a base default hint.

7. The apparatus of claim 5, wherein the packet is a read/write request packet, and wherein the caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

8. The apparatus of claim 6, wherein the I/O module is also to receive a second packet, and wherein the second packet includes a prefetch message packet to include a first prefetching hint.

9. The apparatus of claim 8, wherein the first prefetching hint is to include a reference to a prefetch address and a prefetch parameter, and wherein the prefetch parameter is selected from a group consisting of a length, a number of blocks, a number of blocks within a page, a number of cachelines in a prefetch block, a number of cachelines to the start of a next block, an offset, a stride, and a direction.

10. The apparatus of claim 9, wherein fetch logic is further to prefetch from at least the prefetch address based on the prefetch address and the prefetch parameter.

11. An apparatus comprising:

an input/output (I/O) module including a protocol stack configured to communicate with a device over a serial point-to-point interconnect, the protocol stack being configured to transmit a first packet on the serial point to point interconnect, wherein the first packet is to include a reference to a memroy address and a first caching hint including an intent to read and write hint, and wherein in response to the first cache hint including the intent to read and write hint, the device is to fetch a data element from the memory address and is to transition a memory location to hold the data element to an Exclusive coherency state.

12. The apparatus of claim 11, wherein the protocol stack comprises a transaction layer, a link layer, and a physical layer, and wherein the I/O module is included in a device selected from a group consisting of a processing element, a microprocessor, a root controller, a switch, a controller hub, a graphics accelerator, and a input/output (I/O) device.

13. The apparatus of claim 11, wherein the first packet is a read/write request packet, and wherein the first caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

14. The apparatus of claim 11, wherein the protocol stack is also to transmit a second packet on the serial point-to-point interconnect, wherein the second packet is a prefetch message packet to include a first prefetching hint, and wherein the first prefetching hint is to include a reference to a prefetch address and a prefetch parameter, and wherein the prefetch parameter is selected from a group consisting of a length, a number of blocks, a number of blocks within a page, a number of cachelines in a prefetch block, a number of cachelines to the start of a next block, an offset, a stride, and a direction.

16. The apparatus of claim 11, wherein the protocol stack comprises a transaction layer, a link layer, and a physical layer, and wherein the I/O module is included in a device selected from a group consisting of a processing element, a microprocessor, a root controller, a switch, a controller hub, a graphics accelerator, and a input/output (I/O) device.

17. The apparatus of claim 11, wherein the first packet is a read/write request packet, and wherein the first caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

18. The apparatus of claim 11, wherein the protocol stack is also to transmit a second packet on the serial point-to-point interconnect, wherein the second packet is a prefetch message packet to include a first prefetching hint, and wherein the first prefetching hint is to include a reference to a prefetch address and a prefetch parameter, and wherein the prefetch parameter is selected from a group consisting of a length, a number of blocks, a number of blocks within a page, a number of cachelines in a prefetch block, a number of cachelines to the start of a next block, an offset, a stride, and a direction.

15. An apparatus comprising:

an input/output (I/O) module including a protocol stack configured to communicate with a device over a serial point-to-point interconnect, the protocol stack being configured to transmit a first packet on the serial point to point interconnect, wherein the first packet is to include a reference to a memroy address and a first caching hint including an intent to read only hint, and wherein in response to the first cache hint including the intent to read only hint, the device is to fetch a data element from the memory address and is to transition a memory location to hold the data element to an Exclusive or Shared coherency state.

19. A method comprising:

transmiting a first packet with an input/output (I/O) module including a protocol stack over a serial point-to-point interconnect, the first packet including a reference to a memroy address and a first caching hint;

determining if a data element associated with the memory address is to be fetched based on the first caching hint in response to receiving the first packet from the serial point-to-point interconnect;

fetching the data element in response to determining the data element associated with the memory address is to be fetched based on the first caching hint; and

transitioning a cache memory location associated with the memroy address to a cache coherency state based on the first caching hint.

20. The method of claim 19, wherein the caching hint is selected from a group consisting of an intent to modify hint, an intent to write hint, an intent to writeback hint, an intent to read hint, an intent to read only hint, an intent to read and write hint, an evict hint, a DIO hint, a base read hint, a base write, and a base default hint.

21. The method of claim 19, wherein the caching hint includes an intent to write only hint and the cache cohernecy state includes an Exclusive coherency state, and wherein determining if a data element associated with the memory address is to be fetched based on the first caching hint comprises determining the data element is not to be fetched based on the first caching hint including the intent to write only hint.

22. The method of claim 19, wherein the caching hint includes an intent to read and write hint and the cache cohernecy state includes an Exclusive coherency state, and wherein determining if a data element associated with the memory address is to be fetched based on the first caching hint comprises determining the data element is to be fetched based on the first caching hint including the intent to read and write hint.

23. The method of claim 19, wherein the caching hint includes an intent to read only hint and the cache cohernecy state includes an Exclusive or Shared coherency state, and wherein determining if a data element associated with the memory address is to be fetched based on the first caching hint comprises determining the data element is to be fetched based on the first caching hint including the intent to read only hint.

24. The method of claim 19, wherein the first packet is a read/write request packet, and wherein the first caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

25. A system comprising:

a transmitting device including an input/output (I/O) module configured to implement a protocol stack including at least a transaction layer, a link layer, and a physical layer, the I/O module being configured to transmit a first packet, wherein the first packet is to include a reference to an address and a caching hint

a receiving device coupled to the transmitting device over a serial point-to-point interconnect, the receiving device including a cache memory, fetch logic, and an I/O module configured to implement a protocol stack, the I/O module being configured to receive the first packet, wherein the fetch logic is configured to determine if a data element is to be fetched from the address based on the caching hint and the cache memory is configured to hold a cache location associated with the address in a cache coherency state based on the caching hint.

26. The system of claim 25, wherein the caching hint is selected from a group consisting of an intent to modify hint, an intent to write hint, an intent to writeback hint, an intent to read hint, an intent to read only hint, an intent to read and write hint, an evict hint, a DIO hint, a base read hint, a base write, and a base default hint.

27. The system of claim 25, wherein the packet is a read/write request packet, and wherein the caching hint is to be included within a first portion of the read/write request packet, a first prefetching hint is to be included within an address field of the read/write request packet, and wherein the first portion and the address field are separate portions of the read/write request packet that do not overlap.

28. The system of claim 25, wherein the transmitting device is further configured to transmit a second packet, the receiving device is further configured to receive the second packet, and the second packet is to include a prefetch message packet to include a first prefetching hint, a the first prefetching hint to include a reference to a prefetch address and a prefetch parameter, the prefetch parameter to be selected from a group consisting of a length, a number of blocks, a number of blocks within a page, a number of cachelines in a prefetch block, a number of cachelines to the start of a next block, an offset, a stride, and a direction, and wherein the fetch logic is further configured to prefetch from at least the prefetch address based on the prefetch address and the prefetch parameter.