Cache usage for concurrent multiple streams

• US 20040022094A1
• Filed: 02/05/2003
• Published: 02/05/2004
• Est. Priority Date: 02/25/2002
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Lu, Hui, Creta, Kenneth C., Radhakrishnan, Sivakumar, Natarajan, Chitra, Congdon, Bradford B.
• Application Number:
  US10/358,618
• Publication Number:
  US 20040022094A1
• Time in Patent Office: Days
• US Class Current: 365/200
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/0862 with prefetch

Abstract

In a system supporting concurrent multiple streams that pass through a cache between memory and the requesting devices, various techniques improve the efficient use of the cache. Some embodiments use adaptive pre-fetching of memory data using a dynamic table to determine the maximum number of pre-fetched cache lines permissible per stream. Other embodiments dynamically allocate the cache to the active streams. Still other embodiments use a programmable timer to deallocate inactive streams.

Claims

1. An apparatus, comprising:

a stream monitor to determine stream activity;

a scheduler coupled with the stream monitor to determine a maximum number of cache lines per stream to pre-fetch based upon the stream activity;

a pre-fetch engine coupled with the scheduler to generate pre-fetch requests; and

cache coupled with the scheduler to store pre-fetched cache lines in response to the pre-fetch requests.

2. The apparatus of claim 1, further comprising a pending transaction monitor coupled with the scheduler to limit a number of pending requests.

3. The apparatus of claim 1, further comprising an in-flight transaction monitor coupled with the scheduler to limit a number of in-flight requests.

4. The apparatus of claim 1, further comprising a throttle mechanism coupled with the scheduler to change a rate of generation of the pre-fetch requests.

5. The apparatus of claim 4, wherein the throttle mechanism is to change the rate of the generation of the pre-fetch requests based upon stream traffic.

6. The apparatus of claim 1, further comprising cache logic circuitry coupled with the stream monitor to limit a number of pre-fetched cache lines per stream based upon the stream activity.

7. An apparatus, comprising:

a stream monitor to determine stream activity;

cache logic circuitry coupled with the stream monitor to generate pre-fetch requests and to limit a number of pre-fetched cache lines per stream based upon the stream activity; and

cache coupled with the cache logic circuitry to store pre-fetched cache lines in response to the pre-fetch requests.

8. The apparatus of claim 7, wherein the stream monitor comprises inactivity circuitry to determine that a stream is inactive.

9. The apparatus of claim 7, wherein the stream monitor comprises inactivity circuitry to de-allocate a stream structure for an inactive stream.

10. The apparatus of claim 7, wherein the cache logic circuitry comprises a throttle mechanism to change a rate of generation of pre-fetch requests based upon the stream activity.

11. The apparatus of claim 7, wherein the cache logic circuitry comprises a scheduler to schedule a pre-fetch request for a stream based upon an amount of the cache allocated to the stream.

12. An apparatus, comprising:

a stream monitor to determine stream activity;

cache to store pre-fetched cache lines for active streams in a cache structure of the cache; and

cache logic circuitry coupled with the stream monitor and with the cache to determine the cache structure and to allocate the cache structure to the active streams based upon the stream activity.

13. The apparatus of claim 12, wherein the stream monitor comprises stream type circuitry to determine a type of active stream.

14. The apparatus of claim 12, wherein the stream monitor comprises stream count circuitry to determine a number of active streams.

15. The apparatus of claim 14, wherein the cache logic circuitry comprises allocation circuitry to allocate the cache structure to the active streams.

16. The apparatus of claim 12, wherein the stream monitor comprises circuitry to determine a change in a number of active streams.

17. The apparatus of claim 16, wherein the cache logic circuitry comprises allocation circuitry to change the cache structure for the change in the number of active streams.

18. A system, comprising:

a memory;

a plurality of devices to request data from the memory;

input-output circuitry coupled between the memory and the plurality of devices and comprising

a stream monitor to determine stream activity;

cache to store pre-fetched cache lines for active streams; and

cache logic circuitry coupled with the stream monitor and with the cache to allocate portions of the cache to the active streams based upon the stream activity.

19. The system of claim 18, wherein the stream monitor comprises stream count circuitry to determine a number of active streams.

20. The system of claim 19, wherein the cache logic circuitry comprises allocation circuitry to allocate the portions of the cache to the active streams.

21. The system of claim 18, wherein the stream monitor comprises circuitry to determine a change in a number of active streams.

22. The apparatus of claim 21, wherein the cache logic circuitry comprises allocation circuitry to change the portions of the cache responsive to the change in the number of active streams.

23. An apparatus, comprising:

a timer to determine a stream is inactive;

cache logic circuitry coupled with the timer to de-allocate a cache structure for the stream based upon the determination that the stream is inactive; and

cache coupled with said cache logic circuitry to store data in the cache structure.

24. The apparatus of claim 23, wherein the timer is to determine that a stream is inactive based upon a heuristically determined time between requests associated with an active stream.

25. The apparatus of claim 23, wherein the timer comprises a reset circuit to restart the timer based upon receipt of a request associated with the stream.

26. The apparatus of claim 23, wherein the cache logic circuitry comprises a table to store a value indicating a time between requests of an active stream based upon a number of active streams.

27. The apparatus of claim 23 wherein the cache logic circuitry comprises a table to store a value indicating a time between requests of an active stream based upon a type of the active stream.

28. An apparatus, comprising:

a pre-fetch engine to pre-fetch data from a memory for multiple concurrent streams;

a timer coupled to the pre-fetch engine to determine a particular stream of the multiple concurrent streams is inactive;

cache logic circuitry coupled with the timer to de-allocate a cache structure for the particular stream based upon the determination that the particular stream is inactive; and

cache coupled with said cache logic circuitry to store the data in the cache structure.

29. The apparatus of claim 28, wherein a timeout value for the timer is a heuristically determined timeout value.

30. The apparatus of claim 28, further comprising circuitry to reset the timer upon receipt of a new request associated with the particular stream.

31. The apparatus of claim 28, wherein the cache logic circuitry comprises a table to store a timeout value based upon a number of active concurrent streams.

32. The apparatus of claim 28 wherein the cache logic circuitry comprises a table to store a timeout value based upon a type of the particular stream.

33. An apparatus, comprising:

a stream monitor to determine stream activity,

a scheduler coupled with the stream monitor to determine a maximum number of cache lines per stream to pre-fetch based upon the stream activity;

cache coupled with the scheduler to store pre-fetched cache lines;

cache logic circuitry coupled with the stream monitor to generate pre-fetch requests, to limit a number of pre-fetched cache lines per stream based upon the maximum number, and to allocate the cache based upon the stream activity; and

a timer coupled with the cache logic circuitry to de-allocate a portion of the cache for a particular stream responsive to a determination that the particular stream is inactive.

34. The apparatus of claim 33, further comprising a pre-fetch engine coupled with the scheduler to adjust a generation rate of the pre-fetch requests based upon the stream activity.

35. The apparatus of claim 33, wherein the stream monitor comprises circuitry to determine a change in a number of active streams.