Method and apparatus for prefetching recursive data structures
First Claim
1. A method of scheduling units of work for execution on a computer system including a data cache or data cache hierarchy, the method comprising the steps of:
- buffering a plurality of transactions on a data structure;
scheduling a plurality of said transactions on said data structures in a loop;
issuing prefetch instructions within the body of said loop for the data required to process said transactions.
2 Assignments
0 Petitions
Accused Products
Abstract
Computer systems are typically designed with multiple levels of memory hierarchy. Prefetching has been employed to overcome the latency of fetching data or instructions from or to memory. Prefetching works well for data structures with regular memory access patterns, but less so for data structures such as trees, hash tables, and other structures in which the datum that will be used is not known a priori. The present invention significantly increases the cache hit rates of many important data structure traversals, and thereby the potential throughput of the computer system and application in which it is employed. The invention is applicable to those data structure accesses in which the traversal path is dynamically determined. The invention does this by aggregating traversal requests and then pipelining the traversal of aggregated requests on the data structure. Once enough traversal requests have been accumulated so that most of the memory latency can be hidden by prefetching the accumulated requests, the data structure is traversed by performing software pipelining on some or all of the accumulated requests. As requests are completed and retired from the set of requests that are being traversed, additional accumulated requests are added to that set. This process is repeated until either an upper threshold of processed requests or a lower threshold of residual accumulated requests has been reached. At that point, the traversal results may be processed.
-
Citations
28 Claims
-
1. A method of scheduling units of work for execution on a computer system including a data cache or data cache hierarchy, the method comprising the steps of:
-
buffering a plurality of transactions on a data structure;
scheduling a plurality of said transactions on said data structures in a loop;
issuing prefetch instructions within the body of said loop for the data required to process said transactions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computer system with a cache hierarchy comprising:
-
a) at least one main memory, b) at least one cache coupled to main memory, c) a means for prefetching data into any such cache from main memory, d) a buffer for accumulating traversal requests, e) a buffer for storing traversal results, f) a means of storing traversal requests once prefetch operations have been initiated, g) a buffer for holding an active traversal request, h) a multiplexor to select between the accumulated traversals and the active traversal. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25)
-
-
26. The method of organizing data within the memory on a computer system, the method comprising the steps of:
-
a) determining the cache line boundaries of data structure elements;
b) aligning the base of the data structure on a cache line boundary;
c) homogenizing the data structure;
d) inserting a pad field into data structure elements so that subsequent elements are aligned on cache line boundaries;
e) packing elements so as to maximize the data represented in each cache line by removing pointers to adjacent elements, whereby the program instructions that traverse the data structure are constructed to traverse the adjacent packed elements before traversing non-packed elements, whereby steps b, c, d, and e may be performed in any order and any proper subset of steps c, d, and e can be employed. - View Dependent Claims (27, 28)
-
Specification