Method and system for managing cache injection in a multiprocessor system

• US 8,255,591 B2
• Filed: 09/23/2004
• Issued: 08/28/2012
• Est. Priority Date: 09/23/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bohrer, Patrick Joseph, Gheith, Ahmed, Hochschild, Peter Heiner, Rajamony, Ramakrishnan, Shafi, Hazim, Sinharoy, Balaram
• Primary Examiner(s): Mamo, Elias
• Application Number:
  US10/948,407
• Publication Number:
  US 20060064518A1
• Time in Patent Office: 2,896 Days
• Field of Search: 710/22, 710/23, 712/1, 712/10
• US Class Current: 710/22
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence

Abstract

A method and apparatus for managing cache injection in a multiprocessor system reduces processing time associated with direct memory access transfers in a symmetrical multiprocessor (SMP) or a non-uniform memory access (NUMA) multiprocessor environment. The method and apparatus either detect the target processor for DMA completion or direct processing of DMA completion to a particular processor, thereby enabling cache injection to a cache that is coupled with processor that executes the DMA completion routine processing the data injected into the cache. The target processor may be identified by determining the processor handling the interrupt that occurs on completion of the DMA transfer. Alternatively or in conjunction with target processor identification, an interrupt handler may queue a deferred procedure call to the target processor to process the transferred data. In NUMA multiprocessor systems, the completing processor/target memory is chosen for accessibility of the target memory to the processor and associated cache.

Claims

1. A method for performing a direct memory access (DMA) transfer within a processing system including multiple processors, said method comprising:

initiating said DMA transfer to a DMA transfer target memory image within a DMA transfer target memory over a bus;

determining in a cache controller managing a cache memory associated with a particular one of said multiple processors that said DMA transfer is occurring on said bus;

responsive to determining that said DMA transfer is occurring, copying data being transferred in said DMA transfer to a cache line in said cache memory during said DMA transfer; and

targeting said particular processor for executing a routine that accesses memory addresses of said DMA transfer target memory image, whereby said particular processor processes data transferred by said DMA transfer, wherein said targeting comprises identifying said particular processor subsequent to completion of said DMA transfer, and further comprising queuing said routine for execution by said particular processor.

2. The method of claim 1, further comprising generating an interrupt in response to completion of said DMA transfer, and wherein said identifying and queuing are performed in response to said interrupt, whereby another processor handling said interrupt schedules execution of said routine on said particular processor.

3. The method of claim 1, processing system is a non-uniform memory access (NUMA) multiprocessor system and further comprising selecting said particular processor in conformity with a latency between said cache memory and said DMA transfer target memory.

4. The method of claim 1, wherein said targeting comprises:

selecting said particular processor prior to said initiating;

selecting said cache memory in conjunction with selecting said particular processor; and

directing execution of said routine to said particular processor.

5. The method of claim 4, wherein said processing system is an non-uniform memory access (NUMA) multiprocessor system and wherein said particular processor is selected in conformity with a latency between said cache memory and said DMA transfer target memory.

6. The method of claim 4, further comprising generating an interrupt in response to completion of said DMA transfer, and wherein said directing is performed by setting an interrupt map directing handling of said interrupt to said particular processor.

7. The method of claim 6, wherein said interrupt map is an interrupt register mask in a programmable interrupt controller, and wherein said setting is performed by programming said programmable interrupt controller.

8. The method of claim 1, wherein said copying is performed in response to said cache controller determining that said cache line is present and valid in said cache memory, whereby said cache line is maintained in a valid state at a time of execution of said DMA completion routine.

9. The method of claim 1, wherein said copying is performed in response to said cache controller detecting that said cache line is not present in said cache memory, and further comprising allocating a line for storing said cache line in said cache memory prior to said copying.

10. A multiprocessor system, comprising:

a plurality of processors each coupled to an associated one of a plurality of cache memories;

at least one cache controller coupled to said associated cache memories, said at least one cache controller including a bus snooper for observing transfers on at least one bus coupling said cache memories to a memory and a transfer circuit for transferring data to cache lines managed by said cache controller in response to detecting that a direct memory access (DMA) transfer is taking place to said memory; and

an executive program resident in a memory coupled to an executive processor of said multiprocessor system, comprising program instructions that initiate said DMA transfer to said memory, and program instructions that direct execution of a routine that accesses said data transferred to said cache lines by a particular processor associated with a particular cache containing said cache lines, wherein said executive program further comprises program instructions that identify said particular processor subsequent to completion of said DMA transfer, and program instructions that queue said routine for execution by said particular processor.

11. The multiprocessor system of claim 10, wherein said executive processor is one of said plurality of processors.

12. The multiprocessor system of claim 10, wherein said multiprocessor system is a non-uniform memory access (NUMA) multiprocessor system and wherein said executive program further comprises program instructions that select said particular processor in conformity with a level of transfer latency between said cache memory and said memory.

13. The multiprocessor system of claim 10, wherein said executive program further comprises program instructions that select said particular processor prior to said initiating, whereby said cache memory is selected by selection of said particular processor.

14. The multiprocessor system of claim 13, wherein said multiprocessor system is a non-uniform memory access (NUMA) multiprocessor system and wherein said program instructions that select said particular processor select said particular processor in conformity with a transfer latency between said cache memory and said memory.

15. The multiprocessor system of claim 13, wherein said executive program further comprises program instructions that handle an interrupt generated in response to completion of said DMA transfer, and wherein said program instructions that direct execution of said routine set an interrupt map directing handling of said interrupt to said particular processor.

16. The multiprocessor system of claim 15, wherein said interrupt map is an interrupt register mask in a programmable interrupt controller, and wherein said program instructions that direct execution of said routine set said interrupt map by setting a value of an interrupt mask register in said programmable interrupt controller.

17. The multiprocessor system of claim 10, wherein said bus snooper detects a hint bit present in a bus state decode of said at least one bus that indicates that write to said memory are part of said DMA transfer, and wherein said transfer circuit transfers data to said cache line in response to said bus snooper detecting that said hint bit is present.

18. The multiprocessor system of claim 10, wherein said bus snooper detects that an address of a write to said memory on said at least one bus indicates that said write is part of said DMA transfer, and wherein said transfer circuit transfers data to said cache line in response to said bus snooper detecting that said address indicates that said write is part of said DMA transfer.

19. The multiprocessor system of claim 18, wherein said bus snooper further determines that an address of a write to said memory on said at least one bus indicates that said DMA transfer is targeted for access by said particular processor, and wherein said transfer circuit transfers said values to said cache associated with said particular processor in response to a result of said address determining.

20. The multiprocessor system of claim 18, wherein said bus snooper further determines a processor/cache identifier from a bus state decode associated with said write, and wherein said transfer circuit transfers said values to said cache associated with said particular processor in conformity with said identifier.

21. A computer program product comprising computer-readable storage media encoding program instructions forming part of an executive program for execution by a processor within a multiprocessor system, said multiprocessor system including a cache controller with a cache injection feature that loads values into a cache memory during a direct memory access (DMA) transfer to a memory within said multiprocessor system, said values being at least a portion of data transferred by said DMA transfer, and said program instructions comprising program instructions that:

initiate said DMA transfer; and

direct execution of a routine that accesses said values to a particular processor associated with said cache memory, whereby said particular processor processes said values, wherein said program instructions further comprise program instructions that;

identify said particular processor subsequent to completion of said DMA transfer; and

queue said routine for execution by said particular processor.

22. The computer program product of claim 21, wherein said multiprocessor system is a non-uniform memory access (NUMA) multiprocessor system and wherein said program instructions further comprise program instructions that select said particular processor in conformity with a level of transfer latency between said cache memory and said memory.

23. The computer program product of claim 21, wherein said program instructions further comprise program instructions that:

select said particular processor prior to said initiating, whereby said cache memory is selected by selection of said particular processor; and

direct execution of said routine to said particular processor.

24. The computer program product of claim 23, wherein said multiprocessor system is a non-uniform memory access (NUMA) multiprocessor system and wherein program instructions that select said particular processor select said particular processor in conformity with a transfer latency between said cache memory and said memory.

25. The computer program product of claim 23, wherein said program instructions further comprise program instruction that handle an interrupt generated in response to completion of said DMA transfer, and wherein said program instructions that direct execution of said routine set an interrupt map directing handling of said interrupt to said particular processor.

26. The computer program product of claim 25, wherein said interrupt map is an interrupt register mask in a programmable interrupt controller, and wherein said program instructions for setting program said programmable interrupt controller.

27. A method for performing a direct memory access (DMA) transfer within a processing system including multiple processors, the method comprising:

specifying a particular cache memory and an associated particular processor for handling data received from the DMA transfer using an affinity map that indicates latency relationships between a DMA transfer target memory and the multiple processors;

initiating the DMA transfer to a DMA transfer target memory image within the DMA transfer target memory over a bus using a bus protocol containing an indication of the particular cache memory and particular processor;

first determining in a cache controller managing a particular cache memory associated with a particular one of the multiple processors that the DMA transfer is occurring on the bus;

second determining from the bus protocol of the DMA transfer that the particular cache memory and the particular processor are specified by the bus protocol to handle the data received from the DMA transfer;

responsive to determining that the DMA transfer is occurring and that the particular cache memory is specified by the bus protocol, copying data being transferred in said DMA transfer to a cache line in said particular cache memory during said DMA transfer;

receiving an interrupt indicating completion of the DMA transfer;

executing an interrupt routine on one of the multiple processors other than the particular processor to handle the interrupt and schedule a deferred procedure call targeting said particular processor for execution; and

executing the deferred procedure call on the particular processor to access memory addresses of the DMA transfer target memory image, whereby the particular processor processes data transferred by said DMA transfer.