Method and apparatus for monitoring processes in a non-uniform memory access (NUMA) computer system

• US 20060259704A1
• Filed: 05/12/2005
• Published: 11/16/2006
• Est. Priority Date: 05/12/2005
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Wyman, Blair
• Granted Patent: US 7,383,396 B2
• Time in Patent Office: Days
• US Class Current: 711/141
• CPC Class Codes: G06F 11/3404 for parallel or distributed programming
• CPC Class Codes: G06F 9/3851 from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence

Abstract

A monitoring process for a NUMA system collects data from multiple monitored threads executing in different nodes of the system. The monitoring process executes on different processors in different nodes. The monitoring process intelligently collects data from monitored threads according to the node it which it is executing to reduce the proportion of inter-node data accesses. Preferably, the monitoring process has the capability to specify a node to which it should be dispatched next to the dispatcher, and traverses the nodes while collecting data from threads associated with the node in which the monitor is currently executing. By intelligently associating the data collection with the node of the monitoring process, the frequency of inter-node data accesses for purposes of collecting data by the monitoring process is reduced, increasing execution efficiency.

Claims

1. A computer-executed method for monitoring threads in a non-uniform memory access (NUMA) computer system having a plurality of processors and a plurality of portions of memory, wherein each said processor accesses each memory portion in a respective memory access time, at least some of said respective memory access times being different, the set of respective memory access times for a first processor being different from the set of respective memory access times for a second processor, the method comprising the steps of:

executing a plurality of monitored threads, each monitored thread being associated with a respective said portion of memory;

invoking a monitoring process for automatically collecting data for said plurality of monitored threads;

executing said monitoring process on a first plurality of said processors of said NUMA computer system to collect data for said plurality of monitored threads, wherein said monitoring process executes in each processor of said first plurality of processors to collect data for each monitored thread of a respective subset of said plurality of monitored threads, the respective subset being determined according to the respective portion of memory with which each monitored thread is associated.

2. The method for monitoring threads of claim 1, wherein said step of executing said monitoring process on a first plurality of said processors of said NUMA computer system to collect data for said plurality of monitored threads comprises:

changing an order in which data is collected for monitored threads to group monitored threads associated with the same respective portion of memory together.

3. The method for monitoring threads of claim 1, wherein said step of executing said monitoring process on a first plurality of said processors of said NUMA computer system to collect data for said plurality of monitored threads comprises:

changing a processor on which said monitoring process is executing to reduce memory access time to a portion of memory with which a next monitored thread for which data is to be collected is associated.

4. The method for monitoring threads of claim 3,

wherein said monitoring process has the capability to designate an absolute association with a portion of memory;

wherein a dispatcher dispatching threads in said computer system is bound to follow said absolute association by dispatching a thread of said monitoring process only to a processor associated with the designated portion of memory; and

wherein said step of changing a processor on which said monitoring process is executing comprises;

(a) changing a designated portion of memory with which said monitoring process has said absolute association, and (b) yielding the processor on which said monitoring process is executing.

5. The method for monitoring threads of claim 1, wherein said computer system comprises a plurality of nodes, each node containing a respective subset of said plurality of processors and a respective said portion of memory, wherein memory access time for each processor to access the respective said portion of memory contained in the node containing the processor is less than memory access time for the processor to access any respective portion of memory contained in a different node.

6. The method for monitoring threads of claim 1, wherein each said monitored thread is associated with a respective said portion of memory using at least one of the set of association criteria consisting of:

(a) a portion of memory associated with a processor on which the respective monitoring thread has most recently executed;

(b) a designated association made by the respective thread itself; and

(c) an association made by an operating system at the time the thread is instantiated based on workload balancing considerations.

7. The method for monitoring threads of claim 1, wherein said monitoring process periodically collects data for said monitored threads responsive to an automatic triggering event.

8. A program product for monitoring threads in a non-uniform memory access (NUMA) computer system having a plurality of processors and a plurality of portions of memory, wherein each said processor accesses each memory portion in a respective memory access time, at least some of said respective memory access times being different, the set of respective memory access times for a first processor being different from the set of respective memory access times for a second processor, the program product comprising:

a plurality of computer executable instructions recorded on signal-bearing media, wherein said instructions, when executed by a NUMA computer system, cause the NUMA computer system to perform the steps of;

executing at least one thread of said program product on a first plurality of processors having different sets of respective memory access times to different memory portions of said NUMA computer system, said program product for collecting data for a plurality of monitored threads of said NUMA computer system product, each monitored thread being associated with a respective said portion of memory;

selecting a respective subset of said plurality of monitored threads corresponding to each of said first plurality of processors, said subsets being selected using the memory access time from each respective processor of said first plurality of processors to each respective memory portion with which a respective monitored thread is associated; and

collecting data for each said monitored thread of each respective said subset of said plurality of monitored threads by executing a thread of said program product in a respective said processor corresponding to the respective said subset.

9. The program product for monitoring threads of claim 8, wherein said step of selecting a respective subset of said plurality of monitored threads corresponding to each of said first plurality of processors comprises:

changing an order in which data is collected for monitored threads to group monitored threads associated with the same respective portion of memory together.

10. The program product for monitoring threads of claim 8, wherein said step of selecting a respective subset of said plurality of monitored threads corresponding to each of said first plurality of processors comprises:

changing a processor on which a thread of said program product is executing to reduce memory access time to a portion of memory with which a next monitored thread for which data is to be collected is associated.

11. The program product for monitoring threads of claim 10,

wherein said program product has the capability to designate an absolute association with a portion of memory;

wherein a dispatcher dispatching threads in said NUMA computer system is bound to follow said absolute association by dispatching a thread of said program product only to a processor associated with the designated portion of memory; and

wherein said step of changing a processor on which said program product is executing comprises;

(a) changing a designated portion of memory with which said program product has said absolute association, and (b) yielding the processor on which said thread of said program product is executing.

12. The program product for monitoring threads of claim 8, wherein said computer system comprises a plurality of nodes, each node containing a respective subset of said plurality of processors and a respective said portion of memory, wherein memory access time for each processor to access the respective said portion of memory contained in the node containing the processor is less than memory access time for the processor to access any respective portion of memory contained in a different node.

13. The program product for monitoring threads of claim 8, wherein each said monitored thread is associated with a respective said portion of memory using at least one of the set of association criteria consisting of:

(a) a portion of memory associated with a processor on which the respective monitoring thread has most recently executed;

(b) a designated association made by the respective thread itself; and

(c) an association made by an operating system at the time the thread is instantiated based on workload balancing considerations.

14. The program product for monitoring threads of claim 8, wherein said program product process periodically collects data for said monitored threads responsive to an automatic triggering event.

15. A non-uniform memory access (NUMA) computer system, comprising:

a plurality of nodes, each node comprising at least one processor and a respective portion of main memory of said computer system, wherein a memory access time from each respective processor to a memory location in the portion of memory in the same node as the respective processor is less than a memory access time from the respective processor to a memory location in a portion of memory in a different node than the respective processor;

an operating system including at least one dispatcher, said dispatcher dispatching threads for execution on said processors;

a thread monitor which automatically collects data for a plurality of monitored threads executing on said computer system, each monitored thread being associated with a respective node of said plurality of nodes, wherein said thread monitor preferentially collects data for monitored threads associated with the node containing a processor in which the thread monitor is executing when collecting the data.

16. The computer system of claim 15, wherein said thread monitor changes an order in which data is collected for monitored threads to group monitored threads associated with the same node together.

17. The computer system of claim 15, wherein said thread monitor changes a processor on which said thread monitor executes to coincide with a node with which a next monitored thread for which data is to be collected is associated.

18. The computer system of claim 17,

wherein said thread monitor has the capability to designate an absolute association with a node;

wherein said dispatcher is bound to follow said absolute association by dispatching a thread of said thread monitor only to a processor within the designated node; and

wherein said thread monitor changes a processor on which said thread monitor executs by;

(a) changing a designated node with which said thread monitor has said absolute association, and (b) yielding the processor on which said thread of said thread monitor is executing.

19. The computer system of claim 15, wherein said dispatcher is shared by all nodes of said computer system, and selects a thread for dispatch to an available processor from among a plurality of threads on one or more common dispatch queues, said one or more common dispatch queues not being restricted for dispatching of threads to any particular node of said plurality of nodes.

20. The computer system of claim 15, wherein each said monitored thread is associated with a respective said node using at least one of the set of association criteria consisting of:

(a) a node containing a processor on which the respective monitoring thread has most recently executed;

(b) a designated association made by the respective thread itself; and

(c) an association made by said operating system at the time the thread is instantiated based on workload balancing considerations.