Dynamic memory placement policies for NUMA architecture

• US 6,871,219 B2
• Filed: 03/07/2001
• Issued: 03/22/2005
• Est. Priority Date: 03/07/2001
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Khemani, Prakash, Noordergraaf, Lisa K., Harper, Julia D.
• Primary Examiner(s): PEIKARI, BEHZAD
• Application Number:
  US09/801,474
• Publication Number:
  US 20020129115A1
• Time in Patent Office: 1,476 Days
• Field of Search: 711/147, 711/148, 709/214, 709/213, 709/216, 709/218, 709/226, 707/101, 712/203, 712/225, 718/104
• US Class Current: 709/214
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration

Abstract

A distributed shared memory multiprocessor computer system utilizes page placement policies to reduce data access latencies. Pages of memory are allocated to nodes in a distributed shared memory multiprocessor computer system. A placement policy database is maintained which indicates a placement policy for each page in the system. Upon an access to a page, the placement policy corresponding to the accessed page is determined and the indicated policy is acted upon. A Migrate on Next Touch policy provides that the next access to a page with this policy will cause the page to migrate to the node of the accessing CPU. A Memory Follows Lightweight Process (LWP) policy ensures that pages within a given address range are always local to a specified LWP. A Migrate on Every Touch policy provides that pages within a given address range are migrated to an accessing CPU on every touch. A Replicate on Remote Touch policy provides for replication of a page in the memory of each accessing CPU'"'"'s domain. Finally, a Replicate on All policy provides that upon an access to a given page, that page is replicated on all nodes in the system.

Claims

1. A method of dynamically placing data within a distributed shared memory computer system comprising:

configuring a memory management subsystem to indicate an affinity policy between a first process and a first page of memory of a plurality of pages of memory of said computer system;

scheduling said first process to run on a first node of a plurality of nodes of said computer system; and

placing said first page in said first node of said computer system in response to detecting said scheduling.

2. The method of claim 1, wherein said process is a lightweight process and said affinity policy is a memory_follows_lwp policy.

3. The method of claim 2, wherein said placing is in further response to detecting an access to said first page by said first process.

4. The method of claim 2, wherein a first placement policy is applied to said first page, wherein said first placement policy indicates said first page is to follow a next process which touches memory corresponding to said first page, and wherein said affinity is indicated between said first page and said first process in response to detecting said first process is a next process to touch said first page.

5. The method of claim 4, wherein a default policy is applied to said first page subsequent to said placing said first page.

6. The method of claim 3, further comprising updating said affinity policy in response to detecting said placing.

7. A method of dynamically placing data within a distributed shared memory computer system comprising:

configuring a memory management subsystem to indicate a first data placement policy corresponding to a first page of a plurality of pages of memory, wherein said first page has an initial placement corresponding to a first node of a plurality of nodes of said system;

detecting an access to said first page by a second node of said plurality of nodes; and

placing said first page in said second node of said computer system in response to detecting said access.

8. The method of claim 7, wherein said first data placement policy is a migrate_on_next_touch policy and said first page is migrated to said second node from said first node.

9. The method of claim 7, wherein said first data placement policy is a replicate_on_next_touch policy and said first page is replicated on said second node.

10. The method of claim 7, wherein said first data placement policy is a replicate_on_all policy and said first page is replicated on said plurality of nodes.

11. The method of claim 7, wherein said first data placement policy is a migrate_on_every_touch policy and said first page is migrated to said second node from said first node.

12. The method of claim 7, further comprising updating said first data placement policy in response to detecting said placing.

13. The method of claim 7, further comprising obtaining said first data placement policy from a home node of said first page.

14. A distributed shared memory computer system including a plurality of nodes, said system comprising:

a first node of said plurality of nodes, wherein said first node is configured to indicate a first data placement policy corresponding to a first page of a plurality of pages of memory, wherein said first page has an initial placement corresponding to said first node; and

a second node of said plurality of nodes, wherein said second node is configured to access said first page, and wherein said second node is configured to place said first page in said second node in response to detecting an access to said first page by said second node.

15. The system of claim 14, wherein said first data placement policy is a migrate_on_next_touch policy and said first page is migrated to said second node from said first node.

16. The system of claim 14, wherein said first data placement policy is a replicate_on_next_touch policy and said first page is replicated on said second node.

17. The system of claim 14, wherein said first data placement policy is a replicate_on_all policy and said first page is replicated on said plurality of nodes.

18. The system of claim 14, wherein said first data placement policy is a migrate_on_every_touch policy and said first page is migrated to said second node from said first node.

19. The system of claim 14, further comprising updating said first data placement policy in response to detecting said placing.

20. The system of claim 14, further comprising obtaining said first data placement policy from a home node of said first page.

21. A distributed shared memory computer system including a plurality of nodes, said system comprising:

a first node of said plurality of nodes, wherein said first node is configured to indicate an affinity policy between a first process and a first page of memory of a plurality of pages of memory of said computer system; and

a process scheduler, wherein said scheduler is configured to schedule said first process to run on said first node;

a memory management subsystem, wherein said memory management subsystem is configured to place said first page in said first node in response to detecting said scheduling.

22. The computer system of claim 21, wherein said first process is a lightweight process and said affinity policy is a memory_follows_lwp policy.

23. The computer system of claim 22, wherein said memory management subsystem is configured to place said first page in further response to detecting an access to said first page by said first process.

24. The computer system of claim 23, wherein a first placement policy is applied to said first page, wherein said first placement policy indicates said first page is to follow a next process which touches memory corresponding to said first page, and wherein said affinity is indicated between said first page and said first process in response to detecting said first process is a next process to touch said first page.

25. The computer system of claim 24, wherein a default policy is applied to said first page subsequent to said placing said first page.

26. The computer system of claim 24, further comprising updating said affinity policy in response to detecting said placing.

27. A carrier medium comprising program instructions, wherein the program instructions are computer-executable to:

configure a memory management subsystem to indicate an affinity policy between a first process and a first page of memory of a plurality of pages of memory of a computer system;

schedule said first process to run on a first node of a plurality of nodes of said computer system; and

place said first page in said first node of said computer system in response to detecting said scheduling.

28. The carrier medium of claim 27, wherein said process is a lightweight process and said affinity policy is a memory_follows_lwp policy.

29. The carrier medium of claim 28, wherein said first page is placed in said first node in further response to detecting an access to said first page by said first process.

30. The carrier medium of claim 29, further comprising updating said affinity policy in response to detecting said first page is placed.

31. The carrier medium of claim 28, wherein a first placement policy is applied to said first page, wherein said first placement policy indicates said first page is to follow a next process which touches memory corresponding to said first page, and wherein said affinity is indicated between said first page and said first process in response to detecting said first process is a next process to touch said first page.

32. The carrier medium of claim 31, wherein a default policy is applied to said first page subsequent to placing said first page.

33. A carrier medium comprising program instructions, wherein the program instructions are computer-executable to:

configure a memory management subsystem to indicate a first data placement policy corresponding to a first page of a plurality of pages of memory, wherein said first page has an initial placement corresponding to a first node of a plurality of nodes of a computer system;

detecting an access to said first page by a second node of said plurality of nodes; and

placing said first page in said second node of said computer system in response to detecting said access.

34. The carrier medium of claim 33, wherein said first data placement policy is a migrate_on_next_touch policy and said first page is migrated to said second node from said first node.

35. The carrier medium of claim 33, wherein said first data placement policy is a replicate_on_next_touch policy and said first page is replicated on said second node.

36. The carrier medium of claim 33, wherein said first data placement policy is a replicate_on_all policy and said first page is replicated on said plurality of nodes.

37. The carrier medium of claim 33, wherein said first data placement policy is a migrate_on_every_touch policy and said first page is migrated to said second node from said first node.

38. The carrier medium of claim 33, further comprising updating said first data placement policy in response to detecting said placing.

39. The carrier medium of claim 33, further comprising obtaining said first data placement policy from a home node of said first page.