System and method for memory migration in distributed-memory multi-processor systems

• US 20040019751A1
• Filed: 07/23/2002
• Published: 01/29/2004
• Est. Priority Date: 07/23/2002
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Sharma, Debendra Das, Gupta, Ashish, Bryg, William R.
• Granted Patent: US 7,103,728 B2
• Time in Patent Office: Days
• US Class Current: 711/154
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0817 using directory methods

Abstract

A distributed-memory multi-processor system includes a plurality of cells communicatively coupled to each other and collectively including a plurality of processors, caches, main memories, and cell controllers. Each of the cells includes at least one of the processors, at least one of the caches, one of the main memories, and one of the cell controllers. Each of the cells is configured to perform memory migration functions for migrating memory from a first one of the main memories to a second one of the main memories in a manner that is invisible to an operating system of the system.

Claims

1. A distributed-memory multi-processor system comprising:

a plurality of cells communicatively coupled to each other and collectively including a plurality of processors, caches, main memories, and cell controllers;

each of the cells including at least one of the processors, at least one of the caches, one of the main memories, and one of the cell controllers; and

wherein each of the cells is configured to perform memory migration functions for migrating memory from a first one of the main memories to a second one of the main memories in a manner that is invisible to an operating system of the multi-processor system.

2. The distributed-memory multi-processor system of claim 1, wherein at least one of the cells further includes at least one input/output module with an associated cache.

3. The distributed-memory multi-processor system of claim 1, wherein the system is configured to migrate memory from a first location in a main memory to a second location in the same main memory in a manner that is invisible to an operating system of the multi-processor system.

4. The distributed-memory multi-processor system of claim 1, wherein the system is configured to allow the memory being migrated to be accessed during migration of the memory.

5. The distributed-memory multi-processor system of claim 1, wherein the system is configured to perform the migration of memory on a memory line by memory line basis.

6. The distributed-memory multi-processor system of claim 1, wherein each cell includes a cache coherency directory, and wherein the cache coherency directories are configured to store migration status information.

7. The distributed-memory multi-processor system of claim 6, wherein the migration status information indicates a migration state of a memory line during the migration.

8. The distributed-memory multi-processor system of claim 7, wherein the migration status information indicates at least three migration states, including a first state indicating that a memory line is waiting for migration, a second state indicating that a memory line is in migration, and a third state indicating that a memory line has been migrated.

9. The distributed-memory multi-processor system of claim 1, wherein the system is configured to update address registers of the processors after the migration.

10. The distributed-memory multi-processor system of claim 1, wherein each cell includes firmware for performing memory migration functions.

11. A method of migrating memory in a distributed-memory multi-processor system, the method comprising:

providing a plurality of cells, each cell including at least one processor, at least one cache, a main memory, a cell controller, and a cache coherency directory;

initiating a memory migration transaction between a first one of the cells and a second one of the cells;

copying data from a first memory portion in the main memory of the first cell to a second memory portion in the main memory of the second cell during the memory migration transaction; and

storing migration status information in the cache coherency directories of the first and the second cells, the migration status information indicating migration states of the first and the second memory portions during the memory migration transaction.

12. The method of claim 11, wherein the first and the second cells each include a configuration and status register, the method further comprising:

writing to the configuration and status registers of the first and the second cells to inform the first cell that memory is being migrated from the first cell and to inform the second cell that memory is being migrated to the second cell.

13. The method of claim 11, and further comprising:

sending a fetch request from the second cell to the first cell requesting migration of the first memory portion in the first cell to the second memory portion in the second cell.

14. The method of claim 11, and further comprising:

sending a recall request to caches holding a copy of the first memory portion.

15. The method of claim 11, and further comprising:

redirecting memory accesses originally directed to the first memory portion to the second memory portion after the data has been copied from the first memory portion to the second memory portion.

16. The method of claim 11, wherein each processor includes an address register, the method further comprising:

updating the address registers of the processors based on the memory migration transaction.

17. The method of claim 11, wherein the migration status information indicates at least three migration states, including a first state indicating that a memory portion is waiting for migration, a second state indicating that a memory portion is in migration, and a third state indicating that a memory portion has been migrated.

18. The method of claim 11, wherein the memory migration transaction is performed without operating system intervention.

19. A distributed-memory multi-processor system with directory-based cache coherency comprising:

a plurality of cells, each cell including at least one processor, at least one cache, a main memory, a cell controller, and a cache coherency directory;

each cell configured to perform memory migration functions for migrating a memory portion from the main memory of one cell to the main memory of another cell; and

each cache coherency directory configured to store migration status information representing a migration state of memory portions in the main memory of the cell containing the cache coherency directory.

20. The distributed-memory multi-processor system of claim 19, wherein each cell controller includes firmware for performing the memory migration functions.