Method and system for collective file access using an mmap (memory-mapped file)

• US 20080120474A1
• Filed: 11/17/2006
• Published: 05/22/2008
• Est. Priority Date: 11/17/2006
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Coverston, Harriet G., Hastings, Andrew B., Choudhary, Alok N.
• Granted Patent: US 7,647,471 B2
• Time in Patent Office: Days
• US Class Current: 711/147
• CPC Class Codes: G06F 12/1036 for multiple virtual address spaces, e.g. segmentation G06F12/1045 takes precedence
• CPC Class Codes: G06F 12/1072 Decentralised address translation, e.g. in distributed shared memory systems
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 2212/656 Address space sharing

Abstract

A method for processing using a shared file that includes creating a plurality of mmaps between a shared file and a plurality of address spaces, wherein each of the plurality of mmaps maps at least a portion of the shared file to one of the plurality of address spaces, and wherein each of the plurality of address spaces is associated with one of a plurality of processors, transferring, in parallel, data between the shared file and the address spaces using the plurality of mmaps associated with the plurality of address spaces, processing the data in parallel by the plurality of processors to obtain a result, wherein the plurality of processors access data from the plurality of address spaces, and storing the result in the shared memory.

Claims

1. A method for processing using a shared file comprising:

creating a plurality of mmaps between a shared file and a plurality of address spaces,

wherein each of the plurality of mmaps maps at least a portion of the shared file to one of the plurality of address spaces, and

wherein each of the plurality of address spaces is associated with one of a plurality of processors;

transferring, in parallel, data between the shared file and the address spaces using the plurality of mmaps associated with the plurality of address spaces;

processing the data in parallel by the plurality of processors to obtain a result, wherein the plurality of processors access data from the plurality of address spaces; and

storing the result in the shared memory.

2. The method of claim 1, wherein creating at least one of the plurality of mmaps comprises:

identifying a maximum offset in shared file, wherein the maximum offset is associated with one of the plurality of processors;

mapping a contiguous region of the shared file specified by the maximum offset; and

storing the mapping in the at least one of the plurality of mmaps.

3. The method of claim 1, further comprising:

identifying the global maximum offset for the plurality of processors in a shared memory;

determining whether the global maximum offset corresponds to a memory location in the shared memory in which the shared filed is stored; and

generating an exception if the global maximum offset does not correspond to a memory location in the shared memory in which the shared filed is stored.

4. The method of claim 1, wherein the shared file is located in a shared memory accessible by at least two of the plurality of processors.

5. The method of claim 1, wherein transferring data is performed using one selected from a group consisting of a load operation and a store operation.

6. The method of claim 1, wherein the data for each of the plurality of processors is stored in interleaving portions of the shared file.

7. The method of claim 1, wherein the at least two of the plurality of mmaps map at least one common portion of the shared file.

8. The method of claim 1, wherein transferring data between the shared file and the plurality of address spaces is performed transparently to each processor of the plurality of processors.

9. The method of claim 1, wherein the plurality of processors execute a common application to process the data.

10. A system comprising:

a shared memory for storing a shared file; and

a plurality of processors connected to the shared memory and configured to;

create a plurality of mmaps between the shared file and a plurality of address spaces,

wherein each of the plurality of mmaps maps at least a portion of the shared file to one of the plurality of address spaces, and

wherein each of the plurality of address spaces is associated with one of the plurality of processors;

transfer, in parallel, data between the shared file and the address spaces using the plurality of mmaps associated with the plurality of address spaces;

process the data in parallel by the plurality of processors to obtain a result, wherein the plurality of processors access data from the plurality of address spaces; and

store the result in the shared memory.

11. The system of claim 10, wherein creating at least one of the plurality of mmaps comprises:

identify a maximum offset in shared file, wherein the maximum offset is associated with one of the plurality of processors;

map a contiguous region of the shared file specified by the maximum offset;

store the mapping in the at least one of the plurality of mmaps.

12. The system of claim 10, wherein the plurality of processors are further configured to:

identify the global maximum offset for the plurality of processors in a shared memory;

determine whether the global maximum offset corresponds to a memory location in the shared memory in which the shared filed is stored; and

generate an exception if the global maximum offset does not correspond to a memory location in the shared memory in which the shared filed is stored.

13. The system of claim 10, wherein the shared file is located in a shared memory accessible by at least two of the plurality of processors.

14. The system of claim 10, wherein transferring data is performed using one selected from a group consisting of a load operation and a store operation.

15. The system of claim 10, wherein the data for each of the plurality of processors is stored in interleaving portions of the shared file.

16. The system of claim 10, wherein the at least two of the plurality of mmaps map at least one common portion of the shared file.

17. The system of claim 10, wherein transferring data between the shared file and the plurality of address spaces is performed transparently to each processor of the plurality of processors.

18. The system of claim 10, wherein the plurality of processors execute a common application to process the data.

19. A shared memory computer system comprising a plurality of processors and a shared memory, wherein each processor in the plurality of processors is configured to:

create a mmap between a shared file and an address space associated with a processor;

transfer, in parallel, data between the shared file and the address space using the mmap with the plurality of processors; and

process the data by the processor to obtain processed data,

wherein the processed data is used to obtain a result,

wherein the result is stored in the shared memory, and

wherein the shared file is in the shared memory and accessible by each of the plurality of processors.

20. The shared memory computer system of claim 19, wherein transferring data is performed using one selected from a group consisting of a load operation and a store operation.