Multi-writer in-memory non-copying database (MIND) system and method

• US 8,291,269 B1
• Filed: 11/14/2011
• Issued: 10/16/2012
• Est. Priority Date: 09/20/2011
• Status: Active Grant

• Current Assignee: Advent Software Incorporated (SS&C Technologies Holdings Incorporated)
• Original Assignee: Advent Software Incorporated (SS&C Technologies Holdings Incorporated)
• Inventors: McGrogan, Stephen K., Ro, Kwang T., Andersen, Brian L.
• Primary Examiner(s): Patel, Kamini
• Application Number:
  US13/295,697
• Time in Patent Office: 337 Days
• Field of Search: 714/49
• US Class Current: 714/49
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 2201/80 Database-specific techniques

Abstract

Embodiments of the invention relate to memory management methods and systems for object-oriented databases (OODB). In an embodiment, a database includes a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium and not in main memory. An application program connects to the database with a plurality of writing processes to simultaneously write to an in-memory database, each writing process updating its own disk-based logfile, such that the effective disk writing speed is substantially increased and lock conflicts reduced.

Claims

1. A method for an application program to access an object-oriented database (OODB), the application program being one of a portfolio management application and a trade order management system (TOMS) application, the application program being associated with an operating system, the method comprising:

(a) connecting to a data repository of the database, the data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium, wherein the database is not stored in a main memory of a computer connecting to the data repository, the file segments including objects directly interconnected by memory pointers, wherein each object has an associated stored time stamp, the time stamp indicating at least one of a time when the object first appeared in the data repository and a time when the object became invalid, wherein the data in the data repository is not copied into main memory from the data repository when needed by the application program, and wherein the database is one of a portfolio management database and a TOMS database;

(b) registering a fault handler with the operating system, the fault handler associated with the data repository;

(c) catching, by the fault handler, a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program;

(d) finding a file segment of the data repository that corresponds to the referenced object;

(e) mapping the found file segment into the main memory;

(f) restarting the application program at the interrupt location; and

(g) using a plurality of writing processes to simultaneously update a plurality of objects in the data repository, wherein each writing process writes to one object at a time, and wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object.

2. The method of claim 1, wherein the file segments include at least one portfolio segment and price segment.

3. A method for an application program to manage memory associated with an object-oriented database (OODB) accessed by the application program, the application program associated with an operating system, the method comprising:

(a) connecting to a data repository of the database, the data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium, wherein the database is not stored in a main memory of a computer connecting to the data repository;

(b) registering a fault handler with the operating system, the fault handler associated with the data repository;

(c) catching, by the fault handler, a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program;

(d) finding a file segment of the data repository that corresponds to the referenced object;

(e) mapping the found file segment into the main memory;

(f) restarting the application program at the interrupt location; and

(g) using a plurality of writing processes to simultaneously update a plurality of objects in the data repository, wherein each writing process writes to one object at a time, and wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object, and wherein file segments in the data repository are not copied into the main memory from the data repository when needed by the application program and are instead read, updated, deleted or modified directly in the data repository by the application program.

4. The method of claim 3, further comprising unmapping a second file segment from the main memory before mapping the found file segment.

5. The method of claim 4, wherein file segments are unmapped according to a least-recently-used (LRU) criterion.

6. The method of claim 3, wherein the nonvolatile memory medium is a disk.

7. The method of claim 3, wherein objects in the data repository are directly interconnected by memory pointers.

8. The method of claim 7, wherein the objects comprise C++ objects.

9. The method of claim 7, wherein a linkage in the interconnected objects includes at least one of an X node, a Y node, and a Z node.

10. The method of claim 3, wherein each object includes a virtual function pointer, the pointer pointing to a shared memory area holding virtual function tables associated with object types.

11. The method of claim 3, wherein every object in a class of the data repository points to a same predetermined shared memory address when stored.

12. The method of claim 3, wherein each object includes a pointer to itself.

13. The method of claim 3, wherein each object in the data repository has an associated stored time stamp, the time stamp indicating at least one of a time when the object first appeared in the data repository and a time when the object became invalid.

14. The method of claim 13, wherein the time stamp is stored in a header of the object.

15. The method of claim 3, wherein the application program is a portfolio management application, and the database is a portfolio management database.

16. The method of claim 15, wherein the file segments include at least one portfolio segment and price segment.

17. The method of claim 3, further comprising:

(a) upon a request by the application program to store a new object in the database, creating a segment object in the data repository;

(b) associating a segment identifier with the new object, the segment identifier being one of a default segment identifier, a segment identifier specified by the new object, and a segment identifier specified by another object that owns the new object;

(c) if a current segment file has sufficient memory for the new object, allocating memory to the new object from the current segment file;

(d) if the current segment file has insufficient space for the new object, allocating memory to the new object by extending the current segment file or creating a new segment file; and

(e) storing the new object in the allocated memory.

18. The method of claim 17, wherein memory is allocated in 16 megabyte segments.

19. The method of claim 17, wherein segment files are extended or created in 1 megabyte segments.

20. The method of claim 3, wherein the data repository is connected to a NFS (Network File System) network.

21. The method of claim 20, wherein multiple computers access the data repository via the NFS network.

22. The method of claim 3, wherein the data repository resides in multiple computers.

23. The method of claim 3, wherein the found file segment is stored in a segment library having a two-level directory structure.

24. The method of claim 23, wherein a file name of the found file segment includes a hexadecimal digit sequence representative of a portion of a memory address of the found file segment.

25. The method of claim 23, wherein a directory name of a directory containing the found file segment includes a hexadecimal digit sequence representative of a portion of a memory address of the found file segment.

26. The method of claim 3, wherein the application program and database are associated with a trade order management system (TOMS).

27. A computer system for an application program to manage memory associated with an object-oriented database (OODB) accessed by the application program, the application program associated with an operating system, the computer system for an application program to manage memory comprising:

an application and database server, the server including a secondary storage, the secondary storage including a data repository;

(b) means for connecting to the data repository of the database, the data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium, wherein the database is not stored in a main memory of a computer connecting to the data repository;

(c) means for registering a fault handler with the operating system, the fault handler associated with the data repository;

(d) means for catching, by the fault handler, a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program;

(e) means for finding a file segment of the data repository that corresponds to the referenced object;

(f) means for mapping the found file segment into the main memory;

(g) means for restarting the application program at the interrupt location; and

(h) means for using a plurality of writing processes to simultaneously update a plurality of objects in the data repository, wherein each writing process writes to one object at a time, and wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object, and wherein file segments in the data repository are not copied into the main memory from the data repository when needed by the application program and are instead read, updated, deleted or modified directly in the data repository by the application program.

28. The computer system of claim 27, wherein the application program is a portfolio management application, and the database is a portfolio management database.

29. An object-oriented database (OODB), comprising:

(a) a data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium;

(b) the file segments including objects directly interconnected by memory pointers;

(c) each object having an associated stored time stamp, the time stamp indicating at least one of a time when the object first appeared in the data repository and a time when the object became invalid; and

(d) the data in the data repository not being copied into a main memory of a computer accessing the data,

wherein the database is not stored in the main memory,

wherein the data repository has an associated fault handler, the fault handler registered by an application program with an operating system associated with the application program,

wherein the fault handler is configured to catch a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program,

wherein a file segment of the data repository that corresponds to the referenced object is mapped into the main memory,

wherein the application program is restarted at the interrupt location,

wherein a plurality of writing processes simultaneously update a plurality of objects in the data repository, and each writing process writes to one object, and

wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object.

30. The database of claim 29, wherein the application program is a portfolio management application, and the data repository is a portfolio management data repository.