Multi-writer in-memory non-copying database (MIND) system and method

• US 8,769,350 B1
• Filed: 09/20/2011
• Issued: 07/01/2014
• Est. Priority Date: 09/20/2011
• Status: Active Grant

• Current Assignee: Advent Software Incorporated (SS&C Technologies Holdings Incorporated)
• Original Assignee: Advent Software Incorporated (SS&C Technologies Holdings Incorporated)
• Inventors: Ro, Kwang T., Andersen, Brian L., McGrogan, Stephen K.
• Primary Examiner(s): PATEL, KAMINI B
• Application Number:
  US13/237,240
• Time in Patent Office: 1,015 Days
• Field of Search: 714/49
• US Class Current: 714/49
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 2201/80 Database-specific techniques

Abstract

Embodiments of the invention relate to memory management methods and systems for object-oriented databases (OODB). In an embodiment, a database includes a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium and not in main memory. An application program connects to the database with a plurality of writing processes to simultaneously write to an in-memory database, each writing process updating its own disk-based logfile, such that the effective disk writing speed is substantially increased and lock conflicts reduced.

Claims

1. A system for an application program to manage memory associated with an object-oriented database (OODB) accessed by the application program, the application program associated with an operating system, the system for an application program to manage memory comprising:

1) a non-transitory computer memory; and

2) a programmable processor in electronic communication with the memory, the processor configured to execute one or more modules comprising;

(a) connecting, by the processor, to a data repository of the database, the data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium, wherein the database is not stored in a main memory of a computer connecting to the data repository;

(b) registering, by the processor, a fault handler with the operating system, the fault handler associated with the data repository;

(c) catching, by the fault handler, a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program;

(d) finding, by the processor, a file segment of the data repository that corresponds to the referenced object;

(e) mapping, by the processor, the found file segment into the main memory;

(f) restarting, by the processor, the application program at the interrupt location; and

(g) using, by the processor, a plurality of writing processes to simultaneously update a plurality of objects in the data repository, wherein each writing process writes to one object at a time, and wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object, and wherein file segments in the data repository are not copied into the main memory from the data repository when needed by the application program and are instead read, updated, deleted or modified directly in the data repository by the application program.

2. The system of claim 1, wherein the application program is a portfolio management application, and the database is a portfolio management database.

3. The system of claim 1, wherein the file segments include at least one portfolio segment and price segment.

4. The system of claim 1, further comprising a module programmed for unmapping a second file segment from the main memory before mapping the found file segment.

5. The system of claim 4, wherein file segments are unmapped according to a least-recently-used (LRU) criterion.

6. The system of claim 1, wherein objects in the data repository are directly interconnected by memory pointers.

7. The system of claim 6, wherein a linkage in the interconnected objects includes at least one of an X node, a Y node, or a Z node.

8. The system of claim 7, wherein the X node is programmed to connect to at least one object instance.

9. The system of claim 7, wherein the Y node is programmed to define the start of like types of objects.

10. The system of claim 7, where the Z node is programmed as an object for pointing from a variant of an object to at least one other variant of the object.

11. The system of claim 1, wherein each object includes a virtual function pointer, the pointer pointing to a shared memory area holding virtual function tables associated with object types.

12. The system of claim 1, wherein every object in a class of the data repository points to a same predetermined shared memory address when stored.

13. The system of claim 1, wherein each object includes a pointer to itself.

14. The system of claim 13, wherein each object pointer comprises an object containing its own knowledge begin date and knowledge end date.

15. The system of claim 1, wherein each object in the data repository has an associated stored time stamp, the time stamp indicating at least one of a time when the object first appeared in the data repository and a time when the object became invalid.

16. The system of claim 1, wherein the file segments include at least one portfolio segment and price segment.

17. The system of claim 1, further comprising a module programmed for:

(a) upon a request by the application program to store a new object in the database, creating a segment object in the data repository;

(b) associating a segment identifier with the new object, the segment identifier being one of a default segment identifier, a segment identifier specified by the new object, and a segment identifier specified by another object that owns the new object;

(c) when a current segment file has sufficient memory for the new object, allocating memory to the new object from the current segment file;

(d) when the current segment file has insufficient space for the new object, allocating memory to the new object by extending the current segment file or creating a new segment file; and

(e) storing the new object in the allocated memory.

18. The system of claim 1, wherein the data repository is connected to an NFS (Network File System) network.

19. The system of claim 18, wherein multiple computers access the data repository via the NFS network.

20. The system of claim 1, wherein the data repository resides in multiple computers.

21. The system of claim 1, wherein the found file segment is stored in a segment library having a two-level directory structure.

22. The system of claim 21, wherein a file name of the found file segment includes a hexadecimal digit sequence representative of a portion of a memory address of the found file segment.

23. The system of claim 21, wherein a directory name of a directory containing the found file segment includes a hexadecimal digit sequence representative of a portion of a memory address of the found file segment.

24. The system of claim 1, wherein the application program and database are associated with a trade order management system (TOMS).

25. The system of claim 1, wherein at least one of the writing processes is programmed for updating its own log file.

26. The system of claim 1, wherein at least one of the writing processes is programmed to write to volatile memory and to a disk-based log file.

27. The system of claim 1, wherein the database is configured for access simultaneously by multiple reading processes located on multiple computers.

28. The system of claim 1, further comprising a module programmed for entering an object into the database with a knowledge begin time and a knowledge end time.

29. The system of claim 28, further comprising a module programmed for setting the knowledge end time of an original object to a current time when the object is updated.

30. The system of claim 29, further comprising the module programmed for setting a new knowledge begin time of a new variant of the original object.

31. The system of claim 30, wherein each variant is programmed to point to the next variant older than itself and also to a linkage object that points to the newest variant.

32. The system of claim 30, further comprising a module programmed for creating a past report for audit purposes in association with at least one object variant.

33. The system of claim 1, wherein each genus includes its own writing process, lock, log file, and set of segment files.

34. The system of claim 1, wherein each genus includes its own write clock and read clock.

35. The system of claim 1, wherein each genus includes its own locking object programmed for automatic selection by a virtual function.

36. The system of claim 35, wherein the locking object includes at least a lock and a pointer to a corresponding log file.

37. The system of claim 1, wherein at least one writing process is programmed to ask for a lock object and to sleep for a random period when the lock object is already in use.

38. The system of claim 1, further comprising a module programmed for declaring at least one of the writing processes a penalty writing process which is forced to sleep the next time the penalty writing process asks for a lock object.

39. The system of claim 1, wherein a first genus includes price objects, a second genus includes user session objects, and a third genus includes other objects not included in the first genus or the second genus.

40. The system of claim 1, further comprising a module programmed for using a segment creation lock for locking the database during creation of a new segment.

41. The system of claim 1, wherein the data repository is programmed for applying one or more transaction log files to a most recent checkpoint file for recovery from a computer crash.

42. The system of claim 1, wherein the data repository comprises a price segment representing a time period of prices for one or more investments in an operatively associated portfolio management application.

43. The system of claim 1, wherein the data repository comprises a portfolio segment programmed for holding transactions for individual portfolios and including at least one link pointing to an individual portfolio, at least one object owned by an individual portfolio, and at least one link pointing to an owned object.

44. The system of claim 1, wherein a plurality of objects stored in the memory-mapped file segments of the data repository are divided into species.

45. The system of claim 44, wherein the species include at least one of price species, a portfolio events species, a control objects species, a derived numerical results species, or a core object species.

46. A non-transitory machine-readable medium encoded with a plurality of processor executable instructions for:

(a) connecting to a data repository of an object-oriented database (OODB) accessed by an application program, the application program associated with an operating system, the data repository including a plurality of memory-mapped file segments stored on at least one nonvolatile memory medium, wherein the database is not stored in a main memory of a computer connecting to the data repository;

(b) registering a fault handler with the operating system, the fault handler associated with the data repository;

(c) catching, by the fault handler, a segmentation fault issued for an object referenced by the application program and resident in the data repository, the segmentation fault issued at an interrupt location in the application program;

(d) finding a file segment of the data repository that corresponds to the referenced object;

(e) mapping the found file segment into the main memory;

(f) restarting the application program at the interrupt location; and

(g) using a plurality of writing processes to simultaneously update a plurality of objects in the data repository, wherein each writing process writes to one object at a time, and wherein the objects in the data repository are divided into a plurality of genera, and wherein each genera has its own transaction log and lock object, and wherein file segments in the data repository are is not copied into the main memory from the data repository when needed by the application program and are instead read, updated, deleted or modified directly in the data repository by the application program.

47. The non-transitory machine-readable medium of claim 46, wherein each object in the data repository has an associated stored time stamp, the time stamp indicating at least one of a time when the object first appeared in the data repository and a time when the object became invalid.